I have to import flat files into SQL Server. So far, I’ve used SSIS packages and Delphi programs to do the job but I’m getting more and more files to load. Many are very wide (more than 90 fields) and they all have their own specifics: some are fixed-width, some are delimited with a wide set of characters used as field and row delimiter, some use a “always insert” pattern while others use “update or insert”, etc. many of these files are several gigabytes in size making it even harder to handle them.
Now I’m starting to wonder if there isn’t a better way to handle this: some tool dedicated to doing that kind of work that could be used in a rational way: SSIS is nice but, frankly, the use of a GUI to do 99% of the work makes it completely impractical when dealing with non-trivial tasks.
Would anyone care to suggest a solution ?
You may use the format file bulk load.
Description:
http://msdn.microsoft.com/en-us/library/ms189636.aspx