Question

Right now I am working with a text file that happens to have all the data next to each other like in the following string: AB005103012ES-** OF**MID115748607.83

However, I need to upload this to a MYSQL database, so ideally the translated format would have the code be separated with commas like so:

AB00,51,0301,2,ES-**,  ,O,F,**,M,I,D,1,157486,07.83

I am dealing with around 14 million lines and the commas would appear in the same character spot each time. As in the first comma would appear after the 4th character in each line, the second comma would appear after the 6th comma in each line and so on.

Is there any way to quickly achieve this in MYSQL or failing that, another simple program that could be run easily?

Answer 1

You can use INFILE along with the SUBSTR (substring) command.

http://dev.mysql.com/doc/refman/5.1/en/load-data.html

To load a text file with fixed width columns, I used the form:

LOAD DATA LOCAL INFILE '<file name>' INTO TABLE <table>
(@var1)
SET Date=str_to_date(SUBSTR(@var1,3,10),'%m/%d/%Y'), 
Time=SUBSTR(@var1,14,8), 
WindVelocity=SUBSTR(@var1,26,5),
WindDirection=SUBSTR(@var1,33,3),
WindCompass=SUBSTR(@var1,38,3),
WindNorth=SUBSTR(@var1,43,6),
WindEast=SUBSTR(@var1,51,6),
WindSamples=SUBSTR(@var1,61,4);