Question

I have a pipe-delimited datafile that I want to load into Mysql via LOAD DATA INFILE.

Something like:

a|||d|e
x|y|z|0|1
...

I want replace an empty field with \N so that the data is recognized as a NULL value.

doing

sed ‘s/||/|\\N|/g’

almost works, but not quite – in that first data line, the second pipe is eaten up by sed so that the second contiguous empty field isn’t parsed!!!

My hacky work-around is to pipe the result of the first sed to do another identical search and replace a second time to get rid of those contiguous empty fields.

My question is, what is a more elegant way to get around this problem in sed (preferably in one pass)?

TIA.

Additional Note

If you plan to process your data file with sed, please also watch out for end of line cases like this:

h|i|j|k|

You will need to pipe to

sed ‘s/|$/|\\N/g’

to handle those end-of-line cases.

Also, if you are going from DOS to UNIX systems, you may want to erase ‘\r’ from your data file using sed search-and-replace.

I recommend that you account for all warnings from Mysql when using LOAD DATA INFILE – they will help you catch the evil cases.

Answer 1

You can use the repeat (t) command:

echo 'a|||d|e' | sed ':repeat; s/||/|\N|/g; t repeat'