Question

I have a rather large csv file where each line should end with a pipe (|) and if it doesn’t combine the next line into it until find a pipe again. This need to done using a shell script.

I got an answer as

awk '!/|$/{l=l""$0|next|}{print l""$0|l=""}' file

But it gives me error as size of each line is quite large for me. I found out that I should be using perl to do that and have tried something as below but it does produce the desired result.

perl -pe 's/^\n(|\n)/ /gs' input.csv > output.csv

My data looks like

A|1|abc|<xml/>|
|2|def|<xml
>hello world</xml>|
|3|ghi|<xml/>|

And the desired output should be

A|1|abc|<xml/>|
|2|def|<xml>hello world</xml>|
|3|ghi|<xml/>|

Obviously the line size is quite large than the sample input here.

Any help would be highly appreciated.

Answer 1

This should work:

perl -lne 'unless(/\|$/){$line=$line.$_}else{print $line." $_";undef $line}' your_file

if you want to do an inplace replacement do this:

perl -i -lne 'unless(/\|$/){$line=$line.$_}else{print $line." $_";undef $line}' your_file

check here regarding your comment