Question

I am trying to manipulate a text file. I’ve got most of it figured out myself, but I’m stumped with why sed seems to go into infinite loop mode. The text file can be downloaded from census.gov.

At the moment, I just want a list of states that I can throw into a for loop to do some state-specific processing. So far, I’ve got this. (I’m not a bash expert, suggestions are welcome.

sed 1d tables/ansi.csv | awk -F "," '{print $1}' | uniq | tr \n : | sed s/:/" "/g

I want to put this into $() to use the output in a for loop, but for some reason, sed is hanging up and not exiting. I actually need to add a couple of things to the final sed command, to properly format things, but I want to get this running correctly before I go any further.

In the end – I want something that looks like (just showing the first few):

"AL" "AK" "AZ" "AR" "CA" "CO" ....

Right now, sed returns more of less what I expect and returns (just showing the last few)

...."MP" "PR" "UM" "VI" "

But, rather than exiting, sed hangs and I have to Ctrl-C out of the script. If I remove the final sed statement, the little script runs as I would expect, without hanging.

So, why on earth is this hanging?

Answer 1

I would suggest putting the sed script inside quotes:

sed 1d tables/ansi.csv | awk -F "," '{print $1}' | uniq | tr '\n' : | sed 's/:/" "/g'

The reason that sed seems to “hang” may be that tr has removed the final newline which sed requires. By the way, the newline argument to tr needs to be quoted.

However, the whole thing can be done in AWK:

awk -F, 'NR > 1 {a[$1]=$1} END { delim=":"; num=asort(a); for (i=1;i<=num;i++) printf "\"%s\" ",a[i]; printf "\n"}' tables/ansi.csv