Question

I have some data (separated by semicolon) with close to 240 rows in a text file temp1.
temp2.txt stores 204 rows of data (separated by semicolon).

I want to:

1. Sort the data in both files by field1, i.e. the first data field in every row.
2. Compare the data in both files and redirect the rows that are not equal in separate files.

Sample data:

temp1.txt
1000xyz400100xyzA00680xyz0;19722.83;19565.7;157.13;11;2.74;11.00
1000xyz400100xyzA00682xyz0;7210.68;4111.53;3099.15;216.95;1.21;216.94
1000xyz430200xyzA00651xyz0;146.70;0.00;0.00;0.00;0.00;0.00

temp2.txt
1000xyz400100xyzA00680xyz0;19722.83;19565.7;157.13;11;2.74;11.00
1000xyz400100xyzA00682xyz0;7210.68;4111.53;3099.15;216.95;1.21;216.94

The sort command I’m using:

sort -k1,1 temp1 -o temp1.tmp
sort -k1,1 temp2 -o temp2.tmp

I’d appreciate if someone could show me how to redirect only the missing/mis-matching rows into two separate files for analysis.

Answer 1

You want the difference as described at http://www.pixelbeat.org/cmdline.html#sets

sort -t';' -k1,1 temp1 temp1 temp2 | uniq -u > only_in_temp2
sort -t';' -k1,1 temp1 temp2 temp2 | uniq -u > only_in_temp1

Notes:

• Use join rather than uniq, as shown at the link above if you want to compare only particular fields
• If the first field is fixed width then you don’t need the -t’;’ -k1,1 params above