Question

I have a tab separated file and I want to remove lines (keep one copy) that are identical only in the first three columns. I prefer to do this using unix, for instance awk or uniq.

Input file:

Supercontig_1.1 241783  286397  5677    52
Supercontig_1.1 241783  286397  5678    53
Supercontig_1.1 241783  286397  5679    53
Supercontig_1.2 10500  25700  3000    57
Supercontig_1.2 10500  25700  3001    59
Supercontig_1.2 10500  25700  3002    59
Supercontig_1.3 2000  7000  5686    60
Supercontig_1.3 2000  7000  5687    60

Output:

 Supercontig_1.1 241783  286397  5677    52
 Supercontig_1.2 10500  25700  3000    57
 Supercontig_1.3 2000  7000  5686    60

Answer 1

One way using awk:

awk '!array[$1,$2,$3]++' file.txt

Results:

Supercontig_1.1 241783 286397 5677 52
Supercontig_1.2 10500 25700 3000 57
Supercontig_1.3 2000 7000 5686 60