Question

I want to be able to do this:

INSERT INTO TABLE_1(<LIST OF COLUMNS>)
SELECT <LIST OF ROWS> FROM (SELECT DISTINCT <OTHER COLUMNS> FROM TABLE_2);

How can I do this? I receive an error when I try to do it now.

Note that <LIST OF COLUMNS> is the same in both cases I use it, and also not that the fields in <OTHER ROWS> could, but do not necessarily exist in <LIST_OF COLUMNS>.

Answer 1

OK, a little philosophical treatise here…

“Distinct” means “reject duplicates”. And to detect duplicated rows you have compare them for equality first.

If you just compare rows on all columns, everything is fine: two rows are either equal or they are not, there is no third possibility. So you always know whether a row should be rejected as a duplicate or not.

However, if you try to compare rows by comparing a subset of their columns, you run into trouble: rows that are equal when compared on the subset of their columns may not be equal when compared on all columns. So are they duplicates or not?

• If you consider them duplicates and reject one of the rows, you’ll also be rejecting values that may not exist in the other row. You are effectively losing data (not to mention that which data you loose is random, so even the data you keep is essentially useless).
• If you don’t consider them duplicates, then you are not really making them distinct on the column subset, contradicting what you were trying to do in the first place.

So making rows distinct on the subset of columns can only be done:

• if you don’t keep the the other columns,
• or by “merging” values in other columns by subjecting them to aggregate functions such as MAX, COUNT, SUM etc..

This is the reason why GROUP BY must cover all non-aggregated columns. A DISTINCT is really just a GROUP BY on all columns.