Question

I have a large MySQL table (about 750 million rows) and I just want to extract a couple of columns.

SELECT id, delid FROM tbl_name;

No joins or selection criteria or anything. There is an index on both fields (separately).

In principle, it could just start reading the table and spitting out the values immediately, but in practice the whole system just chews up memory and basically grinds to a halt.

It seems like the entire query is being executed and the output stored somewhere before ANY output is produced…

I’ve searched on unbuffering, turning off caches etc, but just cannot find the answer.

(mysqldump is almost what I want except it dumps the whole table – but at least it just starts producing output immediately)

Answer 1

Have you looked at load data infile and select into?

I think the following is close to what you want to do:

select id,delid 
  into outfile 'C:\\ronnis.csv'
       fields terminated by ','
  from tbl_name;

<perform unix sort>

load data infile 'C:\\ronnis.csv' 
into table new_tbl_name
fields terminated by ',';