Question

I am using ruby to perform a query on a MySQL db. I’m looking for matches of uids with:

WHERE uid in #{VERY_LARGE_COMMA_SEPARATED_LIST}

The VERY_LARGE_COMMA_SEPARATED_LIST is over 30k entries

VERY_LARGE_COMMA_SEPARATED_LIST = ' "one","two","three",...,"30k" '

and it causes sql to dump error:

Mysql2::Error - MySQL server has gone away:

Does anyone know what is going on here?

Answer 1

You must be going over typical 1MB mysql SQL command buffer.

To solve your problem, you should create (possibly temporary) table uids which contains just uid’s, something like

CREATE TABLE uids (uid INT NOT NULL PRIMARY KEY);

(be sure that index exists for uid).

Then INSERT necessary uid’s into uids table in few statements like this:

INSERT INTO uids (uid) VALUES (1),(5),(10),..;

Finally, run your query as:

 WHERE uid IN (SELECT uid FROM uids)

or join like this:

 SELECT m.*
 FROM master m, uids u
 WHERE m.uid = u.uid
 ...

In the end, you may want to drop your uids table.

EDIT: I missed that your uid is not integer. Simply adjust definitions for it to use VARCHAR instead.