I have data split across several mysql tables to prevent data duplication but joining them is too slow when I actually go to serve the data. I am thus building a much smaller cache table of the data which is likely to be needed in the near future (certain data is only relevant for a small time frame so the whole tables are ~700,000 rows but the cache is ~6,000).
Building this cache uses a temp table before copying into the actual cache table so that while the table is built each night, the live cache table doesn’t go down. Sometimes the cache table gets built in about 30 minutes, but other times it gets stuck and will never complete. In the process list it just says Copying to tmp table but it will spin for days if I let it.
What can I do to prevent getting stuck in Copying to tmp table? I have looked around and nowhere online can I find a very clear answer to what causes this except for some of the info here which I have tried to follow but to no avail.
I would think that there might be a better way for me to build the table rather than address memory concerns since the final table is small in rows (although each row is fairly large). The general form of the build query to make the cache table is
Select col1,col2,col3... FROM tbl1 INNER JOIN tbl2 on ... INNER JOIN tbl3 on ...
LEFT JOIN (SELECT col4, col5, col6... FROM tbl4 INNER JOIN tbl5 on ...)
AS tbl6 on ... WHERE col2 > NOW() ORDER BY col3, col4
The left join is needed since I want to have copies of the data in the inner select if it exists, but don’t want to limit by it if it doesn’t.
I’m sorry if this isn’t specific enough but if anyone has any idea how to prevent the Copying to tmp table I would greatly appreciate it.
Have you set up indexes on tables? Indexes on columns which you are using in joins will speed up the select query.
Maybe better solution for you in this case would be materialized view: http://fromdual.com/mysql-materialized-views