Question

I’m trying to store what’s essentially log data in mySQL — potentially large amounts of data in uncompressed form ( 25GB+ month ).

Each row consists of only two columns, a datetime column as a primary key, and a data column containing between 8k – 16k of uncompressed data.

I’ve attempted to use innoDB’s ROW_FORMAT=compressed but it doesn’t seem to have any actual effect on the size of the database. Using my sample data, its 0.53GB without using compressed row formatting, but its still 0.53GB using compressed row format.

I’m checking the size of the stored data with the following query (its in a test database so my test tables will always be the ‘largest’):

SELECT CONCAT(table_schema, '.', table_name),
       CONCAT(ROUND(table_rows / 1000000, 2), 'M')                                    rows,
       CONCAT(ROUND(data_length / ( 1024 * 1024 * 1024 ), 2), 'G')                    DATA,
       CONCAT(ROUND(index_length / ( 1024 * 1024 * 1024 ), 2), 'G')                   idx,
       CONCAT(ROUND(( data_length + index_length ) / ( 1024 * 1024 * 1024 ), 2), 'G') total_size,
       ROUND(index_length / data_length, 2)                                           idxfrac
FROM   information_schema.TABLES
ORDER  BY data_length + index_length DESC
LIMIT  10;

Answer 1

The ARCHIVE storage engine may do what you want. It’s a special engine that stores rows in a compressed flat file format. It’s best for write-once read-infrequently use because it’s not indexed. But it’s very fast and very space-efficient.

http://dev.mysql.com/doc/refman/5.5/en/archive-storage-engine.html