Home/ Questions/Q 7497651
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-29T19:10:35+00:00

I have a table that has reached nearly 3 million records. While I appreciate

  • 0

I have a table that has reached nearly 3 million records. While I appreciate this amount may not be considered large relative to RDBMS, I have noticed slow down in operations that deal with this table. I feel I can optimize or re-architect it.

This is part of the database for the PocketBracket March Madness App. Essentially the table stores meta data for a one to many relationship (Brackets has many Bracket Picks). The twist is the demand on the table is different at times. For example, there is a short period (two weeks) where the table is performing mostly writes. But for the rest of the year it is mostly reads. Furthermore, a far majority of the records are not accessed.

Here is a screenshot of the current structure:

enter image description here

With that, here are some thoughts I have:

  • Place the old records in a separate table. Decreases the amount of records, but would require code modifications.
  • Denormalize the table so the models are 1-1 (i.e. condense all bracket picks into a single serialized column). Decreases the amount of records, but would require code modifications.
  • Swap the table’s engine and or indexes during demand periods (i.e. InnoDB/MyISAM).
  • Something I haven’t thought of…

I would appreciate your direction. In the end, I am fine with code changes, I just want to ensure I’m re-architecting in the right direction.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Before going crazy with “optimizations” like partitioning, sharding, denormalization etc… which will introduce just a lot of additional problems I would first try to detect what is causing the slow down.

    To give you an example I have a table of around 30 millions record and I do quite a lot of inserts, and reads per second, and I can get query results of around 2000 records in less than 300 ms. (still it could be probably improved)

    Additionally please keep in mind that partitioning makes sens when you can split the files between multiple disks and therefore you can increase the overall I/O. So in other words you would need to have a partition file per disk to achieve good performances.

    So first of all it is important to have some more info

    • Amount of server ram
    • Server disk/s type
    • Is there a RAID ? if yes which one ?
    • Is it a virtual server
    • How many queries per second does the mysql handle ?

    It could simply be that the table is just fragmented and the mysql config needs some more tuning.
    First of all you should switch to innodb, like Vyktor suggested, and then you have also to configure the innod db buffer to an higher value. The default one is too low.
    Here an example of my config file. Keep in mind that the parameters are tuned for my type of data and queries and server specs. Additionally I am using a variant of MySQL called percona, which could help you out as well, cause it is proved to be faster. On the site you can find some benchmarks.

    innodb_file_per_table  
innodb_file_format=barracuda

innodb_flush_log_at_trx_commit=2
innodb_buffer_pool_size = 3GB
query_cache_size = 98304
innodb_log_file_size = 10485760
innodb_log_buffer_size = 3145728

    I would also try to run a mysqlcheck. WARNING !!! it locks the table !

    If you need more info about mysql tuning, this is a great blog

    • 0