For a web application I’m developing, I need to store a large number of records. Each record will consist of a primary key and a single (short-ish) string value. I expect to have about 100GB storage available and would like to be able to use it all.
The records will be inserted, deleted and read frequently and I must use a MySQL database. Data integrity is not crucial, but performance is. What issues and pitfalls am I likely to encounter and which storage engine would be best suited to the task?
Many thanks, J
Whatever solution you use, since you say your database will be write-heavy you need to make sure the whole table doesn’t get locked on writes. This rules out MyISAM, which some have suggested. MyISAM will lock the table on an update,delete or insert. That means any client who wants to read from the table will have to wait for the write to finish. Dunno what the INSERT LOW PRIORITY does though, probably some hack around table-locking 🙂
If you simply must use MySQL, you’ll want InnoDB, which doesn’t lock on write. I dunno how MySQL does VACUUM’s InnoDB tables (InnoDB is MVCC like PostgreSQL and so needs to clean up)… but you’ll have to take that into consideration if you are doing a lot of updates or deletes.