Question

I want to design a fast database schema which can handle sorting and filtering columns as good as updating the entries.

For this I created the following scenario:

• An event has exactly one name, status, last-subscription-date, description and one location
• The number of available seats for an event is saved along with the event and will be updated every time an participant subcribes
• every event has exactly one category
• the events can only be listed by categories
• the events can be filtered by name, status or date (no xor)
• the events can be sorted by name, status or date (xor)
• the tables have to handle more than 10 mio entries

For all tests I used MySQL and InnoDB tables. I also tried to use multiple inserts/updates/deletes as often as possible.
Filtering is done by using LIKE ‘%[word]%’

First I tried to use 2 tables: One for the categories, the other one for the events. Indexes were category-name, category-status-name, category-date-name and category-date-status-name.
For this, listing, filtering and sorting was very fast, but inserting, updating or deleting entries was very slow. I also got lock timeouts, because rebuilding the indexes took too much time.

Second try was to have 3 tables: categories, events and locations.
But if the location-table contains 6 mio or more entries it gets also slow. I think because of the indexes for fast catches. Adding 100k entries take ~ 272 seconds. The indexes of locations were primary-index id and zip-street

The next try will be to create an own table for the last-subscription-date and the counter. But what about the possibility to filter for this date or to sort this one?

Is it better to have 3 indexes like: category-name, category-date, category-status or is my solution with the 4 indexes category-name, category-status-name, category-date-name and category-date-status-name the better one for MySQL?

I’m also thinking about the field types: Currently I used VARCHAR for the name. But maybe CHAR is the better one, because every entry has the same length and so it’s faster to jump to a specific position in the index instead of using variable lenghts. What do you think?

Does someone have some tips how a good and fast database schema have to be designed which supports a scenario as described above?

Answer 1

Indexes are fixed length, so CHAR vs VARCHAR won’t matter for indexes, though it does matter for table scans.

I don’t think that I can provide any other definitive answer without specifics. I can give you some general advice.

You should avoid inserting into clustered indexes (InnoDB primary key, or first unique key). Clustered indexes are often used with auto-incremented columns so that the index is only appended to, and nothing is inserted in the middle. This avoids having to rebuild the index.

For non-clustered (secondary) indexes, the larger the index, the more often it must be rebuilt on inserts. Inserts can be performed until a page fills up, then it is rebuilt. Again, appending to the end of an index is fine.

Deletes don’t affect performance, since the index is only marked for deletion, and the index is rebuilt during idle time.

Indexes should not be created on columns with low cardinality since MySQL will not use them. Indexes should only be added as needed, where you weigh the advantage and disadvantage each time.

Multi-column indexes are larger (less entries fit in a page) and require updating more entries. Add multi-column indexes sparingly.

MyISAM is better for frequent reads, but bogs down with frequent updates/inserts in a multi-user environment due to lock contention (table locks). InnoDB is better for updates in a multi-user enviroment due to less lock contention (row locks), but is slower for reads (still requires row locks).

Filtering of the form LIKE '%[word]%' cannot use indexes, though filtering LIKE '[word]%' can use indexes.

On a frequently updated system, indexes are as important for selecting records for updating as they are for reading them. The better the index, the less lock contention, hence better performance and less deadlocks.

The more JOINs, the higher the cost, and the slower the query. JOINs aren’t bad, but JOINs on many rows (a large result set) can be slow.

Some non-performance-related caveats:

With InnoDB, you should be prepared to handle failed transactions due to deadlocks.