Basically we have sales people that request leads to call. Right now it tried a “fresh lead” query to get those.
If there aren’t any fresh leads it moves on to a “relatively new” query. We call these “sources” and essentially a closer will go through sources until they find a viable lead.
These queries all query the same table, just different groups of data. However, there is a lot of complex sorting on each query and between that and inserts/updates to the table (table being InnoDB) we’re experience lots of waits (no deadlocks i’m pretty sure since they don’t show in InnoDB status) so my guess is we have slow selects, coupled with lots of inserts/updates.
NOW, the ultimate question IS:
Should we query the DB for each source and grab about 100ish (obviously variable depending on the system) and cache them in memcached. Then, as closers request leads, send them from cache but update the cache to reflect an “is_acccepted” flag. This way we only call each source as we run out of cached leads so just once as we run out, instead of once per closer requesting a lead?
Then we can use simulated locking with memcached – http://code.google.com/p/memcached/wiki/FAQ#Emulating_locking_with_the_add_command
Does this seem like a viable solution? Any recommendations? We need to minimize the chances of lock waits desperately and quickly.
Sounds viable, but have you looked at your indexes and are you using proper isolation levels on your selects?
Previous SO question may help with the answer your seeking: Any way to select without causing locking in MySQL?
If you perform your select/update in a SP with full transaction’s this could also speed things up quite a bit due to optimization. Of course, there are times when SP’s in MySQL are much slower 🙁
I’d have put this as a comment, but haven’t reached that level yet 🙂
And I did read the part about inno-db, but experience has shown me improvements even with inno when using isolation levels.