Users on my web app generate about 100k search requests per day (product catalogue), each request consists of a search term, category id, users ip and a timestamp. I have a single instance of MySQL – no replication or master / slave setup.
I am worried that all these writes will impact the performance of the app, just wondering how others have handled this situation with similar infrastructure?
I’ve thought about writing the logs to a simple csv file during the day and then a night, when there is very little web traffic, setup a rake task to import these records into the applicable table….this just does not “smell right” though.
Thanks!
There is several common pattern to scaling from the beginning.
First off, split your system to 2 layers: APP, DB. Each of them on independent server (cloud, vps, dedic). The second step is horizontal scaling of the DB (access to data from the APP): write to master, read from slave.
Actually, you just need to start work on it, then you have more questions what is better, what is worth to try.
Some useful resources for you: