Question

with every new visit from a search engine, I save the keyword (e.g. “php performance”) the visitor searched for at the search engine.

To count the number how often the keyword is used per day, I need to check whether somebody else already searched for the term that specific day.

After several months the table gets very big and the mysql check for the existence of the keyword that day takes longer and longer. So I switched to an extra table where I only save the keyword and give each keyword a specific ID to only have the checking process once and not each new day. This specific ID which I related to the keyword I insert in the table with diversification for each day.

This already works better but this table also gets bigger and bigger…

Does somebody has a good solution for this to higher performance?

Best,

Freddy

Answer 1

When it comes to performance there are some ‘tricks’ one can do to improve:

1. Build a caching mechanism. That is what you already did by making a second table. You could take that further and keep the cached table in memory or memcached.
2. Opimize your queries, and make sure there are indexes on the search field(s). Not shure if you did that.
3. Try to (de)normalize your database. Sometimes a different database structure is not optimal from a design point, but optimized for speed. Not sure if you did that.
4. Look at different solutions for searching, perhaps have a look at Lucene.