I’ve got a very simple algorithm that I’m playing with to determine what user-submitted content should display on the front page of a website I’m building. Right now the algorithm is triggered each time the page is loaded. The algorithm goes through all of the posts, calculates the newest weighted score, and displays the posts accordingly.
With no one on the site, this works fine, but seems unlikely to scale well. Is it typical in industry to optimize these cases? If I need to cache the calculations, how should that best be done? I’m very much self-taught so although I can get things to work, I don’t always know if its ideal.
Part of the algorithm is time, which is important here. Aside from time, there are some other variables at play that I weight differently. I add all these "scores" together to form one "weighted score", the higher the score, the higher the post.
There is no standard, really. Some systems run on an interval, like once a day, or once an hour. Others run each time the page is accessed. Caching can be used to reduce the load in the latter case.
It all depends on how efficiently the algorithm scales, how many posts it will have to deal with and how often the information is needed. If the operation is quick and cheap, you may as well just run it every time the page is accessed for your initial version. If it is fast enough in your testing and doesn’t kill the server’s memory usage then doing any more work is a waste of time. If it isn’t good enough, think about caching the result, investing in better hardware or look for opportunities to improve the code.
If the results don’t need to change very often, just schedule it once an hour/inute or so and make sure it meets your needs before shipping.
It is generally better to test the simplest solution before worrying about optimisation.