Question

I’m trying to come up with a weighted algorithm for an application. In the application, there is a limited amount of space available for different elements. Once all the space is occupied, the algorithm should choose the best element(s) to remove in order to make space for new elements.

There are different attributes which should affect this decision. For example:

• T: Time since last accessed. (It’s best to replace something that hasn’t been accessed in a while.)
• N: Number of times accessed. (It’s best to replace something which hasn’t been accessed many times.)
• R: Number of elements which need to be removed in order to make space for the new element. (It’s best to replace the least amount of elements. Ideally this should also take into consideration the T and N attributes of each element being replaced.)

I have 2 problems:

1. Figuring out how much weight to give each of these attributes.
2. Figuring out how to calculate the weight for an element.

(1) I realize that coming up with the weight for something like this is very subjective, but I was hoping that there’s a standard method or something that can help me in deciding how much weight to give each attribute. For example, I was thinking that one method might be to come up with a set of two sample elements and then manually compare the two and decide which one should ultimately be chosen. Here’s an example:

Element A: N = 5, T = 2 hours ago.
Element B: N = 4, T = 10 minutes ago.

In this example, I would probably want A to be the element that is chosen to be replaced since although it was accessed one more time, it hasn’t been accessed in a lot of time compared with B. This method seems like it would take a lot of time, and would involve making a lot of tough, subjective decisions. Additionally, it may not be trivial to come up with the resulting weights at the end.

Another method I came up with was to just arbitrarily choose weights for the different attributes and then use the application for a while. If I notice anything obviously wrong with the algorithm, I could then go in and slightly modify the weights. This is basically a “guess and check” method.

Both of these methods don’t seem that great and I’m hoping there’s a better solution.

(2) Once I do figure out the weight, I’m not sure which way is best to calculate the weight. Should I just add everything? (In these examples, I’m assuming that whichever element has the highest replacementWeight should be the one that’s going to be replaced.)

replacementWeight = .4*T - .1*N - 2*R

or multiply everything?

replacementWeight = (T) * (.5*N) * (.1*R)

What about not using constants for the weights? For example, sure “Time” (T) may be important, but once a specific amount of time has passed, it starts not making that much of a difference. Essentially I would lump it all in an “a lot of time has passed” bin. (e.g. even though 8 hours and 7 hours have an hour difference between the two, this difference might not be as significant as the difference between 1 minute and 5 minutes since these two are much more recent.) (Or another example: replacing (R) 1 or 2 elements is fine, but when I start needing to replace 5 or 6, that should be heavily weighted down… therefore it shouldn’t be linear.)

replacementWeight = 1/T + sqrt(N) - R*R

Obviously (1) and (2) are closely related, which is why I’m hoping that there’s a better way to come up with this sort of algorithm.

Answer 1

What you are describing is the classic problem of choosing a cache replacement policy. Which policy is best for you, depends on your data, but the following usually works well:

First, always store a new object in the cache, evicting the R worst one(s). There is no way to know a priori if an object should be stored or not. If the object is not useful, it will fall out of the cache again soon.

The popular squid cache implements the following cache replacement algorithms:

• Least Recently Used (LRU):
  • replacementKey = -T
• Least Frequently Used with Dynamic Aging (LFUDA):
  • replacementKey = N + C
• Greedy-Dual-Size-Frequency (GDSF):
  • replacementKey = (N/R) + C

C refers to a cache age factor here. C is basically the replacementKey of the item that was evicted last (or zero).

NOTE: The replacementKey is calculated when an object is inserted or accessed, and stored alongside the object. The object with the smallest replacementKey is evicted.

LRU is simple and often good enough. The bigger your cache, the better it performs.

LFUDA and GDSF both are tradeoffs. LFUDA prefers to keep large objects even if they are less popular, under the assumption that one hit to a large object makes up lots of hits for smaller objects. GDSF basically makes the opposite tradeoff, keeping many smaller objects over fewer large objects. From what you write, the latter might be a good fit.

If none of these meet your needs, you can calculate optimal values for T, N and R (and compare different formulas for combining them) by minimizing regret, the difference in performance between your formula and the optimal algorithm, using, for example, Linear regression.