Question

I’m planning to introduce a voting system for my bulletin board and I’m looking for a good and efficent algorithm to award users. The voting system I’m thinking about is quite simple. Let me explain you it.

• Every post made by a user can be voted up or down by every other user on the board only once. Users can not vote their own posts.
• An up vote costs 1 point and a down vote costs 2 points (just to avoid easy bashing and mobbing).

• Every X days, every user on the board receives an amount A of points to spend in up and down voting which is based on a simple formula:

  A = 5 + AwardUserPerformance();
  Where AwardUserPerformance() is a function that returns a value between 0 and 10 depending on the performance of the user during the past X days.
  

I can use the following variables inside AwardUserPerformance() function in order to calculate user’s performance:

1. T = time spent online by the user during the past X days
2. P = number of posts made by the user during the past X days
3. V = the sum of downvotes and upvotes given to user’s posts during the past X days (for example, if the user received 3 upvotes and 2 downvotes, then V=1)

I just can’t figure out a good algorithm to obtain a number between 0 and 10 from those variables.

Answer 1

It’s not a trivial question, and I don’t think you’ll be able to get a “perfect unique answer”, because the function you want depends a lot on how you want your rewards to work.

The most obvious approach would be to find a way to normalize each of your metrics between 0 (terrible) and 1 (perfect). You could then obtain a score between 0 and 1 on each of the measures, multiply them together, and multiply by 10, to get a number that will be by definition between 0 and 10. Time would be fairly easy (0 = no time spent, 1 = entire time spent online), the 2 others would require setting up some min and max thresholds, truncating everything above. Ex: 10 posts and more = 1, 0 posts = 0, in-between is n posts / 10.

There are issues with this, though. First, because of the multiplication effect, a 0 on any of the 3 will tank the total score to zero. More generally, it will make very hard to get a large score, because a low score on any dimension will heavily penalize the total. Then, setting up bounds is either arbitrary (you decide what the min and max is), or somewhat complex (you could base for instance the max on the observed number of posts by users, and pick the 90% fractile as your maximum bound). The reason this is a problem is that the bounds will depend on the activity on your site. If you start with 100 users, getting 99 upvotes will be pretty hard. Once you hit 1 million users, that will become way easier, and the rewards system should adjust (probably) to that.

Instead of using a multiplication between plain linear functions, you can use any increasing function which maps each of your measures to [0; 1]. A function with a decreasing slope (like a log) will give for instance “easy points” for moderately active users, but getting high marks will be harder. This is one of these cases where I would recommend using Excel, to visualize a bit what would happen with various functions.

Alternatively, to avoid the multiplication problem, you can use a separable function, and break the total score in time score + post score + vote score. In that case, you just need each of these functions to start at 0, and their max must sum to 10.

However, I would argue that the best system would simply to use the Votes received. It is likely correlated to the 2 others (you can’t be voted up without posting, and you can’t post without being online).