Question

I’m building a site where I am giving users the ability to drag and drop to order a list of items to rank them for their “personal view.” They can optionally remove an item to hide it from their “personal view.”

My question is how can I fairly implement a ranking algorithm to determine the ordering of the items for a shared view that doesn’t penalize new items.

It would also help if that can also be used to rank where new items would show up in a users personal list.

So if a new item comes along that is highly ranked by other users, we could display it where we predict the user would rank it related to their other rankings.

My initial thoughts is give points to each item ranked by a user = to the position in a users ranked list. (ex. If there are 10 items, give rank 1 10 pts, 2 9 pts, etc, with negative points awarded for items hidden by the user). And the shared view would sort based on total points. But this would not work well for new items that were largely unranked, and would not easily move up the ladder.

So any thoughts on a fair algorithm that can be predictive for new items?

Answer 1

So I think I have a working solution. By combining the approach I mentioned in the question comment, with the lower bound of Wilson’s score confidence interval for a Bernoulli parameter the score seems to align to my expectations.

So to rehash the approach from my comment: user item score = count of items + 1 – rank in the list / count of items. (1 of 3 = 1, 2 of 3 = .667, 2 of 5 = .8).

to give an overall item score I plug into the Wilson formula:
(phat + z*z/(2*n) – z * Math.sqrt((phat*(1-phat)+z*z/(4*n))/n))/(1+z*z/n)

Where phat = average of scores, n is number of rankings, z=1.96 (for a 95% confidence ranking).

I mocked up some data in Excel and played around with different scenarios and liked the results. Will move to implementation. Thanks for the help