launchpad helpWhen working with Launchpad, certain actions such as commenting on a bug or translating a string cause a record to be stored in our database of the action type and the time it occurred. Each action type has a numerical score associated with it. Each action type is associated with a category (bugs, translations, specifications at the moment - 'support' category coming soon I hope).
The score associated with each action type is currently just a best guess. We have yet to do any serious analysis on the real data to determine which actions should be rewarded more and which actions rewarded less.
Each day, we summarize this information and calculate everyones current Karma.
For every user with Karma actions, we total the raw scores for every action they have performed, per category. The raw score for each action is reduced depending on its age; this is currently done linearly, so an action that happened 'today' is worth 100% of its raw score, an action that occurred 182 days ago 50% and an action that occurred more than 365 days ago is worth nothing.
We feel it important that Karma is reduced by time, as otherwise new users may feel that they have no hope of catching up with a long time user, even if that long time user is no longer active. It also means that karma given incorrectly will slowly become irrelevant (there is at least one outstanding bug where users are getting translation Karma when they shouldn't, and our data model does not let us tell this good karma from bad). The way Karma degrades, and the time frame of the degradation, is open for debate. We can plug in other algorithms easily enough to test alternatives - the original events are unaltered so no damage is done when we tweak the algorithm or alter the per action scores.
We now have the issue that the Karma being given for some categories vastly outweighs the Karma being given to other categories. At the moment, for every point being given in the bugs category, 10 points are being given for translations. And for every point being given in the specifications category, 200 points are being given for translations. This is a problem because it doesn't encourage people to do bug triage if they never have a hope of getting a similar Karma to translations.
In the long term, we hope to reduce this problem by adjusting the scores per action so that the all of the categories end up with a similar amount of Karma. However, we now also normalize the categories to ensure that the weight of the various categories automatically tunes so the total number of points in each category across the whole system are equal.
To do this normalization, we total the total amount of Karma given to people in each category (as calculated above - shrunk by time). We then work out the scaling factor for each category, and then multiple everyone's Karma in each category by this scaling factor (bugs around 10, specifications around 200, and translations unaltered since that pool has the most points). Note that because these scaling factors are recalculated daily, it introduces some 'wobble' into the Karma scores although this shouldn't be noticeable in the long term.
We then store the adjusted per category scores per user. These are the per category scores that are visible on your Karma details page (https://launchpad.net/people/stub/+karma). Your total Karma is the sum of these category scores.
Canonical Employees
Karma of Canonical employees is currenly being halved in order to clear up the 'top karma earners' list for community members.
GuilhermeSalgado: Do we keep doing this?
Future & Discussion
Instead of storing total_karma in TotalKarmaCache, we may want to store something like log(total_karma)*1000. This will make being a top karma scorer appear much more obtainable, as well making it appear that karma is not dropping too violently every day for high karma scorers.
- The normalization was introduced because people doing translations were always the ones with the most karma, since they got huge amounts of karma when a POFile was imported into Rosetta. Although the normalization may be a good idea it causes problems when we introduce a new category of karma, since people doing actions on this category will get their karma on that category multiplied by thousands, in order to balance with other category's karma. Also, I think rosetta doesn't give karma when importing POFiles anymore, so the normalization may not be needed anymore.
- Bug #72815 wants to switch of balancing, and suggests we 'assign karma points based on the likely number of minutes something takes to do'.
10:42:58) stub: salgado: I think we will need the normalization forever, but we have the choice of using hard coded values and tweaking them occasionally or the automatic algorithm (or similar) that is currently implemented. (10:43:32) stub: salgado: If the 'wobble' is the issue, there are alternatives such as rounding the scaling factor to the nearest 50 or somesuch, so it doesn't change every day. (10:45:10) Ubugtu: New bug: #70765 in launchpad "Following my person link from a features page doesn't work" [Undecided,Unconfirmed] http://launchpad.net/bugs/70765 (10:45:40) stub: With normalization, we only have to make sure that the karma values make sense within their category. Without scaling, we have to make sure they make sense globally which is a lot more difficult since you have to take into account changing variables such as how busy that particular bit of launchpad is
