Diff for "KarmaCalculation"

Not logged in - Log In / Register

Differences between revisions 3 and 4
Revision 3 as of 2007-04-02 11:51:27
Size: 3412
Editor: 62-30-68-94
Comment:
Revision 4 as of 2007-04-02 13:28:00
Size: 1982
Editor: 62-30-68-94
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
When working with Launchpad, certain actions such as commenting on a bug or translating a string cause a record to be stored in our database of the action type and the time it occurred. Each action type has a numerical score associated with it. Each action type is associated with a category (bugs, translations, specifications at the moment - 'support' category coming soon I hope). Karma is Launchpad's way of showing how active a person is in Launchpad.
Line 3: Line 3:
The score associated with each action type is currently just a best guess. We have yet to do any serious analysis on the real data to determine which actions should be rewarded more and which actions rewarded less. Almost everything you do in Launchpad helps you to build karma. For example: registering bugs, translating strings and answering support requests. Your karma is a reflection of the type of work you've done and when you did it.
Line 5: Line 5:
Each day, we summarize this information and calculate everyones current Karma. == How we calculate your karma ==
Line 7: Line 7:
For every user with Karma actions, we total the raw scores for every action they have performed, per category. The raw score for each action is reduced depending on its age; this is currently done linearly, so an action that happened 'today' is worth 100% of its raw score, an action that occurred 182 days ago 50% and an action that occurred more than 365 days ago is worth nothing. When you work in Launchpad, it records actions that are valuable to the community, such as commenting on a bug or translating a string.
Line 9: Line 9:
We feel it important that Karma is reduced by time, as otherwise new users may feel that they have no hope of catching up with a long time user, even if that long time user is no longer active. It also means that karma given incorrectly will slowly become irrelevant (there is at least one outstanding bug where users are getting translation Karma when they shouldn't, and our data model does not let us tell this good karma from bad). The way Karma degrades, and the time frame of the degradation, is open for debate. We can plug in other algorithms easily enough to test alternatives - the original events are unaltered so no damage is done when we tweak the algorithm or alter the per action scores. Each day, Launchpad looks at all of the work you've done and gives each action a score. The score it gives to an action depends on:
Line 11: Line 11:
We now have the issue that the Karma being given for some categories vastly outweighs the Karma being given to other categories. At the moment, for every point being given in the bugs category, 10 points are being given for translations. And for every point being given in the specifications category, 200 points are being given for translations. This is a problem because it doesn't encourage people to do bug triage if they never have a hope of getting a similar Karma to translations.  * what sort of work you did
 * when you did the work - e.g. filing a bug or editing a blueprint.
Line 13: Line 14:
In the long term, we hope to reduce this problem by adjusting the scores per action so that the all of the categories end up with a similar amount of Karma. However, we now also normalize the categories to ensure that the weight of the various categories automatically tunes so the total number of points in each category across the whole system are equal. For example: if you answer a support request today, Launchpad will give you the full score. If you answered a support request six months ago, Launchpad would award you half the score. After twelve months an action no longer counts towards your karma.
Line 15: Line 16:
To do this normalization, we total the total amount of Karma given to people in each category (as calculated above - shrunk by time). We then work out the scaling factor for each category, and then multiple everyone's Karma in each category by this scaling factor (bugs around 10, specifications around 200, and translations unaltered since that pool has the most points). Note that because these scaling factors are recalculated daily, it introduces some 'wobble' into the Karma scores although this shouldn't be noticeable in the long term. Your karma is the total of the score Launchpad awards you in each category of work.
Line 17: Line 18:
We then store the adjusted per category scores per user. These are the per category scores that are visible on your Karma details page (https://launchpad.net/people/stub/+karma). Your total Karma is the sum of these category scores. == Why is karma time sensitive? ==

Launchpad reduces the karma it gives to older actions for two reasons:

 * to reflect each individual's current work
 * and to give newer users the opportunity to catch up with longer-term users.

== How Launchpad calculates the score for each type of work ==

Overall, Launchpad ensures that the total score in each category of work is the same. For example: if the total karma for bug work were 500, Launchpad would tune each of the other categories so that they also produce a total of 500.

To do this normalisation, launchpad adds up the the total amount of karma given to people in each category. It then works out a scaling factor for each category, and multiplies everyone's karma in each category by this scaling factor. As these scaling factors are recalculated daily, it introduces some 'wobble' into the karma scores, although this evens out in the longer term.

Karma is Launchpad's way of showing how active a person is in Launchpad.

Almost everything you do in Launchpad helps you to build karma. For example: registering bugs, translating strings and answering support requests. Your karma is a reflection of the type of work you've done and when you did it.

How we calculate your karma

When you work in Launchpad, it records actions that are valuable to the community, such as commenting on a bug or translating a string.

Each day, Launchpad looks at all of the work you've done and gives each action a score. The score it gives to an action depends on:

  • what sort of work you did
  • when you did the work - e.g. filing a bug or editing a blueprint.

For example: if you answer a support request today, Launchpad will give you the full score. If you answered a support request six months ago, Launchpad would award you half the score. After twelve months an action no longer counts towards your karma.

Your karma is the total of the score Launchpad awards you in each category of work.

Why is karma time sensitive?

Launchpad reduces the karma it gives to older actions for two reasons:

  • to reflect each individual's current work
  • and to give newer users the opportunity to catch up with longer-term users.

How Launchpad calculates the score for each type of work

Overall, Launchpad ensures that the total score in each category of work is the same. For example: if the total karma for bug work were 500, Launchpad would tune each of the other categories so that they also produce a total of 500.

To do this normalisation, launchpad adds up the the total amount of karma given to people in each category. It then works out a scaling factor for each category, and multiplies everyone's karma in each category by this scaling factor. As these scaling factors are recalculated daily, it introduces some 'wobble' into the karma scores, although this evens out in the longer term.

KarmaCalculation (last edited 2008-06-17 14:21:18 by localhost)