launchpad help|
Size: 5563
Comment:
|
Size: 2078
Comment: karma clarification
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| Karma is Launchpad's way of showing how active a person is in Launchpad. | |
| Line 2: | Line 3: |
| When working with Launchpad, certain actions such as commenting on a bug or translating a string cause a record to be stored in our database of the action type and the time it occurred. Each action type has a numerical score associated with it. Each action type is associated with a category (bugs, translations, specifications at the moment - 'support' category coming soon I hope). | Almost everything you do in Launchpad helps you to build karma. For example: registering bugs, translating strings and answering support requests. Your karma is a reflection of the type of work you've done and when you did it. |
| Line 4: | Line 5: |
| The score associated with each action type is currently just a best guess. We have yet to do any serious analysis on the real data to determine which actions should be rewarded more and which actions rewarded less. | Karma is only attributed for work done in Launchpad and does not apply to imported items from other sources. |
| Line 6: | Line 7: |
| Each day, we summarize this information and calculate everyones current Karma. | = How we calculate your karma = |
| Line 8: | Line 9: |
| For every user with Karma actions, we total the raw scores for every action they have performed, per category. The raw score for each action is reduced depending on its age; this is currently done linearly, so an action that happened 'today' is worth 100% of its raw score, an action that occurred 182 days ago 50% and an action that occurred more than 365 days ago is worth nothing. | When you work in Launchpad, it records actions that are valuable to the community, such as commenting on a bug or translating a string. |
| Line 10: | Line 11: |
| We feel it important that Karma is reduced by time, as otherwise new users may feel that they have no hope of catching up with a long time user, even if that long time user is no longer active. It also means that karma given incorrectly will slowly become irrelevant (there is at least one outstanding bug where users are getting translation Karma when they shouldn't, and our data model does not let us tell this good karma from bad). The way Karma degrades, and the time frame of the degradation, is open for debate. We can plug in other algorithms easily enough to test alternatives - the original events are unaltered so no damage is done when we tweak the algorithm or alter the per action scores. | Each day, Launchpad looks at all of the work you've done and gives each action a score. The score it gives to an action depends on: |
| Line 12: | Line 13: |
| We now have the issue that the Karma being given for some categories vastly outweighs the Karma being given to other categories. At the moment, for every point being given in the bugs category, 10 points are being given for translations. And for every point being given in the specifications category, 200 points are being given for translations. This is a problem because it doesn't encourage people to do bug triage if they never have a hope of getting a similar Karma to translations. | * what sort of work you did * when you did the work. |
| Line 14: | Line 16: |
| In the long term, we hope to reduce this problem by adjusting the scores per action so that the all of the categories end up with a similar amount of Karma. However, we now also normalize the categories to ensure that the weight of the various categories automatically tunes so the total number of points in each category across the whole system are equal. | For example: if you answer a support request today, Launchpad will give you the full score. If you answered a support request six months ago, Launchpad would award you half the score. After twelve months an action no longer counts towards your karma. |
| Line 16: | Line 18: |
| To do this normalization, we total the total amount of Karma given to people in each category (as calculated above - shrunk by time). We then work out the scaling factor for each category, and then multiple everyone's Karma in each category by this scaling factor (bugs around 10, specifications around 200, and translations unaltered since that pool has the most points). Note that because these scaling factors are recalculated daily, it introduces some 'wobble' into the Karma scores although this shouldn't be noticeable in the long term. | Your karma is the total of the score Launchpad awards you in each category of work. |
| Line 18: | Line 20: |
| We then store the adjusted per category scores per user. These are the per category scores that are visible on your Karma details page (https://launchpad.net/people/stub/+karma). Your total Karma is the sum of these category scores. | = Why is karma time sensitive? = |
| Line 20: | Line 22: |
| == Canonical Employees == | Launchpad reduces the karma it gives to older actions for two reasons: |
| Line 22: | Line 24: |
| Karma of Canonical employees is currenly being halved in order to clear up the 'top karma earners' list for community members. | * to reflect each individual's current work * and to give newer users the opportunity to catch up with longer-term users. |
| Line 24: | Line 27: |
| * GuilhermeSalgado: Do we keep doing this? | = How Launchpad calculates the score for each type of work = |
| Line 26: | Line 29: |
| = Future & Discussion = | Overall, Launchpad ensures that the total score in each application is the same. For example: if the total karma for work in the Bug Tracker were 500, but the total for the Answer Tracker were 1,000, Launchpad would tune each category's score so that they were equal. |
| Line 28: | Line 31: |
| 1. Instead of storing total_karma in TotalKarmaCache, we may want to store something like log(total_karma)*1000. This will make being a top karma scorer appear much more obtainable, as well making it appear that karma is not dropping too violently every day for high karma scorers. 1. The normalization was introduced because people doing translations were always the ones with the most karma, since they got huge amounts of karma when a POFile was imported into Rosetta. Although the normalization may be a good idea it causes problems when we introduce a new category of karma, since people doing actions on this category will get their karma on that category multiplied by thousands, in order to balance with other category's karma. Also, I think rosetta doesn't give karma when importing POFiles anymore, so the normalization may not be needed anymore. 1. Bug #72815 wants to switch of balancing, and suggests we 'assign karma points based on the likely number of minutes something takes to do'. {{{ 10:42:58) stub: salgado: I think we will need the normalization forever, but we have the choice of using hard coded values and tweaking them occasionally or the automatic algorithm (or similar) that is currently implemented. (10:43:32) stub: salgado: If the 'wobble' is the issue, there are alternatives such as rounding the scaling factor to the nearest 50 or somesuch, so it doesn't change every day. (10:45:10) Ubugtu: New bug: #70765 in launchpad "Following my person link from a features page doesn't work" [Undecided,Unconfirmed] http://launchpad.net/bugs/70765 (10:45:40) stub: With normalization, we only have to make sure that the karma values make sense within their category. Without scaling, we have to make sure they make sense globally which is a lot more difficult since you have to take into account changing variables such as how busy that particular bit of launchpad is }}} |
To do this normalisation, Launchpad adds up the total amount of karma it has given to people in each category. It then works out a scaling factor for each category and multiplies everyone's karma in each category by this scaling factor. As these scaling factors are recalculated daily, it introduces some 'wobble' into karma scores, although this evens out in the longer term. |
Karma is Launchpad's way of showing how active a person is in Launchpad.
Almost everything you do in Launchpad helps you to build karma. For example: registering bugs, translating strings and answering support requests. Your karma is a reflection of the type of work you've done and when you did it.
Karma is only attributed for work done in Launchpad and does not apply to imported items from other sources.
How we calculate your karma
When you work in Launchpad, it records actions that are valuable to the community, such as commenting on a bug or translating a string.
Each day, Launchpad looks at all of the work you've done and gives each action a score. The score it gives to an action depends on:
- what sort of work you did
- when you did the work.
For example: if you answer a support request today, Launchpad will give you the full score. If you answered a support request six months ago, Launchpad would award you half the score. After twelve months an action no longer counts towards your karma.
Your karma is the total of the score Launchpad awards you in each category of work.
Why is karma time sensitive?
Launchpad reduces the karma it gives to older actions for two reasons:
- to reflect each individual's current work
- and to give newer users the opportunity to catch up with longer-term users.
How Launchpad calculates the score for each type of work
Overall, Launchpad ensures that the total score in each application is the same. For example: if the total karma for work in the Bug Tracker were 500, but the total for the Answer Tracker were 1,000, Launchpad would tune each category's score so that they were equal.
To do this normalisation, Launchpad adds up the total amount of karma it has given to people in each category. It then works out a scaling factor for each category and multiplies everyone's karma in each category by this scaling factor. As these scaling factors are recalculated daily, it introduces some 'wobble' into karma scores, although this evens out in the longer term.
