Just launched an algorithm update that keeps note statuses and contributor impact scores more stable as we expand Community Notes. In addition to helping going forward, it boosts existing contributor impact scores by better recognizing helpful past contributions. Details 🧵
As the program expands, older data comprise an increasingly small fraction of the dataset, and scores on older notes have tended to fluctuate causing some of them to lose Helpful status.
To maintain Helpful note quality as the program continues to grow, we are locking note status once a note is two weeks old. This approach allows us to optimize the algorithm with a focus on current data while persisting helpful contributions on older topics.
As always, this change to our algorithm is open source and can be inspected in Github:
@CommunityNotes What about locking the note itself during the initial two weeks? Are the notes ever edited within that initial time period?
@CommunityNotes Locking a note after two weeks can be a problem: what if new evidence emerges later that contradicts it? An option to vote to reopen a note should be available at any time and a better informed note shall be produced to respond to our improved understanding of the subject matter.
@CommunityNotes Heaven forbid relevant information changes ...
@CommunityNotes Psychology Says Yes NY Times Magazine quotes a well-known psychologist, Daniel Kahneman, "When you make your own sandwich, you anticipate its taste as you're working on it. And when you think of a particular food for a while, you become less hungry for it later.Feb 15, 2018