We broke Topcoder ratings

Revision en1, by EbTech, 2021-02-15 18:53:23

It's long been known that certain ratings systems, namely Glicko-2 and Topcoder, are not monotonic. In other words, there are cases where losing can eventually result in a higher rating. We wanted to know just how severe the issue can be. In joint work with inutard at WWW 2021, we computed how tourist's rating would evolve according to both Topcoder and our custom rating system. The dataset consists of Codeforces rounds up to Looksery Cup 2015, accessed via the Codeforces API. Here, we see that tourist's TopCoder rating is 3284, but could have been as high as 3807 if he were willing to lose on purpose!

Plot

More details on the adversarial strategy: for his first 45 rounds, we simulate tourist playing normally, following historical data. In the next 45 rounds, he purposely becomes last place whenever his Topcoder rating is above 2975, but plays normally otherwise. Then finally, he returns to playing normally for an additional 15 rounds.

A similar strategy recently broke the Pokemon Go Battle League rankings, which seem to be based on Glicko-2: https://www.reddit.com/r/TheSilphRoad/comments/hwff2d/farming_volatility_how_a_major_flaw_in_a/.

Tags rating, topcoder

History

 
 
 
 
Revisions
 
 
  Rev. Lang. By When Δ Comment
en4 English EbTech 2021-02-16 05:33:02 1 Tiny change: 'ain ratings systems, ' -> 'ain rating systems, '
en3 English EbTech 2021-02-15 19:06:30 2 Tiny change: '-15]'s TopCoder ratin' -> '-15]'s Topcoder ratin'
en2 English EbTech 2021-02-15 18:59:51 37
en1 English EbTech 2021-02-15 18:53:23 1403 Initial revision (published)