IvayloS's blog

By IvayloS, 10 years ago, In English

I have noticed that problem score distributions are often unfair and the last Round(FF) in Div 1 is not an exception. Although second and third problem gave the same number of points, third problem was solved by way less competitors.

I have a proposal on how to address this: introduce adaptive problem scoring. The problem score will be determined by the number of successful submissions for this problem. There are two options for this: either have a set rule of the type "If problem A has between X% and Y% accepted submissions it will be a 1000 pointer" or create a more flexible formula able to give other point scores, not only 500, 1000, 1500... The score of each problem will be determined at the end of the competition.

Another good feature of this is that with a bit for data mining rounds can be rated according to their quality. We can set expectations for a round and if a round does not meet the criteria it will be considered "not-good". I personally dislike rounds where only 3 out of 5 problems get any solutions and I believe such rounds should be avoided.

  • Vote: I like it
  • +4
  • Vote: I do not like it

»
10 years ago, # |
  Vote: I like it +8 Vote: I do not like it
»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

codeforces already support dynamic scoring.

»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

I really agree with you but there is something that with Dynamic scoring or anything else like you told that the score of each problem is not fixed and it will change.
That is, If you know what problem has what score it really helps you to solve which problems. But in dynamic scoring you don't know which problem should you start.
To be more clear, Fixed problem scores gives you a lot of information for solving problems. Anyway, It has some problems too, just like the ones you noticed.

»
10 years ago, # |
  Vote: I like it +20 Vote: I do not like it

I dont understand why authors does not use dynamic scoring instead of standart scoring.

  • »
    »
    10 years ago, # ^ |
      Vote: I like it +1 Vote: I do not like it

    I guess using dynamic scoring often causes coinsidences of scores. This effect is undesirable, imho. From the other hand, authors often cannot give us 5 problems with difficultiess which differ enough to say surely that all 5 problems must cost differently.

    • »
      »
      »
      10 years ago, # ^ |
        Vote: I like it +3 Vote: I do not like it

      The point is to give most points for hardest question. If thats not happening who cares about prediction of scores before solve them.

      I mean is this situation better for you: easiest problem gives 3000 points, but no problem we all knew it was 3000 before we decided to solve it(!).

  • »
    »
    10 years ago, # ^ |
    Rev. 3   Vote: I like it 0 Vote: I do not like it

    So far dynamic scoring works not very well. General idea is good, but it is not OK when just 1 additional submit may change problem cost from 1000 to 500 and move somebody few hundred places up/down in the standings.

    upd. Just curious — guys that downvoting me, you think it is good idea to have only 6 possible problem costs? Suppose such scenario: we have B and C of generally same difficulty, some guys solving it in order B-C and some other in order C-B. B is a little bit easier, so 1 minute before the end of contest cost of B is 500 (with AC range very close to 0.5, but a little better, like 500/999), and cost of C is 1000. Most of guys who solved in order C-B have much better scores than guys with B-C. And solving A and C looks much better than solving A and B, even with bad time on C. But then some random folk enters contest and submits A just few seconds before the end. He increases number of participants by 1, so now AC range on B is 500/1000, and it costs 1000 instead of 500. Users with C-B solving order fall from 50th place down to 200th, and users with B-C all move up. And guys with A and slow C may move from 100th place somewhere out of top-300. And that is only because of that random folk submitting A in last second, who had not even read B and C.

    As I said, general idea of dynamic scoring is good, but we need better formula for problem cost, not such rough one as we have now.

    • »
      »
      »
      10 years ago, # ^ |
        Vote: I like it +3 Vote: I do not like it

      I'll take a guess that you're mostly downvoted by trolls who downvote everyone and everything equally (it's not so rare to see external contest invitation blogs downvoted at the start) or by people who just give a comment the same (upvote/downvote/nothing) as has been given by majority. Don't worry about it too much.

      I just realized that I don't know if dynamic scoring depends on pretest AC or on systest AC.

      • »
        »
        »
        »
        10 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        It depends on systest AC. My example was simplified, like "suppose that all solutions later passed system tests".

        Because of that "systest AC!=pretest AC" there are also funny moments during system testing, when you may jump up in standings because of decreasing cost of some problem that you did not solve) At the beginning of system testing every problem costs 3000, like at the beginning of contest, and then cost is decreasing while number of AC increases. And sometimes you may look at "System testing — 96% complete" and think "come on, guys, few more AC on that problem C which i didn't solve, just few more AC for it to change cost from 1500 down to 1000, like problem B has, and i'll move up 50 places in final standings":)