BencilSharpeniro's blog

By BencilSharpeniro, history, 7 weeks ago, In English,

Hi everyone!

I just wanted to get everyone's opinion on the USACO scoring system. For the uninformed, USACO uses a system where you are scored based on the number of test cases you get, rather than using subtasks and batch testing like other OIs.

One problem (that's recently been highlighted) is that it restricts problems that have yes/no output because just writing a program to print yes/no nets you at least 50% of cases. This is probably best shown through this problem: http://www.usaco.org/index.php?page=viewproblem2&cpid=941

So, is the grading system outdated? Should it be changed? Comment your opinion!

Complimentary Meme
 
 
 
 
  • Vote: I like it
  • +17
  • Vote: I do not like it

»
7 weeks ago, # |
  Vote: I like it 0 Vote: I do not like it

As a sidenote: Sometimes USACO does have other constraints for a subset of the test cases (much like Hotels from POI). For example Stamp Painting has 75% of test cases under a different constraint.

  • »
    »
    7 weeks ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    That's a good point. However, a lot of the problems also come from the lack of batch testing, which fails to reflect a lot of other OIs. (meaning you can get some test cases of a subtask and some test cases of another in a single submission)

  • »
    »
    7 weeks ago, # ^ |
    Rev. 2   Vote: I like it -10 Vote: I do not like it

    I hope USACO continues to TELL US more about the nature of their test cases (e.g. what percent have N <= 1000)...

    I think if one submits a simple O(N^2) then one can figure out themselves based on what TLE's, but that just seems... weird?

»
7 weeks ago, # |
Rev. 2   Vote: I like it -28 Vote: I do not like it

From a selfish point of view, I got a 14/15 on one problem last February so it is more beneficial to contestants I think. However, it implicitly harms contestants when other people can cheese answers with some randomness but I think that's a rare scenario... and honestly that's lowkey intelligent.

Knock on wood, hope this doesn't bite me later in the butt — I'd prefer situations where people with an almost correct idea get a majority of points at the cost of what I think to be the rarer situation of other people getting points with mainly incorrect ideas.

(photo credits? ;) )

  • »
    »
    7 weeks ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    One thing to note is point inflation. If it helps a lot of people, they'll probably make the cutoff higher (as seen in the bronze contest I linked.)

    • »
      »
      »
      7 weeks ago, # ^ |
        Vote: I like it -13 Vote: I do not like it

      In my opinion, point inflation is not a bad thing since the organizers coordinate cutoffs anyway

»
7 weeks ago, # |
  Vote: I like it +22 Vote: I do not like it

That's why by now most Yes/No problems use a multitest system.

»
7 weeks ago, # |
  Vote: I like it -24 Vote: I do not like it

problems that have yes/no output

Are bad and should be avoided or modified to have more informative output. Exceptions exist, but they're rare.