KayacanV's blog

By KayacanV, history, 7 weeks ago, In English

Hello, we will held a contest with cms, and we are wondering there is any suggestion to speed up to the evaluation process?

For example, if one of the test cases get the time limit in the subtask, it still continues to test other test cases in the same subtask. Is there an option in cms to stop the evaluation for same subtask after getting the WA/TLE.

Tags cms
  • Vote: I like it
  • +15
  • Vote: I do not like it

7 weeks ago, # |
  Vote: I like it +13 Vote: I do not like it

I don't think there's a straightforward way to "break early" on evaluating CMS subtasks. To my knowledge (running and making custom modifications on CMS 1.4 and previous), CMS focuses on distributing the evaluation of all testcases: so if you have n test cases and n workers, all will be evaluated at the same time, so it's not sequential. (For comparison, from memory, DOMJudge distributes the work of evaluating a single submission to a worker, rather than a single testcase). I'm sure with enough modifications you can try to remove the rest of the testcases for the subtask from the judging queue, but that's somewhat complicated and you would have to be pretty careful around race conditions. It's also specific to the "Subtask" score type.

I think your best bet would be to spin up more workers across multiple machines (e.g. on a cloud hosting platform). This also handles the worst case more effectively, and isn't very expensive if you're only doing it for a few hours.