Fixing the contest management mess

Правка en7, от purplesyringa, 2022-07-24 18:08:42

...or "How legacy systems fight obstacles no sane person would introduce in the first place".

Hello, Codeforces!

I am sure that those of you who tried to host their own programming competitions or write their own utilities for contest management are aware of the state of affairs regarding contest hosting. There are lots of incompatible formats, no one knows exactly how stuff ought to work, Polygon is full of nondescript and seemingly contradictory options, ej-polygon never seems to work quite correctly, modifications have to be applied to jury archives to make them work on your particular installation, non-trivial problem types require constant maintenance and hacks, ad infinitum.

Personally, I stumbled upon all these problems when I tried to get my own judge system working and suddenly discovered that the difficult part is not judging complex problems, but handling artificial complexity introduced by legacy software and standards. So here I am, attempting to solve these problems.


Firstly, I introduce a new format, which I shall call problem.xml format, which, as is perhaps obvious, is based on the Polygon format. I introduced one or two special cases here and there to ensure it's 99% compatible with the archives generated by Polygon presently. Unlike Polygon's format, it's completely documented, and as little leeway is allowed as possible while not compromising efficiency.

This format enables many, almost arbitrary problem types, in addition to the default types: input/output, interactive, run-twice, and problems with graders. For example:

  • Problems with custom scorers (better known as valuers by Ejudge adopters) are supported. This means that the points of a solution are not necessarily equal to the sum of points of each test; rather, any relations are possible, up to negative scores, efficiency-rated programs, and "write a program that prints your username".

  • What I shall call formulaic problems, when the user solution outputs a formula or a universal algorithm that is then executed by the judge.

  • Optional compilation on each test, which would come in handy for some practical development competitions.

  • Output-only problems, meaning that the user is asked to submit a ZIP archive that contains the answers to each test.

  • (Optional support) arbitrary strategies, which allow problemsetters to generalize the above as they see fit: run-thrice problems, compile-time testing, and CTF-like competitions are within reach with just a few lines of code,

  • Arbitration, allowing for marathon problems, which means that the points awarded to a submission may depend on the results of other submissions.

For existing platforms that support Polygon format or alike, few modifications will be necessary to get everything but strategy and arbitration working: I am hoping that major vendors are up to the task. Strategy support is somewhat tricky to implement, so that part is up in the air for the moment (not that this diminishes the rest of the work). Arbitration should theoretically be easy to get working, but might require some modifications to the software architecture.

The draft of the specification is available here: https://github.com/imachug/problem-xml-specs. While I think that the spec is mostly finished and I'm publishing it here for better visibility, I'd be glad to listen to your input and apply changes where necessary!


Tagging some people for better visibility--your input is highly appreciated: geranazavr555, MikeMirzayanov, grphil, andrewzta, dkirienko.

Теги polygon, ejudge, pcms2, codeforces

История

 
 
 
 
Правки
 
 
  Rev. Язык Кто Когда Δ Комментарий
ru9 Русский purplesyringa 2022-07-24 18:13:10 8
en9 Английский purplesyringa 2022-07-24 18:12:55 3 Tiny change: 'visibility--your input' -> 'visibility—your input'
en8 Английский purplesyringa 2022-07-24 18:11:10 0 (published)
ru8 Русский purplesyringa 2022-07-24 18:10:53 0 (опубликовано)
en7 Английский purplesyringa 2022-07-24 18:08:42 4 Tiny change: 'ssary!\n\n\n\n---\n\' -> 'ssary!\n\n---\n\'
ru7 Русский purplesyringa 2022-07-24 18:04:59 7 Мелкая правка: ' программы). Это зна' -> ' программы оценки). Это зна'
ru6 Русский purplesyringa 2022-07-24 18:04:17 275
en6 Английский purplesyringa 2022-07-24 18:02:23 253
ru5 Русский purplesyringa 2022-07-24 17:59:30 525
ru4 Русский purplesyringa 2022-07-24 17:52:21 921
ru3 Русский Death_on_2_Legs 2022-07-24 17:42:50 2 Мелкая правка: 'ьницами»\nПривет, ' -> 'ьницами»\n\nПривет, '
ru2 Русский Death_on_2_Legs 2022-07-24 17:42:00 2889
en5 Английский purplesyringa 2022-07-24 17:24:10 34 Tiny change: 'visibility: [user:ge' -> 'visibility--your input is highly appreciated: [user:ge'
ru1 Русский Death_on_2_Legs 2022-07-24 17:23:54 3084 Первая редакция перевода на Русский (сохранено в черновиках)
en4 Английский purplesyringa 2022-07-24 17:21:03 502
en3 Английский purplesyringa 2022-07-24 17:14:37 212
en2 Английский purplesyringa 2022-07-24 16:55:02 9 Tiny change: 'ems fight issues no sane' -> 'ems fight obstacles no sane'
en1 Английский purplesyringa 2022-07-24 16:43:35 2736 Initial revision (saved to drafts)