Some concerning trends related to system test duration

Revision en2, by stefdasca, 2024-06-04 16:00:31

Hello all,

While waiting to submit my solution for last div3's G, I realized that the system tests last way more as of late in div3, div4 and educational rounds. System tests started around 5 hours ago and we are only around 00:50 into the contest, as of the time I am writing this blog.

Although some of the difference is caused by the ever increasing number of contestants for these rounds, this still doesn't explain what seems to be the huge downgrade of the judging servers. From my memory, this phase used to not last more than 1h, maybe 2h in exceptional cases.

Educational Codeforces Round 166 (Rated for Div. 2) System tests lasted around 5 hours

Codeforces Round 946 (Div. 3) System tests lasted around 3 hours

Codeforces Round 944 (Div. 4) System tests lasted around 8 hours

Codeforces Round 943 (Div. 3) System tests lasted around 3.5 hours

Here is a series of 4 contests around 1 year ago

Codeforces Round 898 (Div. 4) System tests lasted around 2.5 hours

Codeforces Round 895 (Div. 3) System tests lasted around 2 hours

Educational Codeforces Round 154 (Rated for Div. 2) System tests lasted around 1 hour

Codeforces Round 894 (Div. 3) System tests lasted around 30 minutes (Probably due to few hacks)

While this is usually not a huge problem, it can lead to situations where in case of back to back rounds, people get assigned to wrong divisions and in general it leads to an annoying experience for the users.

As some suggestions, maybe the hacking phase can be reduced to 8 hours so that the system tests can be hold around what would be early morning in Europe, when the user activity is the smallest? I guess another option would be to be more selective as far as hack tests go so that we don't see 100 tests which test the same thing (i.e: unordered map hacks in the case of this round)?

I think finding some solution to this problem is very important for the purpose of future rounds.

Tags div3, hacking, system test, queue

History

 
 
 
 
Revisions
 
 
  Rev. Lang. By When Δ Comment
en2 English stefdasca 2024-06-04 16:00:31 13
en1 English stefdasca 2024-06-04 16:00:17 1834 Initial revision (published)