What is wrong with system tests and hacks?

Revision en1, by AVdovin, 2024-02-24 16:40:25

Hello, Codeforces! First of all, I would like to emphasize, that I am writing this post not out of fury or disappointment, but I just want to verify, if there are anyone who share my POV.

Yesterday Educational Codeforces Round 162 (Rated for Div. 2) was held. And I liked it! Problems were very nice and interesting. But one thing happened, that is still on my mind. My B got FST (Failed System Tests). For those unfamiliar with this system, I'll explain it to you. When you send your code, during round it works on some relatively small tests set, after the contest is finished your last successful submission for each problem is retested on expended tests set (it also includes all successful hacks). This system exists to lessen delays in testing queue. In my mind the problem behind this system is that during the round you see if your code failed some test, and you can find an issue and fix it, but after the end of the contest you can't.

In my case, i got FST caused by integer overflow, which is not quiet fair to my mind. No, it's absolutely my fault not to see, that $$$3*10^9$$$ will overflow an integer type, but the consequence of this small fault is just too big to my mind — my expected delta went from $$$+40$$$ or so to $$$-60$$$, which is depressing, when you managed to solve till D... No, I am not hunting for big ratings, I participate only to have fun solving the problems (also to improve skills in problem setting, as i really like making up tasks sometimes), but for some people this delta change can be crucial. I think this is a widespread problem and everyone face it sometimes.

So, I have a suggestion to improve the situation. It would be nice, if when your code fails on system tests (or hacked) you don't get the problem incorrect, but instead you receive some penalty for this hack (it could also depend on problem position in the set — the harder the problem, the more penalty you get). In the standings you will have this problem passed, but with red $$$+$$$, meaning that you successfully solved the problem during the contest, but it ended up being hacked/FST.

Let me know what you think about this problem, situation itself and my suggestion in the comment section. Thanks all of you for reading my post! Have a nice day everyone!

Tags system test cases, suggestions

History

 
 
 
 
Revisions
 
 
  Rev. Lang. By When Δ Comment
en1 English AVdovin 2024-02-24 16:40:25 2390 Initial revision (published)