leafvillageninja's blog

How should I do stress testing of my solution during a contest?
By leafvillageninja, history, 4 years ago, In English

In today's contest, I got a WA in problem 3. So I thought of doing a stress testing of my solution, but my workflow was so slow that it took me around 1 hour to debug my code. Is there anything I should improve so that I can do such debugging fastly in contests? I am describing the steps I did: I wrote a brute force solution. I wrote code for randomly generating the test cases. I then ran both the solutions I had against the generated test-cases. I then used https://www.diffchecker.com/ to find the difference between both the outputs. I then opened the randomly generated tests file (which had 300+ test cases) and then navigated to the test case for which the output was different. Then I ran that particular test case separately and did the debugging.

  • Vote: I like it
  • +10
  • Vote: I do not like it

»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

"I then used https://www.diffchecker.com/ to find the difference between both the outputs. I then opened the randomly generated tests file (which had 300+ test cases) and then navigated to the test case for which the output was different." This part can be automated.

→ Reply
»
4 years ago, # |
  Vote: I like it +9 Vote: I do not like it

Some tips:

  1. Consider finding a minimal test case where the bug can be reproduced. When you find a test that has 300 lines, usually you can find a smaller test with the same bug. This can be done by rerunning the generator with smaller parameters or by manually pruning the test case.

  2. Instead of writing separate brute solution and generator you can implement everything on the same file, two solutions (reusing functions you are confident about), test generation and asserts about expected outcome... Also

→ Reply
»
4 years ago, # |
  Vote: I like it +11 Vote: I do not like it

This video could be helpful for you to automate some of the steps you have done, thus reducing the debugging time during the contest.

→ Reply
»
4 years ago, # |
  Vote: I like it +2 Vote: I do not like it

At least you got it debugged :(

→ Reply
»
4 years ago, # |
  Vote: I like it +3 Vote: I do not like it

Sounds like you jumped to doing stress testing way to soon. In my experience manually testing a bunch of small test cases usually works.

→ Reply