I have wanted for a long time and now, finally, I sat down and did it.
Many of you are familiar with my Testlib project. This is a library that helps you develop problems if you are using C++. In the case of problems for Codeforces rounds, its use is mandatory. It seems to me that it is almost a de facto standard for preparing materials for completely different olympiads and competitions.
The history of С++-Testlib is almost lost in centuries, I published the first version in 2005.
Nowadays, making edits to Testlib has become tricky. The code grown to thousands of lines, also it has too long a history. This increases the likelihood of accidentally changing some aspect of behavior, which will jeopardize the holding of a Codeforces round or another important competition. In short, making a bug in this code or changing behavior anywhere is highly discouraged.
So I recently took the plunge and wrote some tests for Testlib. Here's what tests can do now:
- check that all cpp files in the repository are compiled (test
- check the plausibility of the behavior of a simple sval.cpp validator on a series of files (
- check the plausibility of the behavior of the two checkers fcmp.cpp and wcmp.cpp on a series of files (test
- check that the behavior of the random number generator has not changed (test
- check the behavior of some functions and the behavior of InStream and opt in a series of unit tests (test
More good news. All this was automated through GitHub Actions. Tests are automatically run on any push or pull request. The launch is carried out on:
- 6 different operating systems (ubuntu-18.04, ubuntu-22.04, macos-11, macos-12, windows-server-2019, windows-server-2022),
- different versions of different C++ compilers (g ++, clang ++, msvc),
- different C ++ standards
- and both architectures (32 and 64 bits) are used.
In total tests run in 33 jobs! Look here https://github.com/MikeMirzayanov/testlib/actions/runs/2733832393
Of course, there are still few tests. Something else I will definitely add.
If you understand well how some functionality of Testlib works, then you can add a test for it. First of all, consider adding unit-tests to
test-004_use-test.h (carefully study the design of this test).
Otherwise you might find it useful to use
test-ref script in tests, which checks that a process has exited with the expected exit code, standard output, and standard error.
It is highly desirable to cover all new functionality with tests. If the functionality is such that it is not very difficult to write tests, then the presence of tests is required.
How to run tests locally?
You need to have installed
bash and some tools. After just do
cd tests && bash run.sh.
This will run all tests on all supported installed compilers on multiple C++ standards. Probably, just want to run tests on single
g++ version using
--std=c++11. You can use
bash run.sh g++ v0 g++.
bash run.sh— to run all tests on all compilers on multiple standards
bash run.sh g++— to run all tests on
g++on multiple standards
bash run.sh g++ 11 v10— to run all tests on
bash run.sh g++ 11 test-001_run-sval— to run
bash run.sh test-001_run-sval— to run
test-001_run-svalon all compilers on multiple standards
bash run.sh g++ 11 17— to run all tests on
Also you can use
t.bat) to run shortcut of
run.sh g++ 11 v0 $*. It is good idea to use it for local development to have quick response.