terraformer's blog

By terraformer, history, 2 years ago, In English

Do you use online-judge-tools? If not, I believe it is valuable tool to try.

Your operation in competitive programming will become automated and the competition programming experience will be comfortable.

Good tutorial is here.

online-judge-tool seems simple. It is consists of just 4 subcommands (login, download, test and submit). But if you execute oj test --help, you can find about 20 command line options for oj test. Other subcommand (login, download and etc.) also have many command line options. Actually, these are many features.

Therefore, today I want to introduce my favorite online-judge-tools features.

1. Side-by-Side diff

This is very new feature for oj test. Sometimes, finding differences between your answer and expectation is cumbersome task.

For example,

$ oj test -c './a.out'
[*] 1 cases found

[*] sample-1
[x] time: 0.054696 sec
[-] WA
output:
7
1 5 1 3
7 7 89 9 2 2 6 5
2 4 6 1 8 5
9 10 0 29 56

expected:
7
1 5 1 3
7 7 89 9 2 2 6 5
2 4 6 1 3 5
9 10 0 29 56

Can you find where difference is between output and expected?

If you uses side-by-side display option(-S),

Great! You can find "3" in expect was replaced to "8" in output in a moment.

2. MLE and TLE checking

Since almost all of problems are limits computation time and memory consumption, it's natural that you want to confirm not only that the output matches the expected value, but also these limitations.

You only have to add --tle or --mle options for checking these limitations.

ex.

$ oj t -c './a.out' --tle 2.5  # 2.5sec
[*] 1 cases found

[*] sample-1
[x] time: 2.503372 sec
[-] TLE
$ oj t -c './a.out' --mle 56  # 56MB

Note: Currently, this feature requires gnu time. Therefore, unfortunately, Windows users can't access this feature.

3. Floating point error judge

Some problem requires floating point output. (ex. 2.500000) In many situation, relational and absolute error is allowed for these problems.

oj test can handle these errors by -e option.

If problem expectation 3 and your program outputs 3.00000001

$ oj t -c 'echo 3.00000001'
[*] 1 cases found

[*] sample-1
[x] time: 0.004793 sec
[-] WA
output:
3.00000001

expected:
3

In this situation, oj recognize your output as "Wrong Answer". But if you indicated allowed error by -e option, result will become changed.

$ oj t -c 'echo 3.00000001' -e 0.000001
[*] 1 cases found

[*] sample-1
[x] time: 0.004088 sec
[+] AC

Good, oj result becomes "All Correct". Here, oj t -c 'echo 3.00000001' -e 1e-5 is equivalent with oj t -c 'echo 3.00000001' -e 0.000001.

4. API

This is feature for tool developers.

You can access online-judge-tools inner functions via Python and you can construct powerful your own contest helper.

For example, I implemented "go to next problem" function for my IDE.

Snippet to get next problem URL is here.

from onlinejudge import dispatch
from onlinejudge._implementation.utils import (
    default_cookie_path,
    with_cookiejar,
    new_session_with_our_user_agent,
)

def get_next_problem(cur_problem_url: str) -> str:
    cur_problem = dispatch.problem_from_url(cur_problem_url)
    contest = cur_problem.get_contest()
    with with_cookiejar(new_session_with_our_user_agent(), path=default_cookie_path) as sess:
        problems = contest.list_problems(session=sess)
        for i, problem in enumerate(problems):
            if problem == cur_problem:
                next = problems[(i + 1) % len(problems)].get_url()
                return next

next_url = get_next_problem("https://codeforces.com/contest/1312/problem/B")
print(next_url)

If you logged in Codeforce via online-judge-tools, you will get "https://codeforces.com/contest/1312/problem/C". I hope there are more great features will be developed by you!

Thanks,

 
 
 
 
  • Vote: I like it
  • +38
  • Vote: I do not like it

»
2 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Auto comment: topic has been updated by terraformer (previous revision, new revision, compare).

»
2 years ago, # |
  Vote: I like it 0 Vote: I do not like it

When I type $ pip3 install --user online-judge-tools it says invalid syntax. What should i do? I am using Windows.

  • »
    »
    2 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Could you try oj.exe? You can download it from here

    This is executable file for Windows.

»
2 years ago, # |
  Vote: I like it 0 Vote: I do not like it

cool, this seems nice. I will try using it at the next global round

  • »
    »
    2 years ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    I appreciate if you will try! If you try oj at first, please don't forget "oj login" before downloading problems.

»
2 years ago, # |
  Vote: I like it +5 Vote: I do not like it

Since I mostly only use Codeforces, cf-tool is actually quite competent since (1) I can use problem ID in parsing and submitting (no need to copy problem url), (2) it shows live status of Running on test XX in the command line itself.

Of course it does not have all the cool features that you listed above, but then again, the above two features are way too useful as a compromise.

Is it possible that the Running on test XX feature makes it to online-judge-tools? That would make it a really compelling reason to switch.

  • »
    »
    2 years ago, # ^ |
      Vote: I like it +5 Vote: I do not like it

    Thank you for your opinion!

    (1) Problem ID submitting oj supports guess URL feature. If you use oj submit command after oj download URL, submit URL decided automatically. But you need URL every time when oj download.

    Currently, kimiyuki (Author of oj) started a project of more user friendly tools which uses oj as backend.

    It will download samples of all problem by oj-contest. Users can switch problem by switching directory only. But it is very early project and documented by Japanese only.

    (2) Running on test XX feature After oj submit, oj opens your submission page with your default browser. So you can confirm live status of judge.

    But if any user want us to support live status in command line, I interested in supporting this function.

  • »
    »
    2 years ago, # ^ |
      Vote: I like it +5 Vote: I do not like it

    Hi! I'm the author of online-judge-tools.

    cf-tool is optimized for Codeforces, but online-judge-tools isn't. If you use only Codeforces and you are used to cf-tool, I recommend keep using cf-tool. But when you use other contest sites, please remember online-judge-tools.

    In terms of the features you mentioned:

    1. It is difficult to specify problems or contests with their IDs instead of URLs. For example, someone use the number 123 for this Codeforces contest, but others may use the same number for this Codeforces contest (see the round number), this vjudge contest, or this yukicoder contest. For this situation, I think using URLs is the better way.

    2. Live status of Running on test XX is a cool feature, but it will not be implemented. The reason is that the feature has only little effect for your ranking on contests. We need to implement more effective features first (e.g. toll to test and and make documents for your libraries, tool to generate templates with input/output parts).

    • »
      »
      »
      2 years ago, # ^ |
        Vote: I like it +5 Vote: I do not like it

      Hi, thanks to both of you for your replies!

      I did not notice that only oj download requires URL and not oj submit. That is great, and having to put a download url once at the start of a contest is not a big deal actually.

      I have recently started giving other contest sites too, like AtCoder Beginner Contests and I will definitely give oj-tools a try there.

      Also, I agree with your idea of developing more effective tools first, especially online-judge-verify-helper, which I came across yesterday and will soon start integrating our team's library using it!

      You guys are doing great work, thanks for it!

»
2 years ago, # |
  Vote: I like it 0 Vote: I do not like it

how to test python code

»
8 months ago, # |
  Vote: I like it 0 Vote: I do not like it

kimiyuki It seems like this is not working any longer? Few days ago it was working for me but not any longer. Can anyone confirm?