Please subscribe to the official Codeforces channel in Telegram via the link https://t.me/codeforces_official. ×

-Ahmed_-_Elsayed-'s blog

Is using AI during contests to summarize the problem illegal ?

By -Ahmed_-_Elsayed-, history, 16 months ago, In English

Right now Bing AI can help me during contests by telling me directly what the problem wants me to do without the story which can make me understand the problem faster is that legal or not ? Thank you in advance :)

Tags ai
  • Vote: I like it
  • +49
  • Vote: I do not like it

Comments Comments (19)
| Write comment?
»
16 months ago, # |
  Vote: I like it -20 Vote: I do not like it

Read the contest rules. I think there should be something related to this mentioned. If not I guess we'll wait for an answer from the admins

→ Reply
»
16 months ago, # |
← Rev. 2   Vote: I like it +50 Vote: I do not like it

There is not much difference between this question and asking if it is legal to write a codeforces contest using a topcoder tshirt or if you can participate under the effect of drugs or some other nonsense doubt. This is obviously not enforceable. What is interesting is if AI could actually help you with that as you claim, which I think it doesnt. AIs ability is not consistent and accurate enough to help you with understanding the statement I think

→ Reply
  • »
    »
    16 months ago, # ^ |
      Vote: I like it +17 Vote: I do not like it

    Agree. For instance, Chat GPT can't meow solving a simple div 3A isitacat XD

    so how would it assist in understanding the problem and it answers incorrectly

    → Reply
    • »
      »
      »
      16 months ago, # ^ |
      ← Rev. 3   Vote: I like it +23 Vote: I do not like it

      The problem wants you to determine whether a given string describes a cat meowing or not. A string describes a cat meowing if it follows these rules: It can only contain the letters ‘m’, ‘e’, ‘o’ and ‘w’, in either uppercase or lowercase. It must start with non-empty sequence consisting only of characters ‘m’ or ‘M’. It must be immediately followed by non-empty sequence consisting only of characters ‘e’ or ‘E’. It must be immediately followed by non-empty sequence consisting only of characters ‘o’ or ‘O’. It must be immediately followed by non-empty sequence consisting only of characters ‘w’ or ‘W’, this sequence ends the string, after it immediately comes the string end. You need to output YES if the string is a cat meowing and NO otherwise.

      that was the Bing Ai answer when I asked it to tell me what the problem wants me to do but for this problem, I think the statement is so clear it doesn't need any clearance or summary

      .............

      another example of the same contest problem E2

      The problem wants you to determine whether it is possible to transform one string into another by swapping letters at a certain distance. You are given two strings of equal length and a number k. You can swap any two letters in the first string if their positions differ by k or k+1. You need to output YES if you can transform the first string into the second string by using this operation and NO otherwise.

      I asked the same question (to briefly tell me what the problem wants me to do) and that was the answer

      .............

      I still haven't tested it enough to determine how accurate and consistent it is but till now it's not doing bad

      → Reply
      • »
        »
        »
        »
        16 months ago, # ^ |
          Vote: I like it +33 Vote: I do not like it

        It is harder to read your comment than the problem statement.

        → Reply
        • »
          »
          »
          »
          »
          16 months ago, # ^ |
            Vote: I like it +23 Vote: I do not like it

          sorry about that I tried to edit the comment so that the bold text is written by AI (copied like it is without any edits) while the italic is written by me if it is still not clear inform me

          thanks for the criticism :)

          → Reply
      • »
        »
        »
        »
        16 months ago, # ^ |
          Vote: I like it +19 Vote: I do not like it

        These problems are pretty concisely stated anyway, even with the story about casting (they are 1800A - Is It a Cat? and 1800E2 - Unforgivable Curse (hard version) by the way). For this to be useful, it should also be able to summarize some the infamously long kind of statements. Say this one for example.

        Another problem I foresee is the AI not being able to identify important information. Some problems have small constraints or weird conditions that are vital to solving the problem. Even human beginners disregard them.

        → Reply
»
16 months ago, # |
  Vote: I like it +12 Vote: I do not like it

I think it should be legal. From my experience, I think the solving strength of ChatGPT is <1100, and to use it for harder problems requires you to know what you're doing. For example, I could maybe coax it to solve a 1600 rated problem, but at that point I would have to know the main idea of the problem already. It wouldn't be very useful to me for 2000+ rated problems.

→ Reply
»
16 months ago, # |
  Vote: I like it +11 Vote: I do not like it

Well, the linked blog (you can find the link in the registration form, which appears each time you register to the contest) states that "The contestants are forbidden to talk about subjects, related to the problems, with anybody, including other contestants". The blog is 11 years old, so it says nothing about AIs. However, I find asking an AI pretty similar to asking a random person (even not completely random, AIs answers are in general faster, clearer and AIs are even capable of writing programs).

I don't know, how strong is the impact of an AI nowadays, but it's rather clear that AIs will develop, so that in future they will be able to solve real round problems. Thus, the optimal solution would be to forbid any AI usage during the contests (does this hold for chess in the same manner?)

Formally, now it's not forbidden in any way. However, if you want to compete fairly, I wouldn't recommend you doing it.

→ Reply
»
16 months ago, # |
  Vote: I like it 0 Vote: I do not like it

Though it is not forbidden, I do not recommend the usage of AI, it does not help you, you may be able to solve a problem by asking AI, but you will not progress that way

→ Reply
»
16 months ago, # |
  Vote: I like it -50 Vote: I do not like it

Definitely illegal. Sometimes understanding the problem in the right way is part of the solution. For example in networking, Bing will tell you $$$Max_{Flow} = Min_{Cut}$$$ if you don't already know.

→ Reply
»
16 months ago, # |
  Vote: I like it +75 Vote: I do not like it

I'm pretty appalled by the lack of reading comprehension in this comment section.

→ Reply
»
16 months ago, # |
  Vote: I like it +27 Vote: I do not like it

If an AI can comprehend a problem statement better than your brain, then well things do not seem too bright for you.

→ Reply
»
16 months ago, # |
← Rev. 3   Vote: I like it 0 Vote: I do not like it

I think it's not a good idea because very often it will fall into one of the following buckets:

  1. It will basically just repeat the statement, nowadays we don't have a lot of stories anyway.
  2. It will simplify the statement in a way that makes the problem easier. It is basically impossible to draw a line between "oh this is just the statement in slightly different terms, it doesn't change anything" and "this is a reformulation of a problem which hints at the right solution (if not being the solution itself)".

So in my opinion it is both useless (at the moment) and cheating at the same time. On average it doesn't help but when it helps it's cheating.

Btw, in the future, AI will be a growing problem for competitive programming. Div 1 participants are probably [mostly] safe for a few more years but this or next year there will be easily available AIs which will be able to instantly solve most of div2 A and many div2 B-C.

→ Reply
  • »
    »
    16 months ago, # ^ |
    ← Rev. 4   Vote: I like it 0 Vote: I do not like it
    this or next year there will be easily available AIs which will be able to instantly solve most of div2 A and many div2 B-C.

    LOL. Have you tried testing even div2A problems on any of these AI models? Except it's an implementation problem. Or a well known or it has seen similar problems the solutions are straight trash.

    Even recent leetcode contests can't be solved easily by AI models. I think div2 participants are safe.

    → Reply
    • »
      »
      »
      16 months ago, # ^ |
      ← Rev. 4   Vote: I like it 0 Vote: I do not like it

      Ahem, AlphaCode was better than the average human competitor more than 1 year ago... https://www.deepmind.com/blog/competitive-programming-with-alphacode

      And it's not an especially expensive/complicated model, more like DeepMind quickly tried the most obvious thing and it kinda worked.

      Also, ChatGPT is able to solve 75% of HumanEval tasks from the first try. I think it has a similar difficulty to div2A, maybe just slightly easier.

      → Reply
      • »
        »
        »
        »
        16 months ago, # ^ |
        ← Rev. 3   Vote: I like it 0 Vote: I do not like it
        AlphaCode was better than the average human competitor more than 1 year ago

        Based on what stats?

        I think you should go through this blog. AI still has a long way to go.

        Btw, those HumanEval tasks, are direct implementation problems, and ofcourse if will perform well on those. Most Div2A's these days aren't direct implementation, you need to make some observations before you can implement an optimal solution.

        → Reply
        • »
          »
          »
          »
          »
          16 months ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          Based on rating? The linked blog posts say "Apparently, if these accounts would take part in real competitions, then their rating would be about 1300.".

          It is exactly what I meant, 1300 is very slightly above average.

          Even this level is somewhat problematic if it was freely available.

          Also, not everything in HumanEval is completely trivial, it has such "observation" problems too, not sure how much though, possibly <10%.

          → Reply
»
16 months ago, # |
  Vote: I like it +36 Vote: I do not like it

How about problem setters using AI to shorten the statements? Maybe a better direction to go with?

→ Reply