Olympia's blog

By Olympia, history, 21 month(s) ago, In English

History Lesson

Let's take a look at the solve count for the last problems in the last 4 division 2 contests:

803 (Div. 2): F (7 official solves); G (2 official solve)

802: E (18 official solves): F (11 official solves)

801: E (1 official solve)

Edu 130: F (2 official solves)

There's a trend. The last and hardest div2 problem frequently has very few official solves. This means that the last problem, despite taking a lot of time to create, doesn't really affect rankings that much. The authors spent much time for a problem that didn't make a big difference. But it would've made a difference in div1.

Personally, I think that it's okay if 20 or even 30 people AK a div2 round, so it's okay to have 20-30 people solving the last problem. Then, you can take those hard problems and insert them into a div1 round. That way, we can have more contests.

Basically, make div2 last problems a little easier, that way we can save those hard problems for another contest.

What do you think?

  • Vote: I like it
  • +898
  • Vote: I do not like it

| Write comment?
»
21 month(s) ago, # |
  Vote: I like it +131 Vote: I do not like it

Really makes a lot sense

»
21 month(s) ago, # |
  Vote: I like it +105 Vote: I do not like it

Hi. We didn't think F would be hard. We originally put it at D. Our highest rated tester solved the problem in 5 minutes, and we suspected (and saw) that lower rated people who got to the problem would have a decent chance of solving it too. Sometimes we just can't predict difficulty.

In regards to G, I agree with your post. I don't think it's a great idea to put a really incredible problem at the last position, since not many people will see it. I think G is a very mid problem, which is okay since not many rated people will have the chance to appreciate it.

  • »
    »
    21 month(s) ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    was your highest rated tester a Div.2 contestant though

    • »
      »
      »
      21 month(s) ago, # ^ |
        Vote: I like it +13 Vote: I do not like it

      No, but I don't expect Div. 2 contestants to solve it in 5 minutes. My point was that in the official contest, even high rated people took a very long time on this problem, which was surprising to us. (Had this occurred in testing, we wouldn't have put the problem in the contest.)

  • »
    »
    21 month(s) ago, # ^ |
      Vote: I like it +8 Vote: I do not like it

    I'm also surprised how slow people solve F. I thought that I would definitely miss first AC...

  • »
    »
    21 month(s) ago, # ^ |
      Vote: I like it +64 Vote: I do not like it

    Time to solve a constructive task always has a very high variance.

    • »
      »
      »
      21 month(s) ago, # ^ |
        Vote: I like it +13 Vote: I do not like it

      That's true. From the feedback we got from testers, though, we still expected to have more than 7 solves (we predicted around 100).

»
21 month(s) ago, # |
  Vote: I like it -61 Vote: I do not like it

This only creates a difference of one problem and won't ease the time and effort to create a div1 round that much. This also causes rating inflation. Div2 rounds should continue as it is. This is my opinion.

  • »
    »
    21 month(s) ago, # ^ |
      Vote: I like it +9 Vote: I do not like it

    Yes, but from the other side, get ready then for seeing top 1-5 only alt accounts every contest, as it is now.

    • »
      »
      »
      21 month(s) ago, # ^ |
        Vote: I like it -82 Vote: I do not like it

      So to see some more people on the top we are going to change the traditional structure of a round???? I hope div2 rounds stays the same.

      • »
        »
        »
        »
        21 month(s) ago, # ^ |
        Rev. 2   Vote: I like it +6 Vote: I do not like it

        I see nothing bad in traditional div 2 round with 5 problems, where E has official solves

        • »
          »
          »
          »
          »
          21 month(s) ago, # ^ |
            Vote: I like it +24 Vote: I do not like it

          imo I prefer div 2 rounds with 6 problems. Frequently with these contests A is a basic implementation problem with around 90% solve rate, B is a bit more complicated but still generally approachable with about 60% solve rate, and the last problem is ideally a difficult problem with about 30ish solves, or a clear rate around 0.25% (currently it's about a third of this). With 5 problems, you have two problems to bridge the gap from a basic question the majority of coders can solve to a much harder one with 1/240th the clear rate. This creates a steep difficulty curve and results in contests that a more likely to be Speedforces where for most people, ranking is mainly determined by how fast you solve the first 2 or 3 questions. The way to avoid this issue is either to make the last problem much easier (as in 100-200 solves minimum) or to make Problem E in a 5 problem contest Problem F in a 6 problem contest and add a problem in between B and F to bridge the difficulty gap more smoothly.

          • »
            »
            »
            »
            »
            »
            21 month(s) ago, # ^ |
              Vote: I like it 0 Vote: I do not like it

            Alright, 6-problems contest imo is solution too.

      • »
        »
        »
        »
        21 month(s) ago, # ^ |
          Vote: I like it -9 Vote: I do not like it

        I don't say difficult problems are useless, they're still needed for rounds not to be speed-forces, but the talk ig is about the hardest problem difficulty, for div2 imo it should be possible for 10-50 people to solve it, but not 3 or 7 or 11 people, from which 8 are alt accounts

    • »
      »
      »
      21 month(s) ago, # ^ |
        Vote: I like it +23 Vote: I do not like it

      That argument is completely flawed since that'll happen regardless of lowering difficulty or not.

      • »
        »
        »
        »
        21 month(s) ago, # ^ |
        Rev. 2   Vote: I like it 0 Vote: I do not like it

        Yes, but then among those who solved hardest problem won't be only alts, because from what we have now, hardest div2 problems are solveable only by GM+ (what pretty much identifies persons behind those alts.)

        • »
          »
          »
          »
          »
          21 month(s) ago, # ^ |
            Vote: I like it -20 Vote: I do not like it

          I think in a contest there should be problems that are difficult for the contestents to solve yet solvable.

          • »
            »
            »
            »
            »
            »
            21 month(s) ago, # ^ |
              Vote: I like it 0 Vote: I do not like it

            Is it better in your opinion to see in top standings only alts (and even not seeing there CM's, who are targeted to be the top standings)

        • »
          »
          »
          »
          »
          21 month(s) ago, # ^ |
            Vote: I like it -11 Vote: I do not like it

          Div3 standings tells us otherwise.

»
21 month(s) ago, # |
  Vote: I like it +19 Vote: I do not like it

Personally, I read 803G in the first 2 minutes of the contest and found the solution in a couple of minutes so I think that it is fitting for div2. It for sure isn't hard enough to be a div1D.

  • »
    »
    21 month(s) ago, # ^ |
      Vote: I like it -10 Vote: I do not like it

    So because you solved it in 2 minutes it's fit for div2? May I remind you that you're red, and problems you consider easy may not be easy for div2 participants.

    • »
      »
      »
      21 month(s) ago, # ^ |
        Vote: I like it +14 Vote: I do not like it

      Do you think all problems in a contest should be easy for the contestant? Oh damn, why can't I solve div1E most of the time then?

    • »
      »
      »
      21 month(s) ago, # ^ |
        Vote: I like it +87 Vote: I do not like it

      It's an argument why it's not a fit for Div1

      • »
        »
        »
        »
        21 month(s) ago, # ^ |
          Vote: I like it +14 Vote: I do not like it

        The solve stats show it actually would've been pretty good in Div 1, though. One user saying he found it easy does not take away the fact that many reds and even some well-known LGMs failed to solve it. It was clearly a hard problem for the vast majority of people, even those in Div 1.

        On the other hand, having one or two very difficult problems is no tragedy. Inevitably there will be some candidates in Div 2 who really should be high up in Div 1, and these problems ensure they find their way to their rightful position quickly (alt account nonsense aside).

        • »
          »
          »
          »
          »
          21 month(s) ago, # ^ |
            Vote: I like it +62 Vote: I do not like it

          Thr issue is that it's a pretty standard problem. If you have some knowledge, you mindsolve in 2 minutes. This isn't good for Div1

          • »
            »
            »
            »
            »
            »
            21 month(s) ago, # ^ |
              Vote: I like it +17 Vote: I do not like it

            Fair — although if that’s the case then I’m surprised that it doesn’t have more solves, unless people couldn’t be bothered.

          • »
            »
            »
            »
            »
            »
            21 month(s) ago, # ^ |
            Rev. 2   Vote: I like it +8 Vote: I do not like it

            Hmm..that's interesting. I'm an expert, so I obviously am not in the position to judge problem types for later problems (I can only look at problem count).

            Would you say that most late problems little-solved problems in div2 are standard and therefore ineligible for div1? It never crossed my mind that there are some hard div2 problems that are not good for div1...wondering if it's just this round with standard hard late problems or a general trend

          • »
            »
            »
            »
            »
            »
            21 month(s) ago, # ^ |
              Vote: I like it +33 Vote: I do not like it

            I do not like the idea that standard problems are suited for Div. 2 rounds but not Div. 1 rounds. I think standard problems can appear in Div. 1 rounds

            • »
              »
              »
              »
              »
              »
              »
              21 month(s) ago, # ^ |
                Vote: I like it +4 Vote: I do not like it

              Standard problems shouldn't be in div1 rounds.

              • »
                »
                »
                »
                »
                »
                »
                »
                21 month(s) ago, # ^ |
                  Vote: I like it +18 Vote: I do not like it

                Why

                • »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  21 month(s) ago, # ^ |
                    Vote: I like it -14 Vote: I do not like it

                  Div1 is for experienced people, why you want to put a standard problem there. Standard problems are mainly use to introduce concepts to not much experienced coders.

                • »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  21 month(s) ago, # ^ |
                    Vote: I like it +92 Vote: I do not like it

                  Experienced??? Looking back on when I had only 2100 rating, I don't think I was "experienced people".

              • »
                »
                »
                »
                »
                »
                »
                »
                21 month(s) ago, # ^ |
                  Vote: I like it +40 Vote: I do not like it

                Stfu, fukin alt, don't peepee in your pants and write comments from your main.

                or you are ashamed of your rating?

            • »
              »
              »
              »
              »
              »
              »
              21 month(s) ago, # ^ |
                Vote: I like it +97 Vote: I do not like it

              I also do not like the idea that standard problems are suited for Div.2 rounds but not Div.1 rounds. I think standard problems can not appear in rounds.

              • »
                »
                »
                »
                »
                »
                »
                »
                21 month(s) ago, # ^ |
                  Vote: I like it +84 Vote: I do not like it

                If standard problems don't appear in rounds, people will have less opportunity to learn standard algorithms/techniques/data structures etc.

                • »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  21 month(s) ago, # ^ |
                    Vote: I like it 0 Vote: I do not like it

                  People have more than enough opportunities to learn standard techniques.

»
21 month(s) ago, # |
  Vote: I like it +36 Vote: I do not like it

If the div2 difficulty range is lowered, the overall difficulty gap between the problems is lowered. I think most people would prefer this?

At least looking at some recent div2s, there are some pretty large gaps between problem ratings. I'll use Round #794 (Div. 2) as an example. A is 800, B is 800, C is 1100, and D is 2000. This is bad for higher rated people who can solve A, B, C but not D. This is one of the more extreme examples, but you get the idea.

I guess large gaps benefit fast coders, since it allows them to match with higher rated users. Overall though, I think most people don't like "speedforces" rounds judging from some comments I've seen. I'm quite new to CF so correct me if I'm wrong there.

»
21 month(s) ago, # |
Rev. 3   Vote: I like it -12 Vote: I do not like it

The drop-off in clear rate is quite dramatic past Problem D. These are the clear rates I have tracked from all of the Division 2 and Educational Rounds that I have participated in (counting Div 2 participants only):

Problem A: 87.39%

Problem B: 61.01%

Problem C: 31.21%

Problem D: 7.254%

Problem E: 1.207%

Problem F: 0.0798%

Problem G: 0.0137%

The stats above are from 20 Div 2 level contests I participated in this year. The majority of these contests had 6 problems, and Round 803 was the first Div 2 level contest I participated in that actually had a Problem G. The difficulty curve in the first few problems is fine, but it steepens dramatically at Problems E through G. E has 1/6th the clear rate of D which is already a bit much, but F has 1/15th the clear rate of E, which imo is overkill. Currently Problem F tends to have around 10 solves, so making F a bit easier to bring it more in line with the difficulty gap between D and E sounds good to me.

As for Round 803, I feel that the main reason 7 problems were in this contest were to compensate for the earlier Problems A-D being easier than normal. This is backed up by the point system in this contest giving the earlier questions fewer points than normal, while still scaling up to typical point values in the later questions. All of A through D had higher clear rates than normal with Problem D seeing nearly 18% of coders solve it. This pattern does not really hold for the last 3 problems, and Problems F and G essentially have the same issue of being a huge spike from E that is only compounded by a huge gap in difficulty between D and E. I found the experiment with 7 questions this contest interesting, but Problem G is better off serving in a Div 1 contest since most Problem F's are only attempted by a few dozen people to begin with.

Edit: I keep track of above clear rates here (https://docs.google.com/spreadsheets/d/1aQObaGKSDEtilQrLuxFZ6-naW-EMjr4uTE5LnnNn6tY/edit?usp=sharing). It's provided mainly to show the above values are mostly accurate.

  • »
    »
    21 month(s) ago, # ^ |
    Rev. 3   Vote: I like it +13 Vote: I do not like it

    This is a good point. I think for Round 803, the random sample of testers we selected just turned out to not be very representative of the whole CF audience.

    Initially, the round had 6 problems; there was no problem C, and after B there was D. Then nine testers in a row failed to solve D. Worried, we decided to add C to make the gap smaller. In hindsight, I think was the right call. But based on this, we expected the largest difficulty gap to be between C and D, and not between D and E (most people who solved D during testing solved E shortly after). And of course, we didn't expect such hard F.

    Sometimes random selections just can be unlucky ¯\_(ツ)_/¯. Of course, you can always get more testers, but I think it was just sheer coincidence that the problemset turned out the way it did, and I don't know how much more testers would've changed that.

    I don't want to slander the testers. I think they did a great job, because we had to change the problemset several times and they gave lots of great feedback.

    But yeah, I think sometimes, this kind of thing happens, and there's not much I can do about it. In the future, I'll try to get more testers for my rounds, I guess, but from my point of view very few solves on F and G was completely unprecedented for us.

    Sorry to anyone who feels like they had a bad experience with difficulty curve in this contest. I tried for several weeks to iron out the gap between B and D. I wasn't aware that there was such a gap between D and E. I'll try to make it better next time.

    And to anyone still reading: try problem F. I don't think it's as hard as people are making it out to be. ;)

    • »
      »
      »
      21 month(s) ago, # ^ |
        Vote: I like it +16 Vote: I do not like it

      Hey, don't worry, you did a very nice contest, one of my favourite recent contests!

»
21 month(s) ago, # |
  Vote: I like it +15 Vote: I do not like it

If the difficulty of problems are lowered it would affect the few deserving participants to reach Div 1 quickly though. Then the people who are comparitively slower but can solve the tougher problems will be at an disadvantage. Suppose A is 800, B is 900, C is 1200, D is 1500 and E is 1750 instead of 2000. Then E submissions will be substantially more and those people who are slower but can solve the tougher problems will be affected. I think it is an extreme cases but I guess many face the same issue.

  • »
    »
    21 month(s) ago, # ^ |
    Rev. 2   Vote: I like it +11 Vote: I do not like it

    Well, as I told previously, D 1500 isn't solution for this problem at all. D should leave where it is, in [1800-2100], E should leave where it is, in [2000-2200], question is about necesity of F in [2500+,...), Imo what we need is F in [2100,2300]

    (At least in div2 only rounds)

    • »
      »
      »
      21 month(s) ago, # ^ |
        Vote: I like it -16 Vote: I do not like it

      D should be in the range of 1700 — 1900. F should be in the range of 2500 — 2700. You aren't giving much gap between D and E. They are intersecting too.

      • »
        »
        »
        »
        21 month(s) ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        Imho, 1700 rated D isn't normal (except rounds like 803, where D was actually C)

        • »
          »
          »
          »
          »
          21 month(s) ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          I've seen a very recent round with E = 1700 :)

»
21 month(s) ago, # |
Rev. 2   Vote: I like it +8 Vote: I do not like it

What about making a mashup of all problems with < 101 solves from past few div2's(with author name) making it open for everyone??
High rated then can solve them without wasting their time on searching challenging problems.

»
21 month(s) ago, # |
Rev. 2   Vote: I like it +20 Vote: I do not like it

Personally, I like solving unrated for me Div3 and Div2 and that's interesting and useful because of the last problems (I like solving in the reverse order from G/F/E to A). But if there is no hard (for Div2) tasks — it will be worse. And some other unrated participants I consider think so.

Also you have the same statics on the Div1 and Global rounds but it isn't the reason make the last problem easier. You need give very hard problem to the top people (and to prepare it so interesting as well, I suppose, especially for Div2).

  • »
    »
    21 month(s) ago, # ^ |
      Vote: I like it +9 Vote: I do not like it

    A problem solved by 10 or less partitipants does no relevant change in the standings table. Also that 10 people are Div1 in next or second next contest.

    The basic truth is that there are no top people in Div2.

    • »
      »
      »
      21 month(s) ago, # ^ |
      Rev. 2   Vote: I like it 0 Vote: I do not like it

      Maybe, but the main statement about unofficial people, who participate to solve Div2 F, for example, and without hard last problem it will be worse for them.

      Of course, the main auditory for Div2 don't include these people, but in my opinion you need remember about them too.

      • »
        »
        »
        »
        21 month(s) ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        Yeah, but general idea of post was that those difficult problems are absolutely not worth time spent on them. Though, of course they are interesting for div1 participants, but sadly they're not really relevant for auditory of div2 (real auditory, not thousands of alts, ehhhh...)

»
21 month(s) ago, # |
  Vote: I like it +3 Vote: I do not like it

In my opinion, it would be better to lower the difficulty of Div. 2 E/F in general, but problem setters sometimes misjudge the difficulty of problems, which happens for constructive problems more often. This can cause a too difficult Div. 2 E/F, but I guess this is unavoidable since the performances of testers don't accurately reflect that of all participants.

  • »
    »
    21 month(s) ago, # ^ |
      Vote: I like it +8 Vote: I do not like it

    Agreed.

  • »
    »
    21 month(s) ago, # ^ |
      Vote: I like it -59 Vote: I do not like it

    But if we lower the difficulty of div2 E/F then it will create 10x more candidate master and master participants. And after some time they will complain that there are not enough rated contests for them. Thus creating pressure to make more global and div1 + div2 contests. Creating them are super hard. Also in div1 + div2 contest only the last 2 or 3 problems of div2 are the first problems of div1. The rest 4 or 5 problems needs to be created which requires more time and effort.

    • »
      »
      »
      21 month(s) ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      No, it doesn't work this way — if everyone AKs a contest, people who AK it faster gets more rating, and people who solve everything at the last minute still lose rating.

  • »
    »
    21 month(s) ago, # ^ |
      Vote: I like it +17 Vote: I do not like it

    but problem setters sometimes misjudge the difficulty of problems...

    That's why humanity invented contest coordinators

»
21 month(s) ago, # |
Rev. 4   Vote: I like it +27 Vote: I do not like it

I think that the main problem of making Div1 rounds is to make Div1F problems that are hard even for LGMs, so we can't have more Div1s. Hard problems in Div2 rounds make them more interesting for high-rated users. And CMs who are worried about alts in Div2s can just solve Div1s.

  • »
    »
    21 month(s) ago, # ^ |
      Vote: I like it -65 Vote: I do not like it

    Yes, I was saying the exact same thing but getting downvoted.

»
21 month(s) ago, # |
Rev. 2   Vote: I like it 0 Vote: I do not like it

I don't think so, although sone of our problem can hardly be D2F, it is hard to create D1E or D1F for CM or M problemsetters.

  • »
    »
    21 month(s) ago, # ^ |
    Rev. 2   Vote: I like it +31 Vote: I do not like it

    To the one who downvoted me: Not all Div 2 problemsetters are red. Yes we can sometime create some 2400+ task, but 3000+ problems for Div 1 + 2 is something very different. Don't say some thing like "Invite a red and he will do the rest", we aimed to a div 2 at first. And GM+ are not backups for somebody's plan.

»
17 months ago, # |
  Vote: I like it 0 Vote: I do not like it

I disagree, it's not just about solving problems in contest, it's also about enriching the problem archive which is crucial when we level up and as div-1 contests are very less in number that makes a lot of sense