I am removing my contest proposal from Codeforces.
Note: This is a serious blog post. Also, the opinions presented in this blog are my own and not of my friend BiIIy who wrote the contest with me.
Today I have removed my contest proposal from Codeforces. I should probably have done this a long time ago. The proposal was made in February, but the problems were written and finished by March, and that is when I opened the contest proposal for review and messaged KAN. It is currently December and no progress or development has been made regarding my contest proposal since, so I will be removing it.
I have multiple reasons for removing my contest proposal:
I have no motivation to host an online contest anymore.
I am involved in problem setting with two other high school programming contests in my university, and I write, host, and judge another contest for local high schools.
I have forgotten that I “was in the queue” for writing a Codeforces contest anyway.
Codeforces is not motivated to work with me.
Codeforces does not communicate.
Codeforces is one of the biggest, most important, and most influential platforms in competitive programming so I think it should be held to a high standard. My main motivation writing this blog post is to let other aspiring problem setters know of my experience, and so that Codeforces will look for ways to improve their system for the new year. Specifically, I request Codeforces to be more transparent and honest with those of us waiting in line.
My experience with problem setting for Codeforces is as follows:
Some time in February, Billy and I decide that we want to host a Codeforces contest.
Between then and the end of March, we write five problems that we think would make a good Div2 round. When we finished the set, we opened the contest for review and I messaged KAN.
In May I decide I can write a better B problem so, since I’ve had no response yet, I wrote it and added it to the contest.
Sometime in July this guy writes a blog, I comment on it (click on the link and see context), which attracts the attention of Codechef, and they ask me to send them my problems to them. I do, and they reply quickly with some criticism of the problems. However, at the time I was abroad and school season began, so I decide to give up on writing a contest for the time being.
That guy with the blog was able to host a contest. I assume that is because he had a Div1 contest ready, which may be in higher demand. This is OK… But time continues to pass without any kind of communication from Codeforces. As I was already uninterested in hosting anymore, this is probably when I should have removed my proposal.
October 28th, KAN writes back. This is the only thing I hear from them.
November 7th, I get a comment on (just) one of the problems from 300iq. (image) (For context: in this problem I describe a mincost maxflow solution. I do not think that the feedback I received is useful feedback) I am still not sure what that comment means. No matter, I won’t hear from him again.
Fast forward to today.
I hope to bring these issues to light so that we can have conversation. Currently there are very few posts on this website that provide information regarding contest writing. It is possible that Codeforces simply gives priority to already trusted problem setters and will only use new problem setters as a last resort. I believe Codeforces should not waste our time. It should not take you months to comment on 5 simple problems, provided to you in the most abridged form (as you ask), unless you literally have hundreds of competitions in the queue. If that were the case, Codeforces should make an announcement – “Hey everyone, there are hundreds of contest proposals and it is impossible for our small team to review them all. We will not be looking for any new contest proposals for….” some estimated time.
When I used to try and look up information about Codeforces contest setting, such as what should my expectations be? how long does it take? etc. I could not find anything. Hopefully now, when some other problem setter wishes to learn what I wanted to know they will find this blog.
I will make some suggestions for Codeforces, I hope the team reads these and comments. Ultimately, it is their decision what action to take regarding my complaint. Every reasoning for my arguments is based on assumption. I could not possibly know what happens behind the scenes because Codeforces does not communicate. Therefore, some of these suggestions may be invalid.
My suggestions are as follows:
Codeforces should be more transparent. Some people write poor contests. Their problems may be poorly thought out, vague, uncreative, obvious, or they may have an anime protagonist. In these cases, Codeforces should reply immediately with the issue.
Codeforces may, at some point in time, have too many proposals from novice problem setters. It is perfectly reasonable for Codeforces to be honest and turn away some of these problem setters. It’s like hiring for a job: you have no obligation to hire somebody unqualified. However, it is the company’s ethical and respectful obligation to reply to them promptly and reject the contest proposal. It is a proposal after all.
If Codeforces has too many contest proposals in the queue, they should make an announcement. Something that will let people know that their contest may take longer than usual to be reviewed.
Codeforces should have two separate queues. One for Div1+Div2, and one for Div2 (maybe one for Div3, if Codeforces wishes to allow people to write problems for these in the future). Once you open your contest for review, you should be able to see your position in the queue. This way you can see how long you may have to wait.
When a contest is finally assigned a person to review it, that person should review it promptly. Two months to review five problems, as has happened to me, is absurd. Even then, I only got very vague feedback on a single problem! Concerns, criticism, and compliments (if applicable) should be given to each problem, so that if work needs to be done, the problem setter can get started.
There may be more issues with the system. I wouldn’t know. It took me nine months and I only made it this far.
I have two purposes writing this blog. One is that I hope that Codeforces will look for ways to improve the contest writing system the coming year, and two is to enlighten some other novice problem setters who are still waiting in line for their chance to write their first Codeforces contest by sharing my experiences. The second reason is why I am making a public blog post rather than just PMing Mike, and by making a blog I think it is more likely to attract attention to the issue at hand. By no means is my intention to publicly shame anyone mentioned in this blog.
To reiterate, I do not wish to hold a Codeforces contest anymore, so I am not asking for Codeforces to review my problems. I do ask that Codeforces does not use my problems in a contest without my permission, and that the content of the problems be kept confidential. I may return to contest writing for Codeforces in the future, but not unless changes are made. (At this rate you could have a child before you can have a contest). If you are a new problem setter I would recommend considering Codechef, they respond much faster.
Lastly, it is undeniable that Codeforces is a fantastic competitive programming platform. It is one of the best things to happen to competitive programming period, and it does wonders to the community. I personally owe a huge portion of my programming skill to this website, and I want to see its success continue. It is by far the website I use most frequently in competitive programming. The work that MikeMirzayanov and the team does is admirable, and Codeforces makes tremendous progress every year as the contest programming scene grows. I have great respect for the Codeforces team and I hope that this criticism is not taken negatively. Like I said before, Codeforces should seek to meet a high standard, and it should be more communicative with aspiring problem writers. Otherwise, why even give the ability to propose a contest to all Experts+?
I hope to see a response from the team which will address some of these issues, and that Codeforces will work to make a better experience for contest writers in the coming year.
I am very sorry :( The notification was gone with a bunch of others, and I didn't notice it and the activity on your round because there were a lot of big rounds to work on these months.. You could have written me a personal message! I would be happy to continue the review if you change your mind :)
There are still many issues with the current system. I think once a contest is selected for review it should be reviewed promptly and in its entirety. Meaning, every problem should get some comment on it at first glance. It is pointless to work on one problem at a time when a lot of feedback could be gathered at once and the problemsetter can begin to work on their corrections while someone like you works on a different contest in the meantime.
Besides, this doesn't solve the very long and lonely waiting period between opening the contest for review and being contacted for the first time. At least a confirmation message of "Your contest is now open for review, we have received it and will update you in approximately ________ days regarding your status".
That's probably a single issue. As for me, I've always received feedback for all problems at once.
GreenGrape but your contests sucks
I remembered ~2 years ago I and my friend decided to write contest(s) here. We even changed our handles to match with the theme so we were really excited about it.
But several months had been passed, the year changed, and we decided to change our handles back and let it go. I couldn't recall any replies from coordinators back then.
Anyway, we then gave up and submitted the problems to ACM-ICPC Regional instead (and got accepted there). So if you think you can't wait for Codeforces' long queue, just remember that there are many alternative choices available. (Hey, IOI 2019 is still calling for tasks!)
I agree. Anime will literally RUIN Codeforces. Mike Mirzayanov please ban people with anime tendencies from Codeforces.
No, Mike, please don't ban me!
Touhou isn't anime u dumb weeb
Well it's meaningless to discuss touhou is anime or not on codeforces lol
Hope Mike won't ban people with light novel tendencies :<
I am currently in work with my proposal, and through my experience and through technical views, the communication between authors and coordinators seemed quite poor.
The notification system is somewhat troublesome (unless your comments got replied, you won't receive notifications even for new comments on your very own proposals), and what matter most is the lack of instant communications (like instant messaging or such), causing significant delays in the process of working (like reviewing problems or declaring decisions upon how the contest would be structured).
These are my opinions on how the proposing system work. I hope such issues can be tackled in the coming year.
Lol you wanted to give a mincost maxflow problem at div2 round? thank you for deleting your shitty contest. let me guess other problems:
E) mincost maxflow
D) persistent tree with nontrivial operation
C) a dp with 7 dimensions and 20 transitions for each state
B) an easy to make mistake, hard to debug problem with a lot of hacks
A) very easy problem without hacks
You get a 20%
mincost maxflow problem -> fake accounts winners
Me in mincost maxflow problem -> just take minimum over all sliding windows and let me try submitting it without any proof :D LOL
Question about the queue system though. Do you think someone's position should be weighted based on rating/setting experience or just a first come first serve?
I don't think higher rating means better contests so I think it should be first come first serve. Codeforces should read the problems, and if they absolutely suck send them to the back of the queue. Experienced problem setters will have enough of an advantage as they will know how to prepare a quality contest beforehand, so they will rarely be sent to the back of the queue.
It is disappointing that despite the attention this post received, Codeforces decides to continue being silent.
At least this level of communication is consistent with what I described in the blog. Keep that in mind if you decide to write problems for this website, as it seems they are making no effort to change.
I think you're wrong. They're probably delaying their response because they're trying to figure out how they can improve their system the best. Mike is a great guy after all. There's no way he would just ignore your blog.
I am sure he will ingore
In that case, I think it would be a respectful thing to acknowledge the blog with a comment along the lines of: "We have read your complaints and we are currently discussing solutions internally/expect to see some announcement soon."
At this point I feel like I am lecturing too much on how to handle PR...
What's your problem with anime?
300iq has already answered about this particular case. We're working on improving the proposals and notifications systems to make them more convenient.
As for the general queue, we're doing good progress on resolving it. When the system was introduced, I was the only coordinator and it was out of my capabilities to process the proposals in real time. We thought it was a one-time problem with many people using their first chance to hold the contest. It was partly true, but the queue was still increasing, that's why we tried to find more coordinators and succeeded last summer. Now the queue is consistently shrinking: in summer it was 3 pages of proposals waiting, now it's less than one page. I'm expecting that all the queue will be gone in 1-2 months.
So the current status is that I thing everything will work as expected in 1-2 months. Btw, feel free to write your suggestions on how to improve the system here, we'll consider them all.
Thank you for providing some insight.
Please let authors watch the queue so they can estimate and balance the schedule. Thank you.
I genuinely think this would be a great change, though the fact they haven't commented on this makes me think they don't want to do it for one reason or another.
I have published a proposal on 1st April, still no comments. Is the queue really gone?
a month ago i was said BanRussiaAtIOI will coordinate my proposal, but he hasn't appeared still! is he ill?
Your suggestions are very impressive, CF team must pay attention to these suggestions. Don't worry your hard work will be repaid.
Thanks for the great post.
or they may have an anime protagonist