### crap_the_coder's blog

By crap_the_coder, 4 months ago,

Do you have many accepted solutions in CodeForces/CodeChef/Atcoder, but you can't share a convenient link to your GitHub to flex your submissions? Maybe it would take too long to upload all the files because you 1000+ accepted submissions?

For example, here's my repository that was generated by UpCode: https://github.com/crapthecoder/Competitive-Programming

Disclaimer

#### To use UpCode, perform the following steps:

• ###### Enter credentials as prompted

If all goes well, then you should see some lines like CodeForces/GNU C++20 (64)/1234E | Hello World/123456789.cpp CREATED. After that, you should be able to see the updates in your GitHub repository.

#### NOTE:

• If you are running the CodeForces scraper, a Chrome browser will open. DO NOT close it or interact with it. You can safely ignore it and do something else.

• DO NOT run multiple instances of the program for the same repository. Also, do not turn off or sleep the PC when the code is executing.

• It is recommended to not run the code during a CodeForces contest, as a popup notification could possibly stop the code from scraping properly. Please wait until the contest is over.

• If there are some errors, please ignore them as long as the entire program doesn't stop. The code will go through the failed submissions again to make sure they get added.

• If the code stops executing, please try downloading the latest update again, as there have been many major updates/bug fixes thanks to the community. If there are still any issues, let me know in the comments and I will make sure to help you out!

• CodeForces API is sometimes inconsistent and might not always be available, in which case you will be notified by the program. In that case, please wait for a while and try again.

• Due to the Atcoder API only updating every week, there won't be any changes to the Atcoder folder in the repository if you run UpCode multiple times before it updates.

Special thanks to:

• advik123 for being my mentor in this project.
• ghosty for agreeing to test if he was mentioned in the blog. So here you go nerd. (Also for helping me make this blog)
• aryan12 for existing and sharing his wisdom to the world.
• n00brekt for finding like 20 bugs and testing my project.
• evenvalue for not bullying me (he bullied me into saying this)

Thank you to everyone for pointing out bugs, or suggesting some new feature. The CodeForces scraper was very hard to create, and still not as efficient as I would like it to be. But it should work, though it might take some time. If there are any bugs, please do let me know, as I want to make sure this project works for everyone :D

TL;DR:

• +107

 » 4 months ago, # |   +6 Sriv bhaiya OTZ
 » 4 months ago, # |   +6 orz
 » 4 months ago, # |   +6 orz sir
 » 4 months ago, # | ← Rev. 2 →   +9 Looks pretty cool. Also I am sure, existence of aryan12 was very useful.
•  » » 4 months ago, # ^ |   +6 Yes, his existence gave me motivation. Thank you aryan12.
•  » » » 4 months ago, # ^ |   +12 Bruh.
 » 4 months ago, # |   -12 omg SRIV SIR OTZ
•  » » 4 months ago, # ^ | ← Rev. 2 →   0 Hey! I would like to know if your CodeForces scraper works reliably. In my code, after making around 50 requests, CodeForces starts returning HTML that does not contain the solution. Does CF do the same to your code?EDIT: Found the cause of the issue:If anyone has any information on how to properly make requests such that both CF and the scraper are happy, please let me know!
•  » » » 4 months ago, # ^ |   +3 My nodejs code works completely fine. I wrote another script in google app script, which we can run periodically and make the process completely automatic.Link : https://github.com/aryan57/cp-crawler Old nodejs link : https://github.com/aryan57/cp-crawler-nodejs
 » 4 months ago, # |   +11 Pretty nice tool. Can you add support for Atcoder too.
•  » » 4 months ago, # ^ |   +1 Might do that now :D
•  » » 4 months ago, # ^ |   +1 Atcoder support has been added, take a look! If you find any bugs/issues, do let me know.
 » 4 months ago, # | ← Rev. 2 →   +6 Thank you for this tool! Is it possible for you to add the problem link in a text file in each problem's folder? And maybe name the problem folders with the problems' names and codes both? If not for other judges, at least for codeforces?
•  » » 4 months ago, # ^ |   +3 Thanks for the great suggestion! I will add that soon.
•  » » » 4 months ago, # ^ | ← Rev. 2 →   0 Maybe you could just insert a comment at the top of the code file with the problem link, and mention that the comment was auto-generated by UpCode. Not sure how feasible this would be though.You could start with just C++, maybe.
•  » » » » 4 months ago, # ^ | ← Rev. 2 →   0 I have made it so that a text file will be created, but in the process I found another issue. But a temporary fix has increased the time the program takes. If you are fine with the code taking a while, you can go ahead and use it now.
 » 4 months ago, # |   +8 Very nice tool indeed!I wrote a similar scraper for cses, if you were interested in it. It's currently on my github.
•  » » 4 months ago, # ^ |   0 Amazing tool. When i was trying to download the files i faced an issue, can you tell how do i resolve it ? Spoiler
•  » » » 4 months ago, # ^ |   0 Glad you've found it useful!I've changed something and now it should work. However I don't have this issue and can't tell whether it's fixed or not.If it still doesn't work, check out this question on stackoverflow.
•  » » » » 4 months ago, # ^ |   0 Thanks, its working now !
 » 4 months ago, # |   0 There is a similiar extension for LeetCode aswell,named LEETHUB..do check it out guys:)
 » 4 months ago, # |   +1 I am getting this error for codeforces solutions.
•  » » 4 months ago, # ^ | ← Rev. 2 →   +6 I am sorry, that's because CodeForces lets the code request atmost 80 times before not letting it make any more requests for a while. There was another issue.But no worries, as the code makes sure to go through the failed solutions once again to get it uploaded.I am currently working on a way to make it so that this error will not appear any more.
•  » » 4 months ago, # ^ |   +3 The issue should be (partially) fixed now. I hope to make more improvements in the future, but the updated code should hopefully work better for you.
 » 4 months ago, # |   +3 Is it me or all codes in your repository are empty?
•  » » 4 months ago, # ^ |   +3 Oh god
•  » » » 4 months ago, # ^ |   +3 nice project btw
•  » » » » 4 months ago, # ^ |   0 Thanks a lot! :D
•  » » 4 months ago, # ^ |   +3 Hopefully it's fixed after 7 hours of banging my head ;_;
 » 2 months ago, # |   +3 Very useful, but I stopped saving solutions as files instead just having "A.cpp" and clear it after every contest many red coders do the same thing I noticed
 » 6 weeks ago, # |   0 It is the best thing...well I know u from TLE
•  » » 4 weeks ago, # ^ |   0 Hello! :D
 » 6 weeks ago, # |   0 can you make the extension upload non-uploaded solutions only? and btw thank you !
•  » » 4 weeks ago, # ^ | ← Rev. 2 →   +1 The issue is that to know which one to avoid, I have to scrape it first ;_;For example, to avoid some CodeChef solution, I first need to know its link.
 » 3 days ago, # |   0 Hello,I am facing an issue where not all the problems are uploaded (less than 50%) and when trying to scrape on Code Chef I get ERROR:root:'NoneType' object is not subscriptable FOR None