You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
XXII. Access a third-party web property for the purposes of web scraping, web crawling, web monitoring, or other similar activity through a web client that does not take commercially reasonable efforts to identify itself via a unique User Agent string describing the purpose of the web client and obey the robots exclusion standard (also known as the robots.txt standard), including the crawl-delay directive;
Well, they are sort-of-right ;)
So yeah.. the easiest would be to fork the repo and then deploy the heroku service from there. Button should stay the same.
So in order for the heroku deployment to work, the app.json needs to aim at a repo that is not blacklisted, which should be any fork that is not the main repo.
So, I would close this. It wont help to appeal to heroku since we are doing exactly what they do not want (scraping, ignoring robots.txt etc).
Activity
Bockiii commentedon Apr 26, 2022
Is there more log?
dvikan commentedon Apr 26, 2022
The repo is blacklisted for some reason.
This worked successfully:
Bockiii commentedon Apr 26, 2022
Well, they are sort-of-right ;)
So yeah.. the easiest would be to fork the repo and then deploy the heroku service from there. Button should stay the same.
dvikan commentedon Apr 27, 2022
It's not enough to fork the repo bockiii. I tested with a fork. It was also necessary to apply the diff I pasted above.
Bockiii commentedon Apr 27, 2022
you're right, I implied that.
So in order for the heroku deployment to work, the app.json needs to aim at a repo that is not blacklisted, which should be any fork that is not the main repo.
So, I would close this. It wont help to appeal to heroku since we are doing exactly what they do not want (scraping, ignoring robots.txt etc).