During the 25th of May I got an unfounded abuse by sys/ovh you call it:
We already got more than 20 abuse reports regarding your IPs.
Please take all appropriate actions, including eventually stopping your M*** crawler, to make stop this abuse reports.
If we continue to receive abuse reports related to your IPs, we could have no other choice but to permanently suspend your OVH services and your customer account.
Please keep us updated within the next 24 hours with details about actions you took.
Cordially,
The OVH Abuse team.
This project has crawled over 5 trillions of urls and I still use it with my kimsufi which for some reason today it stopped working ( I've put in contact with support via email as I can't login to the stupid control panel nor retrieve the password) and with OVH 2016. With the latter I've just been asked what I was doing and I answered back stating the project is 10 years old, 100% legit, that respects robots.txt's , etc, etc.
Well I had 4 big servers which suddenly became useless so I stopped the crawlers.
This crawlers are as legit as googlebot is and everyone wants googlebot in their website. We don't even download entire websites, just backlinks ( we have the biggest backlink index worldwide and I just contribute having made almost 4.5 PB of just hyperlinks).
I tried tweeting Oles, Maxime H. ( both follow me in twitter), replying to abuse but no one assumed anything nor gave me explanations of what this "20 abuses are".
Many of our members deal with companies like Leaseweb,Hetzner, WholeSaleInternet and so on and I won't tell you none hasn't had an issue because of an unproperly/lazy IDS ( intrusion detection system) but not at this scale and what pisses me most is the lack of foundation.
We already know ( as we have crawled a big part of the internet ) some lousy providers who claim to be protection providers ( webiron, bitninja and an italian hosting called netsons). None of those reveal their ip ranges to avoid being crawled nor the "offended url" to the point many ISP given the proof we have presented them started trashing their false abuses.
What does OVH/SYS , etc say about Abuse #FBKLHQVNPG] ??
Nothing.
That's really disappointing. Many guys who host pure **** with you and you are fine and you don't even care to give at least a fake answer?
