OVH Community, your new community space.

Attack from OVH Client Servers hitting our server


alvaroag
22-05-2015, 19:41
Quote Originally Posted by Dante
I did many times reporting & there is no E-mail from them

From my situation we got attack from internal OVH so the Anti DDOs not detect it PLUS the Pre Firewall (https://www.ovh.co.uk/anti-ddos/firewall-network.xml) I blocked all ICMP & UDP -except the ones I allowed it- BUT, we see some attacks can bypass(LEAKED) & caught by our internal software firewall ( for both inside & outside OVH traffic ) !
If OVH staaf doesn't answers, you may try contacting Oles on Twitter. He takes abuse cases very seriously, and has previously suspended accounts with large number of servers for sending large amounts of spam or performing large attacks. http://www.twitter.com/olesovhcom

Quote Originally Posted by Dante
Yes , Correct !

Summary , OVH must take action against internal attacks seriously OR offer option to block all incoming connection from OVH network with exception to OVH company services like (uptime ping test ...etc) & other that I want to except from this.
As I said, managing the internal traffic may be almost impossible, and may have a really high cost. What OVH must do is to enhance their staff capacity so they can handle all the abuse complaints they receive. They could also implement a statistical system on traffic usage, so they can keep an eye on server with high volume of outgoing traffic.

Dante
22-05-2015, 13:38
Quote Originally Posted by HuNterukh
Hey. Have you solved the problem? I had a dedicated server at SYS too, and i was mega attacked. Now i want to move to an OVH Dedicated server with Permanent mitigation and good firewall rules applied. Is it worth? Did you dropped all UDP connections in firewall rules?
Yes , it is worth to upgrade If you have frequently Attacks on your server !

We had SYS before ( not the Game Range) we had DDOS attack every 15 min which is enough to connection lost & cause lag to the server, Because the attacker knows when the Mitigation turned off, so as soon as the SYS Mitigatgation turned off he is attacking then stopping when mitigation


In that case SYS Anti DDOS is useless -Game Range have special anti ddos for game server but I haven't test it - so yeah it is worth to upgrade for permanent mitigation is far better



Quote Originally Posted by HuNterukh
Also, i notice in your screenshots that you are attacked for only 1 minute, and then the traffic turns ok, right? I was mega attacked at SYS for hours.
Sorry, The attack is variable sometimes in minutes & sometimes in hours

Quote Originally Posted by alvaroag
I know this may not be the kind of answer you expect, but there's not too much you can do about it. Configuring a really strict firewall is the best you can do. Apart from that, you can always contact the ISP where the traffic is originated(OVH in this case). However, OVH abuse complaints are managed directly from France, and as a large ISP, they might have a lot of requests, probably more than they can handle efficiently, so they may take a while to answer.



About HuNterukh's question. As I said before, I don't think DDOS mitigation will handle in-network traffic. Mostly because the VAC (mitigation system) has a capacity of 160 Gbps, according to OVH website, which is obviously insufficient to handle local traffic. However, no one from OVH has confirmed or denied this, so I cannot affirm it.

Of course, DDOS mitigation should work without problems with traffic from outside OVH.
I did many times reporting & there is no E-mail from them

From my situation we got attack from internal OVH so the Anti DDOs not detect it PLUS the Pre Firewall (https://www.ovh.co.uk/anti-ddos/firewall-network.xml) I blocked all ICMP & UDP -except the ones I allowed it- BUT, we see some attacks can bypass(LEAKED) & caught by our internal software firewall ( for both inside & outside OVH traffic ) !

Quote Originally Posted by HuNterukh
@alvaroag, and, let's say we block all UDP incoming/outgoing traffic in the hardware firewall, there will still be problems?
It depend on your service opened ports & attacks on it most of the time.

Quote Originally Posted by alvaroag
Depending on the traffic volume of the attack, the server could experience network slowness. This could even saturate the server connection, leaving the server disconnected from the internet. This may not affect connection via KVM/IP or vRack interface, if any available. Of course, when the attack ends, or the VAC mitigates the attack, everything goes back to normal imediately.

On the CPU, if you are using a software firewall(iptables-based on Linux, Windows Firewall on Windows), there could be some small impact on CPU performance, but nothing significant. If you use the real hardware firewall (Cisco ASA, offered with additional cost), an attack may no affect CPU performance.

This is, of course, when the attack is against closed ports. Attacks on open ports, for example HTTP(80), can have higher impact on the CPU, depending on specific, application dependant configuration files, for example, Apache HTTPd.
Yes , Correct !

Summary , OVH must take action against internal attacks seriously OR offer option to block all incoming connection from OVH network with exception to OVH company services like (uptime ping test ...etc) & other that I want to except from this.

alvaroag
21-05-2015, 17:49
Depending on the traffic volume of the attack, the server could experience network slowness. This could even saturate the server connection, leaving the server disconnected from the internet. This may not affect connection via KVM/IP or vRack interface, if any available. Of course, when the attack ends, or the VAC mitigates the attack, everything goes back to normal imediately.

On the CPU, if you are using a software firewall(iptables-based on Linux, Windows Firewall on Windows), there could be some small impact on CPU performance, but nothing significant. If you use the real hardware firewall (Cisco ASA, offered with additional cost), an attack may no affect CPU performance.

This is, of course, when the attack is against closed ports. Attacks on open ports, for example HTTP(80), can have higher impact on the CPU, depending on specific, application dependant configuration files, for example, Apache HTTPd.

HuNterukh
21-05-2015, 16:08
@alvaroag, and, let's say we block all UDP incoming/outgoing traffic in the hardware firewall, there will still be problems?

alvaroag
21-05-2015, 15:35
I know this may not be the kind of answer you expect, but there's not too much you can do about it. Configuring a really strict firewall is the best you can do. Apart from that, you can always contact the ISP where the traffic is originated(OVH in this case). However, OVH abuse complaints are managed directly from France, and as a large ISP, they might have a lot of requests, probably more than they can handle efficiently, so they may take a while to answer.

About HuNterukh's question. As I said before, I don't think DDOS mitigation will handle in-network traffic. Mostly because the VAC (mitigation system) has a capacity of 160 Gbps, according to OVH website, which is obviously insufficient to handle local traffic. However, no one from OVH has confirmed or denied this, so I cannot affirm it.

Of course, DDOS mitigation should work without problems with traffic from outside OVH.

HuNterukh
21-05-2015, 14:55
Also, i notice in your screenshots that you are attacked for only 1 minute, and then the traffic turns ok, right? I was mega attacked at SYS for hours.

HuNterukh
21-05-2015, 14:43
Hey. Have you solved the problem? I had a dedicated server at SYS too, and i was mega attacked. Now i want to move to an OVH Dedicated server with Permanent mitigation and good firewall rules applied. Is it worth? Did you dropped all UDP connections in firewall rules?

Dante
20-05-2015, 19:39
Quote Originally Posted by alvaroag
When you say the "hardware firewall", do you mean you have ordered, along with your server, the dedicated firewall?
No, I haven't order Cisco ASA , It is the dedicated firewal.

ASA Is useless against DOS attacks & ASA can't handle less than 500 Mbps !

alvaroag
20-05-2015, 19:01
When you say the "hardware firewall", do you mean you have ordered, along with your server, the dedicated firewall?

Dante
20-05-2015, 18:30
Okay now another attack from this IP : 5.196.94.213

IP: 5.196.94.213
Hostname: ns376922.ip-5-196-94.eu
ISP: OVH SAS
Organization: OVH SAS

May 20 18:02:50 nsXXXXXX kernel: Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=XXXXXXXXXXXXXXXXXXXXXXXXXX SRC=5.196.94.213 DST=XX.XX.XX.XX LEN=60 TOS=0x00 PREC=0x00 TTL=60 ID=28019 DF PROTO=TCP SPT=49985 DPT=22 WINDOW=29200 RES=0x00 SYN URGP=0
May 20 18:02:51 nsXXXXXX kernel: Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=XXXXXXXXXXXXXXXXXXXXXXXXXX SRC=5.196.94.213 DST=XX.XX.XX.XX LEN=60 TOS=0x00 PREC=0x00 TTL=60 ID=28020 DF PROTO=TCP SPT=49985 DPT=22 WINDOW=29200 RES=0x00 SYN URGP=0
May 20 18:02:53 nsXXXXXX kernel: Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=XXXXXXXXXXXXXXXXXXXXXXXXXX SRC=5.196.94.213 DST=XX.XX.XX.XX LEN=60 TOS=0x00 PREC=0x00 TTL=60 ID=28021 DF PROTO=TCP SPT=49985 DPT=22 WINDOW=29200 RES=0x00 SYN URGP=0

================================
Update : few minutes after post this reply

Another IP from OVH client attacking from this IP:37.59.47.101

IP: 37.59.47.101
Hostname: ns3000868.ovh.net
ISP: OVH SAS
Organization: OVH SAS

May 20 19:37:32 nsXXXXXX kernel: Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=XXXXXXXXXXXXXXXXXXXXXXXXXX SRC=37.59.47.101 DST=XX.XX.XX.XX LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=12626 DF PROTO=TCP SPT=48064 DPT=22 WINDOW=14600 RES=0x00 SYN URGP=0

May 20 19:37:33 nsXXXXXX kernel: Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=XXXXXXXXXXXXXXXXXXXXXXXXXX SRC=37.59.47.101 DST=XX.XX.XX.XX LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=12627 DF PROTO=TCP SPT=48064 DPT=22 WINDOW=14600 RES=0x00 SYN URGP=0

May 20 19:37:35 nsXXXXXX kernel: Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=XXXXXXXXXXXXXXXXXXXXXXXXXX SRC=37.59.47.101 DST=XX.XX.XX.XX LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=12628 DF PROTO=TCP SPT=48064 DPT=22 WINDOW=14600 RES=0x00 SYN URGP=0

Dante
18-05-2015, 17:50
Thanks for reply,

But I need to block this in Hardware Firewall which is not allowed to block range IP or rules not enough to do it.

Blocking it via Internal firewall -Iptables for example- is useless since the attack is big and we already using firewall to block attacks but still can take down web & other services.

We have already blocking incoming ICMP through Hardware Firewall & all UDP ports except the ports that we authorize it

The strange thing that we had attack a 5 hours ago and we got hit on UDP port which is not authorized and blocked by the last rule
(Priority: 19 Action: Refuse Protocol: UDP IP source: all Source port: *Empty* Destination port: *Empty* Options: *Empty* )

May 18 13:06:16 nsXXXXXX kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=XXXXXXXXXXXXXXXXXXXXXXXXX SRC=195.101.49.33 DST=XXX.XXX.XXX.XXX LEN=3338 TOS=0x00 PREC=0x00 TTL=53 ID=46987 PROTO=UDP SPT=53 DPT=40891 LEN=3318

Port: 40891 UDP is not authorized in the OVH Hardware Firewall !

My Regards

alvaroag
18-05-2015, 17:21
If you really want to drop traffic from OVH datacenters, you can block all their IP ranges:

5.39.0.0/17
5.135.0.0/16
5.196.0.0/16
8.7.244.0/24
8.18.128.0/24
8.18.136.0/21
8.18.172.0/24
8.20.110.0/24
8.21.41.0/24
8.24.8.0/21
8.26.94.0/24
8.29.224.0/24
8.30.208.0/21
8.33.96.0/21
8.33.128.0/21
8.33.136.0/24
8.33.137.0/24
37.59.0.0/16
37.60.48.0/21
37.60.56.0/21
37.187.0.0/16
46.105.0.0/16
46.105.194.0/23
46.105.198.0/24
62.245.0.0/19
87.98.128.0/17
87.253.232.0/24
91.90.88.0/21
91.121.0.0/16
92.222.0.0/16
94.23.0.0/16
103.5.12.0/22
142.4.192.0/19
149.202.0.0/16
151.80.0.0/16
167.114.0.0/16
176.31.0.0/16
176.31.160.0/22
176.31.176.0/22
176.31.184.0/22
176.31.188.0/22
178.32.0.0/15
178.32.133.0/24
178.32.134.0/24
178.32.135.0/24
178.236.224.0/20
185.10.17.0/24
185.12.32.0/23
188.165.0.0/16
192.95.0.0/18
192.99.0.0/16
193.104.19.0/24
193.104.19.0/25
193.104.56.0/24
193.109.63.0/24
195.110.30.0/23
195.246.232.0/23
198.27.64.0/18
198.50.128.0/17
198.100.144.0/20
198.245.48.0/20
213.186.32.0/19
213.251.128.0/18
And also IPv6 prefixes:

2001:41d0::/32
2402:1F00:2000::/35
2402:1F00:4000::/35
2402:1F00::/32
2402:1F00::/35
2607:5300::/32
Prior to block traffic from OVH network, you must know that, doing so, you will also block ICMP from OVH monitoring. That means OVH will detect your server as offline, causing the backend system to reboot it, unless you disable monitoring on the manager. Also, if you, at any time, reboot your server from the manager, the backend system will never know when your server is online, so you will get an error message; however, the server will boot normally, unless something on the OS goes wrong. The KVM/IP option, if included, can be really handy in this cases.

Remember, this is not a recommended option. Putting a connection rate limit can be the best option here.

Dante
18-05-2015, 17:06
Quote Originally Posted by alvaroag
That would be a really interesting subject to discuss. I'm not sure if DDOS mitigation works for home attacks(I think not, but not really sure), but OVH does have some kind of domestic attack detection. Briefly, they have many "trap" IPs assigned to their security system. Those IPs are not publi8shed anywhere, so any attack against any of them may be considered a real, malicious attack. That works also for OVH customers.

BTW, the problem with a full DDOS mitigation for a large network as OVH is that if would be slow, or really expensive, or both. It's like putting equipments between all your LAN workstations to inspect all the traffic that goes between them.

But, as I said, write to OVH abuse ASAP. For what I know, they take abuse claims very seriously.
Thanks for reply,

I don't want to know their IPs , I just want them to add at least feature to not accept incoming traffic from other OVH client servers except the IPs I allowed only -If I have to- !

Quote Originally Posted by Dani
Hi

For all abuse issues, it may be best to use the contact form on our website as it will allow you to submit all the required information.

The link is: https://abuse.ovh.net

Thanks

Danny
Thanks for reply,

I already did that a few days ago via E-mail & I re-submit through the page form now

They didn't respond via E-mail : abuse@ovh.net

My Regards

Dani
18-05-2015, 11:50
Hi

For all abuse issues, it may be best to use the contact form on our website as it will allow you to submit all the required information.

The link is: https://abuse.ovh.net

Thanks

Danny

alvaroag
18-05-2015, 04:32
That would be a really interesting subject to discuss. I'm not sure if DDOS mitigation works for home attacks(I think not, but not really sure), but OVH does have some kind of domestic attack detection. Briefly, they have many "trap" IPs assigned to their security system. Those IPs are not publi8shed anywhere, so any attack against any of them may be considered a real, malicious attack. That works also for OVH customers.

BTW, the problem with a full DDOS mitigation for a large network as OVH is that if would be slow, or really expensive, or both. It's like putting equipments between all your LAN workstations to inspect all the traffic that goes between them.

But, as I said, write to OVH abuse ASAP. For what I know, they take abuse claims very seriously.

Dante
17-05-2015, 21:51
Quote Originally Posted by alvaroag
You may write to abuse@ovh.net, preferably attaching any related logs.

BTW, I don't think the permanent mitigation will handle attacks from inside the OVH network.
This is not good

Internal network attacks is no difference from being attacked outside

It must be a detection system or at least make additional option to not accept any connection from any OVH servers -except from company itself like ping test/downtime etc- Only !

alvaroag
17-05-2015, 06:15
You may write to abuse@ovh.net, preferably attaching any related logs.

BTW, I don't think the permanent mitigation will handle attacks from inside the OVH network.

Dante
16-05-2015, 18:48
Hello,

we have a Attack from this IP : 5.196.62.122

It happened a few minutes ago, hitting our server & cause its is lagging & all webpages goes down ( web servers down)

Please ban this Ip it bypass the Anti DDOS Pro Permanent Mitigation & cause our work down !

Command used :
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

Over 3000 Connections from this IP : 5.196.62.122

------------------------
Another IP from OVH Client server : 87.98.151.193

Attacking our server which cause our service server down

Detected by our Internal Firewall
--------------------------

Another IP Another IP from OVH Client server : 178.33.38.153

Attacking our server which cause our service server down



Log report from our firewall

A/D IP address Port Dir Time To Live Comment
DENY 178.33.38.153 * inout 47m 26s IP 178.33.38.153 (DE/Germany/-) found to have 27690 connections



They are bypassed the ANTI DDOS PRO (Permanent Mitigation)


ANTI DDOS PRO (Permanent Mitigation) is ON ! it does not protect from others OVH client servers !!


That is disappointed !


My Regards