OVH Community, your new community space.

New mSP - terrible network upload speed - please help!


rickyday
03-10-2013, 11:15
Speeds back to normal now for me in GRA on my mSP.

23MB/s (about 185Mbit) from your test file Fred, things any better for you?

Tz-OVH
03-10-2013, 08:52
http://weathermap.ovh.net/gravelines

http://weathermap.ovh.net/europe

Things don't look saturated...but there might be issues that we aren't aware of.

rickyday
02-10-2013, 18:48
If it is any consolation speeds are rubbish today for Gravelines, and even downloading the test file from proof.ovh.net for RBX is absolutely abysmal.....

On the plus side speeds from SBG into Gravelines are top notch, but please OVH sort out whatever is going on in Gravelines please.

Fred I downloaded your test file onto my server in GRA and got 5MB/s which is pants considering its probably on the rack next door. Strange thing is I download your file onto my RBX 4G and it maxes out the connection.

Looks like all these 2G Kimsufi servers are playing havoc with the network, as I believe most are being deployed in GRA.

Lets hope OVH sort this out soon, they are probably tweaking bandwidth and routing (I hope) seeing as the DC is getting busier and busier with the KS 2G brigade!

MediocreFred
02-10-2013, 17:27
Tech Support created an incident. The technician ran iperf from the server in Rescue mode to a test OVH server and determined that speeds are optimal.

A couple of you responded with servers in GRA. If you have an server in GRA (ideally, an mSP running Windows Server 2008/2012), can you please send me a link to a test file that I can attempt to download from home and compare the speed with what I am getting from my mSP. If they are pretty close, I'll attribute it to sucky routing between GRA and the US. Otherwise, I'll add info regarding external routing to the incident.

Thanks,
MediocreFred.

MediocreFred
01-10-2013, 17:52
The traceroutes look very similar:

KS at RBX - Maxes out my 35 Mbits/sec connection:
1 10.40.0.24 (10.40.0.24) 4.640 ms 4.505 ms 4.833 ms
2 50.38.7.101 (50.38.7.101) 4.975 ms 4.884 ms 4.849 ms
3 ae2---0.cor01.bvtn.or.frontiernet.net (74.40.1.217) 24.981 ms 24.404 ms 24.942 ms
4 ae0---0.cor02.bvtn.or.frontiernet.net (74.40.1.186) 24.964 ms 24.398 ms 25.089 ms
5 ae3---0.cor01.plal.ca.frontiernet.net (74.40.1.225) 37.324 ms 32.017 ms 25.093 ms
6 ae0---0.cbr01.plal.ca.frontiernet.net (74.40.3.150) 24.823 ms 24.437 ms 24.994 ms
7 ge9-26-1G.ar1.pao2.gblx.net (64.215.195.45) 24.941 ms 24.664 ms 24.972 ms
8 xe10-2-1-10G.scr4.SNV2.gblx.net (67.17.111.245) 25.076 ms 29.918 ms 24.957 ms
9 e1-2-10G-ar6.SJC2.gblx.net (67.16.146.58) 24.843 ms 29.388 ms 24.973 ms
10 snj-2-6k.ca.us (178.32.135.1) 25.081 ms * *
11 * * *
12 * * *
13 * * *
14 178.33.100.67 (178.33.100.67) 177.261 ms 177.063 ms 177.512 ms
15 * rbx-1-6k.fr.eu (91.121.131.13) 196.166 ms *
16 rbx-33-m1.fr.eu (213.251.191.94) 173.515 ms 174.827 ms 175.065 ms
17 ks354719.kimsufi.com (91.121.104.167) 177.300 ms 177.318 ms 177.488 ms

mSP at GRA - my problem server:
1 10.40.0.24 (10.40.0.24) 2.854 ms 4.388 ms 4.915 ms
2 50.126.112.61 (50.126.112.61) 4.962 ms 4.661 ms 4.725 ms
3 ae2---0.cor01.bvtn.or.frontiernet.net (74.40.1.217) 24.942 ms 24.128 ms 25.047 ms
4 ae0---0.cor02.bvtn.or.frontiernet.net (74.40.1.186) 24.965 ms 34.349 ms 24.839 ms
5 ae3---0.cor01.plal.ca.frontiernet.net (74.40.1.225) 24.944 ms 24.339 ms 24.841 ms
6 ae0---0.cbr01.plal.ca.frontiernet.net (74.40.3.150) 24.976 ms 24.387 ms 24.961 ms
7 ge9-2-1G.ar1.PAO2.gblx.net (208.49.158.141) 24.969 ms 24.408 ms 25.130 ms
8 ae3-20G.scr3.SNV2.gblx.net (67.17.66.221) 24.923 ms 24.420 ms 25.064 ms
9 e1-4-20G-ar6.SJC2.gblx.net (67.16.146.54) 24.844 ms 24.388 ms 25.045 ms
10 snj-2-6k.ca.us (178.32.135.1) 24.833 ms * 23.382 ms
11 lax-1-6k.ca.us (178.32.135.161) 74.970 ms 74.300 ms 75.055 ms
12 * * *
13 * * *
14 gra-g2-a9.fr.eu (178.33.103.218) 178.647 ms 179.403 ms 180.029 ms
15 * * *
16 ns3367434.ovh.net (37.187.78.229) 182.923 ms 174.537 ms 174.932 ms

And yes, I am located in the US. Have a total of four OVH servers - two KS at RBX, one KS at BHS and now, an mSP at GRA. Of these four, only the mSP struggles to reach even a quarter of my bandwidth.

rickyday
01-10-2013, 17:11
Just got home from work retried download above and rock solid 15MB/s (120Mbit)

Must be a routing issue and only affect some ISPs.

*Edit*

downloaded on to my server in GRA and got 22MB/s
downloaded on to my Kimsufi in RBX and got 11.3MB/s

I would expect OVH > OVH speeds to be half decent though, strange that my Virgin Media connection loves GRA datacentre and good speeds there as well.

Here is a tracert if it sheds any light on to the situation.

Powershell 4
Tracing route to ns336xx34.ovh.net [37.187.78.22x]
over a maximum of 30 hops:

1 15 ms 5 ms 8 ms cpc1-heme11-2-0-gw.9-1.cable.virginmedia.com [81.104.44.1]
2 10 ms 8 ms 8 ms lutn-core-2a-ae5-2550.network.virginmedia.net [80.4.221.93]
3 11 ms 8 ms 8 ms popl-bb-1c-ae2-0.network.virginmedia.net [62.253.174.1]
4 6 ms 13 ms 9 ms tele-ic-5-ae0-0.network.virginmedia.net [213.105.159.117]
5 9 ms * * 195.66.236.220
6 11 ms 10 ms 12 ms gra-g1-a9.fr.eu [37.187.36.211]
7 11 ms * 11 ms gra-3a-6k.fr.eu [178.33.103.224]
8 13 ms 9 ms 13 ms ns336xx34.ovh.net [37.187.78.2xx]

Trace complete.
Fred could you post a tracert to your server you are having the issue with (from home connection)? and even a tracert to the one with no problems, perhaps they can shed some light on the issues you are having

MediocreFred
01-10-2013, 13:08
Quote Originally Posted by rickyday
Just downloaded your test file and have to report absolutely no problems at all, 15MB/s which is maxing out my connection here on Virgin Media (120Mbit)
Interesting! I have the same test file on two servers - this mSP at GRA and another KS at RBX (although this KS is running Linux - not to be confused with my KS at BHS running Win 2K12).

Downloading from the KS, I max out my 35 Mbits/s bandwidth. Downloading from this mSP, I get 1.4 Mbits/sec (drops to 900 Kbits/sec after a few minutes).

rickyday
01-10-2013, 09:00
Just downloaded your test file and have to report absolutely no problems at all, 15MB/s which is maxing out my connection here on Virgin Media (120Mbit)



I will try it again later at peak time.

raxxeh
01-10-2013, 00:10
Quote Originally Posted by MediocreFred
Ah, ok... thanks. What speeds do you normally expect to see from those servers hitting an mSP?
No idea, I don't bother speedtesting tiny msp servers..

One would assume that since those 10gbit servers both download at 9gbit/s (950-1000Megabyte/s) that it would saturate the msp servers upstream with no struggle.

MediocreFred
30-09-2013, 23:59
Quote Originally Posted by raxxeh
M for Mega, m for mbit.

Generic linux.
Ah, ok... thanks. What speeds do you normally expect to see from those servers hitting an mSP?

raxxeh
30-09-2013, 23:50
Quote Originally Posted by MediocreFred
Thanks guys!

@raxxeh: In your test outputs, when you say "M/s" do you mean MB/s or Mbits/s?

From my Kimsufi in BHS, I get a choppy 32 Mbits/sec - which is terrible for an in-network transfer.

@NeddySeagoon: those are about the speeds I get too from my PC at home (35 Mbits/sec fiber connection)
M for Mega, m for mbit.

Generic linux.

MediocreFred
30-09-2013, 23:42
Thanks guys!

@raxxeh: In your test outputs, when you say "M/s" do you mean MB/s or Mbits/s?

From my Kimsufi in BHS, I get a choppy 32 Mbits/sec - which is terrible for an in-network transfer.

@NeddySeagoon: those are about the speeds I get too from my PC at home (35 Mbits/sec fiber connection)

NeddySeagoon
30-09-2013, 23:33
2% [> ] 25,903,728 947KB/s eta 21m 0s

From Scotland over BT wholesale fibre that normally delivers 78Mbit/sec downstream.

raxxeh
30-09-2013, 23:28
rbx2, 10gbit:

--2013-09-30 23:24:30-- http://37.187.78.229/10Gb.dat
Connecting to 37.187.78.229:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1250000000 (1.2G) [application/octet-stream]
Saving to: `/dev/null'

26% [=====================================> ] 337,018,340 9.03M/s eta 93s ^C


rbx2, 10gbit #2:

--2013-09-30 23:24:56-- http://37.187.78.229/10Gb.dat
Connecting to 37.187.78.229:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1250000000 (1.2G) [application/octet-stream]
Saving to: `/dev/null'

33% [===============> ] 418,543,562 10.2M/s eta 83s ^C

rbx 1, msp:

--2013-09-30 23:26:25-- http://37.187.78.229/10Gb.dat
Connecting to 37.187.78.229:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1250000000 (1.2G) [application/octet-stream]
Saving to: `/dev/null'

22% [============> ] 284,534,841 16.7M/s eta 60s ^C


GRA, msp:

--2013-09-30 23:27:10-- http://37.187.78.229/10Gb.dat
Connecting to 37.187.78.229:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1250000000 (1.2G) [application/octet-stream]
Saving to: `/dev/null'

38% [==============> ] 480,815,646 6.75M/s eta 1m 53s ^C


The GRA msp is 37.187.79.x, so it's close to you. (obviously...being in GRA... ;P) I don't have access to this servers manager interface so I couldn't tell you which rack its in.

MediocreFred
30-09-2013, 23:22
The server is back up now.

I ran the network speed test from Rescue Mode. The first part (PC to/from server) completed right away, but, the second part (server to/from ovh network) never finished - even after half an hour. Here's a screenshot after half an hour...


The speeds to/from my PC are about a fifth of what I get from other http/ftp downloads and uploads.

Have sent this to tech support.

Please try to download the test file - http://37.187.78.229/10Gb.dat - and let me know what speeds you get.

Thanks,
MediocreFred

rickyday
30-09-2013, 22:51
Will try the download file when your server is back up.

I don't believe there are any problems with your server or the hardware especially as you say some speed tests are reporting back as fine.

I believe this is a routing/network issue.

MediocreFred
30-09-2013, 21:31
Taking the server down for about half an hour. Apparently, there is a network speed test in Rescue Mode that the tech support folks want me to run.

If the link to the test file above does not work, please try again after a while. I will post back here the results of the network speed test in rescue mode.

Thanks,
MediocreFred.

MediocreFred
30-09-2013, 21:14
Quote Originally Posted by rickyday
Just like to add I haven't upgraded any NIC drivers (default 2012 install from OVH manager), this is the version on my server just to add something else to the mix.
My network driver is newer - since I am running R2. But, with the stock 2012 Standard install, I had the same performance issue.

MediocreFred
30-09-2013, 21:14
Yeah, that's pretty close to what I get! How would I go about finding out if it is a problem with my Windows install (highly doubt it - it was the stock OVH Win Server 2012 Standard install) or with the server or its networking? Don't really want to spend 20 euros on an incident just to be told "it's not us, it's you".

Quote Originally Posted by crowarth
i'm lucky if i get 700K/s on that

wget http://37.187.78.229/10Gb.dat
--2013-09-30 20:05:29-- http://37.187.78.229/10Gb.dat
Connecting to 37.187.78.229:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1250000000 (1.2G) [application/octet-stream]
Saving to: `10Gb.dat'

4% [> ] 54,624,142 669K/s eta 25m 13s

crowarth
30-09-2013, 21:06
i'm lucky if i get 700K/s on that

wget http://37.187.78.229/10Gb.dat
--2013-09-30 20:05:29-- http://37.187.78.229/10Gb.dat
Connecting to 37.187.78.229:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1250000000 (1.2G) [application/octet-stream]
Saving to: `10Gb.dat'

4% [> ] 54,624,142 669K/s eta 25m 13s

MediocreFred
30-09-2013, 21:01
Hi RickyDay,

Thanks for the detailed reply and suggestions.

Here are my results from proof.ovh.net

RBX -

SBG -

Interestingly, my server is physically located very close to yours - GRA1, Rack G111B1_

I have a copy of OVH's test file, 10Gb.dat on my server (downloaded at ~ 130 Mbps). Here's the url - http://37.187.78.229/10Gb.dat - can you and others please test from both, an OVH and non-OVH connection.

Thanks,
MediocreFred.

rickyday
30-09-2013, 17:44
Just like to add I haven't upgraded any NIC drivers (default 2012 install from OVH manager), this is the version on my server just to add something else to the mix.


rickyday
30-09-2013, 17:19
I have a mSP located at GRA and speeds are very good get full speed advertised.

GRA 1 and Rack G111B1*

There should be no need to change duplex settings in this day and age especially on Server 2012 (don't blame you trying anything though)

Hope it gets resolved soon.

Just out of interest what are the results from http://proof.ovh.net

You shouldn't have to tweak anything from a new installation on a new server.
This issue doesn't sound hardware related at all, it looks more like a routing issue which *might* explain the poor speeds.

Perhaps you could link us to a file to test the speed and report back our findings here?
PM if you don't want to post details here? Would be happy to test the speed here in Hertfordshire.

MediocreFred
30-09-2013, 16:37
Yup, that was one of the first things I changed - set it to "1 Gbps Full Duplex". Still the same abysmal upload speeds.

SRT
30-09-2013, 16:18
See if changing you Speed & Duplex settings for your NIC from Auto to Full Duplex.

MediocreFred
30-09-2013, 15:59
Hello,

I recently (mid September) got an mSP at GRA. Installed Windows Server 2012 Standard on it. Tested network speeds using speedtest.net (chose the ookla Amsterdam server as recommended in a thread here). Got slightly higher than 200 Mbps. However, with real-life data transfers, I have been having terrible network speeds with this server. Here are some details of the tests I have been running:

Server1 - mSP at GRA (200 Mbps up/down) - problem server
Server2 - My Kimsufi server at BHS (100 Mbps up/down)
PC at home - 35 Mbps up/down

Bandwidth for all three have been verified using speedtest.net. All three are idle except for the tests being conducted.

Test file - 4 GB Windows Server iso.

Server1 and Server2 are running Windows Server 2012 Standard. I upgraded Server1 to Windows Server 2012 R2 Standard last week to see if the network issue would go away, but, it has remained the same.

PC is running Windows 7 x64.

I tested file copying using both, Windows native SMB and SFTP (using CopSSH). Here are the results:

On Server2, copy from Server1 to Server2 - using SMB - ~35 Mbps average (constant sawtooth fluctuation with ramp up to 60 Mbps followed by sharp drop to 20 Mbps)

On PC, copy from Server1 - using SMB - 10 Mbps (fluctuating speeds between 7 Mbps and 12 Mbps)

On PC, copy from Server1 - using SFTP - 4 Mbps (starting multiple transfers REDUCED speed of each to a TOTAL of 4 Mbps)

On PC, copy from Server2 - using SMB - 23 Mbps (rock solid speed)

On PC, copy from Server2 - using SFTP - 10 Mbps (starting multiple parallel transfers, total speed 33 Mbps - maxed out home PC bandwidth)

I have tried to research this issue online and compare the settings of this new mSP with the older BHS Kimsufi server, but, have not had any luck getting this fixed. All of the network settings look OK and there are no networking-related errors in the Windows logs.

I asked OVH for help (via tech support email) and the reply I got was to start the server in Rescue Mode and run some tests. The only tests I see are related to cpu, ram and hard disks - all of which appear to be fine.

How can I get anything close to the advertised 200 Mbps from this server? I'd really appreciate any suggestions to help troubleshoot and fix this.

Thanks,
MediocreFred.