Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > OVH North America (BHS) has fiber cut
OVH North America (BHS) has fiber cut
| Posted by aww, 10-03-2013, 11:26 AM |
http://status.ovh.net/?do=details&id=5527
Quote:
2 fiber connections have been lost between Montreal and Beauharnois.
Traffic has been redirected through an emergency network route.
The cables were severed in the western part of Montreal, between Montreal and BHS. The teams are actually looking for the exact location where the severing of the pairs of fibers took place, and what caused it.
The emergency route has relayed the main route, but it's saturated, so there may be packet loss.
The cables were severed by the "Monseigneur Langlois" bridge (same location as last time), fibers ripped under the bridge. Currently intervening.
|
We are seeing strange behavior with internal and external connections on our servers. |
| Posted by HostingOwner-SS, 10-03-2013, 12:02 PM |
|
I hope so they will resolve the issue as soon as possible. |
| Posted by thedediguy, 10-03-2013, 12:17 PM |
That is seriously not good!
Hope they get it fixed fast |
| Posted by jcastill, 10-03-2013, 01:29 PM |
|
I know it's not the first time it happens, I wonder if they will make a redundant path to avoid a third time. |
| Posted by Serverbros, 10-03-2013, 01:33 PM |
Yep - exact same location for a second time!!
Not happy but hey ho, that's why we are spreading our servers around co-located now to minimise any single network outage causing issues.
Didn't help my office power was off for maintenence, queue me replying to tickets via the phone app with 1% battery left, wondering why it wasn't charging even though the cable was in....DOH! |
| Posted by Spluut, 10-03-2013, 04:13 PM |
Hope this gets fixed soon.  |
| Posted by gigist, 10-03-2013, 04:47 PM |
Quote:
Comment by OVH - Thursday, 03 October 2013, 22:37PM
Fibers have been spliced. Connectivity is back on main links.
|
Still getting tons of packetloss. Servers can't push more than 2Mbps. Ouch. |
| Posted by Serverbros, 10-03-2013, 05:05 PM |
Quote:
|
Originally Posted by gigist
Still getting tons of packetloss. Servers can't push more than 2Mbps. Ouch.
|
We were the same up until about 20 minutes ago, completely fine for us and all our customers now - I'd maybe check with OVH to see if anything else is up.....hope you get it sorted!
Jordan |
| Posted by Spluut, 10-03-2013, 05:10 PM |
Quote:
|
Originally Posted by gigist
Still getting tons of packetloss. Servers can't push more than 2Mbps. Ouch.
|
Yeah remoting in my servers is hell right now still.. |
| Posted by Spluut, 10-03-2013, 05:13 PM |
|
Things appear to be normal again, for now. |
| Posted by gigist, 10-03-2013, 05:15 PM |
Quote:
|
Originally Posted by Serverbros
We were the same up until about 20 minutes ago, completely fine for us and all our customers now - I'd maybe check with OVH to see if anything else is up.....hope you get it sorted!
Jordan
|
Thanks Jordan. It seems the second I open a support ticket, all is fine. Connectivity is back to normal. |
| Posted by Encrypted, 10-03-2013, 07:35 PM |
|
Seems like a pretty sketchy spot to lay fiber. Why didn't/couldn't they utilize those utility towers? |
| Posted by Spluut, 10-03-2013, 07:37 PM |
Quote:
|
Originally Posted by Encrypted
Seems like a pretty sketchy spot to lay fiber. Why didn't/couldn't they utilize those utility towers?
|
Trolls are great at guarding!  |
| Posted by UplinkVPS, 10-03-2013, 08:56 PM |
Quote:
|
Originally Posted by aww
For the curious: img
|
That is going to be big job to repair. I feel bad for the repair crew. |
| Posted by StevenLeonK, 10-04-2013, 02:28 AM |
|
I am still experiencing terrible packet loss, I called them and made a ticket about the issue and I am being told that there will be nobody able to respond to this issue until morning... I asked if it would be possible to call someone or alert someone and I was pretty much told no... I also asked if there was anyone else I could speak to in regards to this matter, and I was told no one else was there... So I asked if she was the only one working support and she said yes... I am annoyed to say the least... My clients are dropping connection from our servers constantly, and they seem unwilling to do anything at all... I've provided trace routes and so on from and to the server. They all show loss within the OVH network. |
| Posted by gigist, 10-04-2013, 03:23 AM |
Quote:
|
Originally Posted by StevenLeonK
I am still experiencing terrible packet loss, I called them and made a ticket about the issue and I am being told that there will be nobody able to respond to this issue until morning... I asked if it would be possible to call someone or alert someone and I was pretty much told no... I also asked if there was anyone else I could speak to in regards to this matter, and I was told no one else was there... So I asked if she was the only one working support and she said yes... I am annoyed to say the least... My clients are dropping connection from our servers constantly, and they seem unwilling to do anything at all... I've provided trace routes and so on from and to the server. They all show loss within the OVH network.
|
Bad deal all around. When dealing with a budget provider one must consider these situations. I know how you feel... and truthfully circumstances like this allow for a greater sense of redundancy.
I highly doubt she's the only support person available, she most likely misunderstood you. OVH had to employ a certain amount of people when building out BHS (it was in the terms).
Good luck. |
| Posted by critihost, 10-04-2013, 04:03 AM |
|
I'm not following the bashing. I've had much more expensive providers with network issues. It happens. Those resulted in downtime of a couple hours and routing issues for a day or more. Pingdom showed no downtime for me on my BHS server in this case and it slowed for 12 hours with ping hurt more. Overall I'm happy with the way it was handled. |
| Posted by aww, 10-04-2013, 05:00 AM |
Note if you are measuring OVH intranet packet loss via UDP/ICMP pings, their routers give those lowest priority so you won't get real results.
Try using MTR 0.85 with TCP mode instead and put a 5 second interval on it. |
| Posted by StevenLeonK, 10-04-2013, 05:26 AM |
Quote:
|
Originally Posted by aww
Note if you are measuring OVH intranet packet loss via UDP/ICMP pings, their routers give those lowest priority so you won't get real results.
Try using MTR 0.85 with TCP mode instead and put a 5 second interval on it.
|
It doesn't matter, when my clients are losing connection to things that are hosted on the server there is clearly an issue... I would have never done trace routes if that wasn't taking place. |
| Posted by aww, 10-04-2013, 06:03 AM |
Are you on any of these racks? They are replacing switches: http://status.ovh.net/?do=details&id=5360
I once had bad loss problems to OVH but it turned out after much investigation it was my ISP using bad routes.
iweb has a datacenter "down the road" from OVH near their BHS in Montreal - when I have problems to OVH, I test to iweb to compare the routes/latency and other network issues http://speedtest.iweb.com/ |
| Posted by StevenLeonK, 10-04-2013, 06:05 AM |
|
How do I determine If I am on one of these switches? It also says that was last edited on 2013-09-09 which is almost a month ago... |
| Posted by StevenLeonK, 10-04-2013, 06:16 AM |
|
It seems one of my IP's does match with the first 3 numbers of one of the IP's they list but the all my servers were effected by this packet loss... And there still experiencing it as we speak. |
| Posted by aww, 10-04-2013, 06:34 AM |
Well they will probably attempt to explain this to you in the morning but you cannot actually determine packet loss with a ping/traceroute to OVH, it simply will not work with UDP/ICMP packets.
I'm not saying your problem isn't real and I'm not defending OVH but if you are using ping/traceroute to say there is packet loss, it's not actually going to help your argument with them.
They have three level of tech support and that's what you are running into tonight - they will bump you up when the next level gets in.
If you have MTR on your server, try doing this
mtr --interval=10 -T 1.2.3.4
where 1.2.3.4 is the IP of the pc you are working on
Or if you have linux at home use MTR from home to the server.
the -T is the critical part that uses TCP packets
Some linux installs come with mtr 0.75 and I am not sure if it has TCP support (looking at it's help, I think not) but MTR 0.85 does - so you'd have to know how to build it (easy if you know how)
added: Oh here is a good explanation of identifying "real" vs "fake" packet loss
https://library.linode.com/linux-too...ng-packet-loss |
| Posted by atchoooo, 10-04-2013, 07:07 PM |
Quote:
|
iweb has a datacenter "down the road" from OVH near their BHS in Montreal - when I have problems to OVH, I test to iweb to compare the routes/latency and other network issues http://speedtest.iweb.com/
|
OVH's datacenter is at least 40km away (from our closest datacenter). Not sure if routes to get to both networks can be similar, probably depends on your location. |
| Posted by aww, 10-05-2013, 01:43 AM |
Yeah the route is similar the further away you are but iweb has different peering agreements perhaps.
The transfer speed between ovh <-> iweb is phenomenal, iweb should use that for marketing to unhappy ovh users ;-) |
| Posted by StevenLeonK, 10-05-2013, 01:56 AM |
Quote:
|
Originally Posted by aww
Yeah the route is similar the further away you are but iweb has different peering agreements perhaps.
The transfer speed between ovh <-> iweb is phenomenal, iweb should use that for marketing to unhappy ovh users ;-)
|
Well, I'm an unhappy customer... |
| Posted by Mastermind Networks, 10-05-2013, 08:03 AM |
The issue was resolved yesterday morning and yesterday afternoon my servers were working at full speed. I think OVH did a decent job considering the situation.
Fortunately in my case we had some serious back-up planned and were able to shift most of the load. Customers however had mixed opinions  |
| Posted by Serverbros, 10-08-2013, 03:52 PM |
|
Indeed, 50% packet loss here too! |
| Posted by Serverbros, 10-08-2013, 04:06 PM |
|
Back to normal again, for now. |
| Posted by Serverbros, 10-08-2013, 04:13 PM |
Seems like its a DDOS : Comment by OVH - Tuesday, 08 October 2013, 22:10PM
The network team has implemented some measures to mitigate a massive attack. |
| Posted by MAXZEUS, 10-08-2013, 05:29 PM |
Quote:
|
Originally Posted by Serverbros
Seems like its a DDOS : Comment by OVH - Tuesday, 08 October 2013, 22:10PM
The network team has implemented some measures to mitigate a massive attack.
|
No wonder, they said they can handle all DDoS Attacks of the world |
Add to Favourites 
Print this Article
Also Read
Shared Hosting
Shared Hosting
