Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > GNAX/Netdepot vCloud down
GNAX/Netdepot vCloud down
| Posted by 12Nitro, 02-25-2013, 09:20 PM |
7 hours ago Nagios alerted me for yet another downtime at our vCloud, affecting all VMs. After quickly logging in to Director and verifying that I could not access or restart the VMs, tracerouted the servers until Atalanta and then submitted a support ticket with Critical priority.
7 hours later still know very little about the problem - "still troubleshooting" was the clearest explanation offered. No ETA has been provided, still in the dark on this one. Nothing to tell the customers.
...And nothing to do but waiting. Anyone else experiencing the same? Any clues regarding the cause? |
| Posted by NetDepot - Terrence, 02-25-2013, 09:30 PM |
Quote:
|
Originally Posted by 12Nitro
7 hours ago Nagios alerted me for yet another downtime at our vCloud, affecting all VMs. After quickly logging in to Director and verifying that I could not access or restart the VMs, tracerouted the servers until Atalanta and then submitted a support ticket with Critical priority.
7 hours later still know very little about the problem - "still troubleshooting" was the clearest explanation offered. No ETA has been provided, still in the dark on this one. Nothing to tell the customers.
...And nothing to do but waiting. Anyone else experiencing the same? Any clues regarding the cause?
|
Hello,
We are aware of this issue its isolated and our engineering team is working to get this resolved. Can you please pm or email me your ticket ID? |
| Posted by 12Nitro, 02-25-2013, 09:33 PM |
Quote:
|
Originally Posted by NetDepot - Terrence
Hello,
We are aware of this issue its isolated and our engineering team is working to get this resolved. Can you please pm or email me your ticket ID?
|
Thank you Terrence. Already did that 3 hours ago. |
| Posted by 12Nitro, 02-26-2013, 05:16 AM |
At about hour #11 got a ticket update indicating that the cause had been identified. Unfortunately had fallen asleep a few minutes earlier (4am here) and wouldn't see it until waking up at around hour #14. Spent the following hour recovering each VM, one at a time.
So, 15 hours of downtime overall. That speaks for itself. |
| Posted by sfchost, 03-01-2013, 04:11 PM |
|
We've ended up cancelling our vCloud server. Even the ticked created cancelling the service was closed with no response. Wow. |
| Posted by 12Nitro, 03-01-2013, 05:23 PM |
Quote:
|
Originally Posted by sfchost
We've ended up cancelling our vCloud server.
|
I have only been with them for little more than a year. To be fair the service is okay for its cost, and up until now this was by far the worst downtime, both in length and in the way that it was handled. So normally I'm a satisfied customer, even if very aware of GNAX/Netdepot limitations.
Still, already lost count to the tickets I've opened. Probably over fifty, ranging from network issues, storage failure and billing inconsistencies. Usually the response time is actually pretty good, but advertising the service as "reliable" is... well, a bit of a stretch.
Quote:
|
Originally Posted by sfchost
Even the ticked created cancelling the service was closed with no response. Wow.
|
...yeah, generally few explanations are provided even when asked for them. |
| Posted by 12Nitro, 03-07-2013, 01:56 PM |
Oh wow. Same thing happening again.
Terrence, since you will no doubt be patrolling these threads, my ticket number is #WYO-824-10775. Thanks for your dedication and attention. Let's try NOT to beat the downtime record this time, shall we?
Best regards,
Ricardo Nuno Almeida |
| Posted by 12Nitro, 03-07-2013, 06:49 PM |
|
For the past 5 hours the server has been in a rather regular loop of 30-40 minutes working, followed by a 10-minute outage. Tests, maybe? |
| Posted by 12Nitro, 03-12-2013, 09:56 PM |
|
Same thing again, recurring problem. So far "only" 30 minutes. Ticket still unassigned. |
| Posted by gadams999, 03-12-2013, 10:26 PM |
Quote:
|
Originally Posted by 12Nitro
Same thing again, recurring problem. So far "only" 30 minutes. Ticket still unassigned.
|
Down for me too (and twice before over the past two weeks). Opened ticket at 2142 EDT, got the following just a few minutes ago:
We are experiencing some issues with the vcloud at this time. We are working as quickly as possible to resolve the issue and will provide updates as soon as they are available. |
| Posted by 12Nitro, 03-12-2013, 10:30 PM |
|
Called them. The engineer stated that they "were aware" of the problem. |
| Posted by 12Nitro, 03-12-2013, 11:50 PM |
|
2 hours and counting. |
| Posted by 12Nitro, 03-13-2013, 06:14 AM |
"### is UP again at 13-03-2013 10:00:20, after 8h 30m of downtime."
Woke up to see that all VMs were shut down instead of unavailable. No updates to the ticket. No feedback.
Simply turned them on again, fixed the usual "OMG could'nt connect to storage" problems, restarted the network services, and voila.
So, over 15 + 5 + 8.30 downtime over the last 2-3 weeks. Pathetic. I really cannot face our costumers anymore. |
| Posted by gadams999, 03-13-2013, 08:28 AM |
My two VM's (non-critical) came back up at just near 0700 EDT. So downtime from approx 2130 the night before for total of 9.5 hours. The VM's appeared to have restarted a couple times during the course of the outage, but total duration is pretty bad.
I'd just like an RFO and mitigation steps in the future. Has prompted me to look at distributing the load across multiple providers for resiliency.
Here's the latest ticket response:
Quote:
|
Both vms are back at login prompt. Please check let us know if you have access to these vms.
|
|
| Posted by 12Nitro, 03-13-2013, 08:37 AM |
Quote:
|
Originally Posted by gadams999
I'd just like an RFO and mitigation steps in the future.
|
Good luck with that.
Quote:
|
Originally Posted by gadams999
Has prompted me to look at distributing the load across multiple providers for resiliency.
|
Same here. Do you recommend any other particular provider?
My latest update to ticket was just after I had finished booting up all of my VMs. Replied asking for a RFO, nothing yet.
Quote:
|
sorry for the delays in replying. VM is up and responding to ping and ssh.
|
|
| Posted by NetDepot - Terrence, 03-13-2013, 09:29 AM |
Part of our cloud hardware infrastructure was unavailable this AM due to an HP firmware issue, which is now resolved. GNAX leadership and technical staff worked through the night to test the vendor supplied fix with positive results. We have added more hardware resources from one of our other top vendors for good measure, even though we strongly believe that our current fix is sustainable. GNAX cloud is stable and customers are back online with minimal system interaction.
If you are experiencing difficulties, please create a support request at https://portal.gnax.net and our experienced technical staff will be happy to assist you.
Thank you for your patience and input on this matter.
If you have further questions please write to wsouder AT gnax DOT net |
| Posted by 12Nitro, 03-13-2013, 11:10 AM |
Quote:
|
Originally Posted by NetDepot - Terrence
Part of our cloud hardware infrastructure was unavailable this AM due to an HP firmware issue, which is now resolved. GNAX leadership and technical staff worked through the night to test the vendor supplied fix with positive results. We have added more hardware resources from one of our other top vendors for good measure, even though we strongly believe that our current fix is sustainable. GNAX cloud is stable and customers are back online with minimal system interaction.
|
Thank you for the info. Good to know.
Quote:
|
Originally Posted by NetDepot - Terrence
If you are experiencing difficulties, please create a support request at https://portal.gnax.net and our experienced technical staff will be happy to assist you. |
Aha.
Well, everything seems to be working now. Let's see if we can finally check this one as closed. |
Add to Favourites 
Print this Article
Also Read
Shared Hosting
Shared Hosting
