FDCservers - scheduled electrical maintenance - Knowledgebase

Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > FDCservers - scheduled electrical maintenance

FDCservers - scheduled electrical maintenance

Posted by bummer6666, 02-09-2008, 02:42 AM
There has been electrical maintenance scheduled for 2/9/08. The maintenance will start around 8am CST. All customers have been notified via email and announcement has been made on our forums on Wednesday.
Posted by matador, 02-09-2008, 03:52 AM
They sent email on 6th, but I wasnt paying attention to a certain mailbox. Re: http://www.chicagotribune.com/news/l...,2323842.story Commonwealth Edison has inspected all transformers in the building. After completing the inspection several transformers will need repairs. Unfortunately customers will be affected and will experience approximately 3 to 5 hours of down time to repair a high voltage elbow on the transformer that supplies power to our facility. Work is scheduled this Saturday February 9th starting around 8:00 am CST. Good luck getting through this one folks.
Posted by dfh_ie, 02-09-2008, 10:00 AM
after shutting down all the servers hope this doesn't take long, best of luck
Posted by bummer6666, 02-09-2008, 10:27 AM
The maintenance has been delayed by 30-60 minutes
Posted by Arsenico, 02-09-2008, 11:24 AM
So, what's going on now? how much time is it gonna take?
Posted by bummer6666, 02-09-2008, 11:25 AM
the power has been shut down at 9;00 am CST I`ll post further updates as I get them
Posted by Arsenico, 02-09-2008, 11:30 AM
how much time is it gonna take?
Posted by RyanD, 02-09-2008, 11:31 AM
according to the above post 3-5 hours
Posted by nobody24, 02-09-2008, 12:08 PM
bummer6666 any news? pls inform what is going on
Posted by quazi, 02-09-2008, 12:08 PM
Thank you fdc for not emailing me about this!!!!! I really appreciate not receiving an email about this... *#$&*#&
Posted by zenex5ive, 02-09-2008, 12:13 PM
Power maintenance will last around 5 hours. They did email all the clients.
Posted by colohotelm, 02-09-2008, 12:20 PM
yup they said they emailed everyone.
Posted by Krazy, 02-09-2008, 12:27 PM
i got the email from fdc , but what i don;t understand is why they don have second utility conn, as generator is more money
Posted by slypete, 02-09-2008, 12:28 PM
I certainly got an email.
Posted by slypete, 02-09-2008, 12:30 PM
They're located in the CBOT. When that building has no power it affects the economy, so it will never be down for too long. That's what they bank on I believe.
Posted by Arsenico, 02-09-2008, 12:41 PM
Investing some money to do not have downtime because of the power, will be great.
Posted by slypete, 02-09-2008, 12:55 PM
I doubt it will happen. That's how they keep their prices down by banking on their location.
Posted by Arsenico, 02-09-2008, 01:00 PM
price are just a little down than theplanet (Watching regulars server..). anyways FDC gives a good service, but i love to do not have downtimes... i hate downtimes... i spent a lot of money past month for dedicated server issues, and now this... i'll be broke ..
Posted by AMJones, 02-09-2008, 01:05 PM
Planned maintenance for which one has advance notice shouldn't cost you money, given a good business model. If you are in FDC, and promising 100% uptime to your customers... you may want to reconsider one or the other. Additionally, if your uptime guarantee doesn't allow for scheduled maintenance - you really need to look at that and make adjustments. 100% guaranteed uptime is simply not possible. A datacenter with plenty of cash (IE - that charges alot) can minimize the likelyhood of downtime, but along a long enough timeline EVERYONE will have downtime eventually.
Posted by slypete, 02-09-2008, 01:06 PM
Prices are below the Planet and FDC offers unmetered BW.
Posted by Arsenico, 02-09-2008, 01:09 PM
yes it can make me lose customers... server issues caused me lost of money .. i bought 2 servers more, one on hivelocity and one in theplanet, anyways, i still haven't a chance to make a VPN tunel between servers to do not have any kind of downtime. because of server issues, i lost 4 customers, big ones, and they was paying A LOT .. more than 50 U$ e/o ... (Just support, service and tech support from my side), that's why i can't lose more customers or cost will be high and winnings will be negatives.
Posted by dfh_ie, 02-09-2008, 01:41 PM
^^^ look they gave plenty of notice, anyways if you keep all your server eggs in one basket... you only have yourself to blame
Posted by WebFoundation, 02-09-2008, 01:42 PM
I know it /sucks/ per say to loose customers but things happen. They gave you a heads up notice more then a week in advance.
Posted by Arsenico, 02-09-2008, 01:44 PM
yeap but i didn't got the email .. ... =( anyways are 3 hours by now.. still have 2 more O_O, more than that will be crazy.
Posted by Hassan, 02-09-2008, 01:54 PM
Its very bad to have long downtime on weekend...2 Hrs more :|
Posted by quazi, 02-09-2008, 01:56 PM
Well I did not get an email. Period. And i have servers with fdc and zenix... Just saying wish the email would of gone through. Then i could of not been wasting my money on purchased traffic right now.
Posted by T-4-K, 02-09-2008, 02:03 PM
I didn't get an email either. I checked the other box to make sure. Infact I haven't gotten a email from fdc since i've activated the server.
Posted by nitroburn, 02-09-2008, 02:07 PM
crap, I didn't even realize it was Saturday, that makes this suck even more.
Posted by reasonpolice, 02-09-2008, 02:13 PM
Whats silly is that that they have no off-site emergency/downtime page (where this thread should be) eg, I got the email and when i went to check details, theres only a link to a thread I cant reach
Posted by WebFoundation, 02-09-2008, 02:21 PM
If you did not get the e-mail you need to login to your CRM (new thing they rolled out year ago or so) and update your profile. Yes Sundays sucks, and yes FDC should host their site with another provider. I completely agree.
Posted by T-4-K, 02-09-2008, 02:34 PM
I updated my CRM info awhile ago. When I forgot my password.... Well that didn't work out to well. I told them about it. It's neither in the spam filter or shows any sign of it connecting to the mail server it's weird.
Posted by csgo, 02-09-2008, 03:04 PM
Well we're beyond the 5 hour window and everything at FDC appears to be down. Anyone have an update?
Posted by WebFoundation, 02-09-2008, 03:09 PM
I would like to know this too.
Posted by Arsenico, 02-09-2008, 03:11 PM
me too ... we need servers up ! but i think there just need to be 1 hour more.
Posted by zenex5ive, 02-09-2008, 03:17 PM
Maintenance was delayed by 1 hour. Hopefully everything will be back to normal within 2 hours. It will take some time for them to manually boot up all the servers.
Posted by Arsenico, 02-09-2008, 03:24 PM
hopefully? that means ... more than 2 hours more.. 7 hours total?
Posted by nobody24, 02-09-2008, 03:29 PM
>7 hours total? what about 9h? 12? 24? I guess we need to get more info from Bummer6666
Posted by bummer6666, 02-09-2008, 03:35 PM
we just got word that power should be back in about 30 minutes from now ( 2:00PM CST). I`ll make another post once the network goes live
Posted by WebFoundation, 02-09-2008, 03:36 PM
I bet you are having a more stressful Saturday then me, makes me feel better.
Posted by Arsenico, 02-09-2008, 03:47 PM
that's a good new!
Posted by scooby2, 02-09-2008, 04:03 PM
It does not cost all that much to rent generators for a weekend. At my last job (large ISP/webhost) where we ran our own datacenter, the power company was doing a three day weekend maintenance to build out for a new housing complex and a couple retail/commercial complexes. We brought in a couple rental generators on trucks to backup the Cat 500kw that we had on site just in case it had issues. We ended up not needing them but keeping the customers up 24/7/365 was more important to us than the cost of the rentals.
Posted by Arsenico, 02-09-2008, 04:11 PM
Server up and running nope down again...
Posted by WebFoundation, 02-09-2008, 04:12 PM
That's great and all, but completely irrelevant.
Posted by slypete, 02-09-2008, 04:18 PM
How is that irrelevant? That's a great suggestion for this situation. In fact, I'm surprised the company doing the maintenance isn't providing something like that for the CBOT.
Posted by nitroburn, 02-09-2008, 04:18 PM
Server back online, yay!
Posted by csgo, 02-09-2008, 04:20 PM
Mine are up too.
Posted by bummer6666, 02-09-2008, 04:21 PM
power was restored at 2:00 CST it will take another 30-60 minutes to get everything back to fully operational state. If your server is not back online by 3:00PM CST, please be sure to open up a trouble ticket through our helpdesk
Posted by slypete, 02-09-2008, 04:27 PM
Yes, my DRAC card is now accessible too.
Posted by rhsadmin, 02-09-2008, 04:29 PM
showing online currently
Posted by csgo, 02-09-2008, 04:34 PM
I'm back down again.
Posted by nitroburn, 02-09-2008, 04:38 PM
back down again...
Posted by csgo, 02-09-2008, 04:39 PM
Up again. Hope this is OVER!
Posted by Net2Hosting, 02-09-2008, 04:41 PM
In my case 1 out of 3 online...
Posted by WebHostingNeeds, 02-09-2008, 04:42 PM
Got one online, while posting this next come online
Posted by dfh_ie, 02-09-2008, 04:47 PM
yea they are coming back tho networks is sluggish 95% packet loss (@ 2nd hop @ 66.90.127.141 according to mtr) from the servers i managed to login to hopefully this is temporary
Posted by CNSERVERS, 02-09-2008, 04:54 PM
one of my servers didn't get back online. unable to access crm.fdcservers.net at the moment.
Posted by csgo, 02-09-2008, 04:57 PM
Yes, CRM doesn't appear to be up. Their network is very slow and high packet loss since coming back up. I suspect they have some problems that are not power related.
Posted by bummer6666, 02-09-2008, 05:03 PM
yes our CRM is currently offline. It should be online in 15-20 minutes. Network is also being worked on. It should be back to normal in about 30 min
Posted by Dougy, 02-09-2008, 05:04 PM
I spy a few new gray hairs, Petr. Ping statistics for 66.90.66.135: Packets: Sent = 20, Received = 15, Lost = 5 (25% loss), Approximate round trip times in milli-seconds: Minimum = 32ms, Maximum = 121ms, Average = 43ms It's getting better. Last edited by Dougy; 02-09-2008 at 05:08 PM.
Posted by playboy2012, 02-09-2008, 05:06 PM
Hey bummer, I pmed you in regarding an upgrade and still haven't received a reply from you. Can you please take a look at your pm?
Posted by bummer6666, 02-09-2008, 05:09 PM
not really...today`s maintenance is nothing compared to FDC`s early days issues we`d had years ago. When Cogent was our only provider 6 years ago and they pulled plug on us which caused 4 day outage...that alone could cause some people to suffer ED
Posted by bummer6666, 02-09-2008, 05:10 PM
just got word that the network should be back to normal %100 we continue to monitor everything to mitigate any unforseen issues
Posted by tonyric, 02-09-2008, 05:13 PM
Still down for 1 account, 3 domains. still cannot access fdcservers.net
Posted by tonyric, 02-09-2008, 05:15 PM
fdcservers.net is up, but not my domains.
Posted by Net2Hosting, 02-09-2008, 05:16 PM
fdcservers.net is down from my location and one of my server still offline and the other one is telling that my cpanel license is not active
Posted by nobody24, 02-09-2008, 05:20 PM
bummer6666 when can we expect online crm.fdcservers.net ? I have two boxes down...
Posted by chrono-it, 02-09-2008, 05:23 PM
crm.fdcservers.net is now online
Posted by tonyric, 02-09-2008, 05:25 PM
Cool, but my server is still offline. you are not 100% up.
Posted by nobody24, 02-09-2008, 05:28 PM
I can login but can`t create ticket
Posted by playboy2012, 02-09-2008, 05:28 PM
also my cpanel info is expired apparently when i log into it.
Posted by chrono-it, 02-09-2008, 05:31 PM
then open a ticket. since everyone is opening a ticket it's getting a little bogged down usage wise so please be patient.
Posted by Dougy, 02-09-2008, 05:40 PM
Login as root, run /usr/local/cpanel/cpkeyclt and see if it fixes it.
Posted by nobody24, 02-09-2008, 05:50 PM
Just lost one customer if in 30 min another box(50mbit dedicated bw) not to be online I`ll lose one more customer need HELP!!!
Posted by dfh_ie, 02-09-2008, 05:53 PM
^ stop crying, some of us have ALOT more servers and spending ALOT more money with them and are keeping the cool
Posted by Arsenico, 02-09-2008, 05:54 PM
that's bad. contact fdc now!
Posted by nobody24, 02-09-2008, 06:00 PM
I have several boxes still in down but I need UP one ASAP I`m ready to wait with other boxes
Posted by chrono-it, 02-09-2008, 06:19 PM
if you open a ticket please remember to provide your password as we have booted every server in the datacenter and more than likely it is a fsck issue.
Posted by rkm11, 02-09-2008, 06:19 PM
Whats the average ticket wait... I put mine in about 30 minutes ago. Server has been down for 7 hours now, not responding to pings.
Posted by slypete, 02-09-2008, 06:31 PM
My server came up flawlessly @ ~ 2:30CST and has remained up since. This was a truly unfortunate event; however, I couldn't be happier by the way it was managed by FDC. Last edited by slypete; 02-09-2008 at 06:32 PM. Reason: Grammar mistake
Posted by chrono-it, 02-09-2008, 06:32 PM
standard wait time is about 45 minutes-60 minutes. Almost all servers still offline need a fsck or something similar so it takes longer. As of 1 hour ago we had finished rebooting every server by hand.
Posted by tonyric, 02-09-2008, 06:33 PM
The guy with the billing on mine is the one with access to the ticket system, so I cannot get my ticket in and he out of town for the weekend. SIGH Mine are in the same state as you
Posted by rkm11, 02-09-2008, 06:34 PM
I'm forced to disagree, as my server is still down and no personal response from FDC to my ticket. Last edited by rkm11; 02-09-2008 at 06:36 PM. Reason: New Post
Posted by nobody24, 02-09-2008, 06:40 PM
my box still down answer from your techs "Running fsck now" so box fscking one hour already am I right?
Posted by slypete, 02-09-2008, 06:53 PM
Sounds like you didn't prepare too well for the maintenance. I brought my server down before the lights went out. It's boggling the amount of people who are willing to let the power be cut while their servers' are live. I believe most clients still suffering downtime at the moment is a result of thier own careless management.
Posted by tonyric, 02-09-2008, 06:56 PM
Or due to a lack of notice, some of us has notifications heading to the person that handles the billing. Yes, our problem that will be rectified, BUT systems should NEVER just be powered off, there should be poweroff scripts on all servers that the facility has the ability to execute to shut down systems.
Posted by slypete, 02-09-2008, 07:06 PM
I know you're frustrated, but is this not clear to you? Careless management = Notification going to billing personel
Posted by rkm11, 02-09-2008, 07:08 PM
If I would have shut the server down before 800CST, then I would have lost 1 hour and 15 minutes of uptime. Then I would have more to complain about. I thought FDC could handle it, and I was hoping that the power would be cut and when the power was restored the BIOS on the MB would power the server back on. Sounds like it took them a hour to manually power them all on. Thats a hour less to get them working. Last edited by rkm11; 02-09-2008 at 07:19 PM. Reason: grammar
Posted by Net2Hosting, 02-09-2008, 07:58 PM
It seems they are having a lot of problems trying to get everyone on line. It has been more than 2 hours since I open my ticket and no answer. I turned off all my servers before they power off and only one came back up.
Posted by rkm11, 02-09-2008, 08:24 PM
Yeah... I've just broke the 9 hour mark with no response to a ticket that is over 3 hours old.
Posted by WebFoundation, 02-09-2008, 09:17 PM
9 hours of downtime and thats almost 50% because FDC can't reply to two of our tickets.
Posted by csgo, 02-09-2008, 09:27 PM
My servers are fine, but it does disturb me to read some of the comments. I realize that power failures can cause problems... I think we all know that and that's not the part that bothers me. I've found that for the most part customers are reasonable if you communicate with them. Yes, I notified my customers of the outage like FDC did for me, but of course some of them still called, and some were upset. When I keep them informed they usually understand and accept the situation. Communication is the key. If my servers were still down and nobody replied to my ticket I would be livid. I understand your frustration and wish there was a way for me to help. I don't even have any suggestions... sorry. I know the FDC guys are probably overloaded right now, but I hope that they'll take a couple of minutes to answer the open tickets. If nothing else just acknowledge the problem. Good luck!
Posted by Justin, 02-09-2008, 10:58 PM
As the maintenance is over and their support avenues are available at this point if you are still having problems your best bet would be contacting FDC through direct means.

Add to Favourites Print this Article

Knowledgebase

FDCservers - scheduled electrical maintenance

Our Services

Client Menu

Legal