topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Thursday March 28, 2024, 4:14 pm
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: DC Server Down for 3 hours on June 4, 2011 due to datacenter power outage  (Read 17014 times)

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
The Seattle Softlayer datacenter where the DC server is located had an unexpected power loss last night (sat 3am-6am) that took the DC server (and DC member server) offline.

Softlayer experienced a power event which caused the Seattle datacenter to go offline. The UPS's failed to pick up the load and engineers are currently investigating the cause. At this time power has been restored, and we are working on bringing all devices back online into working order.
Best Regards, Eric C., SoftLayer Support

The main DC server/forum is now back online (thanks to DC member Gothic!) and all seems well, and no forum data has been lost.



Note 1: The DC member server is still offline for a bit while we attend to that.
Note 2: We' may take the server briefly offline sometime next week to test and improve our daily backup scripts.

« Last Edit: June 04, 2011, 03:15 PM by mouser »

jgpaiva

  • Global Moderator
  • Joined in 2006
  • *****
  • Posts: 4,727
    • View Profile
    • Donate to Member
I'm now getting this:

An Error Has Occurred!
 The server's resources are temporarily under too high a demand to find all the topics you have not read.

Back
DonationCoder.com | About UsDonationCoder.com Forum | Powered by SMF

[ Page time: 14.093s | Server load: 36.66 ]



[edit] not anymore, apparently [/edit]

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Yep, give the server a bit of time to finish doing background stuff..

cranioscopical

  • Friend of the Site
  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 4,776
    • View Profile
    • Donate to Member
The main DC server/forum is now back online

Thank you Gothic!

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
That is bizarre. Don't they have redundant power? Was it a blackout? Brownout?

I host with Softlayer in Texas. They've been excellent.
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

Gothi[c]

  • DC Server Admin
  • Charter Honorary Member
  • Joined in 2006
  • ***
  • Posts: 873
    • View Profile
    • linkerror
    • Donate to Member
That is bizarre. Don't they have redundant power? Was it a blackout? Brownout?

I host with Softlayer in Texas. They've been excellent.

The entire datacenter went dark completely.

Normally they have UPS'es and battery banks to keep things runing until the generators kick in.
The UPS'es "couldn't handle the load" according to them, and everything went down.

It wasn't a brownout as it took quite a bit for them to restore power.

Ath

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 3,612
    • View Profile
    • Donate to Member
So they are now working on a new disaster recovery plan, I presume :tellme:

kyrathaba

  • N.A.N.Y. Organizer
  • Honorary Member
  • Joined in 2006
  • **
  • Posts: 3,200
    • View Profile
    • Donate to Member
Much appreciation to Gothic, who works tirelessly behind the scenes, keeping DC servers going!  :Thmbsup:

mahesh2k

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 1,426
    • View Profile
    • Donate to Member
Is it just me or some good hosts like host gator, soft layer, media temple are facing problems these days ?

worstje

  • Honorary Member
  • Joined in 2009
  • **
  • Posts: 588
  • The Gent with the White Hat
    • View Profile
    • Donate to Member
I don't know about hosts, but I surely hear a lot of complaints about the place DC is hosted. Professionalism is hard to find, I think, since if they had set it up properly, then they would have had enough juice to last till generators kicked in - or at the very least have had enough juice to power the most important pieces of electronics. Everything going down just means that they messed up big time. :(

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
I've been with The Planet for years, and with SoftLayer and The Planet since they merged.

The number of real problems I have had total exactly zero.

However, I am in the Texas data centers. Maybe that makes a difference.
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

worstje

  • Honorary Member
  • Joined in 2009
  • **
  • Posts: 588
  • The Gent with the White Hat
    • View Profile
    • Donate to Member
With data centers, it isn't about how many problems there are over a period. It is about the effect a single problem can have. Good management keeps up on all their expireable UPSes, keeps matters segmented and so forth. That way, if a problem happens, it only hits some customers, or throughput is lowered, and not the entire location. I stick with my opinion that they should have done more disaster tests and drills. :)

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
I'm seeing tons of complaints from past SoftLayer fans on their forum and on the webhostingtalk.com forum; seems like a general consensus is forming that since the merger with ThePlanet hosting company, SoftLayer is definitely having some issues..

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
I'm seeing tons of complaints from past SoftLayer fans on their forum and on the webhostingtalk.com forum; seems like a general consensus is forming that since the merger with ThePlanet hosting company, SoftLayer is definitely having some issues..

Odd... They were both good companies. Is this just a growing pain do you think?
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,896
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Is this just a growing pain do you think?

Definitely possible, and it would be understandable.  But it does mean that it might be prudent for people thinking about becoming new SoftLayer customers to wait a bit until things settle down.