What the hell happened?

[Big Media]

Did the server fill to capacity?

[genlock]

It's quite the story... a Data Center team's nightmare! There's the blog carrying up-to-the-minute information at: http://forums.theplanet.com/index.php?showtopic=90185

Here's a snippet of what happened:

From Doug Erwin:

This evening (Saturday, 31-May-2008) at 4:55pm CDT in our H1 data center, electrical gear shorted, creating an explosion and fire that knocked down three walls surrounding our electrical equipment room. Thankfully, no one was injured. In addition, no customer servers were damaged or lost.

We have just been allowed into the building to physically inspect the damage. Early indications are that the short was in a high-volume wire conduit. We were not allowed to activate our backup generator plan based on instructions from the fire department.

This is a significant outage, impacting approximately 9,000 servers and 7,500 customers. All members of our support team are in, and all vendors who supply us with data center equipment are on site. Our initial assessment, although early, points to being able to have some service restored by mid-afternoon on Sunday. Rest assured we are working around the clock.

We are in the process of communicating with all affected customers. we are planning to post updates every hour via our forum and in our customer portal. Our interactive voice response system is updating customers as well.

There is no impact in any of our other five data centers.

I am sorry that this accident has occurred and I apologize for the impact.

--------------------

[Lester]

Yeah, it was a doozy. Luckily I only have two paying sites that were beating me over the head for answers.

Sorry for the inconvenience... but there wasn't a whole lot I could do once the towers went dark.

[genlock]

I blame terrorism.

[Rock]

Yup. Definitely Osama.

[genlock]

Or some fat guy with a fish.

[Hoosier Daddy]

Yup. Definitely Osama.

You must be a Hillary man.

[Rock]

Eep!

[The People's DJ]

Yeah, it was a doozy. Luckily I only have two paying sites that were beating me over the head for answers.

Sorry for the inconvenience... but there wasn't a whole lot I could do once the towers went dark.

And here I was thinking that it went down cause I posted that Angry Video Game Nerd video in the flash thread.

[genlock]

The server site firewalls (real fire, real walls) did their job.
Apparently none of the servers or the data contained theirin was lost.
The Nunley philosophy is safe.

[AmpedNow]

Funny, because I was browsing the site when it went down.

Everything worked fine, then it started timing out. And it never came back...

[Dr. Whiplash]

Terrorism? Osama?

Or some fat guy with a fish.

Question: It's obvious we have all been permanently emotionally scarred by the photo in question. Shouldn't we contact Perfader or Jan or Danny or somebody? My brain hurts and somebody needs to PAY!

[David Paleg]

Not terrorism. Not Osama, Obama, or Oprah. The truth should come out. The hamster that powered the server by running in the wheel just got tired and collapsed.