RKNet Blog… thing

Hot:

As I write, the people I work with have realized that our websites are down, that Rackspace’s ticketing area is down, and that Rackspace.com is down. This problem for us began around around 4:15pm EST.

Our head tech is currently on the line with them trying to get through a projected 120 minute wait. Based on the notifications we have been getting from Rackspace recently, they have been doing generator and chiller maintenance. The last being a “DFW Generator Maintenance Notification” on the 26th around 7pm. And before that, “DFW IST Chiller Maintenance” on the 25th around 7pm also.

Note, this downtime applies to the Dallas datacenter, I do not know if it applies elsewhere. More updates as I hear from our head tech and their representatives.

4:38pm EST – At least one person claims they’re already back up: http://twitter.com/evanislam/statuses/2392985312 however based on the screaming, I’d call BS on that.
4:39pm EST – @rackspace breaks their twitter silence: http://twitter.com/Rackspace/status/2393005633 but no news

4:45pm EST – We got through to one of our personal techs and all he has been allowed to say is that their datacenter is offline, and there is no ETA on getting it back up.

And I caught someone who said that the rackspace portal area is back up https://my.rackspace.com/construction/index.html check it out.

Twitter is freaking out about this faster than I can even skim for anything that could approach a nugget of truth. EVERYONE, stop whining long enough for people with actual updates to squeeze through! kthxbai.

4:53pm EST – http://twitter.com/btabke/statuses/2393217225 Webmaster World is back up and thanks @rackspace for their quick action.

4:56pm EST – http://twitter.com/chadpotts/statuses/2393243977 sounds like the problems they had last time.

chadpotts: RT @spmiller talked to a #rackspace rep…power outage in Dallas, battery backup worked fine, but when power came on, switch back didnt work

4:57pm EST – My company’s websites are back up!

5:00pm EST – Someone claims they heard from Rackspace that they were having telecom issues. http://twitter.com/nmiracle/statuses/2393318075

LOTS of people are reporting they are back up. Several are still reporting issues and down time though. News sites are starting to blow up, I’ve seen links to techcrunch, seen Scoble link to his account of events, seen a few others floating around. (What am I, chopped liver? :P)

5:05pm EST – http://twitter.com/Rackspace/status/2393405011 @rackspace provides an update on the situation.

5:10pm EST – and now the “IT MUST BE UP FOR ALL ETERNITY” complaints truly begin http://twitter.com/dingman/statuses/2393466566
5:10pm EST – http://twitter.com/Rackspace/status/2393461248 Rackspace is saying that their phones are working, just very very busy.

5:07pm EST – http://twitter.com/ChrisDrake/status/2393421572 opportunists begin to come out of the woodwork…
5:14pm EST – http://twitter.com/ahyes/statuses/2393496450 and appropriately get called out for it.

5:20pm EST – http://twitter.com/XavierLanier/statuses/2393587390 someone actually points out that having an immediate internet outlet for reporting on status issues, and not using them to update on status issues is, in fact, silly

http://twitter.com/benhschwartz/statuses/2393585920 some people are taking it with more aplomb than others

Also, inexplicably, apparently at about 4:26, Justin Timberlake apparently decided this whole @rackspace thing was worth tweeting about http://twitter.com/jtimberlake/status/2392959228 – That’s weird, right?

5:28pm EST – Another update from Rackspace about those servers that needed extra help getting back up http://twitter.com/Rackspace/status/2393705910

And at 5:44pm EST an official Rackspace email/ticket:

Subject: DFW Power Interruption
Status: New
Account #: [REDACTED]
Date: 06/29/2009 4:44pm CDT
Comment:
————————————————————————
Hello,

We have experienced an interruption in power to a portion of our Dallas-Fort Worth data center facility. Power has been restored and our DC engineers are working on devices that need to be manually brought back online as quickly as possible. Further updates will be made as soon as new information is made available. Please monitor our MyRackspace customer portal as this is the quickest way we can get updates out to you.

Sincerely,

Rackspace
————————————————————————

Potentially Related posts

• No related posts.