Post I.T. - Washington Post Technology Blog Frank Ahrens Sara Goo Sam Diaz Mike Musgrove Alan Sipress Yuki Noguchi Post I.T.
Tech Podcast
The Bloggers
Subscribe to this Blog

Forgive and Forget for RIM?

When you love someone, it's easier to forgive. Looking around Washington yesterday, as people were once again tapping out messages on their Blackberrys, it seems that many have moved on and forgiven Research In Motion for an all-night outage earlier this week.

The company has taken some heat in the press and blogosphere this week for not being very forthcoming about details as to why the outage occurred - basically disrupting service from about 8 p.m. Tuesday night until the Wednesday morning commute. RIM and its public relations team were incredibly tight-lipped about the outage.

Late last night, well past a newspaper deadlines in Washington, RIM issued brief statement (below) noting that service is back to normal and explaining that the outage was caused by a software update that went bad. Security and capacity issues, the company said, were not to blame.

What do you say, Blackberry users? Do you love your Blackberry so much that you've already forgiven RIM? Let us know - in the Comments section below.

RIM Statement, April 19, 2007:

RIM's in-depth diagnostic analysis of the service interruption that
occurred in North America on Tuesday night is progressing well and RIM
will continue to provide further information as it's available. RIM's
first priority during any service interruption is always to restore
service and then establish, monitor and maintain stability. Proper
analysis can take several days or longer and RIM's commitment is to
provide the most accurate and complete information possible in such

RIM is pleased to report that normal conditions returned on Wednesday
and the BlackBerry service continues to operate normally today.

RIM has been able to definitively rule out security and capacity issues
as a root cause. Further, RIM has confirmed that the incident was not
caused by any hardware failure or core software infrastructure.

RIM has determined that the incident was triggered by the introduction
of a new, non-critical system routine that was designed to provide
better optimization of the system's cache. The system routine was
expected to be non-impacting with respect to the real-time operation of
the BlackBerry infrastructure, but the pre-testing of the system routine
proved to be insufficient.

The new system routine produced an unexpected impact and triggered a
compounding series of interaction errors between the system's
operational database and cache. After isolating the resulting database
problem and unsuccessfully attempting to correct it, RIM began it's
failover process to a backup system.

Although the backup system and failover process had been repeatedly and
successfully tested previously, the failover process did not fully
perform to RIM's expectations in this situation and therefore caused
further delay in restoring service and processing the resulting message

RIM apologizes to customers for inconvenience resulting from the service
interruption. RIM's root cause analysis and system enhancement process
with respect to this incident is ongoing and RIM has already identified
certain aspects of its testing, monitoring and recovery processes that
will be enhanced as a result of the incident and in order to prevent

By Sam Diaz  |  April 20, 2007; 12:10 PM ET  | Category:  Sam Diaz
Previous: Online World Grieves for Va. Tech | Next: When Is A Blackberry Not A Blackberry?

Add Post I.T. to Your Site
Stay on top of the latest Post I.T. news! This easy-to-use widget is simple to add to your own Web site and will update every time there's a new installment of Post I.T.
Get This Widget >>


Please email us to report offensive comments.

Treo and Good link rule

Posted by: roger | April 20, 2007 3:17 PM

If updating their software causes a complete loss of service imagine what would happen for a real outage . . .

Posted by: Greg Lygon | April 23, 2007 4:40 PM

The comments to this entry are closed.


© 2010 The Washington Post Company