Technology

Facebook blames outage on error during routine maintenance



The global outage that knocked and its other platforms offline for hours was caused by an error during routine maintenance, the company said.


Santosh Janardhan, Facebook’s vice president of infrastructure, said in a blog post that Facebook, Instagram and WhatsApp going dark was caused not by malicious activity, but an error of our own making.”

The problem occurred as engineers were carrying out day to day work on Facebook’s global backbone network; the computers, routers and software in its data centers around the world along with the fiber-optic cables connecting them.





During one of these routine maintenance jobs, a command was issued with the intention to assess the availability of global backbone capacity, which unintentionally took down all the connections in our backbone network, effectively disconnecting data centers globally, Janardhan said Tuesday.


Facebook’s systems are designed to catch such mistakes but in this case a bug in the audit tool prevented it from properly stopping the command, Janardhan said.


That change also triggered a second problem that made things worse by making it impossible to reach Facebook’s servers even though they were operational.


Engineers scrambled to fix the problem on site, but this took time because of the extra layers of security, Janardhan said. The data centers are hard to get into, and once you’re inside, the hardware and routers are designed to be difficult to modify even when you have physical access to them.


Once connectivity was restored, services were brought back gradually to avoid traffic surges that could cause more crashes.

Dear Reader,

Business Standard has always strived hard to provide up-to-date information and commentary on developments that are of interest to you and have wider political and economic implications for the country and the world. Your encouragement and constant feedback on how to improve our offering have only made our resolve and commitment to these ideals stronger. Even during these difficult times arising out of Covid-19, we continue to remain committed to keeping you informed and updated with credible news, authoritative views and incisive commentary on topical issues of relevance.

We, however, have a request.

As we battle the economic impact of the pandemic, we need your support even more, so that we can continue to offer you more quality content. Our subscription model has seen an encouraging response from many of you, who have subscribed to our online content. More subscription to our online content can only help us achieve the goals of offering you even better and more relevant content. We believe in free, fair and credible journalism. Your support through more subscriptions can help us practise the journalism to which we are committed.

Support quality journalism and subscribe to Business Standard.

Digital Editor



You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *