[imagesource: AP Images]

You open and close Facebook and Instagram. You try a few other apps.

You reset the WiFi router. You switch to data.

You realise Facebook and Instagram are down, so you open WhatsApp.

That doesn’t work either. Total, unadulterated panic.

I was watching Squid Game at the time, so I was fine, but I really feel for all of those meals that were eaten without the benefit of being posted to social media.

For six full hours, Facebook, which owns Instagram and WhatsApp, experienced an outage. According to the company, this wasn’t a hack, but rather “the result of a configuration change to its routers”.

The Verge reports:

The explanation doesn’t give much in the way of detail, but it seems like Facebook’s machines weren’t able to talk to one another — Facebook says that “this disruption to network traffic had a cascading effect on the way our data centers communicate, bringing our services to a halt.”

…The problems appeared to begin with a routine BGP update that went wrong, wiping out the DNS routing information that Facebook needs to allow other networks to find its sites.

Apparently, they tried turning it off and back on again, without success.

The six-hour outage was the longest since 2019, when a technical error affected Facebook and its related sites for 24 hours.

You can see Facebook’s full statement regarding the outage here, and more on what BGP is here.

So that’s all of the technical stuff, but what I really want to know is what sort of chaos was happening inside Facebook HQ during this time.

Thankfully, The Verge also has us covered there:

[The outage] sent the company’s engineers scrambling to fix the issue at one of its data centers in California, according to two people familiar with the situation…

Several employees told The Verge they resorted to talking through their work-provided Outlook email accounts since Facebook mainly runs on an internal version of the social network that is currently not accessible. While employees could email each other, they couldn’t send or receive emails from external addresses…

Further complicating matters, the outage temporarily broke the ability for some employees to access company buildings and conference rooms with their badges.

Imagine their disappointment when the issue was resolved and they could get into the building.

So no Facebook employees turned their back on WhatsApp and switched to Telegram in order to message?

It’s certainly not the sort of thing you would mention publicly.

[sources:verge&verge]