Saewyc Home

A small site for a small family

Outage

Server outage

Ah, the joys of self-hosting at home. A tree fell on a power line. The ensuing power outage lasted longer than my backup power supply. Servers fell down, went boom.

The system as set up is supposed to recover from such an outage. Once power returns, everything reboots, says what the heck happened? and e-mails me a bunch of nastygrams. Based on what arrived in my inbox this mostly happened.

Mostly.

You may remember a few months back the microsd on the proxy server got corrupted, causing an 8 hour outage? Guess which of the pi network seems not to have restarted properly?

Normally, of course, I as sysadmin become aware of the problem, stroll over to physically remove the sd card, reflash it, reinstall, configure, plop it back into the SBC and we are rolling again. It is not going to happen this time. Because I am on the other side of the border, on a boat, in the same storm which knocked out electricity back home. More importantly, I am not going to get back for about 16 days, or longer.

This means a couple of services are in limbo, including the cloud server.

They will be correctly rebooted at some point in the relatively near future. But that 'near' is quite possibly three weeks from now.