Archive for June, 2009

finally back up

Monday, June 1st, 2009

I finally took a few minutes this afternoon to get Apache back up and running.

It failed to come back after the server restarted a 2 weeks ago today. Why did the server restart you might ask. Because the collocation facility where it is house mucked up something with the back-up power. The story is they while doing a weekly test of the generator the switch back to main power failed and the UPS batteries failed before they could get the mains back online or the generators running again.

While I got the most critical functions of the server back as soon as I could (and it was non trivial thanks to an issue with my RAID 1 mirrored config and a corrupted boot file) and restarted critical functions (my mail server and a 2nd mailserver that is relied on for a business). The web server and blog could wait. And wait they did all the way until today.

Now it is back and I am happy.

Oh and two other things the power outage claimed 1 server immediately and I hold it responsible for a firewall failing last week. And I have to say I am no fan of software RAID 1 mirroring on this machine. It made recovering about 10x as hard b/c I had to unmirror the root partition.