Server crash

[theSteveCo]

The server software crashed today at 4 AM after getting hammered by a search engine - I guess looking to refresh content from the previous owner of this IP address. I'll have to figure out exactly *how* that happened, but we're (obviously) back up now.

-Steve

 

[Wolf]

I was wondering what happened. I was at work and fiending for BARF. Had to get my fix from SBN instead.

 

[theSteveCo]

Wolf, considering that you're up late at night, maybe I should put together some sort of mechanism you can use to rehup the server should it fail. Hmmm......

 

[Wolf]

One of our customers here had a neat device. They took an old pager, spliced into the wires that went to the speaker/vibrate thing, and wired those wires up to their server's reset button. So they could just call up the pager when they wanted their server reset. But the pager didn't get reception in the colo, so it never worked.

 

[theSteveCo]

Clever! But why do that when you can just buy devices that will do that? In fact, last time I toured AboveNet I saw rackmounted versions that could power cycle a bunch of different machines.

The server was actually still running when it happened, but the Apache process what butt locked.

www:~ >/etc/rc.d/init.d/httpd status
pid exists but process is dead
www:~ >

(or something like that)

What I was thinking was running a separate server (http, ftp, whatever) that would enable you to touch a file that would flag a cron job to reHUP the process.

[/geek speak]