I feel like I need to document this somewhere, not only for story sake, but for my own sake when I inevitably try to remember when this happened. I would have wrote something this weekend, but I was very busy and I had a lot to do, so I’m doing it now.
I was at work on Wednesday 4/14, and I had just finished my weekly MIS Meeting where we update on projects I’m working on, as well as the direction I should be heading over the next little bit, etc. Just to make sure things are on track.
So I go back and sit at my desk and bring up some code I’m working on for Production, and I hear what sounds like a drill above my head. That’s strange, what is DS working on up there…
So I decide to take a look, after getting up from my desk and heading onto the floor toward the stairs to the attic, I notice the lights are off. Also interesting, that means the sound was something else. I head up the stairs and open the door to the attic to see that the fan sitting next to my network switches which pushes the air off of them so they don’t overheat is ablaze.
I was a little shocked, and begin looking around for a fire extinguisher, but I don’t find one (Turns out we don’t keep one upstairs which has already changed) so I head down in and open the office door and tell everyone there is a fire. Turn around and head back upstairs with a couple of others to show them, and TM heads off to grab an extinguisher. As he’s running up the stairs, I met him halfway, grabbed the extinguisher, and ran toward the fan which is now completely engulfed in flames, and start to douse it with a couple of quick bursts.
Flames go out immediately, Baking soda is all over the upstairs attic, but the fire is out. There was a sprinkler just above the fan, which was probably a few seconds away from triggering, but if it had, it would have been a mess as it would likely have doused my servers and a lot of the electronics which exist just below it.
Turns out what was the drill sound was the motor on the fan burning out, so I was extremely lucky to hear it, and it saves me a lot of time and our company a lot of money over the next couple of weeks had the sprinklers have triggered.
The rest of the day, after the fire dept came and inspected the situation and cleared us for reentry, consisted of diagnosing network connections and stuff (Some cables had unplugged from the switch as we were moving it around and stuff, and it was full of baking soda from the fire extinguisher as well, which I had to blow out and stuff, but everything still works and we got everyone re-connected finally.)
Some other notes which complicated things. We had tested the sprinkler systems just that morning (at 11:30), however, we had forgotten to warn the plant, (though we did warn the security company), so when the alarm sounded the plant just sat there instead of doing like they are supposed to do. Because of the false alarm though, when we had the real fire incident that day, at first when I opened the office door and told them we had a fire, nobody really moved. But they quickly bolted into gear when they saw I was serious.
All in all it was a good thing, and more good things happened than bad things, but it was a wake up call, and though we have disaster recovery contingencies, we can point to a real situation now and evaluate where we would be had it been a worst case scenario.
Unfortunately, I am doing a Server Migration at work and will not be able to attend. Val and the Girls will not be going this time either. I’m super stoked about our opponent though. Oklahoma State is a pretty good team, despite being in a weak conference. Should be another good bowl victory for the Rebels and for the SEC.
Mostly, i spend all day in vi in a terminal window these days. Editing some code for a program that the customer still won’t approve. I think my code is solid now, but we are going to have to change the way we store some data in our database. That’s the only way I’m going to be able to meet their requirements.