The Ides of March. Beware.
I’m sitting in a room with about 30 people planning tonight’s unscheduled maintenance of the Liebert cooling systems which account for 2/3rds of the cooling of our Data Center. Yesterday a leak was identified in the coolant system…
So, we’re turning off servers; planning for the plumbers (requiring overtime to work through the night) to flush the coolant, replace the leaky pipe, replace the coolant, and bleed the lines again to bring these 4 systems back up.
Conversation includes BTU output per server, ambient air temperature through the night, humidity, systems which must stay running…
I love the way this place works as a team. I love how complex problems are broken down into constituent parts and everyone pitches in. No matter what it takes. The Manager of Operations and Engineering came to fill me in on the details as we were planning whether the course management system will be able to stay up at least through the after dinner hours while distance learners are having chat sessions.
The Liebert’s were put in place in April of 2004. The 4 of them were sized for complete redundancy. That is, 2 of them could provide total cooling for the entire Data Center. In just 3 years, because of the number of systems added, all 4 systems together can only maintain 2/3rds of the cooling necessary. The other 1/3rd of cooling necessary is provided by the building’s own AC in the summer time, and by forcing in and drying out ambient air during the winter. In fact, we warm that air up to 55 degrees F before piping it into the Data Center.