How Remote Infrastructure Monitoring Saved A Domain Controller
So imagine this scenario. You have a server that randomly drops off of the network for no apparent reason. I say apparent because there is always a reason. However, we don't always know what it is. Your only option is to visit the data center or server closet to do a cold boot. This is exactly the scenario we walked into at a school.
Needle In A Haystack?
Like many schools, the IT staff was limited in size and had huge responsibilities. From Active Directory and Exchange to internet connectivity and firewalls, they were clearly overwhelmed. They simply needed to know the cause of this ongoing headache without having to become experts in a new technology or product suite.
To there surprise the answer was right in front of them the entire time. Well sort of. The answer was in the event log of the domain controller, but this is a very scary place for many system administrators. All they really needed was a simple log parser that searched for certain events. However, if you don't know what to search for then a parser is useless.
Mystery Solved
In comes remote infrastructure monitoring. Because this was an emergency situation, we simply installed a service on one of their servers which served as a sofware proxy for our cloud based remote monitoring solution. We then entered the IP address of the domain controller in our web console in order to discover the device. Within a couple of hours we recieved the following email alert:
Subject: Critical : DC1 - Disk Becoming Unusable
Matched Event : The file system structure on the disk is corrupt and unusable. Please run the chkdsk utility on the volume \Device\HarddiskVolume1.
In plain English, the physical disk was about to die. It was only a matter of time before the domain controller went down permanently. Armed with this information we were able to instruct them on how to address the problem before the domain controller's imminent death.
A Simple Solution To A Common Problem
Thousands of events are logged in the Windows event logs everyday and it can be a daunting task to know which events to monitor and which to ignore. Even more daunting is getting a soluton in place to catch a problem during a time of crisis. This is the true value of cloud based and SaaS solutions - rapid deployment and near instant results.
Regardless of the solution you choose, infrastructure monitoring is critical to business functions. Ignore it at your own peril. At the same time you want to make sure that the solution allows you to focus on the results and not the complexity of the technology. This is yet another strength of cloud based solutions. In the meantime, at least get a log parser and monitor for the disk event mentioned in this post.
