At this U.S. division of a big Japanese company, many Japanese associates attend training sessions at the main office during weekends, according to an IT pilot fish who works in the data center there.
"One Saturday my network administrator and I both got alert texts that multiple systems were down at our data center, despite automatic failover to UPS and generator power," fish says.
"The net admin traveled 30 minutes to the data center and discovered that systems appeared to be up. However, he could see that many systems were either in the process of booting up or had booted up a short time before."
Fish arrives an hour later and together they start checking the UPS and other power equipment, along with all the data center systems that have problems due to the unscheduled power-cycling. But the UPS and power look fine, and they can't figure out what happened.
But the net admin recalls passing a vehicle leaving the parking lot as he arrived. They check the video surveillance system -- and on the video can see a Japanese associate enter the room containing the UPS and air conditioning equipment for the building. In fact, he enters the room twice within an hour.
They check the access control system and identify the associate, who's visiting from one of the company's remote facilities. So he's not an intruder -- but what was he doing in the HVAC/UPS equipment room?
"We tracked him down on Monday for questioning," says fish. "He explained to HR that he was with a group conducting an English class on Saturday in one of the conference rooms.
"Because the air conditioning system is programmed for weekday operations, it became very warm in the conference room. He decided to enter the equipment room to check the air conditioning systems in hopes of cooling the conference room."
Once in the room, associate explains, he flipped a breaker near the air conditioning systems, thinking it might turn on the cooling, and then returned to the class.
That seemed to improve the cooling situation, and the class continued.
Then, being a responsible company associate, when the class was finished he returned to the equipment room, put the breaker back in the original position and then left the facility.
"What he didn't realize was that he shut down the main breaker for the distribution panel downstream from the data center UPS," fish sighs. "That dropped power to the data center, but the rest of the building still had power. He never knew what he had done.
"The access control system for the main office building was soon expanded to control access to the HVAC/UPS room to keep out curious users."
Get your daily dose of out-takes from the IT Theater of the Absurd delivered directly to your Inbox. Subscribe now to the Daily Shark Newsletter.