Partial power outage in SRCF
Power has now been restored to the datacenter floor, and all the impacted Sherlock nodes are up and running again.
Additional tests and repairs have been performed, and the substation has been successfully re-energized. To make sure everything is stable, the transformer will be allowed to run and heat up overnight, and power restoration on the datacenter floor is scheduled for tomorrow morning.
Electricians are on site and have identified a blown fuse in a high-voltage breaker, in a substation that feeds power to SRCF. More investigation is underway to make sure equipment can be safely re-energized. This is expected to take some time, probably until tomorrow.
We’re currently investigating a partial power outage affecting a significant portion of SRCF, the datacenter where Sherlock is hosted. A number of Sherlock 4.0 compute and service nodes are currently down, which could impact job wait times and potentially network connectivity from compute nodes to external resources.
Engineers are en route to investigate and hopefully restore power as quickly as possible. Updates to follow.
- Job scheduler / Public partitions
- Job scheduler / Owner partitions
- Networks / External connectivity