The /scratch filesystem is experiencing issues

Resolved Mar 1, 2018 3:07 PM PST

Monitoring Mar 1, 2018 3:05 PM PST

Replacement parts have been received, and the defective components have been replaced. The failed disk array is now back in production, and the /scratch file system has been brought back up. Production has resumed, and both $SCRATCH and $PI_SCRATCH are available again.

We’ll continue to monitor the file system to detect any anomaly, but we’re confident the incident has been definitely resolved.

Problem Identified Feb 28, 2018 2:45 PM PST

Replacement parts are scheduled for delivery tomorrow morning. The file system will unfortunately remain unavailable until those parts are replaced.

Problem Identified Feb 28, 2018 11:32 AM PST

The failed component has been identified. Replacement parts have been requested from the hardware vendor.

Investigating Feb 28, 2018 9:23 AM PST

We’ve identified the component at the source of the outage. One of the disk arrays in the storage system is currently not responding, we’re investigating the root cause.

Opened Feb 28, 2018 8:36 AM PST

The /scratch filesystem, that serves both $SCRATCH and $PI_SCRATCH is currently experiencing issues, we’re investigating.