/scratch metadata server outage

Gelöst 4. Sep 2018, 09:07 Uhr PDT

The metadata server issue has been resolved.

Beobachtung 22. Aug 2018, 13:16 Uhr PDT

The filesystem is back online.
We’re still looking into the root cause for those repeated crashes.

Untersuchung 22. Aug 2018, 12:41 Uhr PDT

The metadata server just suffered another crash, we’re investigating.

Gelöst 22. Aug 2018, 08:21 Uhr PDT

The filesystem has been stable overnight, so we’re closing this incident.

Beobachtung 21. Aug 2018, 15:12 Uhr PDT

The filesystem is back online, we’ll continue monitoring it for new occurrences.

Untersuchung 21. Aug 2018, 13:22 Uhr PDT

Another occurrence of this same issue happened, we’re looking into it.

Gelöst 15. Aug 2018, 08:08 Uhr PDT

The issue has been identified and resolved. Access to $SCRATCH and $GROUP_SCRATCH has been restored, and logins should go through normally now. We’ll continue to monitor the state of the storage system, and feel free to report any issue you may notice.

Untersuchung 15. Aug 2018, 07:48 Uhr PDT

The metadata server for the /scratch filesystem is currently unavailable. The current effects include the impossibility to change directories to or list the contents of directories in $SCRATCH or $GROUP_SCRATCH, and possibly hanging logins.
We’re investigating and will post updates as we have them.