Slurm controller
Investigating potential scheduling delays
The scheduling delays are still being investigated.
As mentioned initially, all jobs eventually get execute, so no action is required on the user part, besides a little bit more patience than usual.
We’re aware of the trouble this may cause, and are working with the scheduler developers to identify the problem and find a path to resolution.
We’re currently investigating some scheduling delays with jobs that have recently been submitted on Sherlock.
Under certain circumstances, jobs may take longer to be dispatched and wait in queue for longer than usual. All jobs will eventually start, so we recommend keeping them in queue and to avoid cancelling jobs (re-submitting them later will only put them back at the end of the line).
We’re working with the scheduler support and development teams on this incident, and will post updates when we have them.