The root cause of the outage has been identified and addressed.
The compute nodes provisioning failure was caused by a change in the IP address block of a critical AWS service, which ended up falling outside the range of addresses allowed by our outbound network traffic filtering. For this reason the network security configuration was accidentally preventing the provisioned nodes to get registered in DNS causing the provisioning to fail. The issue has been now corrected.
We have confirmed jobs can now be scheduled normally. All partitions have been re-enabled. Users are welcome to resume their work.
We will keep monitoring the status of the cluster. Please promptly report any other unusual behavior you might experience while working on the cluster.
