Slurm broken

[MilesCranmer]

Is anybody maintaining this package? I haven't been able to get Slurm working for the past month or so... It just ends up stalling on connecting to worker 1 out of <N>:

julia> p = addprocs_slurm(2)
connecting to worker 1 out of <N>

The exact same code seemed to work a month ago. This is slurm 22.05.8. Not sure if this new version is breaking things or not.

[Moelf]

I don't have access to a slurm now, but it would be useful to know if a previous version was okay

[MilesCranmer]

Not sure I know how to test other versions of slurm... I am stuck with whatever my institute cluster has installed

[Moelf]

I guess I this case ask the HPC admin see if they know anything that might be the problem

[kescobo]

Ugh, @MilesCranmer that's annoying. I also don't currently have access to a SLURM cluster... this is the kind of thing that it would be nice if we had #105 that could test on different schedulers 🤦

Would definitely be worth checking in with the cluster admin to see if SLURM was recently updated so we can at least know that that's the culprit.

[MilesCranmer]

We could if this PR gets finished #193

[cnrrobertson]

@kescobo I can confirm that this issue started for me after an upgrade to Slurm on my institution's cluster. Unfortunately, I don't know what the previous version was, but currently the version is 23.11.1

[kescobo]

From the initial post, it looks like it goes back to v22

This is slurm 22.05.8

Does anyone know if SLURM follows SemVer?