You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We can manually configure a pipeline to use job profiling via slurm. However for newer users a --profile option or backend setting to force job profiling can be helpful. Especially if we automatically generate plots and potentially job efficiency metrics.
For the slurm_singularity backend we generate an sbatch script:
Decisions for if we should generate a plot might be more complex than we expect. Initial thoughts suggest that we should generates plot when the task completes. However if a job fails due to memory or walltime, we would likely want the plot in this case as well.
We may also want to consider psrecord as a general option, should only need to insert the following (assuming psrecord is available already, other alternatives may exist):
We can manually configure a pipeline to use job profiling via slurm. However for newer users a
--profile
option or backend setting to force job profiling can be helpful. Especially if we automatically generate plots and potentially job efficiency metrics.For the slurm_singularity backend we generate an sbatch script:
jetstream/jetstream/backends/slurm_singularity.py
Lines 675 to 680 in eca552c
Pseudo implementation:
Decisions for if we should generate a plot might be more complex than we expect. Initial thoughts suggest that we should generates plot when the task completes. However if a job fails due to memory or walltime, we would likely want the plot in this case as well.
We may also want to consider
psrecord
as a general option, should only need to insert the following (assuming psrecord is available already, other alternatives may exist):The text was updated successfully, but these errors were encountered: