Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for running MESSAGEix on UniCC #279

Merged
merged 11 commits into from
Feb 5, 2025
Merged

Add documentation for running MESSAGEix on UniCC #279

merged 11 commits into from
Feb 5, 2025

Conversation

measrainsey
Copy link
Contributor

@measrainsey measrainsey commented Jan 16, 2025

WIP: migrate my internal guide and documentation on how to run MESSAGEix on the cluster (UniCC) and make some modifications

How to review

For @khaeru and/or @glatterf42 : Read the diff and note that the CI checks all pass.

PR checklist

  • Continuous integration checks all ✅
  • Add or expand tests; coverage checks both ✅
  • Add, expand, or update documentation.
  • Update doc/whatsnew.

Copy link

codecov bot commented Jan 16, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.0%. Comparing base (7f94f8a) to head (b920f59).
Report is 12 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##            main    #279     +/-   ##
=======================================
- Coverage   77.9%   77.0%   -0.9%     
=======================================
  Files        213     213             
  Lines      16512   16512             
=======================================
- Hits       12872   12724    -148     
- Misses      3640    3788    +148     

see 7 files with indirect coverage changes

Copy link
Member

@glatterf42 glatterf42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this extensive guide! Skimming over it, I thought of just a few notes :)

doc/distrib/unicc.rst Outdated Show resolved Hide resolved
doc/distrib/unicc.rst Outdated Show resolved Hide resolved
sacct --format=jobid,MaxRSS,MaxVMSize,start,end,CPUTimeRAW,NodeList


Resources to request for reducing MESSAGEix run time
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's great to have a section like this, but if the cluster becomes more heavily used, there are more considerations for setting these parameters: in general, slurm will try to optimize its load so that it's running almost always at full capacity. This makes it harder to request lots of resources for extended amounts of time because the system will not be able to squeeze in such a task with other tasks, thus it will likely take longer for the task to be picked up.
I don't know if UniCC is already experiencing the kind of load to make these considerations necessary, but if/once it does, we would likely want to optimize these parameters further to request the minimum necessary that still reduces the run time :)

@khaeru khaeru self-assigned this Feb 5, 2025
@khaeru khaeru added doc Improvements or additions to documentation enh New features or functionality labels Feb 5, 2025
measrainsey and others added 7 commits February 5, 2025 11:08
- Include text edits from @glatterf42 review.
- Reflow text per code style.
- Use :program: and :file: roles in some places.
- Add "Changes" heading.
- Use sentence case in headings.
- Use Sphinx admonitions instead of ALL CAPS/headings.
@khaeru khaeru marked this pull request as ready for review February 5, 2025 10:45
Copy link
Member

@khaeru khaeru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @measrainsey for this valuable contribution!

Some items for follow up:

  • One thing I believe we discussed orally was to organize this and the other HOWTO guide in their own docs section.
  • I would also like to make sure that the guide teaches users where to find information in the UnICC docs and by using e.g. the man pages for sbatch and similar commands.

But LGTM and make those improvements while addressing #284 et al.

@khaeru khaeru merged commit 78f361c into main Feb 5, 2025
25 checks passed
@khaeru khaeru deleted the unicc branch February 5, 2025 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc Improvements or additions to documentation enh New features or functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants