Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce ncclimo tasks for Anvil to 6 #865

Merged
merged 1 commit into from
Feb 28, 2022

Conversation

xylar
Copy link
Collaborator

@xylar xylar commented Feb 25, 2022

This is needed because of out-of-memory errors that have been seen in some recent runs.

closes #864

This is needed because of out-of-memory errors that have been seen
in some recent runs.
@xylar xylar added the bug label Feb 25, 2022
@xylar xylar requested a review from ytakano3 February 25, 2022 14:23
@xylar xylar self-assigned this Feb 25, 2022
@xylar
Copy link
Collaborator Author

xylar commented Feb 25, 2022

@ytakano3, to review this if you could just verify that this looks consistent with the change you made to get things working on Anvil, I think that's enough.

Copy link
Collaborator

@ytakano3 ytakano3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for updating. I used the ncclimoThreads = 6 but also for the parallelTaskCount = 6 for my BGC testing. I need to check increasing the parallel task will change the behavior.

@xylar
Copy link
Collaborator Author

xylar commented Feb 26, 2022

@ytakano3, if you don't mind re-testing with parallelTaskCount = 12, that would be great! Running with parallelTaskCount = 6 will definitely make the analysis take twice as long, whereas ncclimoThreads = 6 will only make one task (though a big one!) take twice as long.

@ytakano3
Copy link
Collaborator

@xylar I will test with parallelTaskCount = 12 soon and yes I realized parallelTaskCount = 6 was pretty slow. Thank you!

@ytakano3
Copy link
Collaborator

@xylar Tested quickly, parallelTaskCount = 12 works fine so I thing good to go!

@xylar
Copy link
Collaborator Author

xylar commented Feb 28, 2022

Thanks @ytakano3!

@xylar xylar merged commit fa5c0fd into MPAS-Dev:develop Feb 28, 2022
@xylar xylar deleted the reduce_anvil_ncclimo_tasks branch February 28, 2022 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Climatology computation (with BGC) on Anvil has OOM error
2 participants