Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recovery issues in 4.1.12 #1480

Closed
kinow opened this issue Dec 4, 2024 · 3 comments
Closed

Recovery issues in 4.1.12 #1480

kinow opened this issue Dec 4, 2024 · 3 comments
Assignees
Labels
destine DestinE related working on Someone is working on it
Milestone

Comments

@kinow
Copy link
Member

kinow commented Dec 4, 2024

In GitLab by @dbeltrankyl on Dec 4, 2024, 12:43

Hello @isimo00 , @kinow

Whatever the combination of autosubmit create + autosubmit recovery + autosubmit run is prompted, autosubmit will try to download logs already there.

In some cases, this leads to a crash.

The expected outcome is to download the logs during the run if it is required.

@kinow
Copy link
Member Author

kinow commented Dec 4, 2024

In GitLab by @dbeltrankyl on Dec 4, 2024, 12:43

Fixed there https://earth.bsc.es/gitlab/es/autosubmit/-/merge_requests/527

@kinow
Copy link
Member Author

kinow commented Dec 5, 2024

Just writing down what was reported on Mattermost (redacted):

(...) running a test version of autosubmit on the VM... and it's leaking processes (...)

... which means it's consuming memory too.

I freed up 22 gigs of memory by killing your processes

I see that a combination of lumi down with issues and a function that does not exist anymore ( in 4.1.12+, it is part of the log recovery processors rework ) caused that. It could happen also in 4.1.10 and 4.1.9 ( https://earth.bsc.es/gitlab/es/autosubmit/-/merge_requests/475/diffs#ff161687c56ba0a2dd5ce8f3f9c979f7cb2b6dbe_39_38 this one )

ssh sessions to lumi-uan01 for ... has dropped from >1200 to 4

@utigerstedt copied the list of processes to a file on /tmp (thanks!), and I copied it to my home dir in case we need it.

@dbeltrankyl
Copy link
Contributor

According to my previous comment, this was fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
destine DestinE related working on Someone is working on it
Projects
None yet
Development

No branches or pull requests

2 participants