Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Symlinks reduce disk usage #198

Merged
merged 2 commits into from
Dec 5, 2023
Merged

Conversation

mira-miracoli
Copy link
Collaborator

This is a small improvement for the copy tasks in the static directories
As the role currently works, the whole static directory is rsynced into every new subdomain directory, which leads to additional disk usage of ~ 154 MB for each subdomain in a basic (GTN) Galaxy setup.

With this improvement, several static subdirectories are symlinked and all files inside dist are symlinked, which reduces the disk usage per subdomain to ~ 2.3 MB.

However the disadvantage of this is:

  • more extensive logs; each symlink needs to be checked and is logged by ansible, around 200 per subdomain, the galaxy_themes_no_symlink_logs just reduces the extent, but ansible still prints one line per file and subdomain
  • slightly slower playbook runs; synchronize for the whole folder is faster than invoking an ansible operation for each file and we don't get rid of the synchronize fully, because we would need to create multiple subdirectories manually and iterate over them with symlink tasks

To solve this conflict, I added variables that let the user decide weather to use the symlink feature and reduce the logs or not

tasks/static_subdomain_dirs.yml Show resolved Hide resolved
@natefoo natefoo merged commit 06cbe3f into galaxyproject:main Dec 5, 2023
5 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants