Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clean up work dir on self hosted agent #453

Closed
dvescovi1 opened this issue Feb 25, 2020 · 4 comments
Closed

clean up work dir on self hosted agent #453

dvescovi1 opened this issue Feb 25, 2020 · 4 comments

Comments

@dvescovi1
Copy link

My pipeline build on my self hosted agent creates incremental folders in the _work directory (1, 2, 3 etc). These eventually fill up my hard disk as they are not cleaned after pipeline completion. I suspect I can add a final task to clean up but I can not find any documentation on how to do this.
Any suggestion, pointers or documentation?

@f2calv
Copy link

f2calv commented Feb 28, 2020

Specifically you are looking for

workspace:
  clean: all

Yes it's all in the docs, https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema#job

jobs:
- job: string  # name of the job (A-Z, a-z, 0-9, and underscore)
  displayName: string  # friendly name to display in the UI
  workspace:
    clean: outputs | resources | all # what to clean up before the job runs
  steps: [ script | bash | pwsh | powershell | checkout | task | templateReference ]
...

@dvescovi1
Copy link
Author

Yea, but according to the doc you mention "clean: all" cleans up "before" the job runs.
I need something that cleans up everything "after" the job is run to get rid of all the residuals.
I was thinking you could just add an additional job at the end but I think that would still just clean its workspace job, not all the previous ones.
maybe adding a script at the end that just does a "rm -fr ." would do it?

@stale
Copy link

stale bot commented Apr 25, 2020

In order to consolidate to fewer feedback channels, we've moved suggestions and issue reporting to Developer Community. Sorry for any confusion resulting from this move.

@stale stale bot closed this as completed Apr 26, 2020
@EugenMayer
Copy link

Since running pre-cleanup-jobs (workspace.clean) or post-cleanup-jobs some step X to cleanup the agent workdir heavily relies on the pipeline-author of "some pipeline" to fix a potential hard to detect "workdir pollution" issue, we decided to re-implement the idea to guarantee workdir pollution can never happen.

I have created https://github.com/EugenMayer/azure-agent-self-hosted-toolkit which fixes this project on the agent-level, not requiring any changes to the pipelines nor relying on those.

To quote to project idea

The run-once mode is based on Microsoft's `./run.sh --once` which ensures that an agents only runs 1 job and then stops.
This is used to

 - cleanup the workdir in a safe manner after each job
 - ensures each job on an agent runs in a clean workdir
 - starts the agent right after cleanup up (few seconds) to be available for the next job

This repository also offers a toolkit to start / setup x-agents and maintain them using the original tools of Microsoft, but wrapped in convenient scripts.

If this helps anybody else, happy to share it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants