Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revisit condor shadow hooks with plugin staging #730

Closed
joereuss12 opened this issue Jan 31, 2024 · 4 comments · Fixed by #882
Closed

Revisit condor shadow hooks with plugin staging #730

joereuss12 opened this issue Jan 31, 2024 · 4 comments · Fixed by #882
Assignees
Labels
client Issue affecting the OSDF client critical High priority for next release
Milestone

Comments

@joereuss12
Copy link
Contributor

Making this an actual issue so that I can aim to do it. For v7.6.0 I want to revisit condor's shadow job hooks with our Pelican plugin staging to see if we change a file on an origin that we can detect that and change the file. Cole's changes to condor file transfers are making progress but not fully implemented however, we should start working on this again so that we can see if this works how we want it to and just make changes when needed.

@joereuss12 joereuss12 added the client Issue affecting the OSDF client label Jan 31, 2024
@joereuss12 joereuss12 added this to the v7.6.0 milestone Jan 31, 2024
@joereuss12 joereuss12 self-assigned this Jan 31, 2024
@joereuss12
Copy link
Contributor Author

Also we need a unit test for plugin staging, merging those two tickets into this one

@joereuss12 joereuss12 added the critical High priority for next release label Feb 21, 2024
@joereuss12
Copy link
Contributor Author

Making a note here:
We can make a test for this using our federation in a box however, how it is currently written it cannot work. It most likely will work with this PR being merged: #798, because of the changes to how we handle metadata and it would work with an osdf:// url and prefix. It will work because it will load the metadata from config since it should be set already and not completely rely on topology. Therefore, we can set our fed in a box to config and utilize that to run this as a test locally.

@joereuss12
Copy link
Contributor Author

Note: Got a successful test run on HTCondor AP submit4! All that is needed now to complete this PR is a test but currently waiting on #798 to be merged first

@joereuss12 joereuss12 linked a pull request Mar 1, 2024 that will close this issue
@joereuss12
Copy link
Contributor Author

joereuss12 commented Mar 28, 2024

This needs revisiting in terms of the logging. The rest should be good and needs a review soon

@joereuss12 joereuss12 modified the milestones: v7.6.0, v7.7.0 Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client Issue affecting the OSDF client critical High priority for next release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant