Fixes to pelican plugin staging #882
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Note: waiting until I write a unit test for review |
joereuss12
force-pushed
the
shadow-hooks-with-pelican-staging-branch
branch
from
764da0e to
d85e199
Compare
|
Since I have to wait until #798 is merged for a test to work. Marking as ready for review so we can get the bug fixes and refactoring in |
joereuss12
force-pushed
the
shadow-hooks-with-pelican-staging-branch
branch
from
d85e199 to
46fb14e
Compare
Changes: - Found that how condor gives hooks a classad is in a different format than the plugin for some reason. For the hook the classad is split up by newlines but for the plugin it is by ';'. Therefore, made a new function for plugin stage to utilize. - Added function within the plugin to force logging to go to stderr (even though that is the default) as well as a file called `.PelicanPlugin.log`. This extra file is something you can specify in a condor submit file in `transfer_output_files` to get it back but only works if the job does not go on hold/if the plugin does not fail (not super helpful yet)
Separates out a lot of pieces of the main function into smaller functions. This helps with readability as well as for future unit testing.
This causes issues with plugin staging as condor will assume this is a classad being printed out when it in a shadow hook with the -l flag. This needs to be removed or add some sort of check if this is used with staging or the plugin
Just annoying for testing and not really needed, at least for now
joereuss12
force-pushed
the
shadow-hooks-with-pelican-staging-branch
branch
from
46fb14e to
1117218
Compare
jhiemstrawisc
approved these changes
Apr 4, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes:
.PelicanPlugin.log. This extra file is something you can specify in a condor submit file intransfer_output_filesto get it back but only works if the job does not go on hold/if the plugin does not fail (not super helpful yet)