Merge OSDF Client #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Propagate the number of bytes transferred back up to the main function.
Add proper version to classad output
Add forceposix, primarily for windows builds
Adding retry'able error codes
Multi file downloads
Better error message
After the credential generation branch was rebased after the director work, a few build errors popped up.
Follow credential generation advice from topology
Only recently was the OSDF client re-synced with some of the credential generation advice supplied by the Topology service. This primarily affected cases where an origin's base path did not match its local export path (such as in the case of `/chtc/staging/foo` actually exporting the local path `/staging/foo`). This commit connects the Director into the reconfigurations so that it will also work in such cases. A parallel commit was also pushed/merged for the Director to supply the information needed by the OSDF client.
…d-gen Use new cred gen information from Director
Fix GHA deprecation warnings (SOFTWARE-5354)
As we want to have the ability to stand up additional data federation instances (think: test directors), we should split out and centralize the defaults. This introduces the use of viper for the process-wide configuration management. This will allow the use of configuration files in the near future.
In some unit tests, we need to alter the configuration via env var (to ensure the environment-based changes are effective); this adds in re-initialization at the appropriate points. Additionally, this moves the various timeout constants to the config subsystem, allowing us to use much shorter timeouts for test runs (taking about 6 minutes off the time to run `go test`).
Since the binary generated for namespaces_test.go doesn't start with `stash`, the override for the test should be `PELICAN`-prefixed instead.
…erge Begin merge of original osdf-client repository into the pelican CLI; start by merging the entirety of the repository into a sub-directory, allowing future movement and rearrangement.
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR merges the OSDF client into the pelican CLI, integrating it in a way that should be compatible with the OSDF client.
For example, the following are equivalent:
and
Setting $ARG0 to be "stashcp" or "stash_plugin" (e.g., as a symlink above) is equivalent to running the prior respective binaries.