Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save more provenance information #308

Merged
merged 10 commits into from
Aug 8, 2023
Merged

Save more provenance information #308

merged 10 commits into from
Aug 8, 2023

Conversation

tkkuehn
Copy link
Contributor

@tkkuehn tkkuehn commented Jun 15, 2023

Proposed changes

Addresses #205 in a reasonably basic way. It's not guaranteed that a Snakebids app even has a version, so this PR just takes a crack at it by assuming the name of the snakemake directory is also the name of an installed package. If anyone has any other ideas, we can straightforwardly add fallback options to SnakeBidsApp.get_app_version.

Note that this doesn't do anything about the proposed formatting/comments in the output config. Maybe that would be good to raise as a separate issue.

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you are unsure about any of the choices, don't hesitate to ask!

  • Changes have been tested to ensure that fix is effective or that a feature works.
  • Changes pass the unit tests
  • I have included necessary documentation or comments (as necessary)
  • Any dependent changes have been merged and published

Notes

All PRs will undergo the unit testing before being reviewed. You may be requested to explain or make additional changes before the PR is accepted.

@tkkuehn tkkuehn changed the title Provenance Save more provenance information Jun 15, 2023
@tkkuehn tkkuehn linked an issue Jun 15, 2023 that may be closed by this pull request
@kaitj
Copy link
Contributor

kaitj commented Jun 15, 2023

Quickly skimmed the PR (will do a more thorough look later on), and so far it looks good.

Would there be a situation where the app isn't a distribution package installed in the environment?

@tkkuehn
Copy link
Contributor Author

tkkuehn commented Jun 15, 2023

Would there be a situation where the app isn't a distribution package installed in the environment?

An example would be if I find a Snakebids app I want to use on GitHub, clone it, and have Snakebids etc. globally installed in my environment, so I don't bother installing the app to my environment.

A real-world (if kind of weird and likely to change) case is https://github.com/afids/afids-CNN/tree/main/afids_cnn, which wraps a couple of different Snakebids workflows under one package that is not itself a Snakebids workflow.

@kaitj
Copy link
Contributor

kaitj commented Jun 15, 2023

Fair enough, then I think one useful thing to add would be a warning that "...the version of the snakebids app cannot be found as it is likely not installed in the environment" or something to that effect, so its not a surprise that the config shows "Unknown" version.

snakebids/app.py Outdated Show resolved Hide resolved
snakebids/app.py Outdated
@@ -240,6 +250,21 @@ def create_descriptor(self, out_file: PathLike[str] | str) -> None:
)
new_descriptor.save(out_file) # type: ignore

def get_app_version(self) -> str | None:
"""Attempt to get the app version, returning None if we can't.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're moving in this direction, I wonder if we should allow (and encourage) the passing of explicit modules (or string names of modules) to snakemake_dir in addition to paths. This would actually simplify the run.py file a bit. And this would create some extra liberty e.g. if the snakemake package was in some subpackage foo.bar, we'd be able to support that a little better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily a goal for this PR, but maybe something to think about for the future

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't address this for now, but is probably worth a separate issue for discussion.

pyproject.toml Outdated Show resolved Hide resolved
Copy link
Contributor

@pvandyken pvandyken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One concern with leaving a warning if no version is found is that if a developer messes up the implementation preventing versioning, users will get that warning and potentially think something is wrong... But then they may still think something is wrong when they read version=unknown so maybe it doesn't matter.

@tkkuehn
Copy link
Contributor Author

tkkuehn commented Jun 21, 2023

I added a warning where I tried to convey that the missing version will often be nothing for the end user to worry about, but I'm open to changing either or both of the warning's text, and its log level (could make it INFO, but those won't usually ever be seen by end users, as discussed in #184 ).

@tkkuehn tkkuehn requested a review from pvandyken June 22, 2023 18:56
Copy link
Contributor

@kaitj kaitj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This lgtm - the warning might be a little long, but I think it conveys what needs to be conveyed and I don't have any other suggestions there.

Copy link
Contributor

@pvandyken pvandyken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment RE the warning, but once that's resolved no need for re-review

snakebids/app.py Outdated Show resolved Hide resolved
@pvandyken
Copy link
Contributor

@tkkuehn, we had originally said we'd try to get this one in our 0.9 release. Would you still like to? If not, there's no need to maintain py37 support

snakebids/app.py Outdated Show resolved Hide resolved
@tkkuehn
Copy link
Contributor Author

tkkuehn commented Aug 8, 2023

We had originally said we'd try to get this one in our 0.9 release. Would you still like to? If not, there's no need to maintain py37 support

I'm open either way, I don't think this is such a major feature that we need to get it in ASAP, and delaying reduces the amount of work to be done to drop py37 I guess

@pvandyken
Copy link
Contributor

How about we just do a race. If the py37 drop is done before this, I'll release without this, otherwise this can go in

@tkkuehn
Copy link
Contributor Author

tkkuehn commented Aug 8, 2023

How about we just do a race. If the py37 drop is done before this, I'll release without this, otherwise this can go in

I think this is currently ready to go, unless you have any more comments?

@pvandyken
Copy link
Contributor

Nope, looks like just a merge, then you can go ahead

@tkkuehn tkkuehn merged commit e73c908 into khanlab:main Aug 8, 2023
8 checks passed
@pvandyken pvandyken added the enhancement New feature or request label Aug 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Saving more provenance information
3 participants