Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolve/document dependency discrepancies #945

Closed
pont-us opened this issue Mar 12, 2024 · 1 comment · Fixed by #947
Closed

Resolve/document dependency discrepancies #945

pont-us opened this issue Mar 12, 2024 · 1 comment · Fixed by #947
Assignees

Comments

@pont-us
Copy link
Member

pont-us commented Mar 12, 2024

conda-forge releases (e.g. here ) trigger a dependency analysis which always finds a significant number of discrepancies between the dependencies listed in meta.yaml. Automated dependency anaylsis isn't 100% reliable, but we should resolve as many of these as possible and document the rest so we don't waste time looking into them again. for the 1.4.1 release, the list is as follows:

Packages found by source code inspection but not in the meta.yaml:

  • cftime
  • numcodecs
  • botocore

Packages found in the meta.yaml but not found by source code inspection:

  • werkzeug
  • scipy
  • adlfs
  • python-blosc
  • fontconfig
  • lz4
  • requests-oauthlib
Package Comments Action
cftime imported in select.py etc. add to environment.yml and meta.yaml
numcodecs imported in zarrstore/generic.py add to environment.yml and meta.yaml
botocore imported in dsio.py add to environment.yml and meta.yaml
werkzeug only imported in tests remove from meta.yaml
scipy not imported remove from environment.yml and meta.yaml
adlfs not imported, but has comment ‘for azure blob filesystem’ clarify
python-blosc not imported, but has comment ‘required by Coiled’ clarify -- see below
fontconfig not imported remove from environment.yml and meta.yaml
lz4 not imported, but has comment ‘required by Coiled’ clarify -- see below
requests-oauthlib not imported remove from environment.yml and meta.yaml

Notes:

  • We'll also need to update DEFAULT_DEPENDENCY_NAMES in versions.py.
  • From the above list, it appears that we specify explicit dependencies on two transitive (soft?) dependencies of the coiled package, but the coiled package is only a soft dependency (i.e. we try to import locally at runtime if/when it's actually needed and raise an error if it's not available). If these are hard dependencies of coiled, we can probably omit them entirely: if coiled is importable, they must be present. If they're soft dependencies of coiled which we require for our use case, we should handle them the same way we handle the coiled package itself -- i.e. check at runtime if/when required.
@pont-us pont-us self-assigned this Mar 12, 2024
@pont-us
Copy link
Member Author

pont-us commented Mar 15, 2024

Updates:

  • After investigation and discussion, it seems fairly definite that python-blosc and lz4 aren't needed any more (if they ever were), so they should be removed entirely from the dependencies.
  • python-blosc was the only package being installed from a non-conda-forge channel (since the Python-3.12-compatible conda-forge build of python-blosc is currently, and hopefully temporarily, broken), so having removed it we can also remove the defaults channel from environment.yml.
  • adlfs seems to be a soft transitive dependency required only for Azure blob support. We should document:
    • which xcube dependency is optionally using it (currently unknown)
    • the fact that it needs to be explicitly installed if Azure blob support is needed
    • any other optional/soft dependencies which affect xcube's behaviour

pont-us added a commit that referenced this issue Mar 15, 2024
These were commented as "required by Coiled", but this doesn't
seem to be the case any more.

Addresses #945.
pont-us added a commit that referenced this issue Mar 15, 2024
All xcube's dependencies are available in conda-forge, so the
defaults channel is superfluous.

Addresses #945.
pont-us added a commit that referenced this issue Mar 15, 2024
This commit mostly aligns the dependency list in environment.yml
with actual imports in the codebase. Addresses #945.

- Add botocore, cftime, and numcodecs.

- Remove fontconfig, requests-oauthlib, and scipy.
@pont-us pont-us changed the title Resolve/document dependency discrepancies in conda-forge releases Resolve/document dependency discrepancies Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant