Set sharktank package deps during package build reliably. #440
Conversation
We could mirror these between requirements.txt and pyproject.toml or keep them dynamic. For packaging, we don't want to set explicit versions unless absolutely required. Developers may want to pin to a specific version to make testing consistent.
There was a problem hiding this comment.
We could mirror these between requirements.txt and pyproject.toml or keep them dynamic. For packaging, we don't want to set explicit versions unless absolutely required. Developers may want to pin to a specific version to make testing consistent.
@stellaraccident opinions?
There was a problem hiding this comment.
Industry guidance:
-
https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-and-requirements
-
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#dynamic-metadata
Library packagers are discouraged from using overly strict (or “locked”) dependency versions in their dependencies and optional-dependencies.
| @@ -1,7 +1,12 @@ | |||
| iree-turbine | |||
|
|
|||
| # Runtime deps. | |||
| gguf==0.6.0 | |||
| numpy==1.26.3 | |||
There was a problem hiding this comment.
I ran into errors trying to unpin these versions (replacing == with >=). Sample logs: https://github.com/nod-ai/SHARK-Platform/actions/runs/11714432277/job/32629101709#step:6:64
Maybe one of the deps doesn't support numpy 2.0? I can play dependency-updates-whack-a-mole to try unpinning some of the packages I guess...?
There was a problem hiding this comment.
We probably need to, yes :/ Can help with that on Monday. Should probably fill a follow up issue to keep track.
| @@ -1,7 +1,12 @@ | |||
| iree-turbine | |||
There was a problem hiding this comment.
As part of this I added a dep on
iree-turbineto sharktank'srequirements.txtand then switched workflows and user guides to installsharktankfirst and then install a newer/local version from git if desired.
@stellaraccident is this set of changes okay?
| @@ -1,7 +1,12 @@ | |||
| iree-turbine | |||
|
|
|||
| # Runtime deps. | |||
| gguf==0.6.0 | |||
| numpy==1.26.3 | |||
There was a problem hiding this comment.
We probably need to, yes :/ Can help with that on Monday. Should probably fill a follow up issue to keep track.
Co-authored-by: Marius Brehler <marius.brehler@gmail.com>
ScottTodd
force-pushed
the
sharktank-deps-package
branch
from
887fb77 to
7ee48ae
Compare
Progress on #294.
We could mirror requirements between
requirements.txtandpyproject.tomlor keep them dynamic. For packaging, we probably don't want to set explicit versions unless absolutely required. Developers may want to pin to a specific version to make testing consistent though.As part of this I added a dep on
iree-turbineto sharktank'srequirements.txtand then switched workflows and user guides to installsharktankfirst and then install a newer/local version from git if desired.