python312Packages.blis: 0.7.11 -> 1.0.0 #333221

dotlambda · 2024-08-08T12:29:25Z

Description of changes

Diff: explosion/cython-blis@v0.7.11...release-v1.0.0

Changelog: https://github.com/explosion/cython-blis/releases/tag/release-v1.0.0

Version 1.0 depends on numpy 2.0. C.f. #327437.

Things done

Add a 👍 reaction to pull requests you find important.

doronbehar · 2024-08-08T12:35:15Z

Changes look good. Why is this a draft?

dotlambda · 2024-08-08T12:37:30Z

Changes look good. Why is this a draft?

It breaks thinc and we can't depend on numpy_2 inside python3Packages.

doronbehar · 2024-08-08T12:39:11Z

Changes look good. Why is this a draft?

It breaks thinc and we can't depend on numpy_2 inside python3Packages.

And thinc can't depend on numpy_2?

SuperSandro2000 · 2024-08-28T12:30:50Z

It breaks thinc

Then we need an overlay there

we can't depend on numpy_2 inside python3Packages.

I think we can since we catch conflicts now. :) Please give it a shoot.

dotlambda · 2024-08-29T00:42:05Z

It breaks thinc

Then we need an overlay there

Thinc is a Python package, that won't work.

dotlambda · 2024-08-29T00:55:59Z

The package now works with both numpy and numpy_2.

dotlambda · 2024-08-29T00:56:36Z

pkgs/development/python-modules/blis/default.nix

-    # Do not update to BLIS 0.9.x until the following issue is resolved:
-    # https://github.com/explosion/thinc/issues/771#issuecomment-1255825935


@davidak Can thinc's blis pin be removed?

Actually upstream removed it in explosion/thinc@efda3b4 so this should be fine.

SuperSandro2000

LGTM, can be merged of war resolve the question above this

doronbehar · 2024-08-30T06:03:49Z

pkgs/development/python-modules/blis/default.nix

+    numpy
  ];

-  propagatedBuildInputs = [ numpy ];
+  dependencies = [ numpy ];


Isn't that a bit weird to put it in both places?

That's what upstream specifies. IIRC numpy is imported in setup.py.

That's what upstream specifies. IIRC numpy is imported in setup.py.

Hmm if so, the only good reason to still put Numpy also in the native inputs is for cross compilation, which will not work on NixOS with the way upstream uses the native Numpy in their setup.py:

https://github.com/explosion/cython-blis/blob/b33c2ed9cf7ed51d999a926c4950dd8dec4d0286/setup.py#L86C39-L86C58

I think that for cross compilation you'd have to patch that line, and for non-cross compilation, you'd get the host's Numpy available anyway, so there is no need for numpy in the build-system list IMO.

We plan to separate build dependencies and runtime dependencies in the future. #272178
So, we need to add numpy to both build-system and dependencies.

We plan to separate build dependencies and runtime dependencies in the future. #272178 So, we need to add numpy to both build-system and dependencies.

I'm not sure I agree with everything said there. Even if I had agreed, #272179 should have been finished before you'd apply that policy upon other packages.

The order is reversed.
Without proper separation, we cannot start #272179.

you'd get the host's Numpy available anyway, so there is no need for numpy in the build-system list IMO.

There is because in the future dependencies might not be available at build time in order to reduce the number of rebuilds cause by updates, overrides, etc.

you'd get the host's Numpy available anyway, so there is no need for numpy in the build-system list IMO.

There is because in the future dependencies might not be available at build time in order to reduce the number of rebuilds cause by updates, overrides, etc.

I support this motivation and this goal, but in this case, due to the goal of upstream's import numpy in setup.py, it is completely incorrect to add numpy to build-system, but rather you should patch (downstream or upstream) the file to support specifying manually the host machine's Numpy headers.

I agree that cross doesn't work as is. But at least this continues to work if dependencies are unavailable.

I agree that cross doesn't work as is. But at least this continues to work if dependencies are unavailable.

OK I understand now. Still it would have been nice if you had put a TODO comment or something. Also the intentions of #272178 should be communicated in the documentation IMO, even though not everything is implemented yet.

pkgs/development/python-modules/blis/default.nix

Diff: explosion/cython-blis@v0.7.11...release-v1.0.0 Changelog: https://github.com/explosion/cython-blis/releases/tag/release-v1.0.0

Changelog: https://github.com/explosion/thinc/releases/tag/v8.2.4 https://github.com/explosion/thinc/releases/tag/v8.2.5 https://github.com/explosion/thinc/releases/tag/v8.3.0

Changelog: https://github.com/explosion/spaCy/releases/tag/release-v3.7.6

dotlambda · 2024-09-02T20:22:45Z

Result of nixpkgs-review pr 333221 run on x86_64-linux 1

2 packages failed to build:

python311Packages.private-gpt
python311Packages.private-gpt.dist

88 packages built:

aider-chat
aider-chat.dist
python311Packages.blis
python311Packages.blis.dist
python311Packages.fastai
python311Packages.fastai.dist
python311Packages.llama-index
python311Packages.llama-index-agent-openai
python311Packages.llama-index-agent-openai.dist
python311Packages.llama-index-cli
python311Packages.llama-index-cli.dist
python311Packages.llama-index-core
python311Packages.llama-index-core.dist
python311Packages.llama-index-embeddings-gemini
python311Packages.llama-index-embeddings-gemini.dist
python311Packages.llama-index-embeddings-google
python311Packages.llama-index-embeddings-google.dist
python311Packages.llama-index-embeddings-huggingface
python311Packages.llama-index-embeddings-huggingface.dist
python311Packages.llama-index-embeddings-ollama
python311Packages.llama-index-embeddings-ollama.dist
python311Packages.llama-index-embeddings-openai
python311Packages.llama-index-embeddings-openai.dist
python311Packages.llama-index-graph-stores-nebula
python311Packages.llama-index-graph-stores-nebula.dist
python311Packages.llama-index-graph-stores-neo4j
python311Packages.llama-index-graph-stores-neo4j.dist
python311Packages.llama-index-graph-stores-neptune
python311Packages.llama-index-graph-stores-neptune.dist
python311Packages.llama-index-indices-managed-llama-cloud
python311Packages.llama-index-indices-managed-llama-cloud.dist
python311Packages.llama-index-legacy
python311Packages.llama-index-legacy.dist
python311Packages.llama-index-llms-ollama
python311Packages.llama-index-llms-ollama.dist
python311Packages.llama-index-llms-openai
python311Packages.llama-index-llms-openai-like
python311Packages.llama-index-llms-openai-like.dist
python311Packages.llama-index-llms-openai.dist
python311Packages.llama-index-multi-modal-llms-openai
python311Packages.llama-index-multi-modal-llms-openai.dist
python311Packages.llama-index-program-openai
python311Packages.llama-index-program-openai.dist
python311Packages.llama-index-question-gen-openai
python311Packages.llama-index-question-gen-openai.dist
python311Packages.llama-index-readers-database
python311Packages.llama-index-readers-database.dist
python311Packages.llama-index-readers-file
python311Packages.llama-index-readers-file.dist
python311Packages.llama-index-readers-json
python311Packages.llama-index-readers-json.dist
python311Packages.llama-index-readers-llama-parse
python311Packages.llama-index-readers-llama-parse.dist
python311Packages.llama-index-readers-s3
python311Packages.llama-index-readers-s3.dist
python311Packages.llama-index-readers-twitter
python311Packages.llama-index-readers-twitter.dist
python311Packages.llama-index-readers-txtai
python311Packages.llama-index-readers-txtai.dist
python311Packages.llama-index-readers-weather
python311Packages.llama-index-readers-weather.dist
python311Packages.llama-index-vector-stores-chroma
python311Packages.llama-index-vector-stores-chroma.dist
python311Packages.llama-index-vector-stores-google
python311Packages.llama-index-vector-stores-google.dist
python311Packages.llama-index-vector-stores-postgres
python311Packages.llama-index-vector-stores-postgres.dist
python311Packages.llama-index-vector-stores-qdrant
python311Packages.llama-index-vector-stores-qdrant.dist
python311Packages.llama-index.dist
python311Packages.llama-parse
python311Packages.llama-parse.dist
python311Packages.spacy
python311Packages.spacy-lookups-data
python311Packages.spacy-lookups-data.dist
python311Packages.spacy-transformers
python311Packages.spacy-transformers.dist
python311Packages.spacy.dist
python311Packages.textacy
python311Packages.textacy.dist
python311Packages.textnets
python311Packages.textnets.dist
python311Packages.thinc
python311Packages.thinc.dist
python312Packages.blis
python312Packages.blis.dist
python312Packages.thinc
python312Packages.thinc.dist

nixos-discourse · 2024-09-08T08:42:55Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/several-comments-about-priorities-and-new-policies-in-the-python-ecosystem/51790/1

nixos-discourse · 2024-09-09T06:54:12Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/several-comments-about-priorities-and-new-policies-in-the-python-ecosystem/51790/9

dotlambda requested review from doronbehar and NickCao August 8, 2024 12:29

github-actions bot added the 6.topic: python label Aug 8, 2024

ofborg bot added 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Aug 8, 2024

dotlambda force-pushed the python3Packages.blis branch from 07cecaa to 18e8282 Compare August 29, 2024 00:55

dotlambda commented Aug 29, 2024

View reviewed changes

SuperSandro2000 approved these changes Aug 29, 2024

View reviewed changes

doronbehar reviewed Aug 30, 2024

View reviewed changes

doronbehar mentioned this pull request Sep 1, 2024

python3Packages: don't propagate numpy_1 if also numpy_2 is supported #338334

Closed

13 tasks

python312Packages.blis: 0.7.11 -> 1.0.0

8719eb4

Diff: explosion/cython-blis@v0.7.11...release-v1.0.0 Changelog: https://github.com/explosion/cython-blis/releases/tag/release-v1.0.0

dotlambda force-pushed the python3Packages.blis branch from 18e8282 to 8719eb4 Compare September 1, 2024 15:35

dotlambda marked this pull request as ready for review September 2, 2024 00:01

python312Packages.thinc: 8.2.3 -> 8.3.0

1f03dd2

Changelog: https://github.com/explosion/thinc/releases/tag/v8.2.4 https://github.com/explosion/thinc/releases/tag/v8.2.5 https://github.com/explosion/thinc/releases/tag/v8.3.0

dotlambda force-pushed the python3Packages.blis branch from 1fbe72e to 1f03dd2 Compare September 2, 2024 00:04

ofborg bot requested a review from aborsu September 2, 2024 01:17

dotlambda added 2 commits September 2, 2024 13:04

python311Packages.spacy: unpin thinc

83c48b4

python311Packages.spacy: 3.7.5 -> 3.7.6

d3f0837

Changelog: https://github.com/explosion/spaCy/releases/tag/release-v3.7.6

dotlambda force-pushed the python3Packages.blis branch from 01addcc to d3f0837 Compare September 2, 2024 20:05

dotlambda mentioned this pull request Sep 4, 2024

python312Packages.scipy: 1.14.0 -> 1.14.1 #336290

Merged

11 tasks

dotlambda merged commit 70e145e into NixOS:master Sep 5, 2024
25 of 26 checks passed

dotlambda deleted the python3Packages.blis branch September 5, 2024 22:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python312Packages.blis: 0.7.11 -> 1.0.0 #333221

python312Packages.blis: 0.7.11 -> 1.0.0 #333221

dotlambda commented Aug 8, 2024

doronbehar commented Aug 8, 2024

dotlambda commented Aug 8, 2024

doronbehar commented Aug 8, 2024

SuperSandro2000 commented Aug 28, 2024

dotlambda commented Aug 29, 2024

dotlambda commented Aug 29, 2024

dotlambda Aug 29, 2024 •

edited

Loading

dotlambda Sep 2, 2024

SuperSandro2000 left a comment

doronbehar Aug 30, 2024

dotlambda Aug 30, 2024

doronbehar Aug 31, 2024

natsukium Sep 1, 2024

doronbehar Sep 1, 2024

natsukium Sep 1, 2024

dotlambda Sep 9, 2024

doronbehar Sep 9, 2024

dotlambda Sep 9, 2024

doronbehar Sep 9, 2024

dotlambda commented Sep 2, 2024

nixos-discourse commented Sep 8, 2024

nixos-discourse commented Sep 9, 2024

		# Do not update to BLIS 0.9.x until the following issue is resolved:
		# https://github.com/explosion/thinc/issues/771#issuecomment-1255825935

python312Packages.blis: 0.7.11 -> 1.0.0 #333221

python312Packages.blis: 0.7.11 -> 1.0.0 #333221

Conversation

dotlambda commented Aug 8, 2024

Description of changes

Things done

doronbehar commented Aug 8, 2024

dotlambda commented Aug 8, 2024

doronbehar commented Aug 8, 2024

SuperSandro2000 commented Aug 28, 2024

dotlambda commented Aug 29, 2024

dotlambda commented Aug 29, 2024

dotlambda Aug 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SuperSandro2000 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotlambda commented Sep 2, 2024

nixos-discourse commented Sep 8, 2024

nixos-discourse commented Sep 9, 2024

dotlambda Aug 29, 2024 •

edited

Loading