Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python312Packages.plotly: disable checks on darwin, unbreak #347567

Merged
merged 2 commits into from
Oct 10, 2024

Conversation

pbsds
Copy link
Member

@pbsds pbsds commented Oct 9, 2024

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@pbsds pbsds marked this pull request as ready for review October 9, 2024 18:36
@sarahec
Copy link
Contributor

sarahec commented Oct 9, 2024

@ofborg build python311Packages.plotly python312Packages.plotly

@sarahec
Copy link
Contributor

sarahec commented Oct 9, 2024

During a nixpkgs-review on x86-64-Linux when building for python 3.11 Edit: also breaks in the CI, same error):

       > FAILED plotly/tests/test_optional/test_px/test_px_input.py::test_build_df_from_vaex_and_polars[vaex] - ModuleNotFoundError: No module named 'vaex'
       > FAILED plotly/tests/test_optional/test_px/test_px_input.py::test_build_df_from_vaex_and_polars[polars] - ModuleNotFoundError: No module named 'polars'
       > FAILED plotly/tests/test_optional/test_px/test_px_input.py::test_build_df_with_hover_data_from_vaex_and_polars[hover_data0-vaex] - ModuleNotFoundError: No module named 'vaex'
       > FAILED plotly/tests/test_optional/test_px/test_px_input.py::test_build_df_with_hover_data_from_vaex_and_polars[hover_data0-polars] - ModuleNotFoundError: No module named 'polars'
       > FAILED plotly/tests/test_optional/test_px/test_px_input.py::test_build_df_with_hover_data_from_vaex_and_polars[hover_data1-vaex] - ModuleNotFoundError: No module named 'vaex'
       > FAILED plotly/tests/test_optional/test_px/test_px_input.py::test_build_df_with_hover_data_from_vaex_and_polars[hover_data1-polars] - ModuleNotFoundError: No module named 'polars'

@pbsds
Copy link
Member Author

pbsds commented Oct 9, 2024

ah, i misread the logs while fixing #347564, i'll move the fix here

@Pandapip1
Copy link
Contributor

Pandapip1 commented Oct 9, 2024

nixpkgs-review pr 347567 crashes the terminal it's run in? I've replicated this three times across two different terminal emulators.

@pbsds
Copy link
Member Author

pbsds commented Oct 9, 2024

likely oom, you either need to wait for ofborg eval or install more ram

Copy link
Contributor

@Pandapip1 Pandapip1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can confirm that both python311Packages.plotly and python312Packages.plotly build. nixpkgs-review is OOMing.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/2029

@sarahec
Copy link
Contributor

sarahec commented Oct 9, 2024

ah, i misread the logs while fixing #347564, i'll move the fix here

I've done that too. An easy mistake!

P.S. Test error is fixed, just awaiting nixpkgs-review

@sarahec
Copy link
Contributor

sarahec commented Oct 9, 2024

Result of nixpkgs-review pr 347567 run on x86_64-linux 1

8 packages marked as broken and skipped:
  • python311Packages.pymatgen
  • python311Packages.pymatgen.dist
  • python311Packages.sumo
  • python311Packages.sumo.dist
  • python312Packages.pymatgen
  • python312Packages.pymatgen.dist
  • python312Packages.sumo
  • python312Packages.sumo.dist
122 packages built:
  • apkleaks
  • apkleaks.dist
  • checkov
  • checkov.dist
  • cve-bin-tool
  • cve-bin-tool.dist
  • das
  • das.dist
  • deeptools
  • deeptools.dist
  • hp2p
  • jadx
  • python311Packages.ax-platform
  • python311Packages.ax-platform.dist
  • python311Packages.catboost
  • python311Packages.catboost.dist
  • python311Packages.chart-studio
  • python311Packages.chart-studio.dist
  • python311Packages.compressai
  • python311Packages.compressai.dist
  • python311Packages.cufflinks
  • python311Packages.cufflinks.dist
  • python311Packages.dash
  • python311Packages.dash-bootstrap-components
  • python311Packages.dash-bootstrap-components.dist
  • python311Packages.dash.dist
  • python311Packages.experiment-utilities
  • python311Packages.experiment-utilities.dist
  • python311Packages.explorerscript
  • python311Packages.explorerscript.dist
  • python311Packages.igraph
  • python311Packages.igraph.dist
  • python311Packages.kmapper
  • python311Packages.kmapper.dist
  • python311Packages.leidenalg
  • python311Packages.leidenalg.dist
  • python311Packages.meshtastic
  • python311Packages.meshtastic.dist
  • python311Packages.niaarm
  • python311Packages.niaarm.dist
  • python311Packages.optuna
  • python311Packages.optuna.dist
  • python311Packages.orange3
  • python311Packages.orange3.dist
  • python311Packages.plotly
  • python311Packages.plotly.dist
  • python311Packages.pyannote-audio
  • python311Packages.pyannote-audio.dist
  • python311Packages.pyannote-pipeline
  • python311Packages.pyannote-pipeline.dist
  • python311Packages.reflex
  • python311Packages.reflex.dist
  • python311Packages.scikit-tda
  • python311Packages.scikit-tda.dist
  • python311Packages.skytemple-dtef
  • python311Packages.skytemple-dtef.dist
  • python311Packages.skytemple-files
  • python311Packages.skytemple-files.dist
  • python311Packages.skytemple-ssb-debugger
  • python311Packages.skytemple-ssb-debugger.dist
  • python311Packages.synergy
  • python311Packages.synergy.dist
  • python311Packages.textnets
  • python311Packages.textnets.dist
  • python312Packages.ax-platform
  • python312Packages.ax-platform.dist
  • python312Packages.catboost
  • python312Packages.catboost.dist
  • python312Packages.chart-studio
  • python312Packages.chart-studio.dist
  • python312Packages.compressai
  • python312Packages.compressai.dist
  • python312Packages.cufflinks
  • python312Packages.cufflinks.dist
  • python312Packages.dash
  • python312Packages.dash-bootstrap-components
  • python312Packages.dash-bootstrap-components.dist
  • python312Packages.dash.dist
  • python312Packages.experiment-utilities
  • python312Packages.experiment-utilities.dist
  • python312Packages.explorerscript
  • python312Packages.explorerscript.dist
  • python312Packages.igraph
  • python312Packages.igraph.dist
  • python312Packages.kmapper
  • python312Packages.kmapper.dist
  • python312Packages.leidenalg
  • python312Packages.leidenalg.dist
  • python312Packages.meshtastic
  • python312Packages.meshtastic.dist
  • python312Packages.niaarm
  • python312Packages.niaarm.dist
  • python312Packages.optuna
  • python312Packages.optuna.dist
  • python312Packages.orange3
  • python312Packages.orange3.dist
  • python312Packages.plotly
  • python312Packages.plotly.dist
  • python312Packages.pyannote-audio
  • python312Packages.pyannote-audio.dist
  • python312Packages.pyannote-pipeline
  • python312Packages.pyannote-pipeline.dist
  • python312Packages.reflex
  • python312Packages.reflex.dist
  • python312Packages.scikit-tda
  • python312Packages.scikit-tda.dist
  • python312Packages.skytemple-dtef
  • python312Packages.skytemple-dtef.dist
  • python312Packages.skytemple-files
  • python312Packages.skytemple-files.dist
  • python312Packages.skytemple-ssb-debugger
  • python312Packages.skytemple-ssb-debugger.dist
  • python312Packages.synergy
  • python312Packages.synergy.dist
  • quark-engine
  • quark-engine.dist
  • skytemple
  • skytemple.dist
  • theharvester
  • theharvester.dist
  • whisper-ctranslate2
  • whisper-ctranslate2.dist

@sarahec
Copy link
Contributor

sarahec commented Oct 9, 2024

Result of nixpkgs-review pr 347567 run on aarch64-darwin 1

28 packages marked as broken and skipped:
  • python311Packages.orange3
  • python311Packages.orange3.dist
  • python311Packages.pyannote-audio
  • python311Packages.pyannote-audio.dist
  • python311Packages.pymatgen
  • python311Packages.pymatgen.dist
  • python311Packages.skytemple-dtef
  • python311Packages.skytemple-dtef.dist
  • python311Packages.skytemple-files
  • python311Packages.skytemple-files.dist
  • python311Packages.sumo
  • python311Packages.sumo.dist
  • python311Packages.textnets
  • python311Packages.textnets.dist
  • python312Packages.orange3
  • python312Packages.orange3.dist
  • python312Packages.pyannote-audio
  • python312Packages.pyannote-audio.dist
  • python312Packages.pymatgen
  • python312Packages.pymatgen.dist
  • python312Packages.skytemple-dtef
  • python312Packages.skytemple-dtef.dist
  • python312Packages.skytemple-files
  • python312Packages.skytemple-files.dist
  • python312Packages.sumo
  • python312Packages.sumo.dist
  • whisper-ctranslate2
  • whisper-ctranslate2.dist
21 packages failed to build:
  • hp2p
  • python311Packages.compressai
  • python311Packages.compressai.dist
  • python311Packages.dash
  • python311Packages.dash-bootstrap-components
  • python311Packages.dash-bootstrap-components.dist
  • python311Packages.dash.dist
  • python311Packages.niaarm
  • python311Packages.niaarm.dist
  • python311Packages.reflex
  • python311Packages.reflex.dist
  • python311Packages.synergy
  • python311Packages.synergy.dist
  • python312Packages.compressai
  • python312Packages.compressai.dist
  • python312Packages.niaarm
  • python312Packages.niaarm.dist
  • python312Packages.reflex
  • python312Packages.reflex.dist
  • python312Packages.synergy
  • python312Packages.synergy.dist
71 packages built:
  • apkleaks
  • apkleaks.dist
  • checkov
  • checkov.dist
  • cve-bin-tool
  • cve-bin-tool.dist
  • das
  • das.dist
  • deeptools
  • deeptools.dist
  • jadx
  • python311Packages.ax-platform
  • python311Packages.ax-platform.dist
  • python311Packages.catboost
  • python311Packages.catboost.dist
  • python311Packages.chart-studio
  • python311Packages.chart-studio.dist
  • python311Packages.cufflinks
  • python311Packages.cufflinks.dist
  • python311Packages.experiment-utilities
  • python311Packages.experiment-utilities.dist
  • python311Packages.explorerscript
  • python311Packages.explorerscript.dist
  • python311Packages.igraph
  • python311Packages.igraph.dist
  • python311Packages.kmapper
  • python311Packages.kmapper.dist
  • python311Packages.leidenalg
  • python311Packages.leidenalg.dist
  • python311Packages.optuna
  • python311Packages.optuna.dist
  • python311Packages.plotly
  • python311Packages.plotly.dist
  • python311Packages.pyannote-pipeline
  • python311Packages.pyannote-pipeline.dist
  • python311Packages.scikit-tda
  • python311Packages.scikit-tda.dist
  • python312Packages.ax-platform
  • python312Packages.ax-platform.dist
  • python312Packages.catboost
  • python312Packages.catboost.dist
  • python312Packages.chart-studio
  • python312Packages.chart-studio.dist
  • python312Packages.cufflinks
  • python312Packages.cufflinks.dist
  • python312Packages.dash
  • python312Packages.dash-bootstrap-components
  • python312Packages.dash-bootstrap-components.dist
  • python312Packages.dash.dist
  • python312Packages.experiment-utilities
  • python312Packages.experiment-utilities.dist
  • python312Packages.explorerscript
  • python312Packages.explorerscript.dist
  • python312Packages.igraph
  • python312Packages.igraph.dist
  • python312Packages.kmapper
  • python312Packages.kmapper.dist
  • python312Packages.leidenalg
  • python312Packages.leidenalg.dist
  • python312Packages.optuna
  • python312Packages.optuna.dist
  • python312Packages.plotly
  • python312Packages.plotly.dist
  • python312Packages.pyannote-pipeline
  • python312Packages.pyannote-pipeline.dist
  • python312Packages.scikit-tda
  • python312Packages.scikit-tda.dist
  • quark-engine
  • quark-engine.dist
  • theharvester
  • theharvester.dist

@ofborg ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Oct 9, 2024
@pbsds
Copy link
Member Author

pbsds commented Oct 9, 2024

darwin derivation is now unchanged from before the checkphase was added. Can be addressed in a followup, less time-pressed PR

@pbsds
Copy link
Member Author

pbsds commented Oct 9, 2024

seems to be a spike in ofborg eval queue, it might take a while

Copy link
Contributor

@sarahec sarahec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with the expectation you'll unblock Darwin soon. Letting it be deferred due to the urgency of unbreaking the build.

@sarahec
Copy link
Contributor

sarahec commented Oct 10, 2024

darwin derivation is now unchanged from before the checkphase was added

It looks like orca is the blocker (which has been deprecated in favor of kaleido). You'll need to set it up as an optional dependency.

@Pandapip1 Pandapip1 removed their request for review October 10, 2024 00:33
Copy link
Contributor

@sarahec sarahec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it all work properly on Darwin without breaking Linux. The review tool wouldn't let me put optional-dependencies in the right place beneath dependencies, so please do so.

If we're rebuilding a slew of Darwin packages, might as well do it right.

@@ -1,5 +1,6 @@
{
lib,
stdenv,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
stdenv,

Comment on lines +74 to +76
# the check inputs are broken on darwin
doCheck = !stdenv.hostPlatform.isDarwin;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# the check inputs are broken on darwin
doCheck = !stdenv.hostPlatform.isDarwin;
optional-dependencies = {
orca = [ orca ];
};

and change nativeCheckInputs to checkInputs

@@ -116,6 +120,9 @@ buildPythonPackage rec {
"test_dependencies_not_imported"
# FAILED test_init/test_lazy_imports.py::test_lazy_imports - AssertionError: assert 'plotly' not in {'IPython': <module 'IPython' from '...
"test_lazy_imports"
# requires vaex and polars, vaex is not packaged
"test_build_df_from_vaex_and_polars"
"test_build_df_with_hover_data_from_vaex_and_polars"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"test_build_df_with_hover_data_from_vaex_and_polars"
"test_build_df_with_hover_data_from_vaex_and_polars"
# Cannot launch kaleido as a subprocess. This would need a passthru test.
"test_kaleido_engine_to_image_returns_bytes"
"test_kaleido_fulljson"
"test_bytesio"
"test_png_renderer_mimetype"
"test_svg_renderer_show"
"test_pdf_renderer_show_override"
"test_mimetype_combination"
# Cannot run on systes without orca
"test_scraper"
"test_bytesio"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about

Suggested change
"test_build_df_with_hover_data_from_vaex_and_polars"
"test_build_df_with_hover_data_from_vaex_and_polars"
] ++ lib.optionals stdenv.hostPlatform.isDarwin [
# Cannot launch kaleido as a subprocess. This would need a passthru test.
"test_kaleido_engine_to_image_returns_bytes"
"test_kaleido_fulljson"
"test_bytesio"
"test_png_renderer_mimetype"
"test_svg_renderer_show"
"test_pdf_renderer_show_override"
"test_mimetype_combination"
# Cannot run on systes without orca
"test_scraper"
"test_bytesio"
];

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Pandapip1 that's even better.

@Pandapip1
Copy link
Contributor

Pandapip1 commented Oct 10, 2024

I would argue the important thing is getting this merged to fix the broken builds; 11-100 additional rebuilds aren't worth blocking this PR.

As per usual, the aarch64-darwin tests have stalled; this PR should be ready for merge.

@GaetanLepage
Copy link
Contributor

@sarahec
Copy link
Contributor

sarahec commented Oct 10, 2024

@GaetanLepage
Copy link
Contributor

On Darwin? Sandboxed?

It looks like my change did not affect darwin platforms for some reason, so I don't know.
#347732 (comment)

@pbsds
Copy link
Member Author

pbsds commented Oct 10, 2024

darwin was broken since some check inputs dont eval on darwin, disabling checkPhase makes it so the nativeCheckInputs are not passed to the derivation

@sarahec
Copy link
Contributor

sarahec commented Oct 10, 2024

In any case, I have no objection to this getting merged as is plus a follow-up patch. Or the fixes can be added (easily) and this is done the right way.

@pbsds
Copy link
Member Author

pbsds commented Oct 10, 2024

i held of my own cleanup fix due to the slow eval, i'll open a followup

@pbsds pbsds merged commit 6c21cff into NixOS:master Oct 10, 2024
35 of 36 checks passed
@pbsds
Copy link
Member Author

pbsds commented Oct 10, 2024

please view #347763

@Pandapip1
Copy link
Contributor

Pandapip1 commented Oct 10, 2024

It was broken on Darwin for all Python versions.

Edit: Whoops, I see you've already responded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants