Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix update docs workflow step failure #359

Merged
merged 8 commits into from
Feb 11, 2022
Merged

Conversation

earmenda
Copy link
Contributor

@earmenda earmenda commented Feb 11, 2022

Closes #358

Code Changes

  • Set python version for publish_docs to 3.9

Steps to Confirm

  • Will use this PR to test the build

Pre-Merge Checklist

  • All CI Pipelines Succeeded
  • Documentation Updated

Description Of Changes

Will use this PR to trigger a failure and then confirm a fix. Commenting out the deploy part of the action:

name: Publish updated docs to gh-pages

on:
  push:
    branches:
      - earmenda-fix-docs-gh-action
    paths:
      # - "docs/**"
      - "*/**"

jobs:
  publish_docs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
      - run: pip install -r docs/fides/requirements.txt
      - name: Build docs
        run: make docs-build
      # - name: Publish docs
      #   run: mkdocs gh-deploy -v -f docs/fides/mkdocs.yml --force
      #   env:
      #     PROD_PUBLISH: true

Run failure: https://github.com/ethyca/fides/runs/5150462980

@earmenda earmenda added the bug Something isn't working label Feb 11, 2022
@earmenda earmenda added this to the fidesctl 1.4.0 milestone Feb 11, 2022
@earmenda earmenda self-assigned this Feb 11, 2022
@earmenda
Copy link
Contributor Author

Focusing on this part of the failure:

        × Building wheel for numpy (pyproject.toml) did not run successfully.
        │ exit code: 1
        ╰─> [901 lines of output]
            setup.py:67: RuntimeWarning: NumPy 1.19.3 may not yet support Python 3.10.
              warnings.warn(

@earmenda
Copy link
Contributor Author

Setting this in the docs requirement.txt was not enough:

numpy>=1.22.0
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 25.3/25.3 MB 80.1 MB/s eta 0:00:00
Collecting pandas==1.3
  Downloading pandas-1.3.0.tar.gz (4.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB [119](https://github.com/ethyca/fides/runs/5150518616?check_suite_focus=true#step:4:119).4 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: still running...
  Installing build dependencies: finished with status 'error'
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [937 lines of output]
      Ignoring numpy: markers 'python_version == "3.7" and (platform_machine != "arm64" or platform_system != "Darwin") and platform_machine != "aarch64"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.8" and (platform_machine != "arm64" or platform_system != "Darwin") and platform_machine != "aarch64"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.7" and platform_machine == "aarch64"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.8" and platform_machine == "aarch64"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.8" and platform_machine == "arm64" and platform_system == "Darwin"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.9" and platform_machine == "arm64" and platform_system == "Darwin"' don't match your environment
      Collecting setuptools>=38.6.0
        Using cached setuptools-60.8.2-py3-none-any.whl (1.1 MB)
      Collecting wheel
        Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
      Collecting Cython<3,>=0.29.21
        Using cached Cython-0.29.27-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.9 MB)
      Collecting numpy==1.19.3
        Downloading numpy-1.19.3.zip (7.3 MB)

@earmenda
Copy link
Contributor Author

pandas-dev/pandas#43729

I think pandas didnt' update dependencies for 3.10.0 until 1.3.4

@earmenda
Copy link
Contributor Author

I think I'm a little confused on how this works. In order to use a different pandas version it would have to be published to pypi not just change the fidesctl requirements file.

Side note: Doesn't that mean this job should depend on the publish package job?

@earmenda
Copy link
Contributor Author

It's not clear why this started happening but I think something changed between ubuntu versions where this old error is now being propagated and failing the job. If you look at older jobs this error is not new:
https://github.com/ethyca/fides/runs/5028305041

I think rather than trying to figure that out we can just address the error we've been seeing:

ERROR: Failed building wheel for numpy
      Failed to build numpy
      ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
      [end of output]

This is due to using python 3.10 numpy/numpy#19033:

            setup.py:67: RuntimeWarning: NumPy 1.19.3 may not yet support Python 3.10.

The dependency on NumPy 1.19.3 comes from pandas 1.3.0.

Sooo we have two options:

I think to update pandas we would also have to publish a new version of fidesctl to pypi so for now I'm okay just setting the python version.

@earmenda
Copy link
Contributor Author

earmenda commented Feb 11, 2022

@ThomasLaPiana

What do you think of the two options above? I am partially thinking that we should push a new build of fidesctl to pypi with a newer pandas version. It is possible to use fidesctl 1.3.1 with python 3.10 and our hardcoded version of pandas(1.3.0) is not compatible with python 3.10. I don't know why this isnt causing more issues but I'm pretty sure that's why our docs python build is failing.

@ThomasLaPiana
Copy link
Contributor

@earmenda let's both pin the python version and bump the pandas version in this PR. we can remove the python version pin in a later PR and this will also set up the fix for 1.4.0 release

@earmenda
Copy link
Contributor Author

@ThomasLaPiana sounds good to me, updated. Added a comment to remove the explicit version later on also

Copy link
Contributor

@ThomasLaPiana ThomasLaPiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, feel free to merge when tests pass. thanks for this fix!

@earmenda earmenda merged commit fb1e2c8 into main Feb 11, 2022
@earmenda earmenda deleted the earmenda-fix-docs-gh-action branch February 11, 2022 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Publish updated docs workflow step failure
2 participants