Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin test tool versions in CI #2422

Closed
wants to merge 2 commits into from
Closed

Conversation

mthrok
Copy link
Collaborator

@mthrok mthrok commented May 30, 2022

All the unittests jobs are failing due to import error related to protobuf and scipy.
This commit pins the versions of them to an older version.

protobuf

https://app.circleci.com/pipelines/github/pytorch/audio/10979/workflows/42005226-ca7e-471c-80f4-db09f4bd2089/jobs/692078

E   TypeError: Descriptors cannot not be created directly.
E   If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E   If you cannot immediately regenerate your protos, some other possible workarounds are:
E    1. Downgrade the protobuf package to 3.20.x or lower.
E    2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E   More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

protocolbuffers/protobuf#10051
Lightning-AI/pytorch-lightning#13159

scipy (pypocketfft)

1.8.1 is causing issue.

https://app.circleci.com/pipelines/github/pytorch/audio/10980/workflows/470a9361-4cc5-4d7c-9264-28fc8b86f1cb/jobs/692267

```
../env/lib/python3.9/site-packages/librosa/core/audio.py:11: in <module>
    import scipy.signal
../env/lib/python3.9/site-packages/scipy/signal/__init__.py:309: in <module>
    from . import _sigtools, windows
../env/lib/python3.9/site-packages/scipy/signal/windows/__init__.py:41: in <module>
    from ._windows import *
../env/lib/python3.9/site-packages/scipy/signal/windows/_windows.py:7: in <module>
    from scipy import linalg, special, fft as sp_fft
../env/lib/python3.9/site-packages/scipy/fft/__init__.py:91: in <module>
    from ._helper import next_fast_len
../env/lib/python3.9/site-packages/scipy/fft/_helper.py:3: in <module>
    from ._pocketfft import helper as _helper
../env/lib/python3.9/site-packages/scipy/fft/_pocketfft/__init__.py:3: in <module>
    from .basic import *
../env/lib/python3.9/site-packages/scipy/fft/_pocketfft/basic.py:6: in <module>
    from . import pypocketfft as pfft
E   ImportError: /home/circleci/project/env/lib/python3.9/site-packages/torch/lib/../../../.././libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/circleci/project/env/lib/python3.9/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-39-x86_64-linux-gnu.so)

https://app.circleci.com/pipelines/github/pytorch/audio/10979/workflows/42005226-ca7e-471c-80f4-db09f4bd2089/jobs/692078

```
E   TypeError: Descriptors cannot not be created directly.
E   If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E   If you cannot immediately regenerate your protos, some other possible workarounds are:
E    1. Downgrade the protobuf package to 3.20.x or lower.
E    2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E   More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
```

protocolbuffers/protobuf#10051
Lightning-AI/pytorch-lightning#13159
@facebook-github-bot
Copy link
Contributor

@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

1.8.1 is causing issue.

https://app.circleci.com/pipelines/github/pytorch/audio/10980/workflows/470a9361-4cc5-4d7c-9264-28fc8b86f1cb/jobs/692267

```
../env/lib/python3.9/site-packages/librosa/core/audio.py:11: in <module>
    import scipy.signal
../env/lib/python3.9/site-packages/scipy/signal/__init__.py:309: in <module>
    from . import _sigtools, windows
../env/lib/python3.9/site-packages/scipy/signal/windows/__init__.py:41: in <module>
    from ._windows import *
../env/lib/python3.9/site-packages/scipy/signal/windows/_windows.py:7: in <module>
    from scipy import linalg, special, fft as sp_fft
../env/lib/python3.9/site-packages/scipy/fft/__init__.py:91: in <module>
    from ._helper import next_fast_len
../env/lib/python3.9/site-packages/scipy/fft/_helper.py:3: in <module>
    from ._pocketfft import helper as _helper
../env/lib/python3.9/site-packages/scipy/fft/_pocketfft/__init__.py:3: in <module>
    from .basic import *
../env/lib/python3.9/site-packages/scipy/fft/_pocketfft/basic.py:6: in <module>
    from . import pypocketfft as pfft
E   ImportError: /home/circleci/project/env/lib/python3.9/site-packages/torch/lib/../../../.././libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/circleci/project/env/lib/python3.9/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-39-x86_64-linux-gnu.so)
```
@mthrok mthrok changed the title Pin prototbuf version Pin test tool versions in CI May 30, 2022
@facebook-github-bot
Copy link
Contributor

@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@mthrok mthrok marked this pull request as ready for review May 30, 2022 17:33
Copy link
Contributor

@atalman atalman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please fix lint before merging.

@mthrok
Copy link
Collaborator Author

mthrok commented May 30, 2022

LGTM. Please fix lint before merging.

Let's follow up on that one. Lint is broken due to the discrepancy between OSS and fbcode, and we have not figured out what to do yet. (probably disabling it is the way)

@github-actions
Copy link

Hey @mthrok.
You merged this PR, but labels were not properly added. Please add a primary and secondary label (See https://github.com/pytorch/audio/blob/main/.github/process_commit.py)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants