Setting up shop with SetupShop #200

blinkdog · 2021-05-27T07:43:23Z

Improving LTA infrastructure by using the wipac-dev-tools created by @ric-evans

lgtm-com · 2021-05-27T07:56:29Z

This pull request introduces 1 alert when merging 07d6078 into f274bc4 - view on LGTM.com

new alerts:

1 for Module is imported with 'import' and 'import from'

lgtm-com · 2021-05-27T08:23:39Z

This pull request introduces 1 alert when merging 4ded992 into f274bc4 - view on LGTM.com

new alerts:

1 for Module is imported with 'import' and 'import from'

…e-release

blinkdog · 2021-05-27T08:44:12Z

I tried specifying the version of numpy to use in requirements.txt, but the install process "helped" me get the numpy 1.21.0rc1 pre-release that doesn't seem to be compatible with Python 3.6...

Installed /opt/hostedtoolcache/Python/3.6.13/x64/lib/python3.6/site-packages/prometheus_client-0.10.1-py3.6.egg
Searching for numpy>=1.20.3
Reading https://pypi.org/simple/numpy/
Downloading https://files.pythonhosted.org/packages/04/2f/951f612fdac05f36b115c1ee8be62a60498034e03d43700dafa4b69504ff/numpy-1.21.0rc1.zip#sha256=f5e7cd7068df4aa803be68edc7c6fc5e3ed934d53a7ab1f21539fb5925e0172e
Best match: numpy 1.21.0rc1
Processing numpy-1.21.0rc1.zip
Writing /tmp/easy_install-arjbwjxx/numpy-1.21.0rc1/setup.cfg
Running numpy-1.21.0rc1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-arjbwjxx/numpy-1.21.0rc1/egg-dist-tmp-jy0yyi5q
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.6.13/x64/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/opt/hostedtoolcache/Python/3.6.13/x64/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/opt/hostedtoolcache/Python/3.6.13/x64/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
    _execfile(setup_script, ns)
  File "/opt/hostedtoolcache/Python/3.6.13/x64/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-arjbwjxx/numpy-1.21.0rc1/setup.py", line 33, in <module>
    "Topic :: System :: Distributed Computing",
RuntimeError: Python version >= 3.7 required.

D'oh!

ric-evans · 2021-05-27T15:07:45Z

setup.py

+shop = SetupShop(
+ "lta",
+ os.path.abspath(os.path.dirname(__file__)),
+ ((3, 6), (3, 10)),


I'm not sure how the GH Action will act with grabbing 3.10. But I see there's an issue

running the bash lines locally, I get 3.6,3.7,3.8,3.9,3.10 so I'm inclined to think that this is happening because 3.10 has not been released. Or there's a bug in the tool I use (https://github.com/actions/setup-python).

relevant: actions/setup-python#177

ahh I found the issue...

#200 (comment)

ric-evans · 2021-05-27T15:16:35Z

I tried specifying the version of numpy to use in requirements.txt, but the install process "helped" me get the numpy 1.21.0rc1 pre-release that doesn't seem to be compatible with Python 3.6...

Yeah, that's a tricky one:

All required packages (install_requires list) are found by parsing requirements.txt. PyPI packages are assumed to be backwards-compatible, so these use the indicated version as a MINIMAL requirement (== is replaced with >=).

There's a way to hack around this, but that's the last resort, especially for something stable like LTA. I recommend bumping the minimum up to 3.7, since this probably isn't something that numpy will resolve.

ric-evans

see comments

ric-evans · 2021-05-27T17:32:12Z

.github/workflows/try-setup-install.yml

+name: Try Setup Install
+
+on: [push]
+
+jobs:
+
+ gather-py3-versions:
+ runs-on: ubuntu-latest
+ outputs:
+ matrix: ${{ steps.py3_versions.outputs.matrix }}
+ steps:
+ - uses: actions/checkout@v2
+ - name: Python Versions
+ id: py3_versions
+ run: |
+ minmin_maxmax=$(grep -P "\(\(\d+, ?\d+\), ?\(\d+, ?\d+\)\)" -oh setup.py | sed 's/[^0-9]/ /g')
+ IFS=', ' read -r -a array <<< "$minmin_maxmax"
+ min_thru_max_series=$(for i in `seq ${array[1]} ${array[3]}`; do printf "3.$i,"; done | rev | cut -c 2- | rev)
+ echo ::set-output name=matrix::{\"py3_versions\":[$(echo $min_thru_max_series)]}\"
+
+ setup-py-install:
+ needs: gather-py3-versions
+ runs-on: ubuntu-latest
+
+ strategy:
+ max-parallel: 4
+ fail-fast: false
+ matrix: ${{ fromJSON(needs.gather-py3-versions.outputs.matrix) }}
+
+ steps:
+ - uses: actions/checkout@v2
+ - uses: actions/setup-python@v2
+ with:
+ python-version: ${{ matrix.py3_versions }}
+ - run: |
+ echo ${{ matrix.py3_versions }}
+ python setup.py install


Suggested change

name: Try Setup Install

on: [push]

jobs:

gather-py3-versions:

runs-on: ubuntu-latest

outputs:

matrix: ${{ steps.py3_versions.outputs.matrix }}

steps:

- uses: actions/checkout@v2

- name: Python Versions

id: py3_versions

run: |

minmin_maxmax=$(grep -P "$\(\d+, ?\d+$, ?$\d+, ?\d+$\)" -oh setup.py | sed 's/[^0-9]/ /g')

IFS=', ' read -r -a array <<< "$minmin_maxmax"

min_thru_max_series=$(for i in `seq ${array[1]} ${array[3]}`; do printf "3.$i,"; done | rev | cut -c 2- | rev)

echo ::set-output name=matrix::{\"py3_versions\":[$(echo $min_thru_max_series)]}\"

setup-py-install:

needs: gather-py3-versions

runs-on: ubuntu-latest

strategy:

max-parallel: 4

fail-fast: false

matrix: ${{ fromJSON(needs.gather-py3-versions.outputs.matrix) }}

steps:

- uses: actions/checkout@v2

- uses: actions/setup-python@v2

with:

python-version: ${{ matrix.py3_versions }}

- run: |

echo ${{ matrix.py3_versions }}

python setup.py install

name: Try Setup Install

on: [push]

jobs:

gather-py3-versions:

runs-on: ubuntu-latest

outputs:

matrix: ${{ steps.py3_versions.outputs.matrix }}

steps:

- uses: actions/checkout@v2

- name: Python Versions

id: py3_versions

run: |

minmin_maxmax=$(grep -P "$\(\d+, ?\d+$, ?$\d+, ?\d+$\)" -oh setup.py | sed 's/[^0-9]/ /g')

IFS=', ' read -r -a array <<< "$minmin_maxmax"

min_thru_max_series=$(for i in `seq ${array[1]} ${array[3]}`; do printf "'3.$i',"; done | rev | cut -c 2- | rev)

echo ::set-output name=matrix::{\"py3_versions\":[$(echo $min_thru_max_series)]}\"

echo $min_thru_max_series

pip-install:

needs: gather-py3-versions

runs-on: ubuntu-latest

strategy:

max-parallel: 4

fail-fast: false

matrix: ${{ fromJSON(needs.gather-py3-versions.outputs.matrix) }}

steps:

- uses: actions/checkout@v2

- uses: actions/setup-python@v2

with:

python-version: ${{ matrix.py3_versions }}

- run: |

pip install --upgrade pip wheel setuptools

pip install --editable .

I'm not sure what's up with the diff here, but it's a copy of https://github.com/WIPACrepo/rest-tools/blob/master/.github/workflows/try-setup-install.yml

Setting up shop with SetupShop

07d6078

blinkdog requested a review from ric-evans May 27, 2021 07:43

There's only room for one LICENSE in this here town...

6b15fe6

blinkdog added 3 commits May 27, 2021 02:56

Make sure we're on the latest pip

89ca484

Whoops, copypasta from rest-tools isn't going to work

9637a92

Python 3.16.13 is so Feb 15, 2021

4ded992

blinkdog added 2 commits May 27, 2021 03:33

Pacify LGTM, support Python 3.6 again

758ff2d

Lock numpy at 1.20.3 so it doesn't try to grab the numpy 1.21.0rc1 pr…

cee77d3

…e-release

ric-evans reviewed May 27, 2021

View reviewed changes

ric-evans requested changes May 27, 2021

View reviewed changes

ric-evans reviewed May 27, 2021

View reviewed changes

blinkdog added 2 commits June 4, 2021 16:52

Python 3.7 is the new minimum

0f32ae4

Python 3.9 is the new maximum

981d637

ric-evans approved these changes Jun 4, 2021

View reviewed changes

blinkdog merged commit 4a8b81a into master Jun 4, 2021

blinkdog deleted the setupshop branch June 4, 2021 22:32

blinkdog mentioned this pull request Jul 19, 2021

Update LTA to use SetupShop #199

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Setting up shop with SetupShop #200

Setting up shop with SetupShop #200

blinkdog commented May 27, 2021

lgtm-com bot commented May 27, 2021

lgtm-com bot commented May 27, 2021

blinkdog commented May 27, 2021

ric-evans May 27, 2021

ric-evans May 27, 2021 •

edited

Loading

ric-evans May 27, 2021

ric-evans May 27, 2021

ric-evans May 27, 2021

ric-evans commented May 27, 2021

ric-evans left a comment

ric-evans May 27, 2021

ric-evans May 27, 2021

Setting up shop with SetupShop #200

Setting up shop with SetupShop #200

Conversation

blinkdog commented May 27, 2021

lgtm-com bot commented May 27, 2021

lgtm-com bot commented May 27, 2021

blinkdog commented May 27, 2021

ric-evans May 27, 2021

Choose a reason for hiding this comment

ric-evans May 27, 2021 • edited Loading

Choose a reason for hiding this comment

ric-evans May 27, 2021

Choose a reason for hiding this comment

ric-evans May 27, 2021

Choose a reason for hiding this comment

ric-evans May 27, 2021

Choose a reason for hiding this comment

ric-evans commented May 27, 2021

ric-evans left a comment

Choose a reason for hiding this comment

ric-evans May 27, 2021

Choose a reason for hiding this comment

ric-evans May 27, 2021

Choose a reason for hiding this comment

ric-evans May 27, 2021 •

edited

Loading