Run tox in travis instead of Docker #219

manics · 2020-09-16T09:45:02Z

Are there any advantages to running the tests with tox and Docker? To me it seems to make things more difficult, especially if we want to start testing multiple Python versions.

Extracted from #218

joshmoore · 2020-09-16T09:46:34Z

Happy to use travis for the testing or even move to GitHub actions, but I'd like to keep the dockerfile for testing. (Certainly having the tox.ini tied to the dockerfile has annoyed me in the past, but let's find a way to have both)

manics · 2020-09-16T09:49:54Z

Do you want to keep the existing centos:7 Dockerfile, or is moving to e.g. library/python:3.6 an option?

joshmoore · 2020-09-16T09:55:28Z

For the target of tox, I assume one of the python.* images is fine. If we want to then use that directly in a docker-compose or similar, we could also have a multi-layer dockerfile.

manics · 2020-09-16T13:29:14Z

I've force pushed to revert the deletion of the Dockerfile, we can discuss that separately #221

joshmoore · 2020-09-17T09:35:53Z

Docker output 👍

py36 run-test: commands[3] | pytest -rf
============================= test session starts ==============================
platform linux -- Python 3.6.9, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
cachedir: .tox/py36/.pytest_cache
rootdir: /src, configfile: pytest.ini
plugins: forked-1.3.0, xdist-1.34.0
collected 331 items

test/unit/test_filters.py .                                              [  0%]
test/unit/test_marshal.py .............                                  [  4%]
test/unit/test_render_response.py .............                          [  8%]
test/unit/test_settings.py ...                                           [  9%]
test/unit/test_show.py ...................                               [ 14%]
test/unit/test_tree.py .........                                         [ 17%]
test/unit/test_util.py ..............................                    [ 26%]
test/unit/test_web.py ................XxX.x............................. [ 41%]
........................................................................ [ 63%]
........................................................................ [ 85%]
...................................                                      [ 95%]
test/unit/test_webgateway.py ..............                              [100%]

=============================== warnings summary ===============================
test/unit/test_render_response.py: 377 warnings
  /src/.tox/py36/lib/python3.6/site-packages/django/utils/regex_helper.py:139: RemovedInDjango21Warning: Using (?i) in url() patterns is deprecated.
    RemovedInDjango21Warning

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========== 327 passed, 2 xfailed, 2 xpassed, 377 warnings in 19.86s ===========
___________________________________ summary ____________________________________
  py36: commands succeeded
  congratulations :)

Native (OSX) with this setup:

  507  conda create -n py36 python=3.6
  509  conda activate py36
  512  pip install tox
  516  conda install -c ome zeroc-ice36-python
  520  conda install -c ome zeroc-ice36-python
  523  tox --sitepackages

gives:

================================================================================ test session starts ================================================================================
platform darwin -- Python 3.6.12, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
cachedir: .tox/py36/.pytest_cache
rootdir: /opt/omero-web, configfile: pytest.ini
plugins: xdist-1.34.0, forked-1.3.0
collected 331 items

test/unit/test_filters.py .                                                                                                                                                   [  0%]
test/unit/test_marshal.py .............                                                                                                                                       [  4%]
test/unit/test_render_response.py .............                                                                                                                               [  8%]
test/unit/test_settings.py ...                                                                                                                                                [  9%]
test/unit/test_show.py ...................                                                                                                                                    [ 14%]
test/unit/test_tree.py .........                                                                                                                                              [ 17%]
test/unit/test_util.py ..............................                                                                                                                         [ 26%]
test/unit/test_web.py ................XxX.x.................................................................................................................................. [ 72%]
..............................................................................                                                                                                [ 95%]
test/unit/test_webgateway.py ...

but not it does require removing the manylinux wheel which meant I had to run with tox --sitepackages.

Otherwise, 👍

will-moore · 2020-09-17T09:57:21Z

Following @joshmoore example I also tried, with less success:

$ conda create -n py36 python=3.6
$ conda activate py36
$ pip install tox
$ conda install -c ome zeroc-ice36-python
$ tox --sitepackages
GLOB sdist-make: /Users/wmoore/Desktop/WEB/omero-web/setup.py
py36 create: /Users/wmoore/Desktop/WEB/omero-web/.tox/py36
py36 installdeps: flake8, future, numpy>=1.9, pytest, PyYAML, tables, pytest-xdist < 2, restructuredtext-lint, https://github.com/ome/zeroc-ice-py-manylinux/releases/download/0.1.0/zeroc_ice-3.6.5-cp36-cp36m-manylinux2010_x86_64.whl
ERROR: invocation failed (exit code 1), logfile: /Users/wmoore/Desktop/WEB/omero-web/.tox/py36/log/py36-1.log
======================================================================================= log start =======================================================================================
ERROR: zeroc_ice-3.6.5-cp36-cp36m-manylinux2010_x86_64.whl is not a supported wheel on this platform.
WARNING: You are using pip version 20.2.2; however, version 20.2.3 is available.
You should consider upgrading via the '/Users/wmoore/Desktop/WEB/omero-web/.tox/py36/bin/python -m pip install --upgrade pip' command.

======================================================================================== log end ========================================================================================
ERROR: could not install deps [flake8, future, numpy>=1.9, pytest, PyYAML, tables, pytest-xdist < 2, restructuredtext-lint, https://github.com/ome/zeroc-ice-py-manylinux/releases/download/0.1.0/zeroc_ice-3.6.5-cp36-cp36m-manylinux2010_x86_64.whl]; v = InvocationError("/Users/wmoore/Desktop/WEB/omero-web/.tox/py36/bin/python -m pip install flake8 future 'numpy>=1.9' pytest PyYAML tables 'pytest-xdist < 2' restructuredtext-lint https://github.com/ome/zeroc-ice-py-manylinux/releases/download/0.1.0/zeroc_ice-3.6.5-cp36-cp36m-manylinux2010_x86_64.whl", 1)
________________________________________________________________________________________ summary ________________________________________________________________________________________
ERROR:   py36: could not install deps [flake8, future, numpy>=1.9, pytest, PyYAML, tables, pytest-xdist < 2, restructuredtext-lint, https://github.com/ome/zeroc-ice-py-manylinux/releases/download/0.1.0/zeroc_ice-3.6.5-cp36-cp36m-manylinux2010_x86_64.whl]; v = InvocationError("/Users/wmoore/Desktop/WEB/omero-web/.tox/py36/bin/python -m pip install flake8 future 'numpy>=1.9' pytest PyYAML tables 'pytest-xdist < 2' restructuredtext-lint https://github.com/ome/zeroc-ice-py-manylinux/releases/download/0.1.0/zeroc_ice-3.6.5-cp36-cp36m-manylinux2010_x86_64.whl", 1)

manics · 2020-09-17T09:58:46Z

Is tox working for you on omero-py?

will-moore · 2020-09-17T09:59:00Z

I also got the same error when running in my omeroweb conda env where I normally run omero-web

joshmoore · 2020-09-17T10:08:45Z

The manylinux line needs an exception so that it doesn't run on Mac.

will-moore · 2020-09-17T10:09:34Z

Hmm - Running $ tox --sitepackages on master of omero-py...

...
copying target/omero_ext/__init__.py -> build/lib/omero_ext
error: can't copy 'target/omero_ext/functional.py': doesn't exist or not a regular file
ERROR: InvocationError for command /Users/wmoore/Desktop/PY/omero-py/.tox/py36/bin/python setup.py install (exited with code 1)
py37 create: /Users/wmoore/Desktop/PY/omero-py/.tox/py37
ERROR: InterpreterNotFound: python3.7
py38 create: /Users/wmoore/Desktop/PY/omero-py/.tox/py38
py38 installdeps: future, numpy, Pillow, pytest, PyYAML, tables, pytest-rerunfailures, pytest-xdist, restructuredtext-lint, https://github.com/ome/zeroc-ice-py-travis-bionic/releases/download/0.0.1/zeroc_ice-3.6.5-cp36-cp36m-linux_x86_64.whl; platform_system=="Linux" and python_version=="3.6", https://github.com/ome/zeroc-ice-py-travis-bionic/releases/download/0.0.1/zeroc_ice-3.6.5-cp37-cp37m-linux_x86_64.whl; platform_system=="Linux" and python_version=="3.7", https://github.com/ome/zeroc-ice-py-travis-bionic/releases/download/0.0.1/zeroc_ice-3.6.5-cp38-cp38-linux_x86_64.whl; platform_system=="Linux" and python_version=="3.8", pywin32; platform_system=="Windows", mox3, jinja2
py38 inst: /Users/wmoore/Desktop/PY/omero-py/.tox/.tmp/package/1/omero-py-5.7.1.zip
py38 installed: apipkg==1.5,appdirs==1.4.4,attrs==20.2.0,certifi==2020.6.20,chardet==3.0.4,docutils==0.16,execnet==1.7.1,extras==1.0.0,fixtures==3.0.0,future==0.18.2,idna==2.10,iniconfig==1.0.1,Jinja2==2.11.2,linecache2==1.0.0,MarkupSafe==1.1.1,more-itertools==8.5.0,mox3==1.1.0,numexpr==2.7.1,numpy==1.19.2,omero-py @ file:///Users/wmoore/Desktop/PY/omero-py/.tox/.tmp/package/1/omero-py-5.7.1.zip,packaging==20.4,pbr==5.5.0,Pillow==7.2.0,pluggy==0.13.1,py==1.9.0,pyparsing==2.4.7,pytest==6.0.2,pytest-forked==1.3.0,pytest-rerunfailures==9.1,pytest-xdist==2.1.0,python-mimeparse==1.6.0,PyYAML==5.3.1,requests==2.24.0,restructuredtext-lint==1.3.1,six==1.15.0,tables==3.6.1,testtools==2.4.0,toml==0.10.1,traceback2==1.4.0,unittest2==1.1.0,urllib3==1.25.10,zeroc-ice==3.6.5
py38 run-test-pre: PYTHONHASHSEED='1738244306'
py38 run-test: commands[0] | rst-lint README.rst
Traceback (most recent call last):
  File "/Users/wmoore/Desktop/PY/omero-py/.tox/py38/bin/rst-lint", line 5, in <module>
    from restructuredtext_lint.cli import main
ModuleNotFoundError: No module named 'restructuredtext_lint'
ERROR: InvocationError for command /Users/wmoore/Desktop/PY/omero-py/.tox/py38/bin/rst-lint README.rst (exited with code 1)
________________________________________________________________________________________ summary ________________________________________________________________________________________
ERROR:   py36: commands failed
ERROR:  py37: InterpreterNotFound: python3.7
ERROR:   py38: commands failed

manics · 2020-09-17T10:52:04Z

Can you try now?

will-moore · 2020-09-17T11:31:34Z

Yep - looks good 👍

$ tox --sitepackages
...
=============================================================== 327 passed, 2 xfailed, 2 xpassed, 377 warnings in 21.75s ================================================================
________________________________________________________________________________________ summary ________________________________________________________________________________________
  py36: commands succeeded
  congratulations :)

joshmoore · 2020-09-18T09:32:11Z

So probably the only downside is that the Docker is no longer being built. Assuming that will be handled via #221, merging.

manics changed the title ~~Run tox in travis instead of Dockerfile~~ Run tox in travis instead of Docker Sep 16, 2020

Run tox in travis instead of Dockerfile

d25fdb8

manics force-pushed the tox-no-docker branch from 0ed6ac7 to d25fdb8 Compare September 16, 2020 13:28

tox: default to not using-manylinux ice wheel

b741354

manics force-pushed the tox-no-docker branch from a769ab7 to b741354 Compare September 17, 2020 10:51

sbesson approved these changes Sep 18, 2020

View reviewed changes

joshmoore merged commit 0cff8fa into ome:master Sep 18, 2020

joshmoore mentioned this pull request Sep 23, 2020

Move production Dockerfile into this repo #221

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run tox in travis instead of Docker #219

Run tox in travis instead of Docker #219

manics commented Sep 16, 2020

joshmoore commented Sep 16, 2020

manics commented Sep 16, 2020

joshmoore commented Sep 16, 2020

manics commented Sep 16, 2020

joshmoore commented Sep 17, 2020

will-moore commented Sep 17, 2020

manics commented Sep 17, 2020

will-moore commented Sep 17, 2020

joshmoore commented Sep 17, 2020

will-moore commented Sep 17, 2020

manics commented Sep 17, 2020

will-moore commented Sep 17, 2020

joshmoore commented Sep 18, 2020

Run tox in travis instead of Docker #219

Run tox in travis instead of Docker #219

Conversation

manics commented Sep 16, 2020

joshmoore commented Sep 16, 2020

manics commented Sep 16, 2020

joshmoore commented Sep 16, 2020

manics commented Sep 16, 2020

joshmoore commented Sep 17, 2020

will-moore commented Sep 17, 2020

manics commented Sep 17, 2020

will-moore commented Sep 17, 2020

joshmoore commented Sep 17, 2020

will-moore commented Sep 17, 2020

manics commented Sep 17, 2020

will-moore commented Sep 17, 2020

joshmoore commented Sep 18, 2020