[WIP] update python 3.0 for vsc-base #258
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vsoch
commented
May 9, 2018
| @@ -138,7 +138,7 @@ def number(self, otherdate): | |||
| def get_other(self, shift=-1): | |||
| """Return month that is shifted shift months: negative integer is in past, positive is in future""" | |||
| new = self.date.year * 12 + self.date.month - 1 + shift | |||
| return self.__class__(date(new // 12, new % 12 + 1, 01)) | |||
| return self.__class__(date(new // 12, new % 12 + 1, 1)) | |||
There was a problem hiding this comment.
This line, specifically the 01
return self.__class__(date(new // 12, new % 12 + 1, 01))
wasn't clear to me why it would be used instead of 1?
There was a problem hiding this comment.
I don't think there's a good reason for the 01, it's exactly the same as 1...
Maybe because this refers to 1st day of the month? Dunno, but changing it to 1 seems fine to me.
We could double-check that this is covered by the tests to make sure changing it is OK, but I don't see how it could have any impact (famous last words!).
There was a problem hiding this comment.
In python2 01 is the octal representation for 1, this is deprecated in python3 (should be 0o1)
If the 01 is needed for programatic reasons here, replace it with 0o1 instead of 1 (but I can't see a reason why it would make a difference)
There was a problem hiding this comment.
oh interesting! I didn't know this, but it looks like the 1 should be equivalent to 0o1 (see here https://bugs.python.org/issue1715302). And here is a confirmation (in Python 3) that the two produce the same result:
In [8]: datetime.date(2007, 5, 0o1)
Out[8]: datetime.date(2007, 5, 1)
In [9]: datetime.date(2007, 5, 1)
Out[9]: datetime.date(2007, 5, 1)
vsoch
commented
May 9, 2018
lib/vsc/utils/fancylogger.py
Outdated
| if hasattr(logging,'_levelNames'): | ||
| logging._levelNames['EXCEPTION'] = logging.ERROR | ||
| logging._levelNames['FATAL'] = logging.CRITICAL | ||
| logging._levelNames['QUIET'] = logging.WARNING | ||
|
|
There was a problem hiding this comment.
I couldn't find this in the logging 3.0 docs, so I'm guessing it is relevant only to some older version and we can just check and pass over if the data structure doesn't exist?
There was a problem hiding this comment.
In Python 2.7, there's this:
>>> import logging
>>> logging._levelNames
{0: 'NOTSET', 10: 'DEBUG', 'WARN': 30, 20: 'INFO', 'ERROR': 40, 'DEBUG': 10, 30: 'WARNING', 'INFO': 20, 'WARNING': 30, 40: 'ERROR', 50: 'CRITICAL', 'CRITICAL': 50, 'NOTSET': 0}
We are defining additional log levels here, which we are injecting in the _levelNames (hidden) dictionary.
I guess we'll need to figure out what the equivalent approach is in Python 3?
There was a problem hiding this comment.
I think we want this https://docs.python.org/3/library/logging.html#logging.addLevelName and with the print you gave me above, I think all I need is that (the name and the level) for each of the new levels. Let me test it out.
There was a problem hiding this comment.
Python 3 has levelToName as well:
import logging
In [2]: logging._levelToName
Out[2]:
{0: 'NOTSET',
10: 'DEBUG',
20: 'INFO',
30: 'WARNING',
40: 'ERROR',
50: 'CRITICAL'}
There was a problem hiding this comment.
In [1]: import logging
In [2]: logging._levelToName
Out[2]:
{0: 'NOTSET',
10: 'DEBUG',
20: 'INFO',
30: 'WARNING',
40: 'ERROR',
50: 'CRITICAL'}
In [3]: logging.addLevelName(logging.CRITICAL,'PINKYANDTHEBRAIN')
In [4]: logging._levelToName
Out[4]:
{0: 'NOTSET',
10: 'DEBUG',
20: 'INFO',
30: 'WARNING',
40: 'ERROR',
50: 'PINKYANDTHEBRAIN'}There was a problem hiding this comment.
This seems to replace it, which isn't what we want. I'm going to look into adding a custom level.
|
@vsoch tests failing on our Jenkins CI server: |
|
how can I reproduce these locally? |
|
okay I was able to rebase and get it working in python2, now I'm hitting another level of errors (for Python 3) so will work on this a bit and update the PR after I make the changes! Stay tuned! |
…all are required/dependency for this
|
heyo! Just a heads up there are more tweaks that need to be done for Python3 (I'm finding as I test easybuild) I'll commit and push in chunks, so expect more updates to this PR. I'll change the name to reflect this. |
vsoch
changed the title
|
@vsoch thanks for al the patches, but how are you doing this migration? just trial and error? we had the idea to add 2 tests to vsc-install: a pylint py3k test and something with we haven't started this work because this is not a priority by far (i guess rhel7 will have py2 support for the rest of its lifetime, and i'm quite certain rhel8 will ship some py2 version whenever it comes out) i would be very nice to know if the tools at https://portingguide.readthedocs.io/en/latest/tools.html can help you out with the current fixes (in particular |
|
hey @stdweird ! I'm aware of these helper tools, but to be honest I'm uncomfortable with applying some black box script to an entire code base. I want to step through each test / function and see the error, and then look up and decide what is the simplest solution to fix it. For these repos, given the dependency, this means sometimes stepping back and doing another fix to one of the vsc modules (while updating python3 for easybuild, for example). It's ok that it hasn't been a priority! That's one of the ways I can help :) I think minimally there are a lot of users that would like to use something like easybuild and really want/need python3. Without that compatibility, you lose many off the bat. And you know, what happens when python 4 comes out? We have to keep up! |
|
@vsoch The last chunk of changes, are those issues you are hitting when trying to run EasyBuild (tests) on top of Python 3? If that are issues that aren't covered by the |
|
Current failing tests: |
|
yep, but if the fixes are done with vep, then they won't be issues. But you are correct if a user has installed an older version of vep (or two versions, as I had at one point) you will see the bugs pop up. |
|
Thanks, on these new issues! |
…n (will address next)
|
hey @boegel I'm at a loss here - the tests start running and just hang, and it almost has to be the functions that are reimplementing subprocess. Is that really necessary? A lot of this functionality could be improved simply by using core python modules that do these basic things very well. |
|
@vsoch Which tests/functions are hanging for you? Are you referring to the functions provided by |
|
Tests now fail with which means this is blocked by hpcugent/vsc-install#83... |
Merged
|
@vsoch we wrote a lot of this code to have nice subprocessing, logging and option parsing in python 2.4. |
|
oups scratch the last comment, you already did! :) |
…to not catch the 0 return code when running simple.py in runtests
…to handle metaclass (not supported in python 3 with __metaclass__
|
okay tests are running! Now we can see (in this running version) when we hit the run.py test with simply.py how it is essentially an infinite loop: PYTHONPATH=lib/ python3 setup.py test
INFO: This is (based on) vsc.install.shared_setup 0.11.00
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
WARN: cleanup lib/vsc_base.egg-info
INFO: initial packages list: dict_keys(['vsc', 'vsc.utils'])
INFO: generated list: dict_keys(['vsc', 'vsc.utils'])
INFO: generated packages list: dict_keys(['vsc', 'vsc.utils'])
INFO: makesetupcfg set to True, (re)creating setup.cfg
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: setting license LGPLv2+
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: Removing None download_url
INFO: get_name_url returns {'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: using long_description Common tools used within our organization. Originally created by the HPC team of Ghent University (http://ugent.be/hpc).
INFO: generated list: ['bin/logdaemon.py', 'bin/optcomplete.bash', 'bin/startlogdaemon.sh']
INFO: generated scripts list: ['bin/logdaemon.py', 'bin/optcomplete.bash', 'bin/startlogdaemon.sh']
INFO: adding prospector to tests_require
{'license': 'LGPLv2+', 'maintainer': 'Stijn De Weirdt;Jens Timmerman;Andy Georges;Kenneth Hoste', 'maintainer_email': 'stijn.deweirdt@ugent.be, jens.timmerman@ugent.be, andy.georges@ugent.be, kenneth.hoste@ugent.be', 'package_dir': {'': 'lib'}, 'packages': dict_keys(['vsc', 'vsc.utils']), 'install_requires': ['vsc-install >= 0.11.00'], 'extras_require': {'coloredlogs': ['coloredlogs<6.0', 'humanfriendly']}, 'url': 'https://github.com/hpcugent/vsc-base', 'author_email': 'stijn.deweirdt@ugent.be, jens.timmerman@ugent.be, andy.georges@ugent.be, kenneth.hoste@ugent.be', 'setup_requires': ['vsc-install >= 0.11.00'], 'scripts': ['bin/logdaemon.py', 'bin/optcomplete.bash', 'bin/startlogdaemon.sh'], 'description': 'Common tools used within our organization. Originally created by the HPC team of Ghent University (http://ugent.be/hpc).', 'command_packages': ['vsc.install.shared_setup', 'shared_setup_dist_only', 'setuptools.command', 'distutils.command'], 'name': 'vsc-base', 'long_description': "# vsc-base\n\n### Build Status\n\n- Python 2.6 : [](https://jenkins1.ugent.be/job/vsc-base-python26/)\n- Python 2.7 : [](https://jenkins1.ugent.be/job/vsc-base-python27/)\n\n# Description\n\nCommon tools used within our organization.\nOriginally created by the HPC team of Ghent University (http://ugent.be/hpc).\n\n# Documentation\nhttps://jenkins1.ugent.be/job/vsc-base-python26/Documentation/\n\n# Namespaces and tools\n\n## lib/utils\npython utilities to be used as libraries\n\n- __fancylogger__: an extention of the default python logger designed to be easy to use and have a\ncouple of `fancy` features.\n - custom specifiers for mpi loggin (the mpirank) with autodetection of mpi\n - custom specifier for always showing the calling function's name\n - rotating file handler\n - a default formatter.\n - logging to an UDP server (logdaemon.py f.ex.)\n - easily setting loglevel\n- __daemon.py__ : Daemon class written by Sander Marechal (http://www.jejik.com) to start a python script as a daemon.\n- __missing.py__: Small functions and tools that are commonly used but not\n available in the Python (2.x) API.\n- ~~__cache.py__ : File cache to store pickled data identified by a key accompanied by a timestamp.~~ (moved to [vsc-utils](https://github.com/hpcugent/vsc-utils))\n- __generaloption.py__ : A general option parser for python. It will fetch options (in this order) from config files, from environment variables and from the command line and parse them in a way compatible with the default python optionparser. Thus allowing a very flexible way to configure your scripts.\nIt also adds a few other useful extras.\n- __affinity.py__ : Linux cpu affinity.\n - Based on `sched.h` and `bits/sched.h`,\n - see man pages for `sched_getaffinity` and `sched_setaffinity`\n - also provides a `cpuset` class to convert between human readable cpusets and the bit version Linux priority\n - Based on sys/resources.h and bits/resources.h see man pages for `getpriority` and `setpriority`\n- __asyncprocess.py__ : Module to allow Asynchronous subprocess use on Windows and Posix platforms\n - Based on a [python recipe](http://code.activestate.com/recipes/440554/) by Josiah Carlson\n - added STDOUT handle and recv_some\n- __daemon.py__ : [A generic daemon class by Sander Marechal](http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/)\n- __dateandtime.py__ : A module with various convenience functions and classes to deal with date, time and timezone.\n- __nagios.py__ : This module provides functionality to cache and report results of script executions that can readily be\ninterpreted by nagios/icinga.\n- __run.py__ : Python module to execute a command, can make use of asyncprocess, answer questions based on a dictionary\n - supports a whole lot of ways to input, process and output the command. (filehandles, PIPE, pty, stdout, logging...)\n- __mail.py__ : Wrapper around the standard Python mail library.\n - Send a plain text message\n - Send an HTML message, with a plain text alternative\n\n## bin\nA collection of python scripts, these are examples of how you could use fancylogger to log to a daemon, but should not be used directly.\n- __logdaemon.py__: A daemon that listens on a port for udp packets and logs them to file, works toghether with fancylogger.\n- __startlogdaemon.py__ : Script that will start the logdaemon for you and set environment variables for fancylogger.\n\n# License\nvsc-base is made available under the GNU Library General Public License (LGPL) version 2 or any later version.\n\n# Acknowledgements\nvsc-base was created with support of [Ghent University](http://www.ugent.be/en),\nthe [Flemish Supercomputer Centre (VSC)](https://vscentrum.be/nl/en),\nthe [Flemish Research Foundation (FWO)](http://www.fwo.be/en),\nand [the Department of Economy, Science and Innovation (EWI)](http://www.ewi-vlaanderen.be/en).\n\n", 'author': 'Stijn De Weirdt;Jens Timmerman;Andy Georges;Kenneth Hoste', 'dependency_links': ['git+ssh://git@github.ugent.be/hpcugent/vsc-install.git#egg=vsc-install-0.11.00', 'git+ssh://git@github.com/hpcugent/vsc-install.git#egg=vsc-install-0.11.00', 'git+https://github.com/hpcugent/vsc-install.git#egg=vsc-install-0.11.00', 'git+ssh://git@github.ugent.be/hpcugent/vsc-install.git#egg=vsc-install-0.11.00', 'git+ssh://git@github.com/hpcugent/vsc-install.git#egg=vsc-install-0.11.00', 'git+https://github.com/hpcugent/vsc-install.git#egg=vsc-install-0.11.00'], 'tests_require': ['prospector', 'coloredlogs<6.0', 'humanfriendly', 'prospector >= 0.12.1', 'pylint-django == 0.9.1'], 'download_url': '', 'namespace_packages': ['vsc'], 'test_suite': 'test', 'cmdclass': {'install_scripts': <class 'vsc.install.shared_setup.vsc_setup.vsc_install_scripts'>, 'egg_info': <class 'vsc.install.shared_setup.vsc_setup.vsc_egg_info'>, 'sdist': <class 'vsc.install.shared_setup.vsc_setup.vsc_sdist'>, 'vsc_release': <class 'vsc.install.shared_setup.vsc_setup.vsc_release'>, 'bdist_rpm': <class 'vsc.install.shared_setup.vsc_setup.vsc_bdist_rpm'>, 'test': <class 'vsc.install.shared_setup.vsc_setup.VscTestCommand'>}, 'classifiers': ['License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)'], 'version': '2.6.0'}
INFO: running test
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: test_loader set to vsc.install.shared_setup:vsc_setup.VscScanningLoader
INFO: Searching for pylint-django==0.9.1
INFO: Best match: pylint-django 0.9.1
INFO: Processing pylint_django-0.9.1-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/pylint_django-0.9.1-py3.5.egg
INFO: Searching for prospector>=0.12.1
INFO: Best match: prospector 0.12.7
INFO: Processing prospector-0.12.7-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/prospector-0.12.7-py3.5.egg
INFO: Searching for humanfriendly
INFO: Best match: humanfriendly 4.12.1
INFO: Processing humanfriendly-4.12.1-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/humanfriendly-4.12.1-py3.5.egg
INFO: Searching for coloredlogs<6.0
INFO: Best match: coloredlogs 5.2
INFO: Processing coloredlogs-5.2-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/coloredlogs-5.2-py3.5.egg
INFO: Searching for pylint>=1.8.2
INFO: Best match: pylint 1.9.1
INFO: Processing pylint-1.9.1-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/pylint-1.9.1-py3.5.egg
INFO: Searching for pylint-plugin-utils>=0.2.1
INFO: Best match: pylint-plugin-utils 0.2.6
INFO: Processing pylint_plugin_utils-0.2.6-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/pylint_plugin_utils-0.2.6-py3.5.egg
INFO: Searching for pydocstyle>=2.0.0
INFO: Best match: pydocstyle 2.1.1
INFO: Processing pydocstyle-2.1.1-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/pydocstyle-2.1.1-py3.5.egg
INFO: Searching for pep8-naming>=0.3.3
INFO: Best match: pep8-naming 0.7.0
INFO: Processing pep8_naming-0.7.0-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/pep8_naming-0.7.0-py3.5.egg
INFO: Searching for pycodestyle==2.0.0
INFO: Best match: pycodestyle 2.0.0
INFO: Processing pycodestyle-2.0.0-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/pycodestyle-2.0.0-py3.5.egg
INFO: Searching for mccabe>=0.5.0
INFO: Best match: mccabe 0.6.1
INFO: Processing mccabe-0.6.1-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/mccabe-0.6.1-py3.5.egg
INFO: Searching for dodgy>=0.1.9
INFO: Best match: dodgy 0.1.9
INFO: Processing dodgy-0.1.9-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/dodgy-0.1.9-py3.5.egg
INFO: Searching for setoptconf>=0.2.0
INFO: Best match: setoptconf 0.2.0
INFO: Processing setoptconf-0.2.0-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/setoptconf-0.2.0-py3.5.egg
INFO: Searching for requirements-detector>=0.4.1
INFO: Best match: requirements-detector 0.5.2
INFO: Processing requirements_detector-0.5.2-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/requirements_detector-0.5.2-py3.5.egg
INFO: Searching for pylint-common>=0.2.5
INFO: Best match: pylint-common 0.2.5
INFO: Processing pylint_common-0.2.5-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/pylint_common-0.2.5-py3.5.egg
INFO: Searching for pylint-flask>=0.3
INFO: Best match: pylint-flask 0.5
INFO: Processing pylint_flask-0.5-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/pylint_flask-0.5-py3.5.egg
INFO: Searching for pylint-celery>=0.3
INFO: Best match: pylint-celery 0.3
INFO: Processing pylint_celery-0.3-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/pylint_celery-0.3-py3.5.egg
INFO: Searching for isort>=4.2.5
INFO: Best match: isort 4.3.4
INFO: Processing isort-4.3.4-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/isort-4.3.4-py3.5.egg
INFO: Searching for astroid<2.0,>=1.6
INFO: Best match: astroid 1.6.4
INFO: Processing astroid-1.6.4-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/astroid-1.6.4-py3.5.egg
INFO: Searching for flake8-polyfill<2,>=1.0.2
INFO: Best match: flake8-polyfill 1.0.2
INFO: Processing flake8_polyfill-1.0.2-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/flake8_polyfill-1.0.2-py3.5.egg
INFO: Searching for flake8
INFO: Best match: flake8 3.5.0
INFO: Processing flake8-3.5.0-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/flake8-3.5.0-py3.5.egg
INFO: Searching for pyflakes<1.7.0,>=1.5.0
INFO: Best match: pyflakes 1.6.0
INFO: Processing pyflakes-1.6.0-py3.5.egg
INFO:
Using /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.eggs/pyflakes-1.6.0-py3.5.egg
INFO: running egg_info
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: creating lib/vsc_base.egg-info
INFO: writing namespace_packages to lib/vsc_base.egg-info/namespace_packages.txt
INFO: writing requirements to lib/vsc_base.egg-info/requires.txt
INFO: writing lib/vsc_base.egg-info/PKG-INFO
INFO: writing top-level names to lib/vsc_base.egg-info/top_level.txt
INFO: writing dependency_links to lib/vsc_base.egg-info/dependency_links.txt
INFO: writing manifest file 'lib/vsc_base.egg-info/SOURCES.txt'
INFO: reading manifest file 'lib/vsc_base.egg-info/SOURCES.txt'
INFO: reading manifest template 'MANIFEST.in'
WARN: warning: no files found matching 'vsc' under directory 'lib'
WARN: warning: no previously-included files matching '__pycache__' found anywhere in distribution
INFO: writing manifest file 'lib/vsc_base.egg-info/SOURCES.txt'
INFO: looking for extra dist files
INFO: running build_ext
test_import_modules (vsc.install.commontest.CommonTest)
Try to import each module ... INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
/home/vanessa/anaconda3/lib/python3.5/site-packages/vsc_install-0.11.0-py3.5.egg/vsc/install/shared_setup.py:437: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/lib/vsc/__init__.py' mode='r' encoding='UTF-8'>
init = open(os.path.join(root, '__init__.py')).read()
/home/vanessa/anaconda3/lib/python3.5/site-packages/vsc_install-0.11.0-py3.5.egg/vsc/install/shared_setup.py:395: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.gitignore' mode='r' encoding='UTF-8'>
all_patterns = [l for l in [l.strip() for l in open(gitignore).readlines()] if l and not l.startswith('#')]
/home/vanessa/anaconda3/lib/python3.5/site-packages/vsc_install-0.11.0-py3.5.egg/vsc/install/shared_setup.py:437: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/lib/vsc/utils/__init__.py' mode='r' encoding='UTF-8'>
init = open(os.path.join(root, '__init__.py')).read()
INFO: generated list: dict_keys(['vsc.utils.docs', 'vsc.utils.generaloption', 'vsc.utils.dateandtime', 'vsc.utils.daemon', 'vsc.utils.mail', 'vsc.utils.wrapper', 'vsc.utils.missing', 'vsc.utils.asyncprocess', 'vsc.utils.optcomplete', 'vsc.utils.rest', 'vsc.utils.affinity', 'vsc.utils.exceptions', 'vsc.utils.run', 'vsc.utils.fancylogger', 'vsc.utils.testing', 'vsc.utils.run-old', 'vsc.utils.frozendict', 'vsc.utils.patterns', 'vsc.utils.runv2'])
INFO: generated modules list: dict_keys(['vsc.utils.docs', 'vsc.utils.generaloption', 'vsc.utils.dateandtime', 'vsc.utils.daemon', 'vsc.utils.mail', 'vsc.utils.wrapper', 'vsc.utils.missing', 'vsc.utils.asyncprocess', 'vsc.utils.optcomplete', 'vsc.utils.rest', 'vsc.utils.affinity', 'vsc.utils.exceptions', 'vsc.utils.run', 'vsc.utils.fancylogger', 'vsc.utils.testing', 'vsc.utils.run-old', 'vsc.utils.frozendict', 'vsc.utils.patterns', 'vsc.utils.runv2'])
ok
test_import_packages (vsc.install.commontest.CommonTest)
Try to import each namespace ... INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: initial packages list: dict_keys(['vsc', 'vsc.utils'])
INFO: generated list: dict_keys(['vsc', 'vsc.utils'])
INFO: generated packages list: dict_keys(['vsc', 'vsc.utils'])
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
/home/vanessa/anaconda3/lib/python3.5/site-packages/vsc_install-0.11.0-py3.5.egg/vsc/install/shared_setup.py:312: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config' mode='r' encoding='UTF-8'>
txt = open(filename).read()
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: Header is an external compatible license. Leaving the header as-is.
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: Header is an external compatible license. Leaving the header as-is.
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
INFO: Header is an external compatible license. Leaving the header as-is.
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
ERROR: No begin/endyear found, using this year as begin (and end)
INFO: Diff header vs gen_header
+ #
+ # Copyright 2018-2018 Ghent University
+ #
+ # This file is part of vsc-base,
+ # originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
+ # with support of Ghent University (http://ugent.be/hpc),
+ # the Flemish Supercomputer Centre (VSC) (https://www.vscentrum.be),
+ # the Flemish Research Foundation (FWO) (http://www.fwo.be/en)
+ # and the Department of Economy, Science and Innovation (EWI) (http://www.ewi-vlaanderen.be/en).
+ #
+ # https://github.com/hpcugent/vsc-base
+ #
+ # vsc-base is free software: you can redistribute it and/or modify
+ # it under the terms of the GNU Library General Public License as
+ # published by the Free Software Foundation, either version 2 of
+ # the License, or (at your option) any later version.
+ #
+ # vsc-base is distributed in the hope that it will be useful,
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ # GNU Library General Public License for more details.
+ #
+ # You should have received a copy of the GNU Library General Public License
+ # along with vsc-base. If not, see <http://www.gnu.org/licenses/>.
FAIL
test_importscripts (vsc.install.commontest.CommonTest)
Try to import each python script as a module ... INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: generated list: ['bin/logdaemon.py', 'bin/optcomplete.bash', 'bin/startlogdaemon.sh']
INFO: generated scripts list: ['bin/logdaemon.py', 'bin/optcomplete.bash', 'bin/startlogdaemon.sh']
INFO: get_header for script
INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: found match url git@github.com:hpcugent/vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: found match name vsc-base in /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/.git/config
INFO: reg found: ('github.com', 'hpcugent/vsc-base')
INFO: get_name_url returns {'download_url': 'https://github.com/hpcugent/vsc-base/archive/ALL_VERSIONS.tar.gz', 'url': 'https://github.com/hpcugent/vsc-base', 'name': 'vsc-base'}
INFO: found license /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/LICENSE with md5sum 5f30f0716dfdd0d91eb439ebec522ec2
INFO: Found license name LGPLv2+ and classifier License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
ok
test_prospector (vsc.install.commontest.CommonTest)
Run prospector.run.main, but apply white/blacklists to the results ... INFO: run_tests from base dir /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base (using executable /home/vanessa/Documents/Dropbox/Code/Python/easybuild/vsc-base/setup.py)
INFO: No protector tests are ran, install prospector manually first
ok
test_wrapper (test.wrapper.TestWrapper)
Use the tests provided by the stackoverflow page ... ok
test_env_to_boolean_false (test.fancylogger.EnvToBooleanTest) ... ok
test_env_to_boolean_true (test.fancylogger.EnvToBooleanTest) ... ok
test_env_to_boolean_undef_with_default (test.fancylogger.EnvToBooleanTest) ... ok
test_env_to_boolean_undef_without_default (test.fancylogger.EnvToBooleanTest) ... ok
test_logtofile (test.fancylogger.FancyLoggerLogToFileTest)
Test to see if logtofile doesn't fail when logging to a non existing file /directory ... ok
test_classname_in_log (test.fancylogger.FancyLoggerTest)
Do a log and check if the classname is correctly in it ... ERROR
test_deprecated (test.fancylogger.FancyLoggerTest)
Test deprecated log function. ... ERROR
test_fail (test.fancylogger.FancyLoggerTest)
Test fail log method. ... ERROR
test_fancylogger_as_rootlogger_logging (test.fancylogger.FancyLoggerTest) ... ERROR
test_fancyrecord (test.fancylogger.FancyLoggerTest) ... ERROR
test_getDetailsLogLevels (test.fancylogger.FancyLoggerTest) ... ERROR
test_getlevelint (test.fancylogger.FancyLoggerTest)
Test the getLevelInt ... ERROR
test_normal_warning_logging (test.fancylogger.FancyLoggerTest) ... ERROR
test_parentinfo (test.fancylogger.FancyLoggerTest)
Test the collection of parentinfo ... ERROR
test_raiseException (test.fancylogger.FancyLoggerTest)
Test raiseException log method. ... ERROR
test_stream_stdout_stderr (test.fancylogger.FancyLoggerTest) ... ERROR
ERROR
test_uft8_decoding (test.fancylogger.FancyLoggerTest)
Test UTF8 decoding. ... ERROR
test_colorize_always (test.fancylogger.ScreenLogFormatterFactoryTest) ... ok
test_colorize_auto_nontty (test.fancylogger.ScreenLogFormatterFactoryTest) ... ok
test_colorize_auto_tty (test.fancylogger.ScreenLogFormatterFactoryTest) ... FAIL
test_colorize_never (test.fancylogger.ScreenLogFormatterFactoryTest) ... ok
test_fixed_topological_sort (test.missing.TestMissing) ... ok
test_frozendictknownkeys (test.missing.TestMissing)
Tests for FrozenDictKnownKeys. ... FAIL
test_get_class_for (test.missing.TestMissing)
Test get_class_for. ... ok
test_get_subclasses (test.missing.TestMissing)
Test get_subclasses functions. ... ERROR
test_nub_length (test.missing.TestMissing) ... ok
test_nub_membership (test.missing.TestMissing) ... ok
test_nub_order (test.missing.TestMissing) ... ok
test_random_topological_sort (test.missing.TestMissing) ... ERROR
test_tryorfail_no_sleep (test.missing.TestMissing)
test for a retry that succeeds. ... ERROR
test_all_completer (test.optcomplete.OptcompleteTest)
Test the AllCompleter class ... ok
test_base_completer (test.optcomplete.OptcompleteTest)
Test the base Completer class ... ERROR
test_dir_completer (test.optcomplete.OptcompleteTest)
Test DirCompleter ... ok
test_extract_word (test.optcomplete.OptcompleteTest)
Test the extract_word function ... ok
test_file_completer (test.optcomplete.OptcompleteTest)
Test FileCompleter ... ERROR
test_gen_cmdline (test.optcomplete.OptcompleteTest)
Test generation of commndline ... ok
test_known_hosts_completer (test.optcomplete.OptcompleteTest)
Test KnownHostsCompleter ... ok
test_list_completer (test.optcomplete.OptcompleteTest)
Test ListCompleter class ... ERROR
test_none_completer (test.optcomplete.OptcompleteTest)
Test NoneCompleter class ... ERROR
test_regex_completer (test.optcomplete.OptcompleteTest)
Test RegexCompleter ... ERROR
test_configfiles (test.generaloption.GeneralOptionTest)
Test configfiles (base section for empty prefix from auto_section_name) ... ERROR
test_dest_with_dash (test.generaloption.GeneralOptionTest)
Test the renaming of long opts to dest ... ERROR
test_enable_disable (test.generaloption.GeneralOptionTest)
Test the enable/disable prefix ... ERROR
test_error_env_options (test.generaloption.GeneralOptionTest)
Test log error on unknown environment option ... ERROR
test_ext_add (test.generaloption.GeneralOptionTest)
Test add and add_first action ... ERROR
test_ext_add_multi (test.generaloption.GeneralOptionTest)
Test behaviour when 'add' options are used multiple times. ... ERROR
test_ext_date_datetime (test.generaloption.GeneralOptionTest)
Test date and datetime action ... ERROR
test_generate_cmdline (test.generaloption.GeneralOptionTest)
Test the creation of cmd_line args to match options ... ERROR
test_get_by_prefix (test.generaloption.GeneralOptionTest)
Test dict by prefix ... ERROR
test_get_options_by_property (test.generaloption.GeneralOptionTest)
Test get_options_by_property and firends like get_options_by_prefix ... ERROR
test_help (test.generaloption.GeneralOptionTest)
Generate (long) help message ... ERROR
test_help_confighelp (test.generaloption.GeneralOptionTest)
Generate long help message ... ERROR
test_help_long (test.generaloption.GeneralOptionTest)
Generate long help message ... ERROR
test_help_outputformats (test.generaloption.GeneralOptionTest)
Generate (long) rst help message ... ERROR
test_help_short (test.generaloption.GeneralOptionTest)
Generate short help message ... ERROR
test_is_value_a_commandline_option (test.generaloption.GeneralOptionTest)
Test ExtOptionParser is_value_a_commandline_option method ... ERROR
test_loggers (test.generaloption.GeneralOptionTest)
Test getloggers ... ok
test_loglevel (test.generaloption.GeneralOptionTest)
Test the loglevel default setting ... ERROR
test_multiple_init (test.generaloption.GeneralOptionTest)
Test behaviour when creating multiple instances of same GO class ... ERROR
test_nosuchoption (test.generaloption.GeneralOptionTest)
Test catching of non-existing options. ... ERROR
test_optcomplete (test.generaloption.GeneralOptionTest)
Test optcomplete support ... ERROR
test_option_as_value (test.generaloption.GeneralOptionTest)
Test how valid options being used as values are handled. ... ERROR
test_quote (test.generaloption.GeneralOptionTest)
Test quote/unquote ... ok
test_set_columns (test.generaloption.GeneralOptionTest)
Test set_columns function. ... ok
test_store_or_None (test.generaloption.GeneralOptionTest)
Test store_or_None action ... ERROR
test_str_list_tuple (test.generaloption.GeneralOptionTest)
Test strlist / strtuple type ... ERROR
test_assertErrorRegex (test.testing.TestTesting)
Tests for assertErrorRegex method. ... ERROR
test_capture_stdout_stderr (test.testing.TestTesting)
Test capturing of stdout. ... ok
test_convert_exception_to_str (test.testing.TestTesting)
Tests for convert_exception_to_str method. ... ok
test_mock_logmethod (test.testing.TestTesting)
Test the mocked cache logger ... ERROR
test_date_parser (test.dateandtime.DateAndTimeTest)
Test the date_parser ... ok
test_datetime_parser (test.dateandtime.DateAndTimeTest)
Test the date_parser ... ok
test_fancymonth (test.dateandtime.DateAndTimeTest)
Test some of the FancyMonth functions ... ok
test_client (test.rest.RestClientTest)
Do a test api call ... ok
test_request_methods (test.rest.RestClientTest)
Test all request methods ... ok
runv2 (unittest.loader._FailedTest) ... ERROR
test_get_callers_logger (test.exceptions.ExceptionsTest)
Test get_callers_logger function. ... ok
test_loggedexception_callerlogger (test.exceptions.ExceptionsTest)
Test LoggedException custom exception class. ... ERROR
test_loggedexception_defaultlogger (test.exceptions.ExceptionsTest)
Test LoggedException custom exception class. ... ok
test_loggedexception_location (test.exceptions.ExceptionsTest)
Test inclusion of location information in log message for LoggedException. ... ok
test_loggedexception_specifiedlogger (test.exceptions.ExceptionsTest)
Test LoggedException custom exception class. ... ok
test_mk_rst_table (test.docs.DocsTest)
Test mk_rst_table function. ... ok
test_noshell_glob (test.run.TestRun) ... ERROR
test_qa_list_of_answers (test.run.TestRun)
Test qa with list of answers. ... FAIL
test_qa_noqa (test.run.TestRun)
Test noqa ... FAIL
test_qa_regex (test.run.TestRun)
Test regex based q and a (works only for qa_reg) ... FAIL
test_qa_simple (test.run.TestRun)
Simple testing ... FAIL
test_simple (test.run.TestRun) ... ERROR
test_simple_asyncloop (test.run.TestRun) ...Testing this manually, it will print the output (so it runs) but the return code is never parsed / the loop not broken. I tried fixing / working on this a few hours yesterday (down to the level of wiping the entirety of run.py and starting fresh) and wasn't able to make progress, so I need help with here! |
|
@vsoch py36 gives you can test the other moduels usingthe |
|
Yes, that confirms what I've seen when testing - the process just keeps going! Unfortunately run.py is quite a hairball and I wasn't able to debug this. My solution would be a massive highlight and use of the delete key, which I don't think would be received with quite my same enthusiasm, heh. |
|
@vsoch my remark was that with |
|
okay, I deleted my entire anaconda python installation and this has resolved the dependency hell, back to hitting my face against the run.py wall I mean... testing!! :P #neversurrender! |
…"type" so we check that they are included in list instead, and adding basestring
…unctions are trying to accomplish at this point.
|
Anyone care to help with this? I have no idea what these tests are trying to accomplish. To re-implement basic python utils like logging and dictionaries is... not good practice. So the bugs that result, they result from that. That's my main comment. |
|
@vsoch Some of this code indeed reimplements things that were added in python 2.5, 2.6 and 2.7 (like a default dict) I'm all in favour of replacing this code with code from the default python libraries but someone will have to do this work, and we just haven't gotten to it yet... I'm concerned about breaking backwards compatibility, but we could deprecate A LOT of it this code that now can be done by default python. I suggest we add deprecation warnings to the code that can most obviously be replaced by default python, and just not support it in the python3 version anymore. |
|
A lot of this is also adding things to default python that don't make sense later on, like logException This will require a bit more cleanup in other places that use this when we switch that code to python3 compatibility, but the cleanup will involve switching this weird code to using built in python things, which is a good cleanup imho. |
|
@JensTimmerman I strongly agree with this sentiment - it would be more work up front to greatly simplify the modules (and update the software that require them) and I believe this to be the right thing to do to ensure future success of the suite. However I think @boegel has some sort of plan in mind (that he will update here) and so I also respect that line of thinking. TLDR: I will wait for @boegel to comment, and either post his plan OR we can do something more along what you are suggesting. |
|
I largely agree with Jens, there's no point in dragging along stuff that was added for compatibility with Python 2.4 (since One step is the right direction is modernizing W.r.t. this PR: it has become a bit too unwieldy, I think we should use a step-wise approach towards supporting Python 3.x, e.g.:
Meanwhile, we should:
The EasyBuild framework test suite can (also) be used as a smoke test to ensure no functionality is broken in Python 2.x, since it leverages quite a bit of the functionality provided by |
|
i agree that we should tackle module per module; the only requirement here should be that py2 keeps working. EB is however not a good testcase, quite a lot of code in vsc-base is not used in EB (eg vsc.utils.run), and not unittested (enough).
make sure the script works with master first. getting the repos in order, and maybe some easy_install to install the extra test dependencies is probably the largest amount of work. testing with py27 in an el7 box is also more than enough. eb should at some point decide when to drop py26, but i guess el6 has a few more years ahead. when this is done, i have no objections to merge stuff that includes heavy rewrites. if not, i'll probably review things in detail, and revert PRs that i don't like merged (like #265) without full testing. but i have little time, so waiting for my reviews will drag out this effort a lot. |
|
@itkovian who knows, we replaced code without unittests without any further test of code where it have been used. the code looks ok, but that's about it. |
|
@stdweird The code changed in #265 is covered by the tests in W.r.t. testing other stuff that depends on |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here are the minimal changes to vsc-base to have python 3 support, related to hpcugent/vsc-install#83. Again, I only did a test with:
after having installed
vsc-installfrom the other branch! I will add comments on areas I wasn't sure about.