Tests: Check for ModuleNotFoundError on Python 3.6+

[hroncok]

This tests originally checked for ImportError. Since Python 3.6
ModuleNotFoundError is raised in this context instead, the test didn't work
as it is text based (so exception inheritance does not save the day).

The test now simply checks for any *Error, which makes it less specific,
but still valuable.

Fixes #2132

I consider this a trivial fix (I realize it's not in documentation, but it's in tests, so I think the same logic applies).

[coveralls]

Coverage remained the same at 92.832% when pulling ae7a32a on hroncok:i2132 into da40bcf on pytest-dev:master.

[decentral1se]

Looks legit however, it would be nice to squeeze in the changes for #2134 here too?

Travis supports the 3.6-dev environment. Want to add this in the .travis.yml env matrix?

Could also take a look if you want 👍

[RonnyPfannschmidt]

while i normally would disagree to mix infrastructure changes and code change
, it make sense to add the environments that we would fail to this change

[hroncok]

I am able to enable Travis, but I have zero experience with appveyor

[ignatenkobrain]

@hroncok same as travis, add py36 into appveyor.yml

[hroncok]

@ignatenkobrain Figured that out.

First try, let's see what the CIs will say.

Also not sure whether you want py36-trial and so on. And if so, whether I need to copy paste it around in tox.ini (seems ugly).

[hroncok]

Simply adding py36 to .travis.yml will not work. Need to change the matrix to use 3.6-dev.

[nicoddemus]

If AppVeyor proves difficult, I think we can hold until 3.6 is officially released, testing only on Travis for now.

while i normally would disagree to mix infrastructure changes and code change
, it make sense to add the environments that we would fail to this change

I agree, specially keeping them in separate commits (which is the case already).

Also not sure whether you want py36-trial and so on.

Let's keep testing py35-trial and etc. for now. When 3.6 is officially released we will want to update the other test environments to use the latest version if possible.

[coveralls]

Coverage remained the same at 92.832% when pulling db31305 on hroncok:i2132 into da40bcf on pytest-dev:master.

[nicoddemus]

Yep, it these Python versions are installed on AppVeyor:

04/20/2016  02:59 AM    <DIR>          Python26
04/20/2016  02:59 AM    <DIR>          Python26-x64
08/05/2016  02:33 AM    <DIR>          Python27
08/05/2016  02:34 AM    <DIR>          Python27-x64
04/20/2016  02:59 AM    <DIR>          Python33
04/20/2016  02:59 AM    <DIR>          Python33-x64
08/05/2016  02:35 AM    <DIR>          Python34
08/05/2016  02:35 AM    <DIR>          Python34-x64
08/05/2016  02:36 AM    <DIR>          Python35
08/05/2016  02:36 AM    <DIR>          Python35-x64
               0 File(s)              0 bytes
              10 Dir(s)  27,207,507,968 bytes free

As I said, I'm in favor of just skipping testing 3.6 on AppVeyor until it is available as part of the installation.

[hroncok]

In that case, only enabling on Travis CI. Amended.

[coveralls]

Coverage remained the same at 92.832% when pulling d00f811 on hroncok:i2132 into da40bcf on pytest-dev:master.

[nicoddemus]

Thanks a ton @hroncok!

[decentral1se]

Cool 👍

[coveralls]

Coverage remained the same at 92.832% when pulling 8d51ae0 on hroncok:i2132 into da40bcf on pytest-dev:master.

[blueyed]

I think it might be good to keep the match strict here.
a) It could be changed to cause an ImportError for py36, too.
b) it could be asserted looking at result.stdout.lines directly, using a regular expression.
c) there could be an alternative check for py36 and later.

I would prefer c).

[nicoddemus]

I'm not sure @blueyed, it seems a fuzzy matching for the message here is enough. Do you see a situation where we obtain a match but it is the wrong match? Seems unlikely to me.

But I'm not entirely against the idea either, if @hroncok agrees this can easily be changed to:

expected_exc = 'ModuleNotFoundError' if sys.version_info[:2] >= (3, 6) else 'ImportError'
result.stdout.fnmatch_lines([
     "*ERROR collecting hello.py*",
     "*%s: No module named *asdals*" % expected_exc,

[hroncok]

Amended

[hroncok]

(Failed CI tests seem unrelated to me.)

[nicoddemus]

@hroncok, pushed a commit that moved the compatibility code to _pytest.compat. Thanks again!

[coveralls]

Coverage remained the same at 92.832% when pulling 515fb09 on hroncok:i2132 into 3164062 on pytest-dev:master.