Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collecting tests from Python modules not generated by Python files #2369

Closed
Kodiologist opened this issue Apr 16, 2017 · 8 comments
Closed
Labels
status: help wanted developers would like help from experts on this topic topic: collection related to the collection phase type: bug problem that needs to be addressed

Comments

@Kodiologist
Copy link
Contributor

Kodiologist commented Apr 16, 2017

I'm a core developer of the Hy programming language, which is a Lisp syntax for Python, and I'm trying to migrate its test suite from Nose to pytest. The repository has a lot of tests written in Python but also tests written in Hy. Hy itself is implemented in Python and provides a meta-importer so that a Python program can import a Hy module just like a Python module. So in theory, pytest should be able to collect tests from Hy files just as well as Python files, so long as Hy's meta-importer is active. Once a Hy file is loaded, there's basically no difference between Hy and Python; a Hy module is in fact a Python module.

I found a hacky way to run the Hy tests, using this conftest.py, which calls _pytest.python.pytest_pycollect_makemodule in a pytest_collect_file hook. The catch is that this hits assert name.endswith(".py") in _pytest.python, and Hy files end with .hy, not .py. If the assertion is commented out, everything works.

Is there a better way to do this? If not, perhaps you should remove the assertion.

I'm using pytest 3.0.7. Hy is intended to support, and is tested on, Python 2.7 and Python 3.3 and up.

@Kodiologist
Copy link
Contributor Author

I'm now just spoofing the module name to look like it ends with .py, for lack of better ideas.

@RonnyPfannschmidt
Copy link
Member

py.test itself checks the filenames of such modules, in order to override it you will need to create a own collector that handles the import

since you left out the trackback i dont know off-hand whats needed

@Kodiologist
Copy link
Contributor Author

Kodiologist commented Apr 18, 2017

Sorry, here's an example of the backtraces I get when running pytest on my repository without the module-name spoof:

_______________ ERROR collecting tests/native_tests/extra/reserved.hy ________________
../lib/python3.6/site-packages/_pytest/runner.py:163: in __init__
    self.result = func()
../lib/python3.6/site-packages/_pytest/main.py:476: in _memocollect
    return self._memoizedcall('_collected', lambda: list(self.collect()))
../lib/python3.6/site-packages/_pytest/main.py:347: in _memoizedcall
    res = function()
../lib/python3.6/site-packages/_pytest/main.py:476: in <lambda>
    return self._memoizedcall('_collected', lambda: list(self.collect()))
../lib/python3.6/site-packages/_pytest/python.py:412: in collect
    return super(Module, self).collect()
../lib/python3.6/site-packages/_pytest/python.py:332: in collect
    l.sort(key=lambda item: item.reportinfo()[:2])
../lib/python3.6/site-packages/_pytest/python.py:332: in <lambda>
    l.sort(key=lambda item: item.reportinfo()[:2])
../lib/python3.6/site-packages/_pytest/python.py:264: in reportinfo
    modpath = self.getmodpath()
../lib/python3.6/site-packages/_pytest/python.py:238: in getmodpath
    assert name.endswith(".py")
E   AssertionError

@RonnyPfannschmidt RonnyPfannschmidt added type: bug problem that needs to be addressed topic: collection related to the collection phase status: help wanted developers would like help from experts on this topic labels Apr 18, 2017
@RonnyPfannschmidt
Copy link
Member

this one is a bit tricky. the "python module" collector of pytest expects a python module for the filename

for now its a pretty good idea to just copy the method and replace its constants replacing .py with .py

@Kodiologist
Copy link
Contributor Author

Okay, thanks.

@nicoddemus
Copy link
Member

Taking a quick look at the code, it seems to me removing the assertion as @Kodiologist suggests is OK:

        for node in chain:
            if isinstance(node, Instance):
                continue
            name = node.name
            if isinstance(node, Module):
                assert name.endswith(".py")
                name = name[:-3]
                if stopatmodule:
                    if includemodule:
                        parts.append(name)
                    break
            parts.append(name)

I think the assertion is there just because of the name = name[:-3] below it, which could be replaced by a safer os.path.splitext(name)[0]. If all tests pass I don't think this will bring any problems.

Would you care to give it a shot @Kodiologist?

@Kodiologist
Copy link
Contributor Author

@nicoddemus That seems to work fine for Hy's test suite.

@nicoddemus
Copy link
Member

Would you like to open a PR? Then we can test it against pytest test suite. 😉

This was referenced Mar 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: help wanted developers would like help from experts on this topic topic: collection related to the collection phase type: bug problem that needs to be addressed
Projects
None yet
Development

No branches or pull requests

3 participants