Merge pull request pytest-dev#2297 from nicoddemus/init-files-docs

Attempt to clarify the confusion regarding __init__ files and unique test names

doc/en/goodpractices.rst

@@ -30,68 +30,106 @@ Within Python modules, ``pytest`` also discovers tests using the standard
 
 
 Choosing a test layout / import rules
-------------------------------------------
+-------------------------------------
 
 ``pytest`` supports two common test layouts:
 
-* putting tests into an extra directory outside your actual application
- code, useful if you have many functional tests or for other reasons
- want to keep tests separate from actual application code (often a good
- idea)::
+Tests outside application code
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
- setup.py # your setuptools Python package metadata
+Putting tests into an extra directory outside your actual application code
+might be useful if you have many functional tests or for other reasons want
+to keep tests separate from actual application code (often a good idea)::
+
+ setup.py
  mypkg/
  __init__.py
- appmodule.py
+ app.py
+ view.py
  tests/
  test_app.py
+ test_view.py
  ...
 
+This way your tests can run easily against an installed version
+of ``mypkg``.
+
+Note that using this scheme your test files must have **unique names**, because
+``pytest`` will import them as *top-level* modules since there are no packages
+to derive a full package name from. In other words, the test files in the example above will
+be imported as ``test_app`` and ``test_view`` top-level modules by adding ``tests/`` to
+``sys.path``.
 
-* inlining test directories into your application package, useful if you
- have direct relation between (unit-)test and application modules and
- want to distribute your tests along with your application::
+If you need to have test modules with the same name, you might add ``__init__.py`` files to your
+``tests`` folder and subfolders, changing them to packages::
 
- setup.py # your setuptools Python package metadata
+ setup.py
  mypkg/
- __init__.py
- appmodule.py
  ...
- test/
- test_app.py
- ...
+ tests/
+ __init__.py
+ foo/
+ __init__.py
+ test_view.py
+ bar/
+ __init__.py
+ test_view.py
+
+Now pytest will load the modules as ``tests.foo.test_view`` and ``tests.bar.test_view``, allowing
+you to have modules with the same name. But now this introduces a subtle problem: in order to load
+the test modules from the ``tests`` directory, pytest prepends the root of the repository to
+``sys.path``, which adds the side-effect that now ``mypkg`` is also importable.
+This is problematic if you are using a tool like `tox`_ to test your package in a virtual environment,
+because you want to test the *installed* version of your package, not the local code from the repository.
+
+In this situation, it is **strongly** suggested to use a ``src`` layout where application root package resides in a
+sub-directory of your root::
+
+ setup.py
+ src/
+ mypkg/
+ __init__.py
+ app.py
+ view.py
+ tests/
+ __init__.py
+ foo/
+ __init__.py
+ test_view.py
+ bar/
+ __init__.py
+ test_view.py
 
-Important notes relating to both schemes:
 
-- **make sure that "mypkg" is importable**, for example by typing once::
+This layout prevents a lot of common pitfalls and has many benefits, which are better explained in this excellent
+`blog post by Ionel Cristian Mărieș <https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure>`_.
 
- pip install -e . # install package using setup.py in editable mode
+Tests as part of application code
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-- **avoid "__init__.py" files in your test directories**.
- This way your tests can run easily against an installed version
- of ``mypkg``, independently from the installed package if it contains
- the tests or not.
+Inlining test directories into your application package
+is useful if you have direct relation between tests and application modules and
+want to distribute them along with your application::
 
-- With inlined tests you might put ``__init__.py`` into test
- directories and make them installable as part of your application.
- Using the ``pytest --pyargs mypkg`` invocation pytest will
- discover where mypkg is installed and collect tests from there.
- With the "external" test you can still distribute tests but they
- will not be installed or become importable.
+ setup.py
+ mypkg/
+ __init__.py
+ app.py
+ view.py
+ test/
+ __init__.py
+ test_app.py
+ test_view.py
+ ...
+
+In this scheme, it is easy to your run tests using the ``--pyargs`` option::
 
-Typically you can run tests by pointing to test directories or modules::
+ pytest --pyargs mypkg
 
- pytest tests/test_app.py # for external test dirs
- pytest mypkg/test/test_app.py # for inlined test dirs
- pytest mypkg # run tests in all below test directories
- pytest # run all tests below current dir
- ...
+``pytest`` will discover where ``mypkg`` is installed and collect tests from there.
+
+Note that this layout also works in conjunction with the ``src`` layout mentioned in the previous section.
 
-Because of the above ``editable install`` mode you can change your
-source code (both tests and the app) and rerun tests at will.
-Once you are done with your work, you can `use tox`_ to make sure
-that the package is really correct and tests pass in all
-required configurations.
 
 .. note::
 
@@ -144,7 +182,13 @@ for installing your application and any dependencies
 as well as the ``pytest`` package itself. This ensures your code and
 dependencies are isolated from the system Python installation.
 
-If you frequently release code and want to make sure that your actual
+You can then install your package in "editable" mode::
+
+ pip install -e .
+
+which lets you change your source code (both tests and application) and rerun tests at will.
+
+Once you are done with your work and want to make sure that your actual
 package passes all tests you may want to look into `tox`_, the
 virtualenv test automation tool and its `pytest support
 <https://tox.readthedocs.io/en/latest/example/pytest.html>`_.
@@ -154,11 +198,6 @@ options. It will run tests against the installed package and not
 against your source code checkout, helping to detect packaging
 glitches.
 
-Continuous integration services such as Jenkins_ can make use of the
-``--junitxml=PATH`` option to create a JUnitXML file and generate reports (e.g.
-by publishing the results in a nice format with the `Jenkins xUnit Plugin
-<https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin>`_).
-
 
 Integrating with setuptools / ``python setup.py test`` / ``pytest-runner``
 --------------------------------------------------------------------------

tox.ini

@@ -106,6 +106,8 @@ commands=
  pytest -ra {posargs:testing/test_unittest.py}
 
 [testenv:docs]
+skipsdist=True
+usedevelop=True
 basepython=python
 changedir=doc/en
 deps=