Test: test_two_levels_deep fails when /tmp is symlink

[ghost]

Originally reported by: koobs (Bitbucket: koobs, GitHub: koobs)

---

The following test fails when /tmp is a symlink:

lrwxr-xr-x 1 root wheel 8 Apr 14 2013 tmp@ -> /var/tmp

Testing setuptools 5.4.1, could not set issue Version correctly ("5.4.1" option doesn't exist)

#!
======================================================================
FAIL: test_two_levels_deep (setuptools.tests.test_resources.NamespaceTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/home/user/repos/freebsd/ports/devel/py-setuptools/work/setuptools-5.4.1/setuptools/tests/test_resources.py", line 619, in test_two_levels_deep
    os.path.join(self._tmpdir, "site-pkgs2", "pkg1", "pkg2")])
AssertionError: Lists differ: ['/var/tmp/tests-setuptools-xF... != ['/tmp/tests-setuptools-xFtkbJ...

First differing element 0:
/var/tmp/tests-setuptools-xFtkbJ/site-pkgs/pkg1/pkg2
/tmp/tests-setuptools-xFtkbJ/site-pkgs/pkg1/pkg2

First list contains 1 additional elements.
First extra element 2:
/tmp/tests-setuptools-xFtkbJ/site-pkgs/pkg1/pkg2

- ['/var/tmp/tests-setuptools-xFtkbJ/site-pkgs/pkg1/pkg2',
?   ----

+ ['/tmp/tests-setuptools-xFtkbJ/site-pkgs/pkg1/pkg2',
-  '/var/tmp/tests-setuptools-xFtkbJ/site-pkgs2/pkg1/pkg2',
-  '/tmp/tests-setuptools-xFtkbJ/site-pkgs/pkg1/pkg2']
+  '/tmp/tests-setuptools-xFtkbJ/site-pkgs2/pkg1/pkg2']
?                                         +
----------------------------------------------------------------------
Ran 135 tests in 1.947s

FAILED (failures=1, skipped=1)

---

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/231

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Disable test_two_levels_deep when /tmp is a symlink. The results it is returning are suitable (shouldn't cause errors in runtime). Users are invited to trace the problem and find a solution. Fixes #231.

[ghost]

Original comment by koobs (Bitbucket: koobs, GitHub: koobs):

---

The following two tests, which were disabled, but not fixed in previous versions, currently fail in 19.6.2:

#!

pkg_resources/tests/test_resources.py::TestNamespaces::test_two_levels_deep FAILED

>       assert pkg1.pkg2.__path__ == expected
E       assert ['/var/tmp/re...s2/pkg1/pkg2'] == ['/tmp/real-te...s2/pkg1/pkg2']
E         At index 0 diff: '/var/tmp/real-tests-setuptools-Dr_VEg/site-pkgs/pkg1/pkg2' != '/tmp/real-tests-setuptools-Dr_VEg/site-pkgs/pkg1/pkg2'
E         Full diff:
E         - ['/var/tmp/real-tests-setuptools-Dr_VEg/site-pkgs/pkg1/pkg2',
E         ?   ----
E         + ['/tmp/real-tests-setuptools-Dr_VEg/site-pkgs/pkg1/pkg2',
E         -  '/var/tmp/real-tests-setuptools-Dr_VEg/site-pkgs2/pkg1/pkg2']
E         ?   ----
E         +  '/tmp/real-tests-setuptools-Dr_VEg/site-pkgs2/pkg1/pkg2']

pkg_resources/tests/test_resources.py:675: AssertionError


pkg_resources/tests/test_resources.py::TestNamespaces::test_path_order FAILED

>       assert nspkg.__path__ == [os.path.join(self._real_tmpdir, site,
                                               "nspkg")
                                  for site in site_pkgs]
E       assert ['/var/tmp/re...-pkgs3/nspkg'] == ['/tmp/real-te...-pkgs3/nspkg']
E         At index 0 diff: '/var/tmp/real-tests-setuptools-vsJr0e/site-pkgs/nspkg' != '/tmp/real-tests-setuptools-vsJr0e/site-pkgs/nspkg'
E         Full diff:
E         - ['/var/tmp/real-tests-setuptools-vsJr0e/site-pkgs/nspkg',
E         ?   ----
E         + ['/tmp/real-tests-setuptools-vsJr0e/site-pkgs/nspkg',
E         -  '/var/tmp/real-tests-setuptools-vsJr0e/site-pkgs2/nspkg',
E         ?   ----
E         +  '/tmp/real-tests-setuptools-vsJr0e/site-pkgs2/nspkg',
E         -  '/var/tmp/real-tests-setuptools-vsJr0e/site-pkgs3/nspkg']
E         ?   ----
E         +  '/tmp/real-tests-setuptools-vsJr0e/site-pkgs3/nspkg']

pkg_resources/tests/test_resources.py:706: AssertionError

[ghost]

Original comment by koobs (Bitbucket: koobs, GitHub: koobs):

---

I just noticed that @embray attempted to fix the root cause of these failures in f738da264d57, which explains why they've been re-enabled, but not why they're failing :)

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Rewrite setup/teardown methods as pytest fixtures, encapsulating concepts for clarity. Incidentally, this also fixes #231.

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

So once again I've marked this ticket as closed, though I'm not happy with what I found.

I the above changeset, the tests no longer fail, but not because the underlying issue was addressed, but because pytest's tmpdir fixture automatically resolves the tmpdir to the realpath. This reveals two flaws:

1. The use of a symlink tmpdir in the tests isn't sufficient to cause the failure.
2. If tmpdir were to simply resolve to the symlinked path, it would start to fail again.

I sympathize with Erik's logic. I would have thought the use of a symlinked tmpdir would have been sufficient to elicit the failure, but it is not.

I welcome any advice to elucidate the cause.

[ghost]

Original comment by embray (Bitbucket: embray, GitHub: embray):

---

I'm a little confused, because my original version of the test didn't use the py.test tmpdir fixture at all.

Why the test is failing now is confusing to me though, because when I wrote it I did check that it reproduced the original problem, and that my fix fixed it. I'm sure I could have missed a case though.