Pypy failures debug

[qwcode]

I'm unable to recreate 10 of 12 of the pypy failures locally that travis reports.
posting this just to try to get some logging for what I guess might be happening.
thinking it might be related to os.removedirs() call in util.renames

[travisbot]

This pull request passes (merged 626a662 into 6e10adf).

[qwcode]

ok, looks like "empty" pkg dirs are not getting removed by os.removedirs() due to pycache dirs

[travisbot]

This pull request passes (merged 16d62fe into 6e10adf).

[travisbot]

This pull request passes (merged c8fa412 into 6e10adf).

[qwcode]

the installed-files.txt log is showing *.pyc files that were never created, but instead there are pycache folders.

[qwcode]

I wish I knew how to make my local pypy install generate pycache folders?
we're both using v1.9

[travisbot]

This pull request passes (merged 3cb94e4 into 6e10adf).

[travisbot]

This pull request passes (merged 0fbebc9 into 6e10adf).

[travisbot]

This pull request passes (merged 86f757a into 6e10adf).

[travisbot]

This pull request passes (merged 2927078 into 6e10adf).

[qwcode]

posted an email to pypy email list for help on mechanism that determines when/how pypy uses pycache folders.

[pnasrat]

@qwcode http://www.python.org/dev/peps/pep-3147/ gives the PEP for this including flowchart

According to how we use scripttest we have

def _ignore_file(self, fn):
        if fn.endswith('__pycache__') or fn.endswith(".pyc"):
            result = True
        else:
            result = super(TestPipEnvironment, self)._ignore_file(fn)
        return result

Will try look whilst that isn't working.

[qwcode]

got an answer from the pypy list (see below)
it's not a normal pypy behavior to create those, but rather a debian-specific patch.

@pnasrat , this isn't something just the test code needs to ignore.
the real pip uninstall code has to be fixed to deal with removing this.
the patch is causing the installed-files.txt manifest to be incorrect, so we don't remove the pycache internal folder,
and then the os.removedirs() call fails to remove what should be an empty package dir.

we either have to:

1. change the logic to look for pycache folders when adding files to the uninstall set
   or
2. when removing code, always just look for a pycache folder

I'm inclined to do #1.

---

Stefano Rivera stefano@rivera.za.net via python.org
2:44 AM (5 hours ago)

to pypy-dev
Hi Armin (2012.07.31_09:25:08_+0200)

PyPy contains no logic to create or search for pycache. A grep
tells me that neither does CPython 2.7. You are probably confused by
py.test creating them.

A Debian/Ubuntu pypy does, though. And IIRC, that's what travis CI is
using.

It's the pep3147 patches:
http://anonscm.debian.org/gitweb/?p=collab-maint/pypy.git;a=tree;f=debian/patches

I thought it would be a useful feature, to make transition between pypy
versions in Debian easier. Proposed the patches in #pypy, and the
reception was lukewarm, so I never pushed harder.

[qwcode]

the full thread from the pypy list on this.
http://mail.python.org/pipermail/pypy-dev/2012-July/thread.html#10369

[qwcode]

probably won't get back to working on a fix til friday.

[qwcode]

although the immediate concern is to have a fix for this in pip 1.2, I'm inclined to believe the debian patch could be improved to not throw off the setuptools --record option like it's doing. I looked at the setuptools source, and it sure likes like the --record option is just walking pkg dir after a byte compile to get the manifest, so it seems the inline *.pyc files are present for a bit? and then removed by the debian patch? not sure. will look confirm later with some debugging.

[travisbot]

This pull request passes (merged e287e00 into 6e10adf).

[travisbot]

This pull request passes (merged 23cb551 into 6e10adf).

[qwcode]

ok, good, down to the last 2 failures
closing this debug pull