No coverage reports created when running mypy with input file

[KonstiB]

Hello,
I am not that familiar with coverage.py, as I am only using it for part of an evaluation, but I ran into strange behavior and couldn't find anything related here or on stackoverflow. Hopefully I am not doing something completely wrong (or stupid).

- What happened:
I am trying to produce coverage reports for the mypy module, using generated inputs. The problem is whenever I run mypy with an input file no .coverage file is created, but otherwise (meaning no file given) a report is created.

- What you did:
I cloned the mypy repository (https://github.com/python/mypy) and installed it locally in a venv using the pip -e . command. I then run the command coverage run -m mypy foo.py, with foo.py beeing an empty file (or containing real code, does not seem to matter) in some folder that is not a python project(for simplicity). Though mypy returns with exactly the same output as it does, when I run
mypy foo.py, no .coverage file is created.

The weird thing is, when i run the command coverage run -m mypy (without the required file parameter) it as expected, returns the error: mypy: error: Missing target module, package, files, or command., however the .coverage file is created. This is also the case when I run coverage run -m mypy --help but I guess this is just for the same reason, as also no input file is given.

- What kind of help you need:
Can you explain to me, why no coverage file is created under this circumstance or how I can get coverage reports for the mypy module when inserting python files?

[offby1]

Could you provide more detail about what you did? Specifically:

• which commit of mypy was current after you cloned? You can find out by cd-ing into the mypy directory, and then typing git log -1. When I do that, for example, I see

commit 0412590ba13395a04f122a5dd83764db9d001fe6 (HEAD -> master, origin/master, origin/HEAD)
Author: wyattscarpenter <wyattscarpenter@gmail.com>
Date:   2024-09-02 06:26:53 -0700

    [nit] conf.py: annotate the type, instead of ignoring the error (#17727)

• could you describe precisely how you installed "mypy"? You said using the pip -e . command, but pip -e . is not a complete command. I assume you typed pip install -e ., but am not certain; it's important to know what your current directory was when you typed that.
• similarly, please describe how you installed coverage
• tell us exactly what you typed in order to run coverage. For example, I typed "/Users/not-workme/git-repositories/3rd-party/coveragepy-issues-1845/.venv/bin/coverage run -m mypy foo.py but I saw something very different than what you described (namely

mypy: "mypy/mypy/typeshed/stdlib/types.pyi" shadows library module "types"
note: A user-defined top-level module with name "types" is not supported

), so precision is important here.

[KonstiB]

I'm sorry, of course the command i used to install mypy was pip install -e ..

Here is what I do precisely:
First I create a virtual environment with: python3.10 -m venv venv and activate it using:
source venv/bin/activate. I have a folder repos/ containing multiple cloned python packages, also containing mypy (which I cloned with the command: git clone https://github.com/python/mypy.git). After cloning mypy into the repos/ folder, I entered it (via cd mypy/) and run pip install -e .. As far as I can tell the installation succeeded, since pip show mypy shows under editable project location: some_path/repos/mypy.

The commit shown using git log -1 is: fe15ee69b9225f808f8ed735671b73c31ae1bed8, which is the same as shown when running pip show mypy under version.

I installed coverage, using pip in the same virtual environment venv, (command: pip install coverage). The installed version is 7.4.3, however the same happens when I create a completely new environment and install version 7.6.1 (using also pip install coverage)

I though I had seen the error you described and maybe had done something earlier on, to fix it. However using a different computer everything worked the same, as I described.

[offby1]

Thanks, I certainly see what you're talking about. Here's how I reproduced it on recent MacOS:

$ mkdir coverage-1845
$ cd coverage-1845
$ pyenv install 3.10
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.10.15.tar.xz...
-> https://www.python.org/ftp/python/3.10.15/Python-3.10.15.tar.xz
Installing Python-3.10.15...
python-build: use tcl-tk from homebrew
python-build: use readline from homebrew
python-build: use ncurses from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.10.15 to /Users/not-workme/.pyenv/versions/3.10.15
$ ~/.pyenv/versions/3.10.15/bin/python3.10 -m venv venv
$ source venv/bin/activate
$ mkdir repos
$ cd repos
$ git clone https://github.com/python/mypy.git
Cloning into 'mypy'...
remote: Enumerating objects: 97399, done.
remote: Counting objects: 100% (437/437), done.
remote: Compressing objects: 100% (337/337), done.
remote: Total 97399 (delta 193), reused 304 (delta 100), pack-reused 96962 (from 1)
Receiving objects: 100% (97399/97399), 65.95 MiB | 28.27 MiB/s, done.
Resolving deltas: 100% (75387/75387), done.
$ cd mypy/
$ git reset --hard fe15ee69b9225f808f8ed735671b73c31ae1bed8
HEAD is now at fe15ee69b [mypyc] Avoid uses of _PyObject_CallMethodOneArg on 3.13 (#17526)
$ pip install -e .
Obtaining file:///Users/not-workme/git-repositories/me/coverage-1845/repos/mypy
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting mypy-extensions>=1.0.0
  Using cached mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Collecting tomli>=1.1.0
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting typing-extensions>=4.6.0
  Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Building wheels for collected packages: mypy
  Building editable for mypy (pyproject.toml) ... done
  Created wheel for mypy: filename=mypy-1.12.0+dev.fe15ee69b9225f808f8ed735671b73c31ae1bed8-0.editable-py3-none-any.whl size=8239 sha256=6bcc0b76a4eea3e6fcac8fb0a32f5176d642f0740945bd807230cf50c4c5c490
  Stored in directory: /private/var/folders/b0/_p0c_57s5n90cy4njdgd8vhw0000gq/T/pip-ephem-wheel-cache-y9zfb65z/wheels/5d/66/8f/ed18142883942a03238d09c4eeb9c71708f5bba980d4b6db9f
Successfully built mypy
Installing collected packages: typing-extensions, tomli, mypy-extensions, mypy
Successfully installed mypy-1.12.0+dev.fe15ee69b9225f808f8ed735671b73c31ae1bed8 mypy-extensions-1.0.0 tomli-2.0.1 typing-extensions-4.12.2

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
$ pip install coverage
Collecting coverage
  Downloading coverage-7.6.1-cp310-cp310-macosx_11_0_arm64.whl (207 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 207.1/207.1 kB 6.3 MB/s eta 0:00:00
Installing collected packages: coverage
Successfully installed coverage-7.6.1

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
$ cd ..
$ touch foo.py
$ coverage run -m mypy foo.py
mypy: "mypy/mypy/typeshed/stdlib/typing_extensions.pyi" shadows library module "typing_extensions"
note: A user-defined top-level module with name "typing_extensions" is not supported
$ ls -al
total 0
drwxr-xr-x  4 not-workme staff  128 Sep  9 08:34 .
drwxr-xr-x  5 not-workme staff  160 Sep  9 08:33 ..
-rw-r--r--  1 not-workme staff    0 Sep  9 08:34 foo.py
drwxr-xr-x 34 not-workme staff 1088 Sep  9 08:33 mypy
$ coverage run mypy foo.py
Can't find '__main__' module in 'mypy'
$  mypy foo.py
Success: no issues found in 1 source file
$ coverage run foo.py
$ ls -al
total 52
drwxr-xr-x  6 not-workme staff   192 Sep  9 08:38 .
drwxr-xr-x  5 not-workme staff   160 Sep  9 08:33 ..
-rw-r--r--  1 not-workme staff 53248 Sep  9 08:38 .coverage
drwxr-xr-x  5 not-workme staff   160 Sep  9 08:35 .mypy_cache
-rw-r--r--  1 not-workme staff     0 Sep  9 08:34 foo.py
drwxr-xr-x 34 not-workme staff  1088 Sep  9 08:33 mypy
$

[KonstiB]

Ok, using your procedure is was able to create a very similar bug:

$ mkdir tmp
$ cd tmp/
/tmp$ python3.10 -m venv venv
/tmp$ source venv/bin/activate
/tmp$ mkdir repos
/tmp$ cd repos/
/tmp/repos$ git clone https://github.com/python/mypy.git
Cloning into 'mypy'...
remote: Enumerating objects: 97399, done.
remote: Counting objects: 100% (437/437), done.
remote: Compressing objects: 100% (337/337), done.
remote: Total 97399 (delta 194), reused 302 (delta 100), pack-reused 96962 (from 1)
Receiving objects: 100% (97399/97399), 65.95 MiB | 1.70 MiB/s, done.
Resolving deltas: 100% (75396/75396), done.
/tmp/repos$ cd mypy/
/tmp/repos/mypy$ git reset --hard fe15ee69b9225f808f8ed735671b73c31ae1bed8
HEAD is now at fe15ee69b [mypyc] Avoid uses of _PyObject_CallMethodOneArg on 3.13 (#17526)
/tmp/repos/mypy$ pip install -e .
Obtaining file:.../tmp/repos/mypy
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting typing-extensions>=4.6.0
  Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Collecting tomli>=1.1.0
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting mypy-extensions>=1.0.0
  Using cached mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Building wheels for collected packages: mypy
  Building editable for mypy (pyproject.toml) ... done
  Created wheel for mypy: filename=mypy-1.12.0+dev.fe15ee69b9225f808f8ed735671b73c31ae1bed8-0.editable-py3-none-any.whl size=8223 sha256=03e1a63e251dc464296b104428325f40afe5629cfab7fe09e8834422ab61b819
  Stored in directory: /tmp/pip-ephem-wheel-cache-vvnelq28/wheels/b8/a6/6d/4374bfa2c38d232f2607a6f70432a43da17ecfc4208baf971f
Successfully built mypy
Installing collected packages: typing-extensions, tomli, mypy-extensions, mypy
Successfully installed mypy-1.12.0+dev.fe15ee69b9225f808f8ed735671b73c31ae1bed8 mypy-extensions-1.0.0 tomli-2.0.1 typing-extensions-4.12.2
/tmp/repos/mypy$ pip install coverage
Collecting coverage
  Using cached coverage-7.6.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (234 kB)
Installing collected packages: coverage
Successfully installed coverage-7.6.1
/tmp/repos/mypy$ cd ..
/tmp/repos$ touch foo.py
/tmp/repos$ coverage run -m mypy foo.py 
mypy: "mypy/mypy/typeshed/stdlib/sys/__init__.pyi" shadows library module "sys"
note: A user-defined top-level module with name "sys" is not supported

It is not exactly the same error, but maybe my fix still applies. If I just get one more directory up, create foo.py and run coverage there, mypy reports success (though no .coverage file is created, as I mentioned in my initial post):

tmp/repos$ cd ..
/tmp$ touch foo.py
/tmp$ coverage run -m mypy foo.py 
Success: no issues found in 1 source file
/tmp$ ls -a
.  ..  foo.py  .mypy_cache  repos  venv

This is also the case when I specify an Outfile:

/tmp$ coverage run --data-file=".cov" -m mypy foo.py 
Success: no issues found in 1 source file
/tmp$ ls -a
.  ..  foo.py  .mypy_cache  repos  venv

Normaly coverage warns that no data was collected, but that is also not the case here.

Hopefully this is useful in recreating the bug.

(editet because I missed that the command ls does not show hidden files, but the coverage file is also not there using
ls -a)

[KonstiB]

Furthermore when I run:

/tmp$ coverage run --data-file=".cov" -m mypy
usage: mypy [-h] [-v] [-V] [more options; see below]
            [-m MODULE] [-p PACKAGE] [-c PROGRAM_TEXT] [files ...]
mypy: error: Missing target module, package, files, or command.
/tmp$ ls -a
.  ..  .cov  foo.py  .mypy_cache  repos  venv

now there is a coverage file.

[KonstiB]

Any new information?
I am on a deadline and would shift to other programs for what i am testing, if it is unclear why coverage is sometimes not producing reports for mypy.