Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pci.PCIDevices(): Fix Popen("lspci -mn") for Py3 using universal_newlines=True #38

Conversation

bernhardkaindl
Copy link
Collaborator

@bernhardkaindl bernhardkaindl commented May 15, 2023

This PR fixes one of the Python3 fixes requested in #21:

Two issues in the Python3 migration of xcp.pci weren't detected by the new test tests_videoclass() in tests/test_pci.py:

  1. The use of mock to mock a call to Popen() (using open() didn't detect a usage bug.
  2. The use of a the fake tests/data/pci.ids, not the real /usr/share/hwdata/pci.ids.

This PR fixes point 1.:

Add a new test case which does not use mock(), and fix the issue 1.

Mocking xcp.pci.subprocess.Popen("lspci -nm").stdout using open("tests/data/lspci-mn")
does not model the actual behavior of subprocess.Popen().stdout in Python3:

From: https://docs.python.org/3.6/library/subprocess.htmlpopen-constructor:

"If encoding or errors are specified, the file objects stdin, stdout and stderr
are opened in text mode with the specified encoding and errors, as described
above in Frequently Used Arguments.
If universal_newlines is True, they are opened in text mode with default
encoding. >>>Otherwise, they are opened as binary streams.<<<"

By it's nature, mocking Popen() does not validate it's correct use in xcp.pci
and therfore, it failed to detect the Python3 problem in xcp/pci.py

The validation of xcp.pci.PCIDevices() using self.PopenWrapper(), replacing
"lspci -mn" with "cat tests/data/lspci-mn" is able to detect such usage bugs.

Correctly modeling subprocess.Popen() using mock is nontrivial, and would require something like the testfixtures library and even that would still be a mock and use the actual subprocess class:
https://stackoverflow.com/questions/25692440/mocking-a-subprocess-call-in-python
https://chromium.googlesource.com/chromium/tools/depot_tools/+/refs/heads/main/tests/subprocess2_test.py

Mocking is better suited when the test knows how an API must be called, which isn't the case here, espcially the use of open("tests/data/lspci-mn") is wrong, which leads to the test not catching the Python3 bug:

with patch("xcp.pci.subprocess.Popen") as popen_mock, \
     open("tests/data/lspci-mn") as fake_data:   # <- The mock uses open() here,
                                                 # which is wrong and hard to fix using mock.
    popen_mock.return_value.stdout.__iter__ = Mock(return_value=iter(fake_data))
    devs = PCIDevices()

This PR therefore adds a new test case which does not use mock and changes the "mock test" to accept the fix.

But in principle after this PR is merged or as a final commit to it, the old mock-only test could be removed as obsolete and not up to the task.

@codecov
Copy link

codecov bot commented May 15, 2023

Codecov Report

Merging #38 (2eb9203) into master (5c36c6e) will not change coverage.
The diff coverage is 100.00%.

❗ Current head 2eb9203 differs from pull request most recent head 6bdd72b. Consider uploading reports for the commit 6bdd72b to get more accurate results

@@           Coverage Diff           @@
##           master      #38   +/-   ##
=======================================
  Coverage   69.27%   69.27%           
=======================================
  Files          20       20           
  Lines        3417     3417           
=======================================
  Hits         2367     2367           
  Misses       1050     1050           
Flag Coverage Δ
unittest 69.27% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
xcp/pci.py 78.37% <100.00%> (ø)

@bernhardkaindl bernhardkaindl force-pushed the Py3-fix-PCIDevices-Popen-universal_newlines branch 3 times, most recently from 3e5adb0 to 6bbad2b Compare May 15, 2023 19:19
@bernhardkaindl bernhardkaindl changed the title xcp.pci: Fix use of Popen() for "lspci -mn" for Pyhon3: use universal_newlines=True pci.PCIDevices(): Fix Popen("lspci -mn") for Py3 using universal_newlines=True May 15, 2023
Extracts assert_videoclass_devices() for reuse by a new testcase.

Also extract mock_lspci_using_open_testfile() from tests_videoclass:

mock_lspci_using_open_testfile() does one thing and extracting helps
to make the use of the new function assert_videoclass_devices() more
obvious and comprehensible from simply looking at the code quickly.

And also fix the mypy type warning by adding missing type comments.

Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
Fixes one part of xenserver#21 by calling Popen(lspci -nm, universal_newlines=True)

This makes Popen().stdout it return an __inter__: string, which the code
expects (otherwise, it opens the stdout pipe as binary stream):

From: https://docs.python.org/3.6/library/subprocess.html#popen-constructor:

"If encoding or errors are specified, the file objects stdin, stdout and stderr
are opened in text mode with the specified encoding and errors, as described
above in Frequently Used Arguments.
If universal_newlines is True, they are opened in text mode with default
encoding. >>>__Otherwise, they are opened as binary streams.__<<<"

The new test tests xcp.pci.PCIDevices() by setting PATH to a shell script which
simluates "lspci -mn" with "cat tests/data/lspci-mn", so calls the real Popen.

Final thoughts on using mock:

Mocking xcp.pci.subprocess.Popen.stdout using open("tests/data/lspci-mn")
does not model the actual behavior of subprocess.Popen().std* in Python3:

By it's nature, mocking Popen() does not validate it's correct use in xcp.pci
and therfore, it failed to detect the Python3 problem in `xcp/pci.py`

Correctly modeling subprocess.Popen() without really calling it is nontrivial,
and would require something like the testfixtures library and even that would
still be a mock and use the actual subprocess class. Mocking is better suited
when the test knows how an API must be called, which isn't the case here.

Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
@bernhardkaindl bernhardkaindl force-pushed the Py3-fix-PCIDevices-Popen-universal_newlines branch from 2eb9203 to 6bdd72b Compare May 16, 2023 14:55
@bernhardkaindl bernhardkaindl merged commit c583723 into xenserver:master May 16, 2023
bernhardkaindl added a commit to xenserver-next/python-libs that referenced this pull request May 17, 2023
This PR fixes the Popen() in xcp.net.biosdevname like xenserver#38 fixed
the Popen of xcp.pci.PCIDevices() using universial_newlines=True,
to make the Popen return strings as expected by the code, not bytes.

Redirects the call to /sbin/biosdevname to tests/data/biosdevname
to provide the test data to tests/test_biosdevname.py:test_nomock()

Adds a new test case because the old test case is using the same
mock which already failed to find the issue in xenserver#38.

Unlike xenserver#38, the new test can't do this by setting PATH="tests/data"
because /sbin/biosdevname is an absolute path.

To ensure a correct test, the new test avoids mocking Popen() by
installing a wrapper for subprocess.Popen() in tests/conftest.py.

This needs to be done a global wrapper in pytest in tests/conftest.py
because when another test module in the same pytest run imports the
testee module first and does not provide the wrapped subprocess.Popen,
when importing it, the wrapper for Popen is not installed.

The benefit is that this wrapper can be used by all tests for modules
which use subprocess.Popen(), and trace all of them from a central
place. This can e.g. be used to check the subprocess call done during
a testsuite run.
bernhardkaindl added a commit to xenserver-next/python-libs that referenced this pull request May 17, 2023
This PR fixes the Popen() in xcp.net.biosdevname like xenserver#38 fixed
the Popen of xcp.pci.PCIDevices() using universial_newlines=True,
to make the Popen return strings as expected by the code, not bytes.

Redirects the call to /sbin/biosdevname to tests/data/biosdevname
to provide the test data to tests/test_biosdevname.py:test_nomock()

Adds a new test case because the old test case is using the same
mock which already failed to find the issue in xenserver#38.

Unlike xenserver#38, the new test can't do this by setting PATH="tests/data"
because /sbin/biosdevname is an absolute path.

To ensure a correct test, the new test avoids mocking Popen() by
installing a wrapper for subprocess.Popen() in tests/conftest.py.

This needs to be done a global wrapper in pytest in tests/conftest.py
because when another test module in the same pytest run imports the
testee module first and does not provide the wrapped subprocess.Popen,
when importing it, the wrapper for Popen is not installed.

The benefit is that this wrapper can be used by all tests for modules
which use subprocess.Popen(), and trace all of them from a central
place. This can e.g. be used to check the subprocess call done during
a testsuite run.

Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
@bernhardkaindl bernhardkaindl deleted the Py3-fix-PCIDevices-Popen-universal_newlines branch May 17, 2023 22:11
bernhardkaindl added a commit to rosslagerwall/python-libs that referenced this pull request May 8, 2024
…lize-for-now

pre-commit: Don't string-normalize quotes in xen-bugtool for now
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants