Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rec: Upgrade regression tests to use pytest instead of nose #12912

Merged
merged 4 commits into from
Jul 10, 2023

Conversation

omoerbeek
Copy link
Member

One mysterious failure on Debian bookworm spotted: test_EDNS.py does not seem to set the right edns version on the outgoing query. To be investigated.

Short description

Checklist

I have:

  • read the CONTRIBUTING.md document
  • compiled this code
  • tested this code
  • included documentation (including possible behaviour changes)
  • documented the code
  • added or modified regression test(s)
  • added or modified unit test(s)

@omoerbeek omoerbeek added the rec label Jun 13, 2023
@omoerbeek
Copy link
Member Author

This code (from pythons dns package in message.py):

   def want_dnssec(self, wanted: bool = True) -> None:
        """Enable or disable 'DNSSEC desired' flag in requests.

        *wanted*, a ``bool``.  If ``True``, then DNSSEC data is
        desired in the response, EDNS is enabled if required, and then
        the DO bit is set.  If ``False``, the DO bit is cleared if
        EDNS is enabled.
        """

        if wanted:
            self.ednsflags |= dns.flags.DO
        elif self.opt:
            self.ednsflags &= ~dns.flags.DO

Does seem to behave differently on python 3.11 compared to 3.9.
On 3.11 the flags gets reset to 0 if wanted is False

@omoerbeek
Copy link
Member Author

omoerbeek commented Jun 13, 2023

Reproducer:

import enum

class EDNSFlag(enum.IntFlag):
    #: DNSSEC answer OK
    DO = 0x8000

x = 0xffffffff

print(x)
x &= ~EDNSFlag.DO
print(x)

on a python 3.8 system and a 3.10 system:

4294967295
EDNSFlag.2147483648|1073741824|536870912|268435456|134217728|67108864|33554432|16777216|8388608|4194304|2097152|1048576|524288|262144|131072|65536|16384|8192|4096|2048|1024|512|256|128|64|32|16|8|4|2|1

On a python 3.11.2 system:

4294967295
32767

Note that 32767 = 0x7fff

@peterthomassen
Copy link
Contributor

There seem to be differences in flag handling between Python 3.11.3 and 3.11.4 (python/cpython#105497). Which 3.11 minor version are you testing on?

@omoerbeek
Copy link
Member Author

There seem to be differences in flag handling between Python 3.11.3 and 3.11.4 (python/cpython#105497). Which 3.11 minor version are you testing on?

3.11.2 (also edited above)

@omoerbeek
Copy link
Member Author

BTW, setting this PR to be ready for review. The python shenanigans are unrelated to the move to pytest.

@omoerbeek omoerbeek marked this pull request as ready for review June 13, 2023 19:00
@omoerbeek
Copy link
Member Author

Output on a python 3.11.4 system:

4294967295
4294934527

4294934527 = 0xffff7fff, which looks good.

@zeha
Copy link
Collaborator

zeha commented Jun 13, 2023

doesn't seem quite right:

  - generated xml file: /__w/pdns/pdns/regression-tests.recursor-dnssec/pytest.xml -
  ================= 441 passed, 6 warnings in 490.86s (0:08:10) ==================
  + ./printlogs.py
  Traceback (most recent call last):
    File "/__w/pdns/pdns/regression-tests.recursor-dnssec/./printlogs.py", line 8, in <module>
      e = xml.etree.ElementTree.parse('nosetests.xml')
    File "/usr/lib/python3.9/xml/etree/ElementTree.py", line 1229, in parse
      tree.parse(source, parser)
    File "/usr/lib/python3.9/xml/etree/ElementTree.py", line 569, in parse
      source = open(source, "rb")
  FileNotFoundError: [Errno 2] No such file or directory: 'nosetests.xml'

One mysterious failure on Debian bookworm spotted:
test_EDNS.py does not seem to set the right edns version on the outgoing
query. To be investigated.
omoerbeek added a commit to omoerbeek/pdns that referenced this pull request Jun 14, 2023
omoerbeek added a commit to omoerbeek/pdns that referenced this pull request Jun 14, 2023
@omoerbeek
Copy link
Member Author

FileNotFoundError: [Errno 2] No such file or directory: 'nosetests.xml'

Fixed, thanks!

@omoerbeek
Copy link
Member Author

Added code that skip the tests on specific python versions:

if sys.version_info >= (3, 11) and sys.version_info <= (3, 11, 3):
            raise SkipTest("Test skipped, see https://github.com/PowerDNS/pdns/pull/12912")

@omoerbeek omoerbeek added this to the rec-4.10.0 milestone Jun 14, 2023
Co-authored-by: Chris Hofstaedtler <zeha@users.noreply.github.com>
@omoerbeek omoerbeek merged commit e762e4f into PowerDNS:master Jul 10, 2023
@omoerbeek omoerbeek deleted the rec-nose-to-pytest branch July 10, 2023 08:55
@rgacogne
Copy link
Member

Bad news, this seems broken on 3.11.5 again:

$ python badvers.py
4294967295
32767

rgacogne added a commit to rgacogne/pdns that referenced this pull request Sep 29, 2023
rgacogne added a commit to rgacogne/pdns that referenced this pull request Dec 12, 2023
See PowerDNS#12912 for the longer
explanation.

(cherry picked from commit f57b74a)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants