Simplify tests, shorten buildir name on windows

[mattip]

Taken from #135, without the pytest-xdist module name randomization

This should be smaller and easier to review:

• explicitly test c file is not regenerated rather than retesting, should speed up tests
• solve windows test crashes due to too-long filenames

[mattip]

I am not sure why CI is failing. When I run the tests locally they pass.

[arigo]

Force the tests to re-run (unsure how), and see if they still fail? If they do, force a re-run on the base (unsure how) and see if they still pass? If all that is true, then dig deeper.

[arigo]

The failure we get is genuine. I think you can get it too by running both test_verify.py and test_vgen.py in the same run. Then it tries to load a library made in the test_verify mode while assuming it was made in the test_vgen mode, or something.

[mattip]

When I run the tests locally the cffi.verifier.cleanup_tmpdir() call cleans out the old module. I am not sure why in this test only the cleanup does not work.

[arigo]

Maybe the CI runs pytest-xdist anyway? Or some other pytest version, or other plugins, etc.? Something that would make the order of execution different, e.g. test_vgen.setup_module could be called before the actual tests of test_verify run.

[mattip]

I am stumped. No pytest-xdist plugin is used:

platform linux -- Python 3.13.0, pytest-8.3.3, pluggy-1.5.0

and the tests are run in the correct order:

  ../../../project/cffi/testing/cffi0/test_verify.py ...................s. [ 42%]
  .....s.....s............................................................ [ 46%]
  .....................................................s...s.............s [ 50%]
  s...                                                                     [ 50%]
  ../../../project/cffi/testing/cffi0/test_version.py .....                [ 51%]
  ../../../project/cffi/testing/cffi0/test_vgen.py ...................s... [ 52%]
  ...s.....s.............................................................. [ 56%]
  ...................................................s...s.............ss. [ 60%]
  .F                                

When I can I will add debug cruft to cleanup_tempdir(), maybe somehow it is skipping the module in question.

[mattip]

Another strange thing I noticed in the codebase: there is both testing/cffi0/test_verify.py and testing/cffi1/test_verify1.py. These files echo each other: they have many of the same tests with slight variations. Some of these repeated tests are quite "expensive" on PyPy's nightly testing on windows. For instance:

• test_struct_bad_sized_integer takes 101 seconds
• test_global_const_int_size takes ~75 secs

[mattip]

Still stumped. I used cos to make sure the test is using a unique function call and print the remaining files in the cleanup_tmp() call at the beginning of test_vgen.py, there are no remnants that might trip up the call to verify(). It still seems like something is being cached from test_verify into test_vgen somewhere.

[mattip]

CI was passing so I removed the debug cruft and enabled windows tests. Let's see how long it takes. I am not sure why when using a module name I had to create a unique module name but 🤷. I guess #135 avoided this problem by making all the module names unique.

[mattip]

25 minutes for windows tests, I guess that is acceptable? It could be less if the repetitive tests are removed from cffi0/test_verify and cffi1/test_verify1.py

[arigo]

cffi0 checks the verify() call that was in cffi 0.x.y, while cffi1 checks the new compile() call. This is very different pieces of code, even if they both implement a large common part.

[mattip]

Ahh, makes sense, thanks for clarifying. CI is passing, including enabling windows tests.