Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the --duration option to the test suite #119050

Closed
Eclips4 opened this issue May 14, 2024 · 5 comments
Closed

Add the --duration option to the test suite #119050

Eclips4 opened this issue May 14, 2024 · 5 comments
Labels
tests Tests in the Lib/test dir type-feature A feature request or enhancement

Comments

@Eclips4
Copy link
Member

Eclips4 commented May 14, 2024 •
edited by bedevere-app bot
Loading

Feature or enhancement

Proposal:

I propose adding a new option --duration to our test suite. This would print the duration of each test that has run during the test suite, to the file that the user provides.
So actual use case looks like that:
./python.exe -m test --duration tmp/stats -R 3:3
It's can be useful as the --slowest option, which already exists.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs
@Eclips4 Eclips4 added type-feature A feature request or enhancement tests Tests in the Lib/test dir labels May 14, 2024
@bedevere-app bedevere-app bot mentioned this issue May 14, 2024
Closed
@vstinner
Copy link
Member

Why not reusing the JUnit output?

@Eclips4
Copy link
Member Author

Eclips4 commented May 15, 2024

Why not reusing the JUnit output?

JUnit cannot be used in a "hunt refleaks" mode. Although, using JUnit for this purpose would be great, but I do not see an easy solution to making it work with the "hunt refleaks" mode :(

@vstinner
Copy link
Member

JUnit cannot be used in a "hunt refleaks" mode. Although, using JUnit for this purpose would be great, but I do not see an easy solution to making it work with the "hunt refleaks" mode :(

We should write JUnit data into a file, and then load the file to write the XML. Not store these data into memory.

vstinner added a commit to vstinner/cpython that referenced this issue May 18, 2024
@vstinner
pythongh-119050: Add type hints to libregrtest/results.py
8a36960
vstinner added a commit to vstinner/cpython that referenced this issue May 18, 2024
@vstinner
pythongh-119050: Add type hints to libregrtest/results.py
bbab1f6 
Sort also 'omitted' in TestResults.display_result().
@bedevere-app bedevere-app bot mentioned this issue May 18, 2024
Merged
vstinner added a commit to vstinner/cpython that referenced this issue May 18, 2024
@vstinner
pythongh-119050: Add XML support to libregrtest refleak checker
bcbd615 
regrtest test runner: Add XML support to the refleak checker
(-R option).
vstinner added a commit to vstinner/cpython that referenced this issue May 18, 2024
@vstinner
pythongh-119050: Add XML support to libregrtest refleak checker
30ee228 
regrtest test runner: Add XML support to the refleak checker
(-R option).

* run_unittest() now stores XML elements as string, rather than
  objects, in support.junit_xml_list.
* runtest_refleak() now saves/restores XML strings before/after
  checking for reference leaks. Save XML into a temporary file.
@bedevere-app bedevere-app bot mentioned this issue May 18, 2024
Merged
vstinner added a commit that referenced this issue May 18, 2024
@vstinner
gh-119050: Add type hints to libregrtest/results.py (#119144)
30b4e9f 
Sort also 'omitted' in TestResults.display_result().
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 18, 2024
@vstinner @miss-islington
pythongh-119050: Add type hints to libregrtest/results.py (pythonGH-1…
a4d2fb4 
…19144)

Sort also 'omitted' in TestResults.display_result().
(cherry picked from commit 30b4e9f)

Co-authored-by: Victor Stinner <vstinner@python.org>
@bedevere-app bedevere-app bot mentioned this issue May 18, 2024
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 18, 2024
@vstinner @miss-islington
pythongh-119050: Add type hints to libregrtest/results.py (pythonGH-1…
10b802f 
…19144)

Sort also 'omitted' in TestResults.display_result().
(cherry picked from commit 30b4e9f)

Co-authored-by: Victor Stinner <vstinner@python.org>
@bedevere-app bedevere-app bot mentioned this issue May 18, 2024
Merged
vstinner added a commit that referenced this issue May 18, 2024
@miss-islington @vstinner
[3.13] gh-119050: Add type hints to libregrtest/results.py (GH-119144) (
9129614 
#119156)

gh-119050: Add type hints to libregrtest/results.py (GH-119144)

Sort also 'omitted' in TestResults.display_result().
(cherry picked from commit 30b4e9f)

Co-authored-by: Victor Stinner <vstinner@python.org>
vstinner added a commit that referenced this issue May 18, 2024
@miss-islington @vstinner
[3.12] gh-119050: Add type hints to libregrtest/results.py (GH-119144) (
d76d95e 
#119157)

gh-119050: Add type hints to libregrtest/results.py (GH-119144)

Sort also 'omitted' in TestResults.display_result().
(cherry picked from commit 30b4e9f)

Co-authored-by: Victor Stinner <vstinner@python.org>
vstinner added a commit that referenced this issue May 20, 2024
@vstinner
gh-119050: Add XML support to libregrtest refleak checker (#119148)
9257731 
regrtest test runner: Add XML support to the refleak checker
(-R option).

* run_unittest() now stores XML elements as string, rather than
  objects, in support.junit_xml_list.
* runtest_refleak() now saves/restores XML strings before/after
  checking for reference leaks. Save XML into a temporary file.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 20, 2024
@vstinner @miss-islington
pythongh-119050: Add XML support to libregrtest refleak checker (pyth…
b92ae7f 
…onGH-119148)

regrtest test runner: Add XML support to the refleak checker
(-R option).

* run_unittest() now stores XML elements as string, rather than
  objects, in support.junit_xml_list.
* runtest_refleak() now saves/restores XML strings before/after
  checking for reference leaks. Save XML into a temporary file.
(cherry picked from commit 9257731)

Co-authored-by: Victor Stinner <vstinner@python.org>
This was referenced May 20, 2024
Merged
Merged
vstinner added a commit to vstinner/cpython that referenced this issue May 20, 2024
@vstinner
pythongh-119050: Add XML support to libregrtest refleak checker (pyth…
4f0c1fd 
…on#119148)

regrtest test runner: Add XML support to the refleak checker
(-R option).

* run_unittest() now stores XML elements as string, rather than
  objects, in support.junit_xml_list.
* runtest_refleak() now saves/restores XML strings before/after
  checking for reference leaks. Save XML into a temporary file.

(cherry picked from commit 9257731)
@vstinner
Copy link
Member

JUnit cannot be used in a "hunt refleaks" mode. Although, using JUnit for this purpose would be great, but I do not see an easy solution to making it work with the "hunt refleaks" mode :(

It's now supported! Is it enough for your use case?

@Eclips4
Copy link
Member Author

Eclips4 commented May 20, 2024

JUnit cannot be used in a "hunt refleaks" mode. Although, using JUnit for this purpose would be great, but I do not see an easy solution to making it work with the "hunt refleaks" mode :(

It's now supported! Is it enough for your use case?

Yes, it is! Thanks Victor!

@Eclips4 Eclips4 closed this as completed May 20, 2024
vstinner added a commit that referenced this issue May 20, 2024
@miss-islington @vstinner
[3.13] gh-119050: Add XML support to libregrtest refleak checker (GH-…
42a8d11 
…119148) (#119270)

gh-119050: Add XML support to libregrtest refleak checker (GH-119148)

regrtest test runner: Add XML support to the refleak checker
(-R option).

* run_unittest() now stores XML elements as string, rather than
  objects, in support.junit_xml_list.
* runtest_refleak() now saves/restores XML strings before/after
  checking for reference leaks. Save XML into a temporary file.
(cherry picked from commit 9257731)

Co-authored-by: Victor Stinner <vstinner@python.org>
vstinner added a commit that referenced this issue May 20, 2024
@vstinner
[3.12] gh-119050: Add XML support to libregrtest refleak checker (#11…
ec5743b 
…9148) (#119272)

gh-119050: Add XML support to libregrtest refleak checker (#119148)

regrtest test runner: Add XML support to the refleak checker
(-R option).

* run_unittest() now stores XML elements as string, rather than
  objects, in support.junit_xml_list.
* runtest_refleak() now saves/restores XML strings before/after
  checking for reference leaks. Save XML into a temporary file.

(cherry picked from commit 9257731)
@JelleZijlstra JelleZijlstra mentioned this issue May 28, 2024
Closed
estyxx pushed a commit to estyxx/cpython that referenced this issue Jul 17, 2024
@vstinner @estyxx
pythongh-119050: Add type hints to libregrtest/results.py (python#119144
63e27ae 
)

Sort also 'omitted' in TestResults.display_result().
estyxx pushed a commit to estyxx/cpython that referenced this issue Jul 17, 2024
@vstinner @estyxx
pythongh-119050: Add XML support to libregrtest refleak checker (pyth…
d0892ad 
…on#119148)

regrtest test runner: Add XML support to the refleak checker
(-R option).

* run_unittest() now stores XML elements as string, rather than
  objects, in support.junit_xml_list.
* runtest_refleak() now saves/restores XML strings before/after
  checking for reference leaks. Save XML into a temporary file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
tests Tests in the Lib/test dir type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vstinner @Eclips4