Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test the root and roottest repos at scale #18338

Draft
wants to merge 6,413 commits into
base: master
Choose a base branch
from

Conversation

dpiparo
Copy link
Member

@dpiparo dpiparo commented Apr 9, 2025

No description provided.

dpiparo and others added 30 commits March 14, 2024 23:47
Add unit test for issue root-project#11411, because it is fixed with the cppyy
upgrade.
Fix the following error with `roottest-root-io-issue-7754-read_mwe_0` and `roottest-root-io-issue-7754-read_mwe_1` on Windows:
```
1702: Processing C:/root-dev/git/roottest/root/io/issue-7754/mwe/combined.cxx+(1)...
1702: Info in <TWinNTSystem::ACLiC>: creating shared library C:/root-dev/build/x64/debug/roottest/root/io/issue-7754/combined_cxx.dll
1702: In file included from input_line_9:6:
1702: In file included from C:/root-dev/git/roottest/root/io/issue-7754/mwe/combined.cxx:13:
1702: C:/root-dev/git/roottest/root/io/issue-7754/mwe/read_updated.cxx:27:5: error: member access into incomplete type 'TBranch'
1702:   br->GetEntry(0);
1702:     ^
1702: C:/root-dev/build/x64/debug/include/ROOT/TIOFeatures.hxx:17:7: note: forward declaration of 'TBranch'
1702: class TBranch;
1702:       ^
```
Create a single set of testing scripts to run according to different parameters for distributed execution.

The biggest change with this commit is moving from two separate sets of files for Spark and Dask backends to a single set that can run with either backend connection depending on the current parameter in the `payload` fixture set via the `conftest.py` file.

Also, create a directory with input TTree datasets used throughout the tests. This has two advantages. First, it avoids spending extra time in creating and writing the dataset to disk. Second, it sets the ground for a future update of the testing suite to include the data source as another parameter (i.e. running the tests reading either TTree or RNTuple data).

For reproducibility, the `_create_datasets.py` script is added to the input data directory.
The `common` folder tests for a very specific, admittedly extremely rare or inexistent, use case. That is, an application that sends multiple tasks concurrently to different resource schedulers at the same time. Currently, the test submits the same RDataFrame graph to both a Spark scheduler and a Dask scheduler concurrently. So far, this scenario has never occurred in practice and perhaps the test is overzealous. Furthermore, it sporadically fails on the CI due to timeouts that do not provide any evidence of the root cause. For lack of concrete necessity, we can disable the test and eventually re-enable it at a later stage if required.
Co-authored-by: Philippe Canal <pcanal@fnal.gov>

Update runLoadDim.ref
SpectreVert and others added 19 commits March 25, 2025 16:28
* implemented the tests in cmake

* remove Makefile

* Avoid variable shadowing
Ignore Python warnings in `cmdLineUtils` tests to avoid noise when
comparing against reference files.
The test is not adding anything new to already existing tests of TF1,
for example in stressHistoFit.
This is redundant. The `root` Python kernel itself should give a good
error is `metakernel` is not available.
This makes the tests work in general, no matter which memory policy is
used (in particular, make the case of the "strict" memory policy work).
To avoid warnings such as
/Users/vpadulan/Programs/rootproject/roottest/root/treeformula/references/TreeFormulaReferencesGeneration.cxx:146:3: warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
  146 |   sprintf(fEventName,"Event%d_Run%d",ev,200);
      |   ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk/usr/include/_stdio.h:274:1: note: 'sprintf' has been explicitly marked deprecated here
  274 | __deprecated_msg("This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead.")
      | ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk/usr/include/sys/cdefs.h:218:48: note: expanded from macro '__deprecated_msg'
  218 |         #define __deprecated_msg(_msg) __attribute__((__deprecated__(_msg)))
Commit b1d0efe "split stl and numerical conversions from main
evolution-make"
but a subdirectory of root. All the code in the classic Makefile driven
tests that rely on the aforementioned assumption need to be adapted.
@dpiparo dpiparo self-assigned this Apr 9, 2025
@dpiparo dpiparo requested a review from bellenot as a code owner April 9, 2025 14:06
@dpiparo dpiparo marked this pull request as draft April 9, 2025 14:07
@dpiparo dpiparo added the clean build Ask CI to do non-incremental build on PR label Apr 9, 2025
Copy link

github-actions bot commented Apr 9, 2025

Test Results

    19 files      19 suites   4d 10h 43m 52s ⏱️
 2 705 tests  2 704 ✅ 0 💤 1 ❌
49 948 runs  49 947 ✅ 0 💤 1 ❌

For more details on these failures, see this check.

Results for commit 16178eb.

♻️ This comment has been updated with latest results.

@dpiparo dpiparo closed this Apr 9, 2025
@dpiparo dpiparo reopened this Apr 9, 2025
@ferdymercury ferdymercury linked an issue Apr 10, 2025 that may be closed by this pull request
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clean build Ask CI to do non-incremental build on PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New CI does not rebase roottest branch on top of roottest master