Skip to content

Simplify GitHub Action for regression tests (backport from dev-tc)#2825

Merged
andrew-platt merged 2 commits intoOpenFAST:rc-4.0.5from
deslaughter:gh_action_backport
May 22, 2025
Merged

Simplify GitHub Action for regression tests (backport from dev-tc)#2825
andrew-platt merged 2 commits intoOpenFAST:rc-4.0.5from
deslaughter:gh_action_backport

Conversation

@deslaughter
Copy link
Collaborator

This PR is ready to be merged.

Feature or improvement description

This PR introduces a slightly more streamlined process for the regression tests run in Github Actions. It reduces the amount of caching used in the build process by compiling most of the executables in one step and then using them to run the tests so only one workspace is cached. The same regression tests are still being run, but some have been grouped into new jobs.

This change also separates out the Matlab Simulink build and test which is currently failing due to an outage in Matlab's servers. That issue is being tracked at matlab-actions/setup-matlab#146.

Impacted areas of the software

automated-dev-tests.yml

Additional supporting information

This PR also revealed an issue with the 5MW_OC4Jckt_DLL_WTurb_WavesIrr_MGrowth regression test which has to be run with OMP_NUM_THREADS not specified to match the test baseline. This will be corrected in the future by setting OMP_NUM_THREADS=1 and regenerating the baseline.

Test results, if applicable

No tests were changed and the Simulink test cases will start working once the Matlab outage is resolved.

@deslaughter deslaughter added this to the 4.0.5 milestone May 21, 2025
@deslaughter deslaughter requested a review from andrew-platt May 21, 2025 22:54
@deslaughter deslaughter self-assigned this May 21, 2025
@andrew-platt
Copy link
Collaborator

andrew-platt commented May 21, 2025

With 4 threads on OpenMP, the 5MW_OC4Jckt_DLL_WTurb_WavesIrr_MGrowth test is currently failing (the original baseline might have been generated with 2 threads). So we might need to change the baseline for that case now instead of waiting.

Not sure if it just ran on different hardware or if something else changed (memory layout for example).

needs: build-openfast-release
rtest-OF-offshore:
runs-on: ubuntu-24.04
needs: build-all-release
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
needs: build-all-release
needs: build-all-release
OMP_NUM_THREADS: 2

Setting to 2 threads here to match old method. Might get the 5MW_OC4Jckt_DLL_WTurb_WavesIrr_MGrowth to match the baseline (we don't have any artifacts from this case passing on main or .log files in the repo to check against).

@andrew-platt
Copy link
Collaborator

andrew-platt commented May 21, 2025

Regarding the 5MW_OC4Jckt_DLL_WTurb_WavesIrr_MGrowth case: my current suspicion is that something in the marine growth bit of code is causing different results in the simulation depending on the number of threads used (this is the only case using marine growth and showing this behavior). The marine growth section was rewritten in dev (for v4.1.0), so we don't particularly want to address it here.

@deslaughter deslaughter force-pushed the gh_action_backport branch from d4c2fcb to e4935e7 Compare May 22, 2025 12:28
@deslaughter
Copy link
Collaborator Author

There is something in the baseline for 5MW_OC4Jckt_DLL_WTurb_WavesIrr_MGrowth that is specific to using ubuntu-22.04, gfortran-12, and leaving OMP_NUM_THREADS unset. For OpenFAST v4.1 we should update to ubuntu-24.04, gfortran-14, and set OMP_NUM_THREADS=1 and redo the baseline for this case so it corresponds to a given environment.

Other than the issue with the Simulink test, which is caused by Matlab's mpm servers and beyond our control, the PR is ready to be merged.

@andrew-platt andrew-platt merged commit 407e7b4 into OpenFAST:rc-4.0.5 May 22, 2025
11 of 12 checks passed
@andrew-platt andrew-platt mentioned this pull request Jun 11, 2025
35 tasks
@deslaughter deslaughter deleted the gh_action_backport branch October 27, 2025 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants