VPN-5560 - Do not run auth_tests on PRs

[brizental]

No description provided.

[brizental]

Someone that has admin right on this repository needs to add the auth_tests job to the checks run on merge to main and release, I think. cc @lesleyjanenorton, @oskirby

[brizental]

Someone that has admin right on this repository needs to add the auth_tests job to the checks run on merge to main and release, I think.

Actually doesn't look like there is anything that needs to be changed 🤷‍♀️

[oskirby]

I don't think this is going to achieve the desired result, the list of tests in CMake is generated by the add_test() function and get run by ctest. However, in the tests/auth_tests/CMakeLists.txt we are still adding the tests, so they will are still being run on the Unit test job.

What we really need is a way to split off the auth tests so they are not included by ctest unless we explicitly build for them. This could be done by adding something like this to the top-level CMakeLists.txt:

option(BUILD_E2E_TESTS "Build end-to-end tests" FALSE)
if(BUILD_E2E_TESTS)
  add_subdirectory(tests/auth_tests EXCLUDE_FROM_ALL)
endif()

[oskirby]

I don't think this is going to achieve the desired result, the list of tests in CMake is generated by the add_test() function and get run by ctest. However, in the tests/auth_tests/CMakeLists.txt we are still adding the tests, so they will are still being run on the Unit test job.

What we really need is a way to split off the auth tests so they are not included by ctest unless we explicitly build for them. This could be done by adding something like this to the top-level CMakeLists.txt:

option(BUILD_E2E_TESTS "Build end-to-end tests" FALSE)
if(BUILD_E2E_TESTS)
  add_subdirectory(tests/auth_tests EXCLUDE_FROM_ALL)
endif()

Oh wait! I am dumb, I see how you got around it, you are using the --test-dir argument on ctest to run a subset of tests.

[brizental]

I don't think this is going to achieve the desired result, the list of tests in CMake is generated by the add_test() function and get run by ctest. However, in the tests/auth_tests/CMakeLists.txt we are still adding the tests, so they will are still being run on the Unit test job.

That was actually on purpose. I started out and was going to do something similar to what you proposed, but then I decided to keep this minimal and actually just changed the CI job instead.