Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CMake: Add CMake to mbed-psa greentea tests #14828

Merged
merged 11 commits into from
Jun 29, 2021

Conversation

hazzlim
Copy link
Contributor

@hazzlim hazzlim commented Jun 24, 2021

Summary of changes

  • Changed mbed_greentea_add_test macro to require TEST_SOURCES for all files, removing assumption of main.cpp
  • Refactored all pre-existing greentea tests to add main.cpp using TEST_SOURCES
  • Added a CMakeLists.txt for each test suite in Mbed OS PSA.
  • Moved /pal and /val directories into a separate /test_abstraction_layer directory and combined them into one CMake target, which is linked to by the test suites that require them.

Fixes #14680

Impact of changes

None.

Migration actions required

None.

Documentation

None.


Pull request type

[x] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
[] Feature update (New feature / Functionality change / New API)
[] Major update (Breaking change E.g. Return code change / API behaviour change)

Test results

[] No Tests required for this change (E.g docs only update)
[] Covered by existing mbed-os tests (Greentea or Unittest)
[x] Tests / results supplied as part of this PR

Manual testing for pre-existing greentea tests:

All pre-existing tests built successfully using below commands, or fail as expected due to a [NOT_SUPPORTED] error.

Manual testing for PSA tests:
Successfully able to build the test suite with below manual steps:

  • mbedtools configure -t <toolchain> -m <target> --mbed-os-path <path-to-mbed-os> --app-config TESTS/configs/experimental.json (requires this PR)
  • cd cmake_build/<target>/develop/<toolchain>/ && cmake ../../../.. -G Ninja -DMBED_TEST_LINK_LIBRARIES=mbed-os && cmake --build .
  • mbedhtrun -f <path/to/image> -p <port> -d <mount>

All built successfully with both GCC_ARM and ARMC6, and run successfully on target K66F, except:

  • entropy_inject: can only run with MBEDTLS_ENTROPY_NV_SEED defined; build fails with #error [NOT_SUPPORTED] PSA entropy injection tests can run only with MBEDTLS_ENTROPY_NV_SEED enabled. (Build successful if the macro is defined, but test crashes when run on K66F)
  • compliance_its/test_s003: can only run with PS_ALLOW_ENTIRE_STORAGE_FILL defined; build fails with #error [NOT_SUPPORTED] Test is too long for CI, thus always fails on timeout. (Build successful if the macro is defined, and test runs to {{result;success}})

Reviewers

@Patater @LDong-Arm


@hazzlim hazzlim changed the title CMake: Add CMake to Mbed-psa greentea tests CMake: Add CMake to Mbed-psa greentea tests - Closes #14680 Jun 24, 2021
@hazzlim hazzlim changed the title CMake: Add CMake to Mbed-psa greentea tests - Closes #14680 CMake: Add CMake to Mbed-psa greentea tests Jun 24, 2021
@hazzlim hazzlim changed the title CMake: Add CMake to Mbed-psa greentea tests CMake: Add CMake to mbed-psa greentea tests Jun 24, 2021
@hazzlim hazzlim marked this pull request as ready for review June 24, 2021 13:42
@ciarmcom ciarmcom requested review from LDong-Arm, Patater and a team June 24, 2021 14:00
@ciarmcom
Copy link
Member

@hazzlim, thank you for your changes.
@Patater @LDong-Arm @ARMmbed/mbed-os-core @ARMmbed/mbed-os-security @ARMmbed/mbed-os-maintainers please review.

Copy link
Contributor

@LDong-Arm LDong-Arm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hazzlim Thanks for the PR, it looks pretty good to me! I will try a few tests locally.

@LDong-Arm LDong-Arm requested a review from a team June 24, 2021 14:22
@LDong-Arm LDong-Arm linked an issue Jun 24, 2021 that may be closed by this pull request
@hazzlim hazzlim force-pushed the add-cmake-supports-psa-tests branch 2 times, most recently from 341080e to 49e3182 Compare June 28, 2021 13:28
Copy link
Contributor

@LDong-Arm LDong-Arm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @hazzlim, it looks almost perfect to me - nice improvement to mbed_greentea_add_test.
Just one remark on a line of comment.

@hazzlim hazzlim force-pushed the add-cmake-supports-psa-tests branch from 49e3182 to 53d9e7f Compare June 28, 2021 14:07
LDong-Arm
LDong-Arm previously approved these changes Jun 28, 2021
Copy link
Contributor

@LDong-Arm LDong-Arm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mergify mergify bot added needs: CI and removed needs: review labels Jun 28, 2021
@mergify mergify bot added needs: work and removed needs: CI labels Jun 29, 2021
@@ -11,6 +11,11 @@ include(${MBED_PATH}/tools/cmake/mbed_greentea.cmake)
project(${TEST_TARGET})

mbed_greentea_add_test(
TEST_NAME ${TEST_TARGET}
TEST_REQUIRED_LIBS mbed-ble mbed-events
TEST_NAME
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The style change - was it done by a tool or just follow CMake docs style?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was done to follow style pointers from Lingkai for previous commits

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly a bit of an over-interpretation - is it best as it was?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the kind of indentation we have when calling a macro function with parameters, so it looks good to me. I guess what @0xc0170 means is whether it was done manually or by some tool, and the former is the case.

Copy link
Contributor

@rwalton-arm rwalton-arm Jun 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should set-up cmake-format as a pre-commit check and specify the style we want to use. It would be great to run the cmake-format check as part of the CI. That way we will auto-enforce the style and it will always be consistent. We can add a new issue to add cmake-format to the CI in a separate PR.

@hazzlim hazzlim force-pushed the add-cmake-supports-psa-tests branch from 53d9e7f to b4fdb5d Compare June 29, 2021 12:15
@mergify mergify bot dismissed Patater’s stale review June 29, 2021 12:15

Pull request has been modified.

MBED_PATH set to /platform directory incorrectly. Updated to set to
/mbed-os directory.
CMakeLists.txt file in /hal/tests/TESTS/mbed_hal_fpga_ci_test_shield
directory was non-functional as it used the greentea_add_test macro,
which expects a main.cpp in current directory, but no main.cpp exists
there. I checked with @rajkan01 who confirmed that this CMake file is
serving no purpose and is there erroneously. All tests in subdirectories
of this directory have their own CMakeLists.txt that successfully build
them.
The mbed_greentea_add_test macro previously set a variable in order to
use the un-prefixed TEST_NAME to refer to the argument in the macro
body. Whilst pair-programming with LDong, this was identified and
determined to be unecessary (maybe it was a failed attempt to fix
something, that was never reversed?) and so it has been removed.
Assumption that greentea test file is always named main.cpp is
incorrect. Updated mbed_greentea_add_test() macro to make TEST_SOURCES
parameter compulsory, which is used to specify greentea test
file(s). This allows tests to use C, or have a different name.
Therefore also updated all pre-existing greentea test CMake files to
explicity add main.cpp to TEST_SOURCES.
The greentea test for mbed-psa attestation can now be built with CMake.
The greentea test for mbed-psa crypto_init can now be built with CMake.
The greentea test for mbed-psa its_ps can now be built with CMake.
The greentea test for PSA entropy_inject can now be built with CMake.
Note: requires MBEDTLS_ENTROPY_NV_SEED enabled, so not tested on target.
Move /val and /pal directories into /test_abstraction_layers directory
and combine into one CMake target, mbed-psa-tal. Moved into seperate
directory in order to have own CMakeLists.txt, rather than adding to
/TARGET_MBED_PSA_SRV CMake file.
Greentea test for PSA compliance_attestation can now build with CMake.
PSA compliance_its tests can now be built with CMake.
@hazzlim hazzlim force-pushed the add-cmake-supports-psa-tests branch from b4fdb5d to f69a375 Compare June 29, 2021 12:34
@mergify mergify bot dismissed Patater’s stale review June 29, 2021 12:34

Pull request has been modified.

Copy link
Contributor

@Patater Patater left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@0xc0170
Copy link
Contributor

0xc0170 commented Jun 29, 2021

CI started

@mergify mergify bot added needs: CI and removed needs: work labels Jun 29, 2021
@mbed-ci
Copy link

mbed-ci commented Jun 29, 2021

Jenkins CI Test : ✔️ SUCCESS

Build Number: 1 | 🔒 Jenkins CI Job | 🌐 Logs & Artifacts

CLICK for Detailed Summary

jobs Status
jenkins-ci/mbed-os-ci_unittests ✔️
jenkins-ci/mbed-os-ci_cmake-cloud-example-ARM ✔️
jenkins-ci/mbed-os-ci_build-cloud-example-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_build-cloud-example-ARM ✔️
jenkins-ci/mbed-os-ci_cmake-cloud-example-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_build-greentea-ARM ✔️
jenkins-ci/mbed-os-ci_build-greentea-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_cmake-example-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_cmake-example-ARM ✔️
jenkins-ci/mbed-os-ci_build-example-ARM ✔️
jenkins-ci/mbed-os-ci_build-example-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_greentea-test ✔️
jenkins-ci/mbed-os-ci_tfm-integration ✔️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CMake supports missing for Mbed OS PSA tests
8 participants