Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow unit tests defined anywhere in mbed-os #13253

Merged
merged 2 commits into from
Jul 17, 2020

Conversation

LDong-Arm
Copy link
Contributor

@LDong-Arm LDong-Arm commented Jul 8, 2020

Summary of changes

Currently unit tests are inside the top-level UNITTESTS. But in our new directory structure proposal, unit tests will be moved into libraries. To make it possible, this PR is to allow unittest.cmake to sit anywhere inside Mbed OS.

Before:

  • UNITTESTS/foo/test_A/unittest.cmake -> BUILD/unittests/foo-test_A
  • bar/UNITTESTS/test_B/unittest.cmake -> not built

After:

  • UNITTESTS/foo/test_A/unittest.cmake -> BUILD/unittests/UNITTESTS-foo-test_A
  • bar/UNITTESTS/test_B/unittest.cmake -> BUILD/unittests/bar-UNITTESTS-test_B

IMO we can fine tune the test binary names in the future, but they don't really matter in my opinion.

Impact of changes

Migration actions required

Documentation


Pull request type

[x] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
[] Feature update (New feature / Functionality change / New API)
[] Major update (Breaking change E.g. Return code change / API behaviour change)

Test results

[] No Tests required for this change (E.g docs only update)
[x] Covered by existing mbed-os tests (Greentea or Unittest)
[] Tests / results supplied as part of this PR

Reviewers

@0xc0170 @jamesbeyond @ARMmbed/mbed-os-core


UNITTESTS/CMakeLists.txt Outdated Show resolved Hide resolved
@LDong-Arm
Copy link
Contributor Author

@jamesbeyond @0xc0170 We will need this for the directory structure work. Any suggestions?

@jamesbeyond
Copy link
Contributor

after the change, is the unit-test still required to be is a UNITTESTS folder ?
e.g.

foo/bar/UNITTESTS/test_A/unittest.cmake

@LDong-Arm
Copy link
Contributor Author

after the change, is the unit-test still required to be is a UNITTESTS folder ?
e.g.

foo/bar/UNITTESTS/test_A/unittest.cmake

For a test to compile & run, the directory does not have to be named UNITTESTS - it's not a magic keyword, unlike TESTS for Greentea. Hopefully our CI scripts don't look for UNITTESTS as a magic name?

@ciarmcom ciarmcom requested review from 0xc0170, jamesbeyond and a team July 8, 2020 17:00
@ciarmcom
Copy link
Member

ciarmcom commented Jul 8, 2020

@LDong-Arm, thank you for your changes.
@jamesbeyond @0xc0170 @ARMmbed/mbed-os-maintainers please review.

@@ -178,7 +178,7 @@ foreach(testfile ${unittest-file-list})

file(RELATIVE_PATH
TEST_SUITE_NAME # output
${PROJECT_SOURCE_DIR} # root
"${PROJECT_SOURCE_DIR}/.." # root
Copy link
Contributor Author

@LDong-Arm LDong-Arm Jul 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The relative path (to mbed-os) is the binary name of a test case. ${PROJECT_SOURCE_DIR} is UNITTESTS.

@0xc0170
Copy link
Contributor

0xc0170 commented Jul 9, 2020

CI started

@mbed-ci
Copy link

mbed-ci commented Jul 9, 2020

Test run: SUCCESS

Summary: 1 of 1 test jobs passed
Build number : 1
Build artifacts

@jamesbeyond
Copy link
Contributor

after the change, is the unit-test still required to be is a UNITTESTS folder ?
e.g.

foo/bar/UNITTESTS/test_A/unittest.cmake

For a test to compile & run, the directory does not have to be named UNITTESTS - it's not a magic keyword, unlike TESTS for Greentea. Hopefully our CI scripts don't look for UNITTESTS as a magic name?

I see, Shall we keep the consistency of keeping the UNITTESTS folder to identify these are the unit tests?

@LDong-Arm
Copy link
Contributor Author

LDong-Arm commented Jul 9, 2020

As we can see, the test case names have changed: https://mbed-os.mbedcloudtesting.com/blue/organizations/jenkins/mbed-os-ci_unittests/detail/mbed-os-ci_unittests/6431/tests
Names (i.e. paths) used to be relative to UNITTESTS but now to the top directory of mbed-os. This is needed so that tests outside the top-level UNITTESTS (to be created in the future) won't start with .. in their names.

@LDong-Arm
Copy link
Contributor Author

I see, Shall we keep the consistency of keeping the UNITTESTS folder to identify these are the unit tests?

I agree personally. We keep the name UNITTESTS for now as we re-structure some libraries, and plan to change it once we switch to a new build tool.

@ladislas
Copy link
Contributor

will that work for unit tests outside of mbed-os directory?

@LDong-Arm
Copy link
Contributor Author

will that work for unit tests outside of mbed-os directory?

Not yet with this change, which is part of our refactoring work with fairly minimal impact.

@ladislas
Copy link
Contributor

Not yet with this change, which is part of our refactoring work with fairly minimal impact.

Thanks @LDong-Arm! So does that mean it will be worked on in the future? Would be glad to help as it is something we're looking for doing ourselves anyway.

@0xc0170
Copy link
Contributor

0xc0170 commented Jul 16, 2020

CI restarted (old results here)

@mbed-ci
Copy link

mbed-ci commented Jul 16, 2020

Test run: SUCCESS

Summary: 1 of 1 test jobs passed
Build number : 2
Build artifacts

@adbridge adbridge merged commit 29ca32e into ARMmbed:master Jul 17, 2020
@adbridge adbridge added release-type: patch Indentifies a PR as containing just a patch and removed ready for merge labels Jul 17, 2020
@mbedmain mbedmain added release-version: 6.2.1 Release-pending and removed release-type: patch Indentifies a PR as containing just a patch Release-pending labels Aug 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants