Show requirements statistics and test results in documentation#77
Show requirements statistics and test results in documentation#77eelcoem wants to merge 9 commits intoeclipse-score:mainfrom
Conversation
License Check Results🚀 The license check job ran with the Bazel command: bazel run //:license-checkStatus: Click to expand output |
|
The created documentation from the pull request is available at: docu-html |
|
@ Reviewers: After merging, it will look like this: https://eelcoem.github.io/lifecycle/main/statistics.html |
MODULE.bazel
Outdated
There was a problem hiding this comment.
Not having the required properties inside your tests will result in errors / crashes in future versions if I remember correctly.
For the 2.3.3 you using here I think this might still be fine. But you will not have any links.
The links are only generated through 'PartiallyVerifies' & 'FullyVerifies' attributes.
If those are not at your tests we can't link it, just so you are aware.
There was a problem hiding this comment.
Thanks, yeah I think the upgrade to score_docs_as_code 3.0.0 should be a separate PR
There was a problem hiding this comment.
Updated to 3.0.1 already and adapted accordingly
| @@ -28,10 +28,16 @@ bazel_dep(name = "googletest", version = "1.17.0.bcr.1") | |||
|
|
|||
There was a problem hiding this comment.
Is there a reason to not upgrade further?
As for example Platform & Tooling are also not on newest version
There was a problem hiding this comment.
I didn't upgrade score_docs_as_code to v3 as that would break the build, but the bump to 2.3.3 was necessary because 2.3.0 didn't allow linking test results in the pipeline. As for the other requirements, I think there is no reason not to bump them, I'll have a look
There was a problem hiding this comment.
Bumped versions. Also updated to docs-as-code 3.0.1 and adapted requirements as necessary
There was a problem hiding this comment.
3.0.1 might be bugged. Not sure, you have to test that if it works with your build.
if it doesn't 3.0.0 should for sure work
There was a problem hiding this comment.
rename file to test_and_docs, as workflow now does both
| @@ -29,7 +29,12 @@ on: | |||
| types: [checks_requested] | |||
|
|
|||
| jobs: | |||
| unit-tests: | |||
| uses: ./.github/workflows/tests.yml | |||
| secrets: inherit | |||
| build-docs: | |||
There was a problem hiding this comment.
add a job to do docs check, take a look how it is done in docs-as-code:
https://github.com/eclipse-score/docs-as-code/blob/main/.github/workflows/test_and_docs.yml
There was a problem hiding this comment.
added docs-verify step
docs/conf.py
Outdated
There was a problem hiding this comment.
take a look into rglob method from pathlib, no need to mix it with os https://docs.python.org/3/library/pathlib.html#pathlib.Path.rglob
You can simplify it into something like
from itertools import chain
for log_file in chain((ws_root / "bazel-testlogs").rglob("test.log"), (ws_root / "tests-report").rglob("test.log")):
...There was a problem hiding this comment.
Thanks for the tip, changed as suggested
There was a problem hiding this comment.
general comment to all cpp tests:
If its requirements-based why there is no mapping to the requirement that is tested by this testcase?
There was a problem hiding this comment.
All tests that are suppose to be parsed by the testlinker are required to either have 'PartiallyVerified' or 'FullyVerified' as properties that links existing requirements
There was a problem hiding this comment.
Right now there are only dummy requirements, so I didn't link any yet. I planned to either link requirements or adapt the TestType as soon as we have requirements, should I do it differently?
There was a problem hiding this comment.
if there are no real requirements just go with explorative-testing derivation technique and update test type accordingly.
There was a problem hiding this comment.
You have to test then if docs-as-code breaks in the version you have.
Not sure anymore if it is demanding the correct signature there already or only in 3.0
There was a problem hiding this comment.
changed to explorative-testing and interface-test
|
|
||
|
|
||
| def setup(app): | ||
| app.add_directive("display-test-logs", DisplayTestLogs) |
There was a problem hiding this comment.
This might be really slow at some point depending on how large the logs are and how many of these directives you have.
There was a problem hiding this comment.
Yes I would kick this out as soon as it slows down the doc build significantly, but I think it's useful to have as long as it's feasible
|
This will imho have to wait a bit for a resolution on the general issues doc + cov - > eclipse-score/cicd-workflows#63 |
| "@score_platform//:needs_json", | ||
| "@score_process//:needs_json", # This allows linking to requirements (wp__requirements_comp, etc.) from the process_description repository. | ||
| ], | ||
| source_dir = "docs", |
There was a problem hiding this comment.
I would have expected some like deps = [":test_logs"], here.
There was a problem hiding this comment.
Yeah I suppose it will work either way, decided to go with this approach as it is already done like this in persistency
No description provided.