Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update build system to allow code coverage during testing to be optional #194

Closed
koomie opened this issue Oct 27, 2023 · 1 comment
Closed

Update build system to allow code coverage during testing to be optional #194

koomie opened this issue Oct 27, 2023 · 1 comment
Assignees
@koomie
Labels
enhancement New feature or request Omniperf Revamp Ticket related to the redesign of Omniperf

Comments

@koomie
Copy link
Collaborator

koomie commented Oct 27, 2023

Add a cmake option to enable code coverage when running tests. Default to off, but we will enable during CI.
@koomie koomie added enhancement New feature or request Omniperf Revamp Ticket related to the redesign of Omniperf labels Oct 27, 2023
@koomie koomie self-assigned this Oct 27, 2023
koomie added a commit that referenced this issue Oct 31, 2023
@koomie
Introduce a cmake option to enable python code coverage. It now
cdaa8f2 
defaults to being disabled and can be enabled via a
-DENABLE_COVERAGE=ON option (#194).  Introduce CI on mi100 that
leverages code coverage and publishes results along with a testing
report.

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
koomie added a commit that referenced this issue Oct 31, 2023
@koomie
Introduce a cmake option to enable python code coverage. It now
fc43ae5 
defaults to being disabled and can be enabled via a
-DENABLE_COVERAGE=ON option (#194).  Introduce CI on mi100 that
leverages code coverage and publishes results along with a testing
report.

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
koomie added a commit that referenced this issue Oct 31, 2023
@koomie
Introduce a cmake option to enable python code coverage. It now
a39514f 
defaults to being disabled and can be enabled via a
-DENABLE_COVERAGE=ON option (#194).  Introduce CI on mi100 that
leverages code coverage and publishes results along with a testing
report.

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
@koomie
Copy link
Collaborator Author

koomie commented Oct 31, 2023

Landed on 2.x. To enable this option, include -DENABLE_COVERAGE=ON with cmake configuration.

@koomie koomie closed this as completed Oct 31, 2023
coleramos425 added a commit that referenced this issue Dec 13, 2023
@JoseSantosAMD @koomie @coleramos425
Pytest (#212)
6d7c631 
* Initial commit for pytest

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Check if csvs populated

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* pytest now checks soc for soc specific testing

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Roof only csvs updated

-  Removes dir if workload dir already present

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Tests create difference logs calculated from a baseline

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* pytest scripts updated to revamp test cases

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Fix dispatch notation

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Introduce a cmake option to enable python code coverage. It now
defaults to being disabled and can be enabled via a
-DENABLE_COVERAGE=ON option (#194).  Introduce CI on mi100 that
leverages code coverage and publishes results along with a testing
report.

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* update branch for CI -> 2.x

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* add profile execution test in cmake and enable in CI

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* run all ci tests

Signed-off-by: Karl W. Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Initial overhaul of Profile mode

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Modify vcopy to enable multi iteration and multi kernel for CI

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Update docs and --help menu to reflect new vcopy usage

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Fix typo in IPBlock logging

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Cache ascii art in source and remove ascii dependency

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Update testing collateral to reflect name change of kernel-verbose option

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Small typo corrections in docs

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Fixing more typos...

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* run profiler tests through queue

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* bump wallclock runlimit for test

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* add mi100 badge

Signed-off-by: Karl W. Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* include step to build hip binary

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* add additional top-level vars for number of kernels expected and
dispatch id; tests updated to use these vars so that vcopy binary can
be used instead of mixbench

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* add requirements-test.txt file to organize python testing dependencies (#205)

* add a requirements-test.txt file to organize additional python
packages needed to support testing; update CI to install python
packages from dependency files

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Dynamically load dependency checker from requirements.txt

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Add comments

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Replace any - with _ for checker logic

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Update cmake config with Fix for pyyaml package installation check

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* update mi100 CI config

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* upgrade jsdom version (#206)

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Adding the log_metric differ function

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Added missing hasattr checker

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Enable running on mi100 and fixed typos
- Also enables RHEL9 roofline

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* update CMakeLists.txt

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Remove pytest raises SystemExit

Add default thresholds

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Initial overhaul of Analyze mode. Basic CLI is enabled.

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Revert "Remove pytest raises SystemExit"

This reverts commit 9762728.

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* restore formal call to sys.exit() in top-level main; this is needed
for companion testing collateral which expects SystemExit exceptions
to be raised.

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Update return code error checks for roofline-only analysis on MI100

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* disable coverage comment

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* fix comment typo

Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Move error logging to util.py

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Move perfmon_configs dir into omniperf_soc dir. Rename config dirs for clarity

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Add a supported_archs property to Omniperf base class

Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Remove pytest raises SystemExit

Add default thresholds

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Revert "Remove pytest raises SystemExit"

This reverts commit 9762728.

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Add Absolute Difference to report-diff output
-  Pytest regex updated to read Absolute Difference
-  Pytest creates Baseline (Assumes vcopy binary present)
-  Adding Metric logging

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* vcopy binary created before running tests

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Adding the log_metric differ function

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>

* Update test_profile_general.py

remove extra log_metric function

Signed-off-by: JoseSantosAMD <87447437+JoseSantosAMD@users.noreply.github.com>

* Update CMakeLists.txt

soc_params was merged into a class, it isn't a standalone file anymore

Signed-off-by: JoseSantosAMD <87447437+JoseSantosAMD@users.noreply.github.com>

---------

Signed-off-by: JoseSantosAMD <Jose.Santos@amd.com>
Signed-off-by: Karl W Schulz <karl.schulz@amd.com>
Signed-off-by: Karl W. Schulz <karl.schulz@amd.com>
Signed-off-by: colramos-amd <colramos@amd.com>
Signed-off-by: JoseSantosAMD <87447437+JoseSantosAMD@users.noreply.github.com>
Co-authored-by: Karl W Schulz <karl.schulz@amd.com>
Co-authored-by: colramos-amd <colramos@amd.com>
Co-authored-by: Karl W. Schulz <koomie@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@koomie koomie

Labels
enhancement New feature or request Omniperf Revamp Ticket related to the redesign of Omniperf
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@koomie