Skip to content

Commit

Permalink
Add docs for checking/improving test coverage (#884)
Browse files Browse the repository at this point in the history
* Add docs for checking/improving test coverage
  • Loading branch information
enya-yx authored Dec 1, 2022
1 parent 4ad70f2 commit cae6a99
Show file tree
Hide file tree
Showing 2 changed files with 47 additions and 0 deletions.
Binary file added docs/dev_guide/images/coverage_res.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
47 changes: 47 additions & 0 deletions docs/dev_guide/test_coverage_guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
layout: default
title: Feathr Test Coverage Guide
parent: Developer Guides
---

# Feathr Test Coverage Check Guide

## Background
To maintain and improve codes quality of feathr, we expect test coverage ratio to be equal to or above 90% in general. For any code changes, please make sure related test cases are added and check that test coverage can meet our request.

## How to conduct test coverage
### Through github workflows pipeline:
We already added this coverage checking into our CI pipeline. For every pull request, push and scheduled jobs, github will check the coverage when runing 'pytest' automatically. You can find the result for 'azure_synapse', 'databricks' and 'local spark', respectively from each PR and commit.

An example of test coverage result:
![test coverage example](./images/coverage_res.png)

From the example above we can see the result can show coverage ratio for each file and also the missing lines of codes that current test cases fail to cover. Based on the result, we can add related test cases to improve the total test coverage ratio.

### Test locally:
We can also check the coverage locally by simply running the command under `feathr/` directory:
`pytest --cov-report term-missing --cov=feathr_project/feathr feathr_project/test/`
This will provide a coverage report for all codes of python client by running all python test cases.

You may need to install `pytest-cov` locally:
`pip install pytest-cov`

If you just want to check coverage situation under some folders by running some of test cases, you can also specify paths for each of them. Eg. if you only want to check coverage ratio of local spark, you may run:
`pytest --cov-report term-missing --cov=feathr_project/feathr/spark_provider feathr_project/test/test_local_spark_e2e.py`

In addition, if you want the report ignore some files under the specified folder, you can set it through config file. Eg:
```
[run]
omit = feathr_project/feathr/registry/_feature_registry_purview.py
feathr_project/feathr/spark_provider/_synapse_submission.py
feathr_project/feathr/spark_provider/_localspark_submission.py
```

The default config file is `.coveragerc`. You can also specify one by adding `--cov-config={your_coverage_config}` into the above `pytest` command.

Coverage config example:
[coverage config to ignore synapse when running by databricks](../../.github/workflows/.coveragerc_db)

More reference for pytest coverage:
[pytest coverage](https://pypi.org/project/pytest-cov/)

0 comments on commit cae6a99

Please sign in to comment.