Skip to content

Commit

Permalink
rename tool
Browse files Browse the repository at this point in the history
  • Loading branch information
RedContritio committed Nov 6, 2023
1 parent 98de5f3 commit fb04838
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 14 deletions.
2 changes: 0 additions & 2 deletions tools/callvariety/.gitignore

This file was deleted.

2 changes: 2 additions & 0 deletions tools/validate_unittest/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
validation_report.md
validation.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## CallVariety Check
## validate_unittest Check

单测 api 调用多样性检查,用于检查是否满足以下情况:

Expand All @@ -11,29 +11,29 @@
首次使用时,需要先构建测试数据,在项目根目录下执行:

```bash
python tools/callvariety/callvariety_check.py tests
python tools/validate_unittest/validate_unittest.py tests
```

*注意:即使单测出错,也可以正常收集数据。等待全部执行完即可,会生成 `tools/callvariety/unittest_data.json` 数据文件*
*注意:即使单测出错,也可以正常收集数据。等待全部执行完即可,会生成 `tools/validate_unittest/validation.json` 数据文件*

此时默认会生成 `tools/callvariety/callvariety_report.md` 作为多样性检测报告,其中仅包含多样性不符合的 api。
此时默认会生成 `tools/validate_unittest/validation_report.md` 作为多样性检测报告,其中仅包含多样性不符合的 api。

当修改后需要**重新检测**时,用法和 `pytest` 基本一致,支持三种更新方式:

1. 全局重新生成

```bash
python tools/callvariety/callvariety_check.py -r tests
python tools/validate_unittest/validate_unittest.py -r tests
```

2. 重新生成单个单测的数据

```bash
python tools/callvariety/callvariety_check.py -r tests/test_Tensor_amax.py
python tools/validate_unittest/validate_unittest.py -r tests/test_Tensor_amax.py
```

3. 重新生成若干个单测的数据

```bash
python tools/callvariety/callvariety_check.py -r tests/test_Tensor_div*
python tools/validate_unittest/validate_unittest.py -r tests/test_Tensor_div*
```
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@

import apibase

project_dir = os.path.join(os.path.dirname(__file__), "../..")
output_dir = os.path.dirname(__file__)

whitelist_pattern = [
Expand Down Expand Up @@ -254,12 +255,12 @@ def check_call_variety(test_data, api_mapping):

args = parser.parse_args()

with open("paconvert/api_mapping.json", "r") as f:
with open(os.path.join(project_dir, "paconvert/api_mapping.json"), "r") as f:
api_mapping = json.load(f)
with open("paconvert/api_alias_mapping.json", "r") as f:
with open(os.path.join(project_dir, "paconvert/api_alias_mapping.json"), "r") as f:
api_alias_mapping = json.load(f)

test_data_path = os.path.join(output_dir, "unittest_data.json")
test_data_path = os.path.join(output_dir, "validation.json")

if os.path.exists(test_data_path) and os.path.isfile(test_data_path):
with open(test_data_path, "r") as f:
Expand Down Expand Up @@ -300,9 +301,9 @@ def check_call_variety(test_data, api_mapping):
report = check_call_variety(test_data, api_mapping)

if args.report:
report_outpath = os.path.join(output_dir, "callvariety_report.md")
report_outpath = os.path.join(output_dir, "validation_report.md")
with open(report_outpath, "w") as f:
f.write("# Call Variety Report\n\n")
f.write("# Unittest Validation Report\n\n")

columns = ["api", "all args", "all kwargs", "kwargs out of order"]
f.write(f'| {" | ".join(columns)} |\n')
Expand Down

0 comments on commit fb04838

Please sign in to comment.