Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add short summary table for non-specialists to convey quickly what the results are #667

Merged
merged 3 commits into from
May 23, 2024
Merged

Add short summary table for non-specialists to convey quickly what the results are #667

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7d61285
Add short summary table to convey quickly what the results are
dd-BeeNee May 23, 2024
b8aa291
Rename RTL_Freeze_v2.0.0_summary.md to README.md
dd-BeeNee May 23, 2024
ed595e3
Update README.md
dd-BeeNee May 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Project-Descriptions-and-Plans/CV32E40Pv2/Milestone-data/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ The OpenHW Group asserts that the documentation, implementation and verification
The directories and files below this point store the completed checklists, reports (and waivers) in support of this claim.

**RTL_Freeze_v2.0.0** :
- README.md : overview summary tables for RTL Freeze results
- OpenHWGroup_TRL5_for_COREV_RTL_Cheklist-CV32E40P.xls: checklists for the v2.0.0 tag of CV32E40P.
- CV32E40Pv2_regression_known_failure.xls
- CV32E40Pv2_uncovered_coverage_explanation.xls
Expand Down
Binary file modified ...iptions-and-Plans/CV32E40Pv2/Milestone-data/RTL_Freeze_v2.0.0/CV32E40Pv2_waiver_list.xlsx
View file
Open in desktop
Binary file not shown.
56 changes: 56 additions & 0 deletions ...ct-Descriptions-and-Plans/CV32E40Pv2/Milestone-data/RTL_Freeze_v2.0.0/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
## RTL_Freeze_v2.0.0_summary
Short summary table for the RTL_Freeze_v2.0.0 results overview

### Formal Verification
Control and Datapath assertions checking runs launched on 3 configurations
*430 assertions. (TODO:update the number of assertions)*
| Configurations | Status |
----------------------------------------- | ------------------------------------------------ |
PULP | Successful unbounded check (11 days) |
PULP_FPU (0 cycle latency) | still running after 18 days. No error so far. |
PULP_FPU_ZFINX_2CYCLAT (2 cycles latency) | still running after 18 days. No error so far. |

### Regression Results
*Some testcases run multiple seeds in one regression*
|Configurations | xplup || F + xpulp || F + xpulp (lat. 1) || F + xpulp (lat. 2) || Xpulp + Zfinx || Xpulp + Zfinx (lat 1) || Xpulp + Zfinx (lat. 2) || **Total All Cfg** ||
----------------------------|------|------|------|------|------|--------------|------|--------------|------|---------|-------|----------------|------|------------------|----------|----------|
**Regress File** | Pass | Fail | Pass | Fail | Pass | Fail | Pass | Fail | Pass | Fail | Pass | Fail | Pass | Fail | **Pass** | **Fail** |
cv32e40pv2_fpu_instr | NA | NA | 1504 | 0 | 1504 | 0 | 1504 | 0 | 1504 | 0 | 1504 | 0 | 1504 | 0 | 9024 | 0 |
cv32e40pv2_interrupt_debug | 1701 | 0 | 1950 | 2 | 1951 | 1 | 1951 | 1 | 1951 | 1 | 1952 | 0 | 1949 | 3 | 13405 | 8 |
cv32e40pv2_xpulp_instr | 1433 | 0 | 1433 | 0 | 1433 | 0 | 1433 | 0 | 1433 | 0 | 1433 | 0 | 1433 | 0 | 10031 | 0 |
cv32e40pv2_legacy_v1 | 29 | 0 | 29 | 0 | 29 | 0 | 29 | 0 | 29 | 0 | 29 | 0 | 29 | 0 | 203 | 0 |
**Total number of tests** | 3134 | 0 | 4887 | 2 | 4888 | 1 | 4888 | 1 | 4888 | 1 | 4889 | 0 | 4886 | 3 | 32663 | 8 |

### Riscof Architecture Test
| Configurations | Status |
-------------------------------|--------|
PULP configuration | Pass |
PULP_FPU_0CYCLAT configuration | Pass |
PULP_FPU_1CYCLAT configuration | Pass |
PULP_FPU_2CYCLAT configuration | Pass |

### RTL Code Coverage
*Only left with holes in cv32e40p_controller (12 causes resulting in 24 holes).
Seeking help from Openhwgroup community*
| Configurations | Statement | Branch | Condition |
-------------------------------|-----------|--------|-----------|
PULP Configuration | 99.8% | 99.5% | 98.7% |
PULP_FPU_0CYCLAT configuration | 99.9% | 99.5% | 98.5% |

### Functional Coverage
(1) **Combined from all 7 configurations** using PULP_FPU_0CYCLAT as master
| Covergroups | Status |
-----------------------|------------------------------------------------------|
FPU | 100% |
HWLOOP | 100% |
Debug | 100% |
Interrupts | 100% |
OBI | 100% |
Assertions & Directive | 100% |
riscvISACOV | 95.01% (*optional. We use Formal OneSpin tool to verify instructions.*)|

(2) **Combined from 3 ZFINX configurations** using PULP_ZFINX_0CYCLAT as master
| Covergroups | Status |
-----------------------|------------------------------------------------------|
ZFINX | 100% |

108 changes: 1 addition & 107 deletions ...-Plans/CV32E40Pv2/Milestone-data/RTL_Freeze_v2.0.0/Reports/2024-05-06/Coverage/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,26 +25,6 @@ <h4>CV32E40P v2 RTL Code Coverage Reports</h4>
<ul><li><a href="Code_coverage/CFG_P_F0/htmlcovreport/index.html">CFG_P_F0 Code Coverage</a></li></ul>
<ul><li><a href="Code_coverage/CFG_P_F0/waivers/CFG_P_F0_code_cov_waivers_report.txt">CFG_P_F0 Code Coverage Waivers Report</a></li></ul>
<br><br>
<table border="1">
<tr>
<th></th>
<th>Statement Coverage</th>
<th>Branch Coverage</th>
<th>Condition Coverage</th>
</tr>
<tr>
<th>CFG_P</th>
<td>99.80%</td>
<td>99.48%</td>
<td>98.73%</td>
</tr>
<tr>
<th>CFG_P_F0</th>
<td>99.87%</td>
<td>99.51%</td>
<td>98.48%</td>
</tr>
</table>
</ul>

<br><br>
Expand All @@ -59,93 +39,7 @@ <h4>CV32E40P v2 Functional Coverage Reports</h4>
<ul><li><a href="Function_coverage/CFG_P_Z0/htmlcovreport/index.html">CFG_P_Z0 Function Coverage</a></li></ul>
<ul><li>no waiver</li></ul>
<br><br>
<li> Other results are presented also in the table below for reference only: FPU covergroup in CFG_P_F1/F2, Zfinx covergroup in CFG_P_Z1/Z2.</p>
<table border="1">
<tr>
<th></th>
<th>FPU Covergroups</th>
<th>HWLOOP Covergroups</th>
<th>Debug Covergroups</th>
<th>Interrupt Covergroups</th>
<th>OBI Covergroups</th>
<th>RISCV ISA Covergroups</th>
<th>Assertion</th>
<th>Directive</th>
<th>Zfinx Covergroups</th>
</tr>
<tr>
<th>CFG_P_F0</th>
<td><span style="background: green">100%</span></td>
<td><span style="background: green">100%</span></td>
<td><span style="background: green">100%</span></td>
<td><span style="background: green">100%</span></td>
<td><span style="background: green">100%</span></td>
<td>94.93%</td>
<td><span style="background: green">100%</span></td>
<td><span style="background: green">100%</span></td>
<td></td>
</tr>
<tr>
<th>CFG_P_Z0</th>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>99.99%</td>
</tr>
<tr>
<th>CFG_P_F1</th>
<td>99.74%</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<th>CFG_P_F2</th>
<td>99.70%</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<th>CFG_P_Z1</th>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>99.91%</td>
</tr>
<tr>
<th>CFG_P_Z2</th>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>99.89%</td>
</tr>
</table>
<li> Other configurations also have their Functional Coverage collected for reference, especially for: FPU covergroup in CFG_P_F1/F2, Zfinx covergroup in CFG_P_Z1/Z2.</p>
</ul>


Expand Down