Merge pull request #574 from simon5656/Jan-Update

Jan Update

TGs/verification-task-group/projects/ARVM-Advanced-RISCV-Verification-Methodologies-TopProject/meetings/2023/2022-Aug10-ARVM-Advanced-riscv-Verification-Methodologies-OpenHW-Project-Concept-Proposal.(PASSED).md

@@ -0,0 +1,138 @@
+# OpenHW Project Concept Proposal:
+# Advanced RISC-V Verification Methodology (ARVM)
+## Presented at OpenHW TWG 2022-July-25, reviewed 10-Aug-2022, and Approved 06-Sep-2022
+
+## Summary of Project
+This project is to be a project under the VTG.
+
+This project aims to enhance the capabilities and efficiency of the RISC-V verification available to all RISC-V core developers and thereby improve the quality of the available RISC-V cores and reduce the risk of RISC-V market fragmentation, disarray, and slow growth.
+
+## Summary of market
+Today, many RISC-V processor implementations use a relatively simple subset of the RISC-V ISA. However, more advanced features and capabilities are brought forward as implementations, supporting features such as Out-of-Order, multi-issue, multi-core, multi-hart, multi-thread, and new ISA extensions such as vectors.
+
+To support this evolution in processor capabilities, processor verification tools, technology, VIPs and methodologies must continually improve and be adopted for delivery of quality RISC-V processors.
+
+By working on and succeeding with this ARVM project, OpenHW can lead the industry in evolving best-in-class RISC-V processor DV solutions and impact the entire RISC-V community.
+
+### Goals of Project
+The project will address RISC-V processor verification with an initial sub-project list of:
+
+- **ARVM-Methodologies**: improving the capabilities and available methodologies available for verification environments
+- **ARVM-Promotion**: educating / informing processor verification teams of choices and techniques available across the RISCV community and verification ecosystem (for example tutorials and videos)
+- **ARVM-TestbenchQuality**: developing quality measurement of test benches - so quality of cores can be predicted (for example defining fault models and tests and relating those to TRL levels)
+- **ARVM-Standards**: defining and implementing evolving interface standards for test bench components to enable better test bench component reuse and potentially stimulate availability of compatible VIPs
+- **ARVM-FunctionalCoverage**: developing open-source VIPs (such as functional coverage) that can be used for many different core configurations/implementations
+- **ARVM-SoCIntegration** : consider requirements and solutions for SoC core integration verification (for example cache coherency with uncore components)
+- **ARVM-Roadmap**: ongoing roadmap to accommodate new innovations in RISC-V designs, new ratified extensions, and new tool developments
+
+To be clear - this project / group is focused on advancing high quality industrial strength verification and is not targeting specific OpenHW core implementations / teams / groups - but it is expected that members of the various OpenHW core verification teams will participate actively in, and benefit from, this project.
+
+## Project Organization
+This project and its sub-projects will be tracked as follows in OpenHW:
+- ARVM (Advanced riscv Verification Methodologies) will be the "carrier project" within VTG that goes through the Project Concept gate in the OpenHW TWG to setup the basic idea and structure
+- Each of the sub-projects as listed above (or others to be determined) will traverse the Project Launch, Plan Approve, and Project Freeze gates in the OpenHW TWG under the guidance of the ARVM leaders
+- ARVM will work with OpenHW staff to develop the Project Launch, Plan Approve and Project Freeze criteria for the sub-projects as suits their nature
+
+## Who would make use of the developments in this project
+This project is focused on advancing high quality industrial strength verification and is not targeting specific OpenHW core implementations / teams / groups - but it is expected that members of the various OpenHW core verification teams will participate actively in, and benefit from, this project. 
+
+The outputs from this project range from education & methodology, to VIPs and interface standards - all of which will enhance DV teams capabilities and thereby drive RISC-V to more success.
+
+All 'internal' OpenHW core verification teams can participate and benefit and so can external core verification teams, including commercial partners. Nothing will be done which will be specific to OpenHW cores - the focus is ecosystem wide.
+
+OpenHW is a collaborative community / organization and for a sub-project to become an approved ARVM project an OpenHW member must propose it and provide resources to work on the sub-project.
+
+There is no requirement that a sub-project is directly related to an existing OpenHW core / project.
+
+### Other verification projects (not this project, but part of other VTG projects)
+Here are some suggestions about other projects that could be done in VTG if / when member companies come forward to develop project and resource plans for them (but note - these are not currently part of this ARVM project planning):
+- Continued promotion of SV/UVM testbenches for CORE-V cores and SoCs
+- Reduction of the integration effort of new cores into core-v-verif
+- Build on the core-v-mcu-uvm project to develop an SoC verification methodology
+- Review and update of Strategic Goals for core-v-verif
+- Standing up a multi-simulator automatic CI flow for all supported CORE-V cores
+- Creation of an open-source formal verification testbench for at least one CORE-V core
+- Refactoring of CORE-V-VERIF into per-core verification repos
+- Make ISACOV standalone VIP, able to connect to any Instruction Fetch Bus Agent
+- Improved processes for creating, writing and tracking DVplans
+- Toolchain independence
+- Update riscv-dv to make it a true UVM component (and integrate it into core-v-verif)
+- Update the verification coding style guidelines
+- Lint checking to create automated checks for the above guidelines
+
+However these may become part of the ARVM sub projects as appropriate.
+
+## Initial Estimate of Timeline
+ARVM is an ongoing carrier project. The following are initial steps:
+
+August / September 2022
+- Refine and enhance the sub-projects list, develop the sub-projects focus
+- Encourage participation in ARVM sub-projects both among existing OpenHW members and new prospective members
+- for ARVM sub-projects setup recurring meetings on the OpenHW Calendar, chaired by the ARVM Technical Project Leader (named)
+- for ARVM sub-projects setup MatterMost channels
+- for ARVM sub-projects setup GitHub repos
+
+October 2022 - onwards - get on with it
+
+December 2022 - target for an initial progress update at the RISC-V Summit
+
+Provide regular feedback on progress to VTG, TWG etc...
+
+## Explanation of why OpenHW should do this project
+As an open-source industry forum already focusing on industry quality verification of CORE-V processor cores, publishing results in open-source while making use of best available tools, it is natural for the OpenHW Group to foster continued development of verification methodologies specific to that application domain. While the ARVM project would increase the scope of verification methodology within OpenHW to consider requirements beyond those of the CORE-V Processor Cores IP, the increased verification methodology focus will be beneficial for CORE-V Cores IP and for OpenHW members.
+
+## Industry landscape: description of competing, alternative, or related efforts in the industry
+RISC-V has created opportunities for new entrants to develop their own processors.  Few of these new entrance have experience with processor verification.
+
+As there is no industry accepted flow for processor verification - every core verification team is forced to invent & develop their own verification ideas, components, tools, simulators, models, interfaces, frameworks, flows, etc. This is inefficient, costly, error prone, and is unnecessary.
+
+RISC-V International is focused on the ISA and the standardization of the ISA. Yes RVI has a compliance group - but this is solely focused on ISA compliance (architecture) and not, repeat not, on processor implementation verification (micro-architecture).
+
+CHIPS Alliance is focused on open-source EDA tooling, for example Verilator, and RISCV-DV - yes components as part of verification. 
+
+This new OpenHW ARVM group is needed and focused on industrial grade processor verification.
+
+## OpenHW Members/Participants committed to participate
+- Imperas 
+- Codasip (name to be provided)
+- Other EDA tool companies under discussion to join OpenHW
+- Current and future OpenHW members actively verifying one or more CORE-V projects
+
+All OpenHW core projects with verification targeting TRL5 should be participating.
+
+## Project Leader(s)
+### Technical Project Leader(s)
+Simon Davidmann (Imperas)
+
+Each sub-group will have a 'leader' - tbd.
+
+## Resource Requirements
+Imperas will provide leadership for the ARVM carrier project as well as contribute to the engineering resources for the sub-projects
+
+Other members will participate and contribute in the new project and sub-projects
+
+Engineering resources are mainly needed to help drive the sub-projects
+
+## Project license model
+All deliverables including documents and code will be open-source on https://github.com/openhwgroup/ and developed under Apache 2.0 and/or Solderpad 2.1 licenses
+
+## Description of initial code contribution, if required
+It is not currently thought that ARVM will be part of the Eclipse CORE-V Cores project. If required any initial code contributions will follow OpenHW/EF contribution questionnaire rules.
+
+## Repository Requirements
+Github repos for specs, project documents, milestones, reports etc.
+
+## Project distribution model
+Code Releases will be made available on http://downloads.openhwgroup.org/ under an ARVM heading.
+
+Output documentation will be made available on ReadTheDocs at https://github.com/openhwgroup/advanced-riscv-verification-methodologies
+
+## Preliminary Project plan
+as above
+
+## Risk Register
+Main risk for the carrier project initially is lack of participation and interest during the refinement of the sub-project list. To succeed, ARVM requires a considerable critical mass of expertise on verification. 
+
+To manage this, the ARVM project requires "lobbying" for participation by the current OpenHW processor companies and organizations. 
+
+

TGs/verification-task-group/projects/ARVM-FunctionalCoverage/meetings/2023/monthly-report-2023-Jan-16.md

@@ -0,0 +1,50 @@
+
+[comment]: # "this template is for ARVM projects"
+
+# **ARVM-FunctionalCoverage** Monthly Report for 16-January-2023
+
+Project leader : Simon Davidmann   
+Lead company :  Imperas  
+Participating companies : SiLabs, Dolphin  
+
+## Overview
+There are almost 1,000 instructions in RV64 (inc. all ratified and soon to be ratified extensions).  
+For each instruction somebody will need to write SystemVerilog covergroups and coverpoints…   
+Maybe 10-40 lines of SystemVerilog for each instruction…   
+That is 10,000-40,000 lines of SystemVerilog code to be written for each core… (and be correct and working…)  
+And that is just for the basic un-privilege mode ISA…   
+
+This sub-project is to collect requirements to enable the developing of VIPs (such as functional coverage) that can be used for many different core configurations/implementations.  
+
+## Current Status
+cv32e40p included and used first generation of SystemVerilog for RV32I generated by Imperas in 2020.  
+
+Initial focus of this project is F,Zfinx (FPU) functional coverage for cv32e40pv2 (Dolphin) and Zc for cv32e40s (SiLabs).   
+This second generation architecture is auto generated and works directly from RVVI-TRACE core tracer testbench interface.   
+
+Uses machine readable ISA definition and generates examples as compliance level functional coverage for RV32I.  
+
+This riscvISACOV is now documented and RV32I is available as open source from github: [https://github.com/riscv-verification/riscvISACOV](https://github.com/riscv-verification/riscvISACOV)
+
+Currently soliciting input / requirements on what needs to be covered in verification plans for different ISA extensions.   
+
+Also, beta code has been generated for many extensions, including: 32 bit I, E, M, C, B, K.  237 instructions covered in 2030 coverpoints.  
+
+## Key activities / tasks completed this month
+- F,Zfinx (FPU): First SystemVerilog functional coverage code for FPU has been generated and feedback from Dolphin has meant many more cross coverage and fixed values have been included.
+- Zc (code-size-reduction): has been coded up and generated coverage is being reviewed by SiLabs - with feedback on cross coverage - and requirements for interrupt coverage across multi-cylce instructions like push/pop/mv  
+    - This is now included in pull request for [core-v-verif/cv32e40s/vendor_lib/imperas/riscvISACOV/source/coverage/](https://github.com/openhwgroup/core-v-verif/tree/960880333a53d44b403c4d094d77b37652718880/cv32e40s/vendor_lib/imperas/riscvISACOV/source/coverage)  
+- output of code generator has sections for verification plan in .csv - currently being reviewed  
+- output also .md files of summaries of each extension
+
+## Planned activities / tasks for coming month
+- reviews of resultant functional coverage (RV32I, F, Zc)   
+- addition of more requirements for F and Zc
+- complete interrupt coverage over multi-cycle instructions  
+- addition of more design verification (DV) coverage such as hazards (and micro-architectural)   
+- discussions regarding working with isacov
+
+## Issues / items that are slowing progress
+- Zc functional coverage is currently awaiting update to RVVI to handle async trap handler notifications for cross coverage
+
+

TGs/verification-task-group/projects/ARVM-Promotion/meetings/2023/monthly-report-2023-Jan-16.md

@@ -0,0 +1,63 @@
+
+[comment]: # "this template is for ARVM projects"
+
+# **ARVM-Promotion** Monthly Report for 16-January-2023
+
+Participating companies : OpenHW members
+
+## Overview
+Educating and informing processor verification teams of choices and techniques available across the RISCV community and verification ecosystem (for example conference talks, tutorials and videos).
+
+Purpose is to provide more information on RISC-V verification.
+
+## Current Status
+Many members actively promoting RISC-V Verification - though we always need more...
+
+## Issues / items that are slowing progress
+- need more members, companies, organizations talking publicly about verification
+- please post on matter most upcoming talks and other promotions...
+
+## Future conference papers/presentations
+- HiPEAC - Jan 2023  
+    - "The evolution of RISC-V and the adoption of new open verification standards" (Imperas)  
+    - Panel: "European RISC-V in HPC prospective" (SemiDynamics, Codasip, Cortus, Imperas)    
+- DVCon  - Feb/Mar 2023
+    - "The Evolution of RISC-V Processor Verification: Open Standards and Verification IP" (Imperas)
+- Embedded World - March 2023
+    - "Advanced methodologies to address RISC-V verification for all adopters" (OpenHW, Imperas)
+    - "New ecosystem leads RISC-V mainstream adoption with innovation ready software development and processor verification tools" (Imperas, Dolphin, Intel)
+    - "Example of Extending RISC-V for AI/ML Domain Specific Processors" (Dolphin, Imperas, OpenHW)
+    - "Introduction to RISC-V Processor Verification" (RISC-V International/Imperas)
+    - "Getting started with RISC-V custom instructions" (RISC-V International/Imperas)
+
+## Talks already given (most recent first)
+- RISC-V Summit 2022
+    - Keynote: "Improving RISC-V Quality with Verification Standards and Advanced Methodologies" Simon Davidmann
+        - [Watch on YouTube](https://www.youtube.com/watch?v=CojvlnbGD-A)
+    - "RISC-V Models for Verification, Software Development and Architectural Exploration"
+        - [Watch on YouTube](https://www.youtube.com/watch?v=JJT8Uk_U_Wo)
+    - "The New Verification Ecosystem that Supports RISC-V Verification for all Adopters" (Breker, Imperas)
+        - [Watch on YouTube](https://www.youtube.com/watch?v=QGfHeccfOfA)
+    - "Introduction to RISC-V Verification with the Open Standard RVVI (RISC-V Verification Interface)"
+        - [Watch on YouTube](https://www.youtube.com/watch?v=JECxAFE0Yho)
+    - "Building a Global CORE-V Cores Ecosystem"
+        - [Watch on YouTube](https://www.youtube.com/watch?v=i6bTIhrIUek)
+    - "The Continuum of RISC-V Compliance and Verification Testing"
+        - [Watch on YouTube](https://www.youtube.com/watch?v=VU9MjjprkBc) (RISC-V International Compliance, Imperas)
+    - Tutorial: "Choosing Appropriate Verification Techniques for Desired RISC-V Processor Quality"
+        - [Watch on YouTube](https://www.youtube.com/watch?v=c2H3iRl2WEc) (Imperas)
+- DVClub Europe Nov 2022
+    - "RISC-V processor verification with new open standard RVVI-based methodology"
+        - [Watch on YouTube](https://www.youtube.com/watch?v=1YBT5rYEAZI)
+- DVClub Oct 2022
+    - "The Art and Science of Automating Verification Checking"
+        - [Watch on YouTube](https://www.youtube.com/watch?v=SV2_9rRg7rk)
+- RISC-V Summit 2021
+    - "Open-Source RISC-V Cores with Industrial Strength Verification"
+        - [Watch on YouTube](https://www.youtube.com/watch?v=SZGApzOGsFw)
+- YouTube Video 2021 ["Better Quality RTL"](https://www.youtube.com/watch?v=wwSEIEfxysc) from Philippe Luc of Codasip 
+- RISC-V Summit 2020
+    - Tutorial: "Tutorial Getting Started with RISC-V Verification"
+        - [Watch on YouTube](https://www.youtube.com/watch?v=G2IE7DM-tjE)
+
+

TGs/verification-task-group/projects/ARVM-SoCIntegration/meetings/2023/monthly-report-2023-Jan-16.md

@@ -0,0 +1,41 @@
+
+[comment]: # "this template is for ARVM projects"
+
+# **ARVM-SocIntegration** Monthly Report for 16-January-2023
+
+Project leader : Simon Davidmann   
+Lead company :  Imperas  
+Participating companies : (starting exploring working with non-member Breker)
+
+## Overview
+To consider requirements and solutions for SoC core integration verification (for example cache coherency with uncore components)
+
+## Current Status
+Starting to have discussion on graph based test program generation like PSS and Breker
+
+Current view:  
+There are several levels of verification (read from bottom):
+- SoC integration level - verifying the integration of the core IP - SoC level integration test
+- core DV level - verifying at micro-architecture level - is the implementation good
+- core compliance level - basically ISA - has the spec been understood - verifying architecture understanding  
+
+## Key activities / tasks completed this month
+- had further discussions 
+- created demonstration between processor core and SoC and memory coherency tests
+- presentation and demonstrations at RISC-V Summit Dec 2022
+    - "The New Verification Ecosystem that Supports RISC-V Verification for all Adopters" (Breker, Imperas)
+        - [Watch on YouTube](https://www.youtube.com/watch?v=QGfHeccfOfA)
+
+- Still early days - just starting to collect information  and try things and get ecosystem feedback
+
+
+## Planned activities / tasks for coming month
+- on going discussions
+
+## Issues / items that are slowing progress
+- none - just starting up...
+
+## Other information
+
+
+