Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gnovm): Test Coverage Support #2616

Open
wants to merge 44 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
ccf344c
basic statement coverage
notJoon Jul 23, 2024
78226e2
add control flow graph
notJoon Jul 23, 2024
509d36b
wip: branch
notJoon Jul 23, 2024
9f5e551
fix branch test
notJoon Jul 23, 2024
9a06318
use cfg later. check more conditions
notJoon Jul 23, 2024
f9e44ce
revert
notJoon Aug 29, 2024
225aac9
Merge branch 'master' into coverage
notJoon Aug 29, 2024
fea3044
coverage type
notJoon Aug 29, 2024
b43bd02
something works
notJoon Aug 29, 2024
6cbd004
save
notJoon Aug 29, 2024
7ec4d19
save
notJoon Aug 29, 2024
4295d5f
update logic
notJoon Sep 5, 2024
6aa2cfd
fix: file line handler
notJoon Sep 6, 2024
c5854d7
inspect only the package that contains the executed test file
notJoon Sep 6, 2024
b8f9e0c
feat: more precise coverage and add colored output command
notJoon Sep 9, 2024
c16d806
Merge branch 'master' into coverage
notJoon Sep 9, 2024
6a458ba
json output
notJoon Sep 9, 2024
38e7be4
Record coverage for variable assignments and expressions in op_assign…
notJoon Sep 9, 2024
75431a0
calculate precise executable lines
notJoon Sep 10, 2024
12fbad6
Refactor coverage data handling in Machine
notJoon Sep 10, 2024
0870a7b
remove unnecessat recodings
notJoon Sep 11, 2024
ecd76ba
update CLI
notJoon Sep 11, 2024
8a35fc6
fix minor bugs
notJoon Sep 12, 2024
3ec1df7
fix test
notJoon Sep 14, 2024
d994325
update docs
notJoon Sep 18, 2024
7efca2c
function coverage save
notJoon Sep 19, 2024
19dd206
update non-executable elements
notJoon Sep 19, 2024
a326278
remove func coverage
notJoon Sep 20, 2024
2fde27e
lint
notJoon Sep 20, 2024
9c09457
Merge branch 'master' into coverage
notJoon Oct 2, 2024
1d02c9a
increase coverage
notJoon Oct 5, 2024
c77ed87
state handler
notJoon Oct 21, 2024
6ac0426
disable recording when not enabled
notJoon Oct 21, 2024
b3a0413
fix failed test
notJoon Oct 22, 2024
cc0b24a
fix
notJoon Oct 22, 2024
e7c0d52
Merge branch 'master' into coverage
notJoon Oct 25, 2024
0d06a6f
fix
notJoon Oct 25, 2024
c59c4a7
fix
notJoon Oct 25, 2024
ae248da
Merge branch 'master' into coverage
notJoon Oct 28, 2024
56cc61f
fix
notJoon Oct 28, 2024
d2fac1d
Merge branch 'master' into coverage
notJoon Nov 14, 2024
003b798
fix lint error
notJoon Nov 14, 2024
abdd0b3
Merge branch 'master' into coverage
notJoon Nov 27, 2024
88128a4
decouple
notJoon Nov 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions gnovm/pkg/coverage/analyze.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
package coverage

import (
"go/ast"
"go/parser"
"go/token"
)

// detectExecutableLines analyzes the given source code content and returns a map
// of line numbers to boolean values indicating whether each line is executable.
func DetectExecutableLines(content string) (map[int]bool, error) {
fset := token.NewFileSet()
node, err := parser.ParseFile(fset, "", content, parser.ParseComments)
if err != nil {
return nil, err
}

executableLines := make(map[int]bool)

ast.Inspect(node, func(n ast.Node) bool {
if n == nil {
return true
}

if isExecutableLine(n) {
line := fset.Position(n.Pos()).Line
executableLines[line] = true
}

return true
})

return executableLines, nil
}

// countCodeLines counts the number of executable lines in the given source code content.
func CountCodeLines(content string) int {
lines, err := DetectExecutableLines(content)
if err != nil {
return 0
}

Check warning on line 41 in gnovm/pkg/coverage/analyze.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/analyze.go#L37-L41

Added lines #L37 - L41 were not covered by tests

return len(lines)

Check warning on line 43 in gnovm/pkg/coverage/analyze.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/analyze.go#L43

Added line #L43 was not covered by tests
}

// isExecutableLine determines whether a given AST node represents an
// executable line of code for the purpose of code coverage measurement.
//
// It returns true for statement nodes that typically contain executable code,
// such as assignments, expressions, return statements, and control flow statements.
//
// It returns false for nodes that represent non-executable lines, such as
// declarations, blocks, and function definitions.
func isExecutableLine(node ast.Node) bool {
switch n := node.(type) {
case *ast.AssignStmt, *ast.ExprStmt, *ast.ReturnStmt, *ast.BranchStmt,
*ast.IncDecStmt, *ast.GoStmt, *ast.DeferStmt, *ast.SendStmt:
return true
case *ast.IfStmt, *ast.ForStmt, *ast.RangeStmt, *ast.SwitchStmt,
*ast.TypeSwitchStmt, *ast.SelectStmt:
return true
case *ast.CaseClause:
// Even if a `case` condition (e.g., `case 1:`) in a `switch` statement is executed,
// the condition itself is not included in the coverage; coverage only recorded for the
// code block inside the corresponding `case` clause.
return false
case *ast.LabeledStmt:
return isExecutableLine(n.Stmt)

Check warning on line 68 in gnovm/pkg/coverage/analyze.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/analyze.go#L62-L68

Added lines #L62 - L68 were not covered by tests
case *ast.FuncDecl:
return false
case *ast.BlockStmt:
return false
case *ast.DeclStmt:
// check inner declarations in the DeclStmt (e.g. `var a, b = 1, 2`)
// if there is a value initialization, then the line is executable
genDecl, ok := n.Decl.(*ast.GenDecl)
if ok && (genDecl.Tok == token.VAR || genDecl.Tok == token.CONST) {
for _, spec := range genDecl.Specs {
valueSpec, ok := spec.(*ast.ValueSpec)
if ok && len(valueSpec.Values) > 0 {
return true
}

Check warning on line 82 in gnovm/pkg/coverage/analyze.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/analyze.go#L73-L82

Added lines #L73 - L82 were not covered by tests
}
}
return false

Check warning on line 85 in gnovm/pkg/coverage/analyze.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/analyze.go#L85

Added line #L85 was not covered by tests
case *ast.ImportSpec, *ast.TypeSpec, *ast.ValueSpec:
return false
case *ast.InterfaceType:
return false

Check warning on line 89 in gnovm/pkg/coverage/analyze.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/analyze.go#L88-L89

Added lines #L88 - L89 were not covered by tests
case *ast.GenDecl:
switch n.Tok {
case token.VAR, token.CONST:
for _, spec := range n.Specs {
valueSpec, ok := spec.(*ast.ValueSpec)
if ok && len(valueSpec.Values) > 0 {
return true
}

Check warning on line 97 in gnovm/pkg/coverage/analyze.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/analyze.go#L96-L97

Added lines #L96 - L97 were not covered by tests
}
return false
case token.TYPE, token.IMPORT:
return false
default:
return true

Check warning on line 103 in gnovm/pkg/coverage/analyze.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/analyze.go#L102-L103

Added lines #L102 - L103 were not covered by tests
}
default:
return false
}
}
90 changes: 90 additions & 0 deletions gnovm/pkg/coverage/analyze_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
package coverage

import (
"testing"

"github.com/stretchr/testify/assert"
)

func TestDetectExecutableLines(t *testing.T) {
t.Parallel()
tests := []struct {
name string
content string
want map[int]bool
wantErr bool
}{
{
name: "Simple function",
content: `
package main

func main() {
x := 5
if x > 3 {
println("Greater")
}
}`,
want: map[int]bool{
5: true, // x := 5
6: true, // if x > 3
7: true, // println("Greater")
},
wantErr: false,
},
{
name: "Function with loop",
content: `
package main

func loopFunction() {
for i := 0; i < 5; i++ {
if i%2 == 0 {
continue
}
println(i)
}
}`,
want: map[int]bool{
5: true, // for i := 0; i < 5; i++
6: true, // if i%2 == 0
7: true, // continue
9: true, // println(i)
},
wantErr: false,
},
{
name: "Only declarations",
content: `
package main

import "fmt"

var x int

type MyStruct struct {
field int
}`,
want: map[int]bool{},
wantErr: false,
},
{
name: "Invalid gno code",
content: `
This is not valid Go code
It should result in an error`,
want: nil,
wantErr: true,
},
}

for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got, err := DetectExecutableLines(tt.content)
assert.Equal(t, tt.wantErr, err != nil)
assert.Equal(t, tt.want, got)
})
}
}
170 changes: 170 additions & 0 deletions gnovm/pkg/coverage/coverage.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
package coverage

import (
"io"
"path/filepath"
)

// Collector defines the interface for collecting coverage data
type Collector interface {
RecordHit(loc FileLocation)
SetExecutableLines(filePath string, lines map[int]bool)
AddFile(filePath string, totalLines int)
}

// Coverage implements the Collector interface and manages coverage data
type Coverage struct {
enabled bool
rootDir string
currentPath string
currentFile string
files fileCoverageMap
pathCache pathCache
}

// FileCoverage stores coverage information for a single file
type FileCoverage struct {
totalLines int
hitLines map[int]int
executableLines map[int]bool
}

type (
fileCoverageMap map[string]FileCoverage
pathCache map[string]string
)

func (m fileCoverageMap) get(path string) (FileCoverage, bool) {
fc, ok := m[path]
return fc, ok
}

func (m fileCoverageMap) set(path string, fc FileCoverage) {
m[path] = fc
}

// NewFileCoverage creates a new FileCoverage instance
func NewFileCoverage() FileCoverage {
return FileCoverage{
totalLines: 0,
hitLines: make(map[int]int),
executableLines: make(map[int]bool),
}
}

// New creates a new Coverage instance
func New(rootDir string) *Coverage {
return &Coverage{
rootDir: rootDir,
files: make(fileCoverageMap),
pathCache: make(pathCache),
}
}

// Configuration methods
func (c *Coverage) Enabled() bool { return c.enabled }
func (c *Coverage) Enable() { c.enabled = true }
func (c *Coverage) Disable() { c.enabled = false }
func (c *Coverage) SetCurrentPath(path string) { c.currentPath = path }
func (c *Coverage) CurrentPath() string { return c.currentPath }

Check warning on line 69 in gnovm/pkg/coverage/coverage.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/coverage.go#L69

Added line #L69 was not covered by tests
func (c *Coverage) SetCurrentFile(file string) { c.currentFile = file }
func (c *Coverage) CurrentFile() string { return c.currentFile }

Check warning on line 71 in gnovm/pkg/coverage/coverage.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/coverage.go#L71

Added line #L71 was not covered by tests

// RecordHit implements Collector.RecordHit
func (c *Coverage) RecordHit(loc FileLocation) {
if !c.enabled { return }

path := filepath.Join(loc.PkgPath, loc.File)
cov := c.getOrCreateFileCoverage(path)

if cov.executableLines[loc.Line] {
cov.hitLines[loc.Line]++
c.files.set(path, cov)
}
}

// SetExecutableLines implements Collector.SetExecutableLines
func (c *Coverage) SetExecutableLines(filePath string, executableLines map[int]bool) {
cov, exists := c.files.get(filePath)
if !exists {
cov = NewFileCoverage()
}

cov.executableLines = executableLines
c.files.set(filePath, cov)
}

// AddFile implements Collector.AddFile
func (c *Coverage) AddFile(filePath string, totalLines int) {
if IsTestFile(filePath) || !isValidFile(c.currentPath, filePath) {
return
}

Check warning on line 101 in gnovm/pkg/coverage/coverage.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/coverage.go#L100-L101

Added lines #L100 - L101 were not covered by tests

cov, exists := c.files.get(filePath)
if !exists {
cov = NewFileCoverage()
}

cov.totalLines = totalLines
c.files.set(filePath, cov)
}

// Report generates a coverage report using the given options
func (c *Coverage) Report(opts ReportOpts, w io.Writer) error {
reporter := NewReporter(c, opts)
if opts.pattern != "" {
return reporter.WriteFileDetail(w, opts.pattern, opts.showHits)
}
return reporter.Write(w)

Check warning on line 118 in gnovm/pkg/coverage/coverage.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/coverage.go#L113-L118

Added lines #L113 - L118 were not covered by tests
}

// FileLocation represents a specific location in source code
type FileLocation struct {
PkgPath string
File string
Line int
Column int
}

// Helper methods
func (c *Coverage) getOrCreateFileCoverage(filePath string) FileCoverage {
cov, exists := c.files.get(filePath)
if !exists {
cov = NewFileCoverage()
}

Check warning on line 134 in gnovm/pkg/coverage/coverage.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/coverage.go#L133-L134

Added lines #L133 - L134 were not covered by tests
return cov
}

// GetStats returns coverage statistics for a file
func (c *Coverage) GetStats(filePath string) (hits, total int, ok bool) {
cov, exists := c.files.get(filePath)
if !exists {
return 0, 0, false
}
return len(cov.hitLines), cov.totalLines, true

Check warning on line 144 in gnovm/pkg/coverage/coverage.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/coverage.go#L139-L144

Added lines #L139 - L144 were not covered by tests
}

// GetFileHits returns the hit counts for a file (primarily for testing)
func (c *Coverage) GetFileHits(filePath string) map[int]int {
if cov, exists := c.files.get(filePath); exists {
return cov.hitLines
}
return nil

Check warning on line 152 in gnovm/pkg/coverage/coverage.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/coverage.go#L148-L152

Added lines #L148 - L152 were not covered by tests
}

// GetExecutableLines returns the executable lines for a file (primarily for testing)
func (c *Coverage) GetExecutableLines(filePath string) map[int]bool {
if cov, exists := c.files.get(filePath); exists {
return cov.executableLines
}
return nil

Check warning on line 160 in gnovm/pkg/coverage/coverage.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/coverage.go#L156-L160

Added lines #L156 - L160 were not covered by tests
}

// GetFiles returns a list of all tracked files
func (c *Coverage) GetFiles() []string {
files := make([]string, 0, len(c.files))
for file := range c.files {
files = append(files, file)
}
return files

Check warning on line 169 in gnovm/pkg/coverage/coverage.go

View check run for this annotation

Codecov / codecov/patch

gnovm/pkg/coverage/coverage.go#L164-L169

Added lines #L164 - L169 were not covered by tests
}
Loading
Loading