Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: global var dependencies #2077

Open
wants to merge 46 commits into
base: master
Choose a base branch
from
Open

fix: global var dependencies #2077

wants to merge 46 commits into from

Conversation

petar-dambovaliev
Copy link
Contributor

@petar-dambovaliev petar-dambovaliev commented May 13, 2024

fixes this by adding missing logic in in findUndefined2

@github-actions github-actions bot added the 📦 🤖 gnovm Issues or PRs gnovm related label May 13, 2024
@petar-dambovaliev petar-dambovaliev marked this pull request as ready for review May 13, 2024 12:16
Copy link

codecov bot commented May 13, 2024

Codecov Report

Attention: Patch coverage is 55.17241% with 65 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
gnovm/pkg/gnolang/preprocess.go 55.17% 57 Missing and 8 partials ⚠️

📢 Thoughts on this report? Let us know!

Copy link
Member

@zivkovicmilos zivkovicmilos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some nitty comments 🙏

Overall looks good, thank you for the fix 🙏

gnovm/pkg/gnolang/preprocess.go Outdated Show resolved Hide resolved
gnovm/pkg/gnolang/preprocess.go Show resolved Hide resolved
gnovm/pkg/gnolang/preprocess.go Show resolved Hide resolved
gnovm/pkg/gnolang/preprocess.go Show resolved Hide resolved
gnovm/pkg/gnolang/preprocess.go Outdated Show resolved Hide resolved
Copy link
Member

@thehowl thehowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also add more tests to check for different cases, thanks!

@github-actions github-actions bot added 🧾 package/realm Tag used for new Realms or Packages. 📦 🌐 tendermint v2 Issues or PRs tm2 related 📦 ⛰️ gno.land Issues or PRs gno.land package related labels Jun 12, 2024
Copy link
Contributor

@deelawn deelawn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the first part of my review. I want to make sure I understand the intentions before completing it.

Tell me if I have the main idea of what this is solving. Previously, names referenced by globally defined function literals were not being verified to have been defined. These changes use this unique case as an entrypoint to recursively analyze all of the function literal's statements to ensure that all names referenced are actually defined.

Otherwise, please see the couple of comments I've left.

gnovm/pkg/gnolang/preprocess.go Show resolved Hide resolved
gnovm/pkg/gnolang/preprocess.go Show resolved Hide resolved
@petar-dambovaliev
Copy link
Contributor Author

This is the first part of my review. I want to make sure I understand the intentions before completing it.

Tell me if I have the main idea of what this is solving. Previously, names referenced by globally defined function literals were not being verified to have been defined. These changes use this unique case as an entrypoint to recursively analyze all of the function literal's statements to ensure that all names referenced are actually defined.

Otherwise, please see the couple of comments I've left.

Yes, that is correct.

ltzmaxwell added a commit that referenced this pull request Oct 22, 2024
# Problem Definition

The problem originates from the issue described in
[#1135](#1135). While the full
scope of the issue is broader, it fundamentally relates to the concept
of loop variable escapes block where it's defined.

e.g. 1:
```go
package main

import "fmt"

var s1 []*int

func forLoopRef() {
	defer func() {
		for i, e := range s1 {
			fmt.Printf("s1[%d] is: %d\n", i, *e)
		}
	}()

	for i := 0; i < 3; i++ {
		z := i + 1
		s1 = append(s1, &z)
	}
}

func main() {
	forLoopRef()
}
```

e.g. 2:
```go
package main

type f func()

var fs []f

func forLoopClosure() {
	defer func() {
		for _, f := range fs {
			f()
		}
	}()

	for i := 0; i < 3; i++ {
		z := i
		fs = append(fs, func() { println(z) })
	}
}

func main() {
	forLoopClosure()
}
```

e.g. 3:
```go
package main

func main() {
	c := 0
	closures := []func(){}
loop:
	i := c
	closures = append(closures, func() {
		println(i)
	})
	c += 1
	if c < 10 {
		goto loop
	}

	for _, cl := range closures {
		cl()
	}
}
```



# Solution ideas

- **identify escaped vars in preprocess**:
Detect situations where a loop variable is defined within a loop
block(including `for/range` loops or loops constructed using `goto`
statements), and escapes the block where it's defined.

- **runtime allocation**:
Allocate a new heap item for the loop variable in each iteration to
ensure each iteration operates with its unique variable instance.

- **NOTE1**: this is consistent with Go's Strategy:
"Each iteration has its own separate declared variable (or variables)
[Go 1.22]. The variable used by the first iteration is declared by the
init statement. The variable used by each subsequent iteration is
declared implicitly before executing the post statement and initialized
to the value of the previous iteration's variable at that moment."

- **NOTE2**: the `loopvar` feature of Go 1.22 is not supported in this
version, and will be supported in next version.

    not supporting capture `i` defined in for/range clause;
```go
	for i := 0; i < 3; i++ {
		s1 = append(s1, &i)
	}
```

# Implementation Details

**Preprocess Stage(Multi-Phase Preprocessor)**:

- **Phase 1: `initStaticBlocks`**: Establish a cascading scope structure
where `predefine` is conducted. In this phase Name expressions are
initially marked as `NameExprTypeDefine`, which may later be upgraded to
`NameExprTypeHeapDefine` if it is determined that they escape the loop
block. This phase also supports other processes as a
prerequisite[#2077](#2077).
   
- **Phase 2: `preprocess1`**: This represents the original preprocessing
phase(not going into details).
   
- **Phase 3: `findGotoLoopDefines`**: By traversing the AST, any name
expression defined in a loop block (for/range, goto) with the attribute
`NameExprTypeDefine` is promoted to `NameExprTypeHeapDefine`. This is
used in later phase.
   
- **Phase 4: `findLoopUses1`**: Identify the usage of
`NameExprTypeHeapDefine` name expressions. If a name expression is used
in a function literal or is referrnced(e.g. &a), and it was previously
defined as `NameExprTypeHeapDefine`, the `used` name expression is then
given the attribute `NameExprTypeHeapUse`. This step finalizes whether a
name expression will be allocated on the heap and used from heap.
`Closures` represent a particular scenario in this context. Each
closure, defined by a funcLitExpr that captures variables, is associated
with a HeapCaptures list. This list consists of NameExprs, which are
utilized at runtime to obtain the actual variable values for each
iteration. Correspondingly, within the funcLitExpr block, a list of
placeholder values are defined. These placeholders are populated during
the doOpFuncLit phase and subsequently utilized in the `doOpCall` to
ensure that each iteration uses the correct data.


- **Phase 5: `findLoopUses2`**: Convert non-loop uses of loop-defined
names to `NameExprTypeHeapUse`. Also, demote `NameExprTypeHeapDefine`
back to `NameExprTypeDefine` if no actual usage is found. Also , as the
last phase, attributes no longer needed will be cleaned up after this
phase.

**Runtime Stage**:

1. **Variable Allocation**:
- Modify the runtime so that encountering a `NameExprTypeHeapDefine`
triggers the allocation of a new `heapItemValue` for it, which will be
used by any `NameExprTypeHeapUse`.

2. **Function Literal Handling**:
- During the execution of `doOpFuncLit`, retrieve the `HeapCapture`
values (previously allocated heap item values) and fill in the
placeholder values within the `funcLitExpr` block.
- When invoking the function (`doOpCall`), the `placeHolder`
values(fv.Captures) are used to update the execution context, ensuring
accurate and consistent results across iterations.

---------

Co-authored-by: ltzMaxwell <ltz.maxwell@gmail.com>
Co-authored-by: Morgan <morgan@morganbaz.com>
@petar-dambovaliev petar-dambovaliev added the in focus Core team is prioritizing this work label Nov 22, 2024
@Kouteki Kouteki requested review from zivkovicmilos, deelawn and aeddi and removed request for zivkovicmilos and deelawn November 25, 2024 02:09
@Kouteki
Copy link
Contributor

Kouteki commented Nov 25, 2024

Added @aeddi to work in parallel on test coverage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in focus Core team is prioritizing this work 📦 🌐 tendermint v2 Issues or PRs tm2 related 📦 ⛰️ gno.land Issues or PRs gno.land package related 📦 🤖 gnovm Issues or PRs gnovm related 🧾 package/realm Tag used for new Realms or Packages.
Projects
Status: In Progress
Status: No status
Status: In Review
Development

Successfully merging this pull request may close these issues.

in variable initialization, dependencies within functions are not recursively resolved
9 participants