[Merged by Bors] - Try-catch-block control flow fix/refactor #2568

nekevss · 2023-01-27T03:49:27Z

This Pull Request is meant to address #1900. While working on it, there was a decent amount of refactoring/restructuring. Initially, I had kept with the current approach of just keeping track of a kind and counter on the environment stack, especially because that kept the size of each stack entry to a minimum.

I did, however, make a switch to having the opcode create the EnvStackEntry with a start address and exit address for a bit more precision.

It changes the following:

Consolidates loop_env_stack and try_env_stack into one EnvStackEntry struct.
Changes Continue, Break, LoopStart, LoopContinue, FinallyStart, FinallyEnd and various others. Most of this primarily revolves around the creating of EnvStackEntry and interacting with the env_stack.
Changes/updates the try-catch-finally, break and continue statement compilations as necessary
Adds an AbruptCompletionRecord in place of the finally_jump vector.
Adds some tests for try-catch-finally blocks with breaks.

codecov · 2023-01-27T15:09:26Z

Codecov Report

Merging #2568 (2ca88f3) into main (3f9f6f0) will increase coverage by 0.28%.
The diff coverage is 80.96%.

@@            Coverage Diff             @@
##             main    #2568      +/-   ##
==========================================
+ Coverage   48.52%   48.80%   +0.28%     
==========================================
  Files         387      391       +4     
  Lines       38603    38919     +316     
==========================================
+ Hits        18733    18996     +263     
- Misses      19870    19923      +53

Impacted Files	Coverage Δ
boa_engine/src/vm/code_block.rs	`31.00% <ø> (ø)`
boa_engine/src/vm/flowgraph/mod.rs	`0.00% <0.00%> (ø)`
boa_engine/src/vm/opcode/generator/mod.rs	`0.00% <0.00%> (ø)`
boa_engine/src/vm/opcode/iteration/for_await.rs	`0.00% <0.00%> (ø)`
boa_engine/src/vm/opcode/jump/mod.rs	`39.13% <ø> (ø)`
boa_engine/src/vm/opcode/mod.rs	`45.45% <ø> (ø)`
boa_engine/src/bytecompiler/statement/continue.rs	`39.36% <45.45%> (-31.48%)`	⬇️
boa_engine/src/vm/opcode/control_flow/finally.rs	`59.34% <59.34%> (ø)`
boa_engine/src/vm/opcode/return_stm/mod.rs	`66.66% <65.00%> (+59.25%)`	⬆️
boa_engine/src/vm/mod.rs	`58.13% <76.74%> (+9.37%)`	⬆️
... and 30 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

raskad · 2023-01-27T21:38:46Z

Test result	main count	PR count	difference
Total	94,205	94,205	0
Passed	70,686	70,656	-30
Ignored	18,622	18,622	0
Failed	4,897	4,927	+30
Panics	0	0	0
Conformance	75.03%	75.00%	-0.03%

Fixed tests (4):

test/language/statements/for-of/continue-label-from-try.js [strict mode] (previously Failed)
test/language/statements/for-of/continue-label-from-try.js (previously Failed)
test/language/statements/for-of/continue-label-from-catch.js [strict mode] (previously Failed)
test/language/statements/for-of/continue-label-from-catch.js (previously Failed)

Broken tests (34):

test/language/statements/class/subclass/derived-class-return-override-catch-finally.js [strict mode] (previously Passed)
test/language/statements/class/subclass/derived-class-return-override-catch-finally.js (previously Passed)
test/language/statements/class/subclass/derived-class-return-override-catch-finally-arrow.js [strict mode] (previously Passed)
test/language/statements/class/subclass/derived-class-return-override-catch-finally-arrow.js (previously Passed)
test/language/statements/try/completion-values-fn-finally-normal.js [strict mode] (previously Passed)
test/language/statements/try/completion-values-fn-finally-normal.js (previously Passed)
test/language/statements/try/S12.14_A3.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A3.js (previously Passed)
test/language/statements/try/completion-values-fn-finally-return.js [strict mode] (previously Passed)
test/language/statements/try/completion-values-fn-finally-return.js (previously Passed)
test/language/statements/try/S12.14_A7_T3.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A7_T3.js (previously Passed)
test/language/statements/try/S12.14_A13_T3.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A13_T3.js (previously Passed)
test/language/statements/try/S12.14_A7_T2.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A7_T2.js (previously Passed)
test/language/statements/try/S12.14_A11_T2.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A11_T2.js (previously Passed)
test/language/statements/try/S12.14_A9_T5.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A9_T5.js (previously Passed)
test/language/statements/try/completion-values-fn-finally-abrupt.js [strict mode] (previously Passed)
test/language/statements/try/completion-values-fn-finally-abrupt.js (previously Passed)
test/language/statements/try/S12.14_A10_T2.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A10_T2.js (previously Passed)
test/language/statements/try/S12.14_A9_T2.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A9_T2.js (previously Passed)
test/language/statements/try/S12.14_A2.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A2.js (previously Passed)
test/language/statements/try/S12.14_A10_T5.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A10_T5.js (previously Passed)
test/language/statements/try/S12.14_A12_T2.js [strict mode] (previously Passed)
test/language/statements/try/S12.14_A12_T2.js (previously Passed)
test/language/statements/break/S12.8_A8_T2.js [strict mode] (previously Passed)
test/language/statements/break/S12.8_A8_T2.js (previously Passed)

raskad · 2023-01-27T21:41:22Z

@nekevss Are the new broken tests expected? Where they previously false positives?

Btw, you should be able to push your branches to the boa-dev/boa origin instead of your own fork, that way the test results are posted automatically.

nekevss · 2023-01-28T00:12:07Z

Are the new broken tests expected?

Not entirely, I had actually missed some logic on compiling continues that broke some tests. There's probably some of these that could be valid bugs/missed logic, so I can look at those this weekend and test them. There were some of these tests however that I do think may have been false positives (although I don't have any evidence), especially due to some tests specifically testing CompletionRecord values and outputs. I was trying to stop just short of implementing CompletionRecords too along with changing up the approach to handling control-flow as that seemed like a lot of changes for a single PR.

I mostly wanted to get some eyes on this early since it takes a different approach as to how environments are tracked by the Vm at runtime.

Here's the most recent results (Updated to current).

Test result	main count	PR count	difference
Total	94,205	94,205	0
Passed	70,748	70,752	+4
Ignored	18,622	18,622	0
Failed	4,835	4,831	-4
Panics	0	0	0
Conformance	75.10%	75.10%	+0.00%

Fixed tests (6):

test/language/statements/for-of/continue-label-from-finally.js [strict mode] (previously Failed)
test/language/statements/for-of/continue-label-from-finally.js (previously Failed)
test/language/statements/for-of/continue-label-from-try.js [strict mode] (previously Failed)
test/language/statements/for-of/continue-label-from-try.js (previously Failed)
test/language/statements/for-of/continue-label-from-catch.js [strict mode] (previously Failed)
test/language/statements/for-of/continue-label-from-catch.js (previously Failed)

Broken tests (2):

test/language/statements/break/S12.8_A8_T2.js [strict mode] (previously Passed)
test/language/statements/break/S12.8_A8_T2.js (previously Passed)

raskad · 2023-01-31T01:56:18Z

There were some of these tests however that I do think may have been false positives (although I don't have any evidence), especially due to some tests specifically testing CompletionRecord values and outputs. I was trying to stop just short of implementing CompletionRecords too along with changing up the approach to handling control-flow as that seemed like a lot of changes for a single PR.

I did not have time to look at these change in depth, but I have some thoughts on the topic of possible errors and CompletionRecords:

The first thing is that we do not have to implement CompletionRecords as described in the spec. We only have to implement the behaviour that they describe. So we should not limit ourselves to data structures from the spec.

We currently have some bugs related to completions and control flow that are closeley related to break, 'continue' and try/catch. I'm sure you're aware of them, but I think this is a good opportunity to write them down and maybe find a solution. First, some statements like if and do...while currently do not return values like they should:

// boa
eval('if (true) {"a";}'); // undefined
eval('do {"a";} while (false)'); // undefined

// Node
eval('if (true) {"a";}'); // "a"
eval('do {"a";} while (false)'); // "a"

This in itself is not a big problem, but when combined with control flow breaking statements it gets a little tricky:

eval('do {"a"; if (true) { "b"; continue }; "c";} while (false);'); // "b"

In this example we have to somehow make sure that we return either the "b" from inside the if statements body or the "c" if the if statement's condition evaluates to false. I currently have no good solution for this in mind and I think this behaviour is probably good to keep in mind while working on these control flow statements.

Also if there are some false positives that you find while working on this, it would not surprise me if they are related to these bugs @nekevss

nekevss · 2023-01-31T02:53:27Z

@raskad Agreed on not implementing the CompletionRecord directly to spec. And to be frank and in my opinion, I'm not entirely sure it's something we should implement in Rust as is. Maybe, I'm reading the spec wrong, but to me it's laid out like an overloaded Result enum.

I do think that there is a way to implement something akin to CompletionRecords, which sort of ended up becoming the current version of this branch. Although, this is more of a stepping stone, then it is a full implementation.

The approach I thought might work is to first ensure that control flow is stable and predictable. That way we can guarantee when a break occurs that it will end in x place. Once that's done, then we could either store the value in a record on the CallFrame or manage the value on the stack (although I think the latter may be better and is a bit closer to spec at least where abrupt completions are concerned...normal completions should most likely always live on the stack), so that when Break or Continue are called, then the evaluation value is stored with it. From there, it would mostly be making sure that we are adhering to spec evaluations of CompletionRecords.

I think this branch does address some of/if not most of the control flow bugs. ~~Although now there are 4 panics that I have to double check - one of which when I run both tests on a local file, does throw a SyntaxError.~~

raskad

I like the logic changes. It's still hard to understand, but way better than the old logic.

I've got some suggestions, mainly to avoid panics and to make some things more readable.

boa_engine/src/vm/opcode/mod.rs

boa_engine/src/vm/mod.rs

boa_engine/src/vm/opcode/control_flow/break.rs

raskad · 2023-02-02T01:13:27Z

boa_engine/src/bytecompiler/statement/loop.rs

@@ -22,7 +22,7 @@ impl ByteCompiler<'_, '_> {
        configurable_globals: bool,
    ) -> JsResult<()> {
        self.context.push_compile_time_environment(false);
-        let push_env = self.emit_and_track_decl_env();
+        let push_env = self.emit_opcode_with_two_operands(Opcode::PushDeclarativeEnvironment);
        self.push_empty_loop_jump_control();


This did not change in the PR, but I just noticed that pushing the loop control info here, before the initializer, is theoretically wrong, right? It should only be pushed after the Opcode::LoopStart where we currently do set_label and set_start_address.

This was a small change that was added during the JumpControlInfo being added. Set label and start address occur down further from lines 46-51. The initial idea was to make everything read a bit more uniform and not insert pushing the jump info 15+ lines into the compilation. But I think it could be taken or left either way.

boa_engine/src/vm/opcode/iteration/loop_ops.rs

boa_engine/src/vm/opcode/control_flow/labelled.rs

raskad

I think this is a great improvement. We can start working on some of the more tricky control flow issues when this is merged.

jedel1043

Very nice work! Just some suggestions and questions

boa_engine/src/bytecompiler/statement/break.rs

boa_engine/src/vm/call_frame/abrupt_record.rs

boa_engine/src/vm/opcode/control_flow/break.rs

boa_engine/src/tests.rs

jedel1043

Found some whitespace you missed

boa_engine/src/tests.rs

boa_engine/src/vm/call_frame/abrupt_record.rs

jedel1043

Everything looks good! Nice work!

jedel1043 · 2023-02-05T04:11:15Z

bors r+

This Pull Request is meant to address #1900. While working on it, there was a decent amount of refactoring/restructuring. Initially, I had kept with the current approach of just keeping track of a kind and counter on the environment stack, especially because that kept the size of each stack entry to a minimum. I did, however, make a switch to having the opcode create the `EnvStackEntry` with a start address and exit address for a bit more precision. It changes the following: - Consolidates `loop_env_stack` and `try_env_stack` into one `EnvStackEntry` struct. - Changes `Continue`, `Break`, `LoopStart`, `LoopContinue`, `FinallyStart`, `FinallyEnd` and various others. Most of this primarily revolves around the creating of `EnvStackEntry` and interacting with the `env_stack`. - Changes/updates the try-catch-finally, break and continue statement compilations as necessary - Adds an `AbruptCompletionRecord` in place of the `finally_jump` vector. - Adds some tests for try-catch-finally blocks with breaks.

bors · 2023-02-05T04:21:25Z

Pull request successfully merged into main.

Build succeeded:

nekevss added bug Something isn't working vm Issues and PRs related to the Boa Virtual Machine. labels Jan 27, 2023

nekevss force-pushed the control-flow-refactor branch from 1405510 to e9694c6 Compare January 27, 2023 14:57

nekevss force-pushed the control-flow-refactor branch from 7a092ce to b23f8fb Compare January 31, 2023 01:33

nekevss force-pushed the control-flow-refactor branch from ad5d6b6 to 03500d3 Compare February 1, 2023 03:19

nekevss requested review from raskad, jedel1043, HalidOdat, jasonwilliams, RageKnify and Razican February 1, 2023 03:49

raskad reviewed Feb 2, 2023

View reviewed changes

nekevss force-pushed the control-flow-refactor branch from 03500d3 to d6f3f44 Compare February 3, 2023 01:03

nekevss requested a review from raskad February 3, 2023 01:13

nekevss added this to the v0.17.0 milestone Feb 3, 2023

raskad approved these changes Feb 3, 2023

View reviewed changes

jedel1043 requested changes Feb 4, 2023

View reviewed changes

boa_engine/src/bytecompiler/statement/break.rs Outdated Show resolved Hide resolved

boa_engine/src/vm/call_frame/abrupt_record.rs Outdated Show resolved Hide resolved

boa_engine/src/vm/opcode/control_flow/break.rs Show resolved Hide resolved

boa_engine/src/tests.rs Show resolved Hide resolved

nekevss added 7 commits February 4, 2023 19:44

Initial work on fix

912b1fd

Committing changes to work on idea

4a38317

Working draft of refactor complete

d35dba6

More work complete - A lot more tests stable

ca6e206

More work completed on concept

68c2e06

Alot of refactoring

517a317

Committing more work that was completed

162e40e

nekevss added 15 commits February 4, 2023 19:44

Further progress made

4b042bf

Finished work on a working draft version

59088c7

Add some tests, fmt, and clippy

75e7189

Fixed panics and clean up tests

923e114

Remove env tracker from jump_control

98cad36

Doc cleanup

23108f9

Add missed logic and adjustments for changes

0ceddef

More work completed around throw and return with finally chains

8c65cb3

Rework handling throw and small fix for break

18b5a3a

Cargo fmt

84ad6f1

Panic fixes

b4329e6

Some cleanup and remove thrown flag

4611f1d

Review changes

4032c59

Review changes part 2

807ce49

Cargo fmt

75dfdbe

nekevss force-pushed the control-flow-refactor branch from 6aeef16 to 75dfdbe Compare February 5, 2023 00:44

jedel1043 requested changes Feb 5, 2023

View reviewed changes

boa_engine/src/tests.rs Outdated Show resolved Hide resolved

boa_engine/src/tests.rs Outdated Show resolved Hide resolved

adjustments to AbruptCompletionRecord and remove whitespace

617e956

nekevss requested a review from jedel1043 February 5, 2023 01:23

jedel1043 reviewed Feb 5, 2023

View reviewed changes

boa_engine/src/vm/call_frame/abrupt_record.rs Outdated Show resolved Hide resolved

Rename methods on AbruptCompletionRecord

2ca88f3

jedel1043 approved these changes Feb 5, 2023

View reviewed changes

bors bot changed the title ~~Try-catch-block control flow fix/refactor~~ [Merged by Bors] - Try-catch-block control flow fix/refactor Feb 5, 2023

bors bot closed this Feb 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Merged by Bors] - Try-catch-block control flow fix/refactor #2568

[Merged by Bors] - Try-catch-block control flow fix/refactor #2568

nekevss commented Jan 27, 2023

codecov bot commented Jan 27, 2023 •

edited

Loading

raskad commented Jan 27, 2023

raskad commented Jan 27, 2023

nekevss commented Jan 28, 2023 •

edited

Loading

raskad commented Jan 31, 2023 •

edited

Loading

nekevss commented Jan 31, 2023 •

edited

Loading

raskad left a comment

raskad Feb 2, 2023

nekevss Feb 2, 2023

raskad left a comment

jedel1043 left a comment

jedel1043 left a comment

jedel1043 left a comment

jedel1043 commented Feb 5, 2023

bors bot commented Feb 5, 2023

[Merged by Bors] - Try-catch-block control flow fix/refactor #2568

[Merged by Bors] - Try-catch-block control flow fix/refactor #2568

Conversation

nekevss commented Jan 27, 2023

codecov bot commented Jan 27, 2023 • edited Loading

Codecov Report

raskad commented Jan 27, 2023

raskad commented Jan 27, 2023

nekevss commented Jan 28, 2023 • edited Loading

raskad commented Jan 31, 2023 • edited Loading

nekevss commented Jan 31, 2023 • edited Loading

raskad left a comment

Choose a reason for hiding this comment

raskad Feb 2, 2023

Choose a reason for hiding this comment

nekevss Feb 2, 2023

Choose a reason for hiding this comment

raskad left a comment

Choose a reason for hiding this comment

jedel1043 left a comment

Choose a reason for hiding this comment

jedel1043 left a comment

Choose a reason for hiding this comment

jedel1043 left a comment

Choose a reason for hiding this comment

jedel1043 commented Feb 5, 2023

bors bot commented Feb 5, 2023

codecov bot commented Jan 27, 2023 •

edited

Loading

nekevss commented Jan 28, 2023 •

edited

Loading

raskad commented Jan 31, 2023 •

edited

Loading

nekevss commented Jan 31, 2023 •

edited

Loading