Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Function instrumentation and effect handlers transformation #257

Merged
merged 104 commits into from
Jan 22, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
9e2c4cd
Changes IR
AnsonYeung Dec 8, 2024
e7813a9
initial changes
CAG2Mark Dec 10, 2024
4444d4e
Update elaborator
CAG2Mark Dec 10, 2024
4f26189
update lowering
CAG2Mark Dec 10, 2024
942922d
fix whitespace
CAG2Mark Dec 10, 2024
827c5e7
refactor some code
CAG2Mark Dec 10, 2024
eef7131
Simplify test cases
AnsonYeung Dec 12, 2024
5b26719
Hide handlers behind a flag
AnsonYeung Dec 12, 2024
421b640
Implement transformation for non-resumptive effects
AnsonYeung Dec 16, 2024
76695ba
Continuation class transformation
AnsonYeung Dec 16, 2024
2263ddf
Port tests to new branch
AnsonYeung Dec 16, 2024
bdbee0a
Resume non tail resumptive handlers
AnsonYeung Dec 16, 2024
e9bab2b
Implement final pieces for nested handlers
AnsonYeung Dec 17, 2024
4bb2d73
Implement Match codegen for multiple arms
AnsonYeung Dec 17, 2024
7395b4c
Moved most wiring code to Predef
AnsonYeung Dec 17, 2024
5ef77f2
Changes to IR
AnsonYeung Dec 19, 2024
018dedd
Changes IR
AnsonYeung Dec 19, 2024
dac193e
Patch for JSBuilder
AnsonYeung Dec 19, 2024
68be072
Fix link error
AnsonYeung Dec 19, 2024
ab69c37
Merge branch 'hkmc2' into ir-handler
AnsonYeung Dec 19, 2024
6a71972
Changes to handler syntax
AnsonYeung Dec 19, 2024
ea621a9
Add class inheritance in IR
AnsonYeung Dec 20, 2024
d07371e
Add class inheritance in IR and fixed spacing
AnsonYeung Dec 20, 2024
6b74a07
Merge branch 'hkmc2' into ir-handler
AnsonYeung Dec 20, 2024
be15486
Remove unnecessary newline in JS Codegen
AnsonYeung Dec 20, 2024
db5aa6d
Random typo fix
AnsonYeung Dec 20, 2024
2b67aba
Merge branch 'ir-handler' into ir-handler-transform
AnsonYeung Dec 20, 2024
ef1eabc
Small fix
AnsonYeung Dec 20, 2024
61d2a27
Add mildly interesting test
LPTK Dec 21, 2024
564f020
Add more test cases and reveal some issues
LPTK Dec 21, 2024
9bc81df
Merge branch 'hkmc2' into ir-handler-transform
AnsonYeung Dec 21, 2024
4905f8a
Use subclass for effect and separate internal class
AnsonYeung Dec 24, 2024
c66c73d
Change how elaboration of handlers work
AnsonYeung Dec 24, 2024
f876483
add test
CAG2Mark Dec 24, 2024
879eece
Fixed one example
AnsonYeung Dec 25, 2024
f4e728e
add minimal example of effects in handler bug
CAG2Mark Dec 25, 2024
643ec6f
Fixed another test
AnsonYeung Dec 25, 2024
dc516cd
Fix missing thisProxy
AnsonYeung Dec 25, 2024
e6cd1ec
Fix generator
AnsonYeung Dec 25, 2024
8a6d15c
Fix typo
AnsonYeung Dec 25, 2024
2e07058
Fix function calls and clases sometimes not being in scope after tran…
CAG2Mark Dec 27, 2024
6339e8b
Update hkmc2/shared/src/test/mlscript/handlers/RecursiveHandlers.mls
CAG2Mark Dec 27, 2024
a2900ec
Update hkmc2/shared/src/main/scala/hkmc2/codegen/HandlerLowering.scala
CAG2Mark Dec 27, 2024
4b837eb
Changed and added broken test
AnsonYeung Jan 6, 2025
0864f27
Update tests
AnsonYeung Jan 6, 2025
e6a819b
Minor style changes (? debatable improvement)
LPTK Jan 6, 2025
af17307
Fix elaboration scoping
AnsonYeung Jan 7, 2025
08a4d4c
Merge branch 'hkmc2' into ir-handler-transform
AnsonYeung Jan 7, 2025
2da74c0
Fixed some bugs
AnsonYeung Jan 7, 2025
80e9fba
Fix consistency
AnsonYeung Jan 7, 2025
964c146
Fixed comments
AnsonYeung Jan 7, 2025
b8c6120
move helper function outside
CAG2Mark Jan 7, 2025
2393ea5
Remove usage of dummy class symbol
AnsonYeung Jan 8, 2025
810a639
Remove some unused code
AnsonYeung Jan 10, 2025
74fb276
Merge branch 'hkmc2' into ir-handler-transform
AnsonYeung Jan 10, 2025
b04031a
revert parser workaround
AnsonYeung Jan 10, 2025
e8113c7
predef clean up
AnsonYeung Jan 10, 2025
78f6f54
Simplify Predef
AnsonYeung Jan 10, 2025
d3fedd2
Rewriting most of Predef for clean up
AnsonYeung Jan 10, 2025
b30d8ca
Change test case to todo
AnsonYeung Jan 10, 2025
c9057ea
remove unused code
AnsonYeung Jan 11, 2025
4b73be5
Merge branch 'hkmc2' into ir-handler-transform
AnsonYeung Jan 13, 2025
235a6ec
Added codegen examples
AnsonYeung Jan 13, 2025
ed895eb
use SymbolSubst
CAG2Mark Jan 13, 2025
be37739
Update tests
CAG2Mark Jan 13, 2025
b8d0da8
fix whitespace
CAG2Mark Jan 13, 2025
7da8101
Fix symbol map issues
CAG2Mark Jan 13, 2025
21843b4
Fix overriding hack
LPTK Jan 14, 2025
862bd6a
Address PR comments
CAG2Mark Jan 14, 2025
ac70aea
Address PR comments
CAG2Mark Jan 14, 2025
051bc09
Fix inaccurate inheritance clause.
LPTK Jan 14, 2025
e4f4922
style change
AnsonYeung Jan 14, 2025
b89eb90
revert whitespace change
CAG2Mark Jan 14, 2025
b22ac0b
Merge branch 'ir-handler-transform' of github.com:CAG2Mark/mlscript i…
CAG2Mark Jan 14, 2025
05a8623
Merge branch 'hkmc2' into ir-handler-transform
AnsonYeung Jan 15, 2025
2b3e83e
Eliminate many tail calls to Lowering#term to reduce stack usage
LPTK Jan 16, 2025
e901993
Make Handle a statement, as it should be
LPTK Jan 16, 2025
c7ca7da
Merge branch 'hkmc2' into ir-handler-transform
LPTK Jan 16, 2025
de72285
fix merge
CAG2Mark Jan 16, 2025
444e9b0
update tests
CAG2Mark Jan 16, 2025
83eda13
Address PR comments
CAG2Mark Jan 16, 2025
267c57c
rename
CAG2Mark Jan 16, 2025
8d3ef9f
fix bad comment
CAG2Mark Jan 16, 2025
c8dad1e
move helpers to Block
CAG2Mark Jan 16, 2025
0e19ba1
move extensions to class
CAG2Mark Jan 16, 2025
2107ea8
Tiny JS fix
AnsonYeung Jan 16, 2025
2241bb8
Add some tests, including problematic ones that need some fixes
LPTK Jan 17, 2025
d56a882
Fix for effectful constructor
AnsonYeung Jan 17, 2025
1d54ec8
Style changes
AnsonYeung Jan 17, 2025
0e285a1
Add binding test for non-resumptive handler
AnsonYeung Jan 17, 2025
28a29bc
Small codegen improvements
AnsonYeung Jan 17, 2025
a42e2d3
Change test
AnsonYeung Jan 17, 2025
ad2dd67
Implement BlockTransformer
AnsonYeung Jan 19, 2025
f9f8a89
implement BlockTransformer
AnsonYeung Jan 21, 2025
86a9b38
Fixed typo and translation of first pass
AnsonYeung Jan 21, 2025
68ba0f0
Add missing case
AnsonYeung Jan 21, 2025
8b7f167
refactor to use BlockTransformer
CAG2Mark Jan 21, 2025
cba1195
Fix formating
AnsonYeung Jan 21, 2025
95d7a25
Merge branch 'hkmc2' into ir-handler-transform
AnsonYeung Jan 21, 2025
10d6ebf
Fix error
AnsonYeung Jan 21, 2025
80ba502
check instantiate for effects
CAG2Mark Jan 21, 2025
938dd24
merge
CAG2Mark Jan 21, 2025
438bb33
merge from hkmc2
CAG2Mark Jan 21, 2025
1a01a7e
FIx
AnsonYeung Jan 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion core/shared/main/scala/utils/package.scala
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ package object utils {
def TODO(msg: Any): Nothing = throw new NotImplementedError(
msg.toString + s" (of class ${msg.getClass().getSimpleName()})")
def TODO(msg: Any, cond: Bool): Unit = if (cond) TODO(msg)
def die: Nothing = lastWords("Program reached and unexpected state.")
def die: Nothing = lastWords("Program reached an unexpected state.")
def lastWords(msg: String): Nothing = throw new Exception(s"Internal Error: $msg")
def wat(msg: String, wat: Any): Nothing = lastWords(s"$msg ($wat)")

Expand Down
2 changes: 1 addition & 1 deletion hkmc2/shared/src/main/scala/hkmc2/MLsCompiler.scala
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ class MLsCompiler(preludeFile: os.Path):

val (blk, newCtx) = elab.importFrom(mainParse.resultBlk)
val low = ltl.givenIn:
codegen.Lowering()
codegen.Lowering(lowerHandlers = false)
val jsb = codegen.js.JSBuilder()
val le = low.program(blk)
val baseScp: utils.Scope =
Expand Down
102 changes: 92 additions & 10 deletions hkmc2/shared/src/main/scala/hkmc2/codegen/Block.scala
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import hkmc2.semantics.{Term => st}
import syntax.{Literal, Tree}
import semantics.*
import semantics.Term.*
import sem.Elaborator.State


case class Program(
Expand All @@ -36,13 +37,13 @@ sealed abstract class Block extends Product with AutoLocated:
case Break(_) => Set.empty
case Continue(_) => Set.empty
case Define(defn, rst) => rst.definedVars
case HandleBlock(lhs, res, cls, hdr, bod, rst) => bod.definedVars ++ rst.definedVars + lhs
case HandleBlock(lhs, res, par, cls, hdr, bod, rst) => bod.definedVars ++ rst.definedVars + lhs
case HandleBlockReturn(_) => Set.empty
case TryBlock(sub, fin, rst) => sub.definedVars ++ fin.definedVars ++ rst.definedVars
case Label(lbl, bod, rst) => bod.definedVars ++ rst.definedVars

lazy val size: Int = this match
case _: Return | _: Throw | _: End | _: Break | _: Continue => 1
case _: Return | _: Throw | _: End | _: Break | _: Continue | _: HandleBlockReturn => 1
case Begin(sub, rst) => sub.size + rst.size
case Assign(_, _, rst) => 1 + rst.size
case AssignField(_, _, _, rst) => 1 + rst.size
Expand All @@ -51,15 +52,16 @@ sealed abstract class Block extends Product with AutoLocated:
case Define(_, rst) => 1 + rst.size
case TryBlock(sub, fin, rst) => 1 + sub.size + fin.size + rst.size
case Label(_, bod, rst) => 1 + bod.size + rst.size
case HandleBlock(lhs, res, par, cls, handlers, bdy, rst) => 1 + handlers.map(_.body.size).sum + bdy.size + rst.size

// TODO conserve if no changes
def mapTail(f: BlockTail => BlockTail): Block = this match
case b: BlockTail => f(b)
case Begin(sub, rst) => Begin(sub, rst.mapTail(f))
case Assign(lhs, rhs, rst) => Assign(lhs, rhs, rst.mapTail(f))
case Define(defn, rst) => Define(defn, rst.mapTail(f))
case HandleBlock(lhs, res, cls, handlers, body, rest) =>
HandleBlock(lhs, res, cls, handlers.map(h => Handler(h.sym, h.resumeSym, h.params, h.body.mapTail(f))), body.mapTail(f), rest.mapTail(f))
case HandleBlock(lhs, res, par, cls, handlers, body, rest) =>
HandleBlock(lhs, res, par, cls, handlers.map(h => Handler(h.sym, h.resumeSym, h.params, h.body)), body, rest.mapTail(f))
case Match(scrut, arms, dflt, rst: End) =>
Match(scrut, arms.map(_ -> _.mapTail(f)), dflt.map(_.mapTail(f)), rst)
case Match(scrut, arms, dflt, rst) =>
Expand All @@ -80,11 +82,68 @@ sealed abstract class Block extends Product with AutoLocated:
case Assign(lhs, rhs, rest) => Set(lhs) ++ rhs.freeVars ++ rest.freeVars
case AssignField(lhs, nme, rhs, rest) => lhs.freeVars ++ rhs.freeVars ++ rest.freeVars
case Define(defn, rest) => defn.freeVars ++ rest.freeVars
case HandleBlock(lhs, res, cls, handlers, body, rest) =>
(body.freeVars - lhs) ++ rest.freeVars ++ handlers.flatMap(_.freeVars)
case HandleBlock(lhs, res, par, cls, hdr, bod, rst) =>
(bod.freeVars - lhs) ++ rst.freeVars ++ hdr.flatMap(_.freeVars)
case HandleBlockReturn(res) => res.freeVars
case End(msg) => Set.empty

def floatOutDefns =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please document what this function does.

def rec(b: Block, acc: List[Defn]): (Block, List[Defn]) = b match
case Match(scrut, arms, dflt, rest) =>
val (armsRes, armsDefns) = arms.foldLeft[(List[(Case, Block)], List[Defn])](Nil, acc)(
(accc, d) =>
val (accCases, accDefns) = accc
val (cse, blk) = d
val (resBlk, resDefns) = rec(blk, accDefns)
((cse, resBlk) :: accCases, resDefns)
)
dflt match
case None =>
val (rstRes, rstDefns) = rec(rest, armsDefns)
(Match(scrut, armsRes, None, rstRes), rstDefns)

case Some(dflt) =>
val (dfltRes, dfltDefns) = rec(dflt, armsDefns)
val (rstRes, rstDefns) = rec(rest, dfltDefns)
(Match(scrut, armsRes, S(dfltRes), rstRes), rstDefns)

case Return(res, implct) => (b, acc)
case Throw(exc) => (b, acc)
case Label(label, body, rest) =>
val (bodyRes, bodyDefns) = rec(body, acc)
val (rstRes, rstDefns) = rec(rest, bodyDefns)
(Label(label, bodyRes, rstRes), rstDefns)
case Break(label) => (b, acc)
case Continue(label) => (b, acc)
case Begin(sub, rest) =>
val (subRes, subDefns) = rec(sub, acc)
val (rstRes, rstDefns) = rec(rest, subDefns)
(Begin(subRes, rstRes), rstDefns)
case TryBlock(sub, finallyDo, rest) =>
val (subRes, subDefns) = rec(sub, acc)
val (finallyRes, finallyDefns) = rec(rest, subDefns)
val (rstRes, rstDefns) = rec(rest, finallyDefns)
(TryBlock(subRes, finallyRes, rstRes), rstDefns)
case Assign(lhs, rhs, rest) =>
val (rstRes, rstDefns) = rec(rest, acc)
(Assign(lhs, rhs, rstRes), rstDefns)
case a @ AssignField(path, nme, result, rest) =>
val (rstRes, rstDefns) = rec(rest, acc)
(AssignField(path, nme, result, rstRes)(a.symbol), rstDefns)
case Define(defn, rest) => defn match
case ValDefn(owner, k, sym, rhs) =>
val (rstRes, rstDefns) = rec(rest, acc)
(Define(defn, rstRes), rstDefns)
case _ =>
val (rstRes, rstDefns) = rec(rest, defn :: acc)
(rstRes, rstDefns)
case HandleBlock(lhs, res, par, cls, handlers, body, rest) =>
val (rstRes, rstDefns) = rec(rest, acc)
(HandleBlock(lhs, res, par, cls, handlers, body, rstRes), rstDefns)
case HandleBlockReturn(res) => (b, acc)
case End(msg) => (b, acc)
rec(this, Nil)

end Block

sealed abstract class BlockTail extends Block
Expand Down Expand Up @@ -119,7 +178,7 @@ case class AssignField(lhs: Path, nme: Tree.Ident, rhs: Result, rest: Block)(val

case class Define(defn: Defn, rest: Block) extends Block with ProductWithTail

case class HandleBlock(lhs: Local, res: Local, cls: Path, handlers: Ls[Handler], body: Block, rest: Block) extends Block with ProductWithTail
case class HandleBlock(lhs: Local, res: Local, par: Path, cls: ClassSymbol, handlers: Ls[Handler], body: Block, rest: Block) extends Block with ProductWithTail
case class HandleBlockReturn(res: Result) extends BlockTail

sealed abstract class Defn:
Expand Down Expand Up @@ -149,7 +208,7 @@ final case class ValDefn(
final case class ClsLikeDefn(
sym: MemberSymbol[? <: ClassLikeDef],
k: syntax.ClsLikeKind,
parentSym: Opt[Path],
parentPath: Opt[Path],
methods: Ls[FunDefn],
privateFields: Ls[TermSymbol],
publicFields: Ls[TermDefinition],
Expand All @@ -171,7 +230,7 @@ case class End(msg: Str = "") extends BlockTail with ProductWithTail

enum Case:
case Lit(lit: Literal)
case Cls(cls: ClassSymbol | ModuleSymbol, path: Path)
case Cls(cls: ClassLikeSymbol, path: Path)
case Tup(len: Int, inf: Bool)

lazy val freeVars: Set[Local] = this match
Expand All @@ -198,7 +257,9 @@ case class Call(fun: Path, args: Ls[Arg])(val isMlsFun: Bool) extends Result

case class Instantiate(cls: Path, args: Ls[Path]) extends Result

sealed abstract class Path extends Result
sealed abstract class Path extends Result:
def selN(id: Tree.Ident) = Select(this, id)(N)
def asArg = Arg(false, this)

case class Select(qual: Path, name: Tree.Ident)(val symbol: Opt[FieldSymbol]) extends Path with ProductWithExtraInfo:
def extraInfo: Str = symbol.mkString
Expand All @@ -212,3 +273,24 @@ enum Value extends Path:

case class Arg(spread: Bool, value: Path)

extension (k: Block => Block)

def chain(other: Block => Block): Block => Block = b => k(other(b))
def rest(b: Block): Block = k(b)
def transform(f: (Block => Block) => (Block => Block)) = f(k)

def assign(l: Local, r: Result) = k.chain(Assign(l, r, _))
def assignFieldN(lhs: Path, nme: Tree.Ident, rhs: Result) = k.chain(AssignField(lhs, nme, rhs, _)(N))
def break(l: Local): Block = k.rest(Break(l))
def continue(l: Local): Block = k.rest(Continue(l))
def define(defn: Defn) = k.chain(Define(defn, _))
def end = k.rest(End())
def ifthen(scrut: Path, cse: Case, trm: Block): Block => Block = k.chain(Match(scrut, cse -> trm :: Nil, N, _))
def label(label: Local, body: Block) = k.chain(Label(label, body, _))
def ret(r: Result) = k.rest(Return(r, false))
def staticif(b: Boolean, f: (Block => Block) => (Block => Block)) = if b then k.transform(f) else k

def blockBuilder: Block => Block = identity

extension (l: Local)
def asPath: Path = Value.Ref(l)
Loading
Loading