feat(hydroflow_plus, stageleft)!: allow cluster self ID to be referenced as a global constant

[shadaj]

This eliminates the need to store cluster.self_id() in a local variable first, instead you can directly reference CLUSTER_SELF_ID.

Stack created with Sapling. Best reviewed with ReviewStack.

• refactor(paxos): generalize quorum logic #1583
• feat(hydroflow_plus)!: allow runtime context to be referenced as a global constant #1575
• -> feat(hydroflow_plus, stageleft)!: allow cluster self ID to be referenced as a global constant #1574

[cloudflare-workers-and-pages]

Deploying hydroflow with    Cloudflare Pages

Latest commit:	a865b44
Status:	✅  Deploy successful!
Preview URL:	https://5e5014c4.hydroflow.pages.dev
Branch Preview URL:	https://pr1574.hydroflow.pages.dev

View logs

[shadaj]

Guide for review. The first big change is to stageleft, where we create a richer version of Quoted that takes a Ctx type parameter. The idea is that Ctx captures some metadata about where the quoted snippet will be spliced, which can be used to augment how free variables are made available inside the quoted snippet.

In Hydroflow+, the context for UDFs is the location for the stream, so this allows us to have a general global CLUSTER_SELF_ID value which when used as a free variable will return the appropriate ClusterId based on the provided context. This preserves the type safety guarantees, but avoids needing to call cluster.self_id(). In fact, this API was previously unsafe, since you could call c1.self_id() and reference that in a UDF on c2, which would result in malformed generated code. Now, because we rely on the context, we know that the spliced free variable will always be well-formed.

[MingweiSamuel]

Is CLUSTER_SELF_ID just a magic name? In the future probably best to namespace it to something, ctx.CLUSTER_SELF_ID, if/when we add other magic names

[shadaj]

Is CLUSTER_SELF_ID just a magic name? In the future probably best to namespace it to something, ctx.CLUSTER_SELF_ID, if/when we add other magic names

Ah no, it's actually defined as a const CLUSTER_SELF_ID in the cluster module!