Passing result to subsequent steps, implement JavaScript runner #70

v9n · 2024-12-12T17:03:27Z

This PR implements:

Data from previous step will be feeded to next step using the name as variable. if name has space or invalid character, it will be replace with _
Switch expr -> javascript. no more expr
Implement javascript runner. We cannot use module yet but all primitive js is workin. js is es6 so we can use async/await and arrow function.

v9n · 2024-12-27T20:25:25Z

examples/example.js

+            method: "POST",
+            body: `JSON.stringify({
+              chat_id:-4609037622,
+              text: \`Node IP is: \${getIpAddress.data.ip}.\nCurrent USDC liquidity rate in RAY unit is \${getReserveUSDC.data.reserves[0].liquidityRate} \`


@chrisli30 This shows how to access previous step otput. Notice getIpAddress is the name of a step. That name will be standarized to a var. if the name is get ip address we will convert to get_ip_address any non valid char is converted to _.

getIpAddress is result of a rest call
getReserveUSDC is result of a graphql call

okay, all node Ids from studio are Ulid, so we just need to make sure all Ulid could be valid variable names, right?

User Input like Get IP Address can only be node names, and is it true that we don’t use node.name for reference?

okay, all node Ids from studio are Ulid, so we just need to make sure all Ulid could be valid variable names, right?

nah not just that. it's hard to find out(not just type out) what variable is what with ulid when you type it out in later step.

User Input like Get IP Address can only be node names, and is it true that we don’t use node.name for reference?

it's true.

We can also force the name to be a valid variable identifier like retool. It's auto convert space to _ (where i got this inspiration) and it prohibit the first char to not be a number

chrisli30

Approved! but I have left a few comments for review.

chrisli30 · 2024-12-29T21:54:29Z

aggregator/aggregator.go

@@ -122,8 +123,8 @@ func NewAggregator(c *config.Config) (*Aggregator, error) {
 		clients, err := clients.BuildAll(chainioConfig, c.EcdsaPrivateKey, c.Logger)
 		if err != nil {
 			c.Logger.Errorf("Cannot create sdk clients", "err", err)
-			panic(err)
-			//return nil, err
+			// EigenLayer has update the contract and we cannot fetch the slasher anymore, we should upgrade the EigenSDK, right now we don't use it so it's ok to ignore this error


okay, this was the culprit of the slashing error, makes sense. 👌

I’d add a // TODO: to the beginning, so we can search all TODOs in codebase later.

chrisli30 · 2024-12-29T21:56:02Z

core/taskengine/vm_runner_customcode_test.go

+	}
+}
+
+// Temp disable until we figured out the event loop


Add a prefix // TODO: ?

chrisli30 · 2024-12-29T21:59:27Z

examples/example.js

+            method: "POST",
+            body: `JSON.stringify({
+              chat_id:-4609037622,
+              text: \`Node IP is: \${getIpAddress.data.ip}.\nCurrent USDC liquidity rate in RAY unit is \${getReserveUSDC.data.reserves[0].liquidityRate} \`


okay, all node Ids from studio are Ulid, so we just need to make sure all Ulid could be valid variable names, right?

User Input like Get IP Address can only be node names, and is it true that we don’t use node.name for reference?

chrisli30 · 2024-12-29T22:00:07Z

operator/operator.go

@@ -297,7 +297,8 @@ func NewOperatorFromConfig(c OperatorConfig) (*Operator, error) {
 	)
 	if err != nil {
 		logger.Error("Cannot create AvsWriter", "err", err)
-		return nil, err
+		// EigenLayer has update the contract and we cannot fetch the slasher anymore, we should upgrade the EigenSDK, right now we don't use it so it's ok to ignore this error


Add prefix // TODO: so it’s easy to find all pending code.

chrisli30

Just saw the description "Data from previous step will be feeded to next step using the name as variable. if name has space or invalid character, it will be replace with _"

I’d avoid manipulating the id of the node cause that’s error-prone. There’s no reason that a node id is not fixed nor deterministic. Could we use ulid as the id of the node, for referencing previous nodes, to avoid and string modification?

We are good to merge this PR and iterate on this previous step identifier function.

v9n · 2024-12-30T00:07:30Z

@chrisli30

Just saw the description "Data from previous step will be feeded to next step using the name as variable. if name has space or invalid character, it will be replace with _"

I’d avoid manipulating the id of the node cause that’s error-prone. There’s no reason that a node id is not fixed nor deterministic.

No, you mis-understood it. We don't manipulating the id or anyhing at all.

This is how you refer to it as the variable.

Could we use ulid as the id of the node, for referencing previous nodes, to avoid and string modification?
no, we couldnot use ulid or uuid. this is the variable name. it cannot start with number for example

It's this

Imagine you had this

[
{ id: '12345abcdef',
  name: 'get price",}
{ id: 78910',
  javascriptcode: ' if (get price.data.value > 0) { a = 2 }'
}
]

let me know how you want to handle that case. we cannot use uuild or ulid because it may start with number .

on top of that, it's really hard to keep track of a variable if you have to use ulid/uuid as variable name. examplle, user has to write this code:

[
{ id: '12345abcdef',
  name: 'get price",}
{ id: 78910',
  javascriptcode: ' if (12345abcdef.data.value > 0) { a = 2 }'
}
]

Take a look at my example https://github.com/AvaProtocol/EigenLayer-AVS/pull/70/files#diff-a0e0283e24c8bc8fe874fde781384a5ce414c126f85aa7e2453b60ec44931388R730-R746 what would you expect an end user use as variable name for the previous step?

v9n · 2024-12-30T00:33:00Z

@chrisli30 please check retool as well. they auto convert the space to _ when you type on front-end. and they disallow number as first character.

disallow starting with number

force using name as a valid identifier name

CleanShot.2024-12-29.at.16.31.21.mp4

so on the front-end studio you can do the same to force that.

v9n changed the base branch from main to rename-things December 12, 2024 17:18

v9n marked this pull request as draft December 12, 2024 18:54

v9n had a problem deploying to Test December 14, 2024 10:48 — with GitHub Actions Failure

v9n added 13 commits December 17, 2024 08:13

delete the stale task storage once status changed

981bfc6

Merge branch 'main' into support-batch-data-fetching-of-tasks-and-execs

56c0694

support fetching data for multiple account/workflow

4090666

Fix #78 to add type to trigger metadata

275799a

implement GetExecution

11a11e0

set cursor to blank if no more item

6b25303

support filter smart wallet by factory address

c5a0f36

rename TriggerMark -> TriggerMetadata

e7f17be

rename more files

d21ec36

implement accessing to previous node data

ad81242

migrate trigger from expr -> goja

76c2bcd

add test for trigger with contract read func binding

53dd37d

implement javascript runner with goja

0a2ab8a

v9n force-pushed the passing-result-to-subsequent-steps branch from aa80fe1 to 0a2ab8a Compare December 27, 2024 19:28

v9n temporarily deployed to Test December 27, 2024 19:28 — with GitHub Actions Inactive

v9n changed the base branch from rename-things to multiple-fix-and-improvement December 27, 2024 19:28

adding more test for js runner

d995c9a

v9n had a problem deploying to Test December 27, 2024 19:32 — with GitHub Actions Failure

v9n temporarily deployed to Test December 27, 2024 20:14 — with GitHub Actions Inactive

v9n changed the title ~~Passing result to subsequent steps~~ Passing result to subsequent steps, implement JavaScript runner Dec 27, 2024

v9n marked this pull request as ready for review December 27, 2024 20:17

v9n temporarily deployed to Test December 27, 2024 20:20 — with GitHub Actions Inactive

v9n added 3 commits December 27, 2024 12:22

temp workaround to disable slashing on testnet

b31ab79

disable wip test

ed4aa3e

standarize name

ef7bf2c

v9n force-pushed the passing-result-to-subsequent-steps branch from fcaaee2 to ef7bf2c Compare December 27, 2024 20:22

v9n temporarily deployed to Test December 27, 2024 20:22 — with GitHub Actions Inactive

v9n self-assigned this Dec 27, 2024

v9n commented Dec 27, 2024

View reviewed changes

chrisli30 approved these changes Dec 29, 2024

View reviewed changes

chrisli30 requested changes Dec 29, 2024

View reviewed changes

chrisli30 approved these changes Dec 29, 2024

View reviewed changes

Base automatically changed from multiple-fix-and-improvement to main December 30, 2024 12:55

Merge branch 'main' into passing-result-to-subsequent-steps

239a9ab

v9n temporarily deployed to Test December 30, 2024 12:56 — with GitHub Actions Inactive

v9n added 2 commits December 30, 2024 04:58

cleanup

b02c212

add todo

17c354e

v9n had a problem deploying to Test December 30, 2024 12:59 — with GitHub Actions Failure

cleanup

3449d77

v9n temporarily deployed to Test December 30, 2024 13:21 — with GitHub Actions Inactive

v9n merged commit 3b8201c into main Dec 30, 2024
4 checks passed

v9n deleted the passing-result-to-subsequent-steps branch December 30, 2024 20:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Passing result to subsequent steps, implement JavaScript runner #70

Passing result to subsequent steps, implement JavaScript runner #70

v9n commented Dec 12, 2024 •

edited

Loading

v9n Dec 27, 2024

chrisli30 Dec 29, 2024

v9n Dec 30, 2024

chrisli30 left a comment

chrisli30 Dec 29, 2024

chrisli30 Dec 29, 2024

chrisli30 Dec 29, 2024

chrisli30 Dec 29, 2024

chrisli30 Dec 29, 2024

chrisli30 left a comment •

edited

Loading

v9n commented Dec 30, 2024 •

edited

Loading

v9n commented Dec 30, 2024 •

edited

Loading

Passing result to subsequent steps, implement JavaScript runner #70

Passing result to subsequent steps, implement JavaScript runner #70

Conversation

v9n commented Dec 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chrisli30 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chrisli30 left a comment • edited Loading

Choose a reason for hiding this comment

v9n commented Dec 30, 2024 • edited Loading

v9n commented Dec 30, 2024 • edited Loading

v9n commented Dec 12, 2024 •

edited

Loading

chrisli30 left a comment •

edited

Loading

v9n commented Dec 30, 2024 •

edited

Loading

v9n commented Dec 30, 2024 •

edited

Loading