Gateway Node proxies user requests to Vault Node #18420

DeividasK · 2025-07-01T15:27:46Z

This PR introduces a new VaultHandler implementation and updates to error handling.

TODOs

Propagate new handler error
Cleanup expired pending requests
Switch to using beholder over prometheus

core/services/gateway/handlers/vault/service.go

system-tests/tests/smoke/cre/vault_test.go

…way-vault-flow-with-mocks

cedric-cordenier · 2025-07-09T16:46:38Z

core/services/gateway/handlers/vault/handler.go

+		cfg.RequestTimeoutSec = 30
+	}
+
+	userRateLimiter, _ := ratelimit.NewRateLimiter(cfg.UserRateLimiterConfig)


What's the second value that is being omitted here? (Not an error I hope 😬 ?)

It is error. we should handle it

cedric-cordenier · 2025-07-09T16:51:04Z

core/services/gateway/handlers/vault/handler.go

+	ErrNotAllowlisted = errors.New("sender not allowlisted")
+	ErrRateLimited    = errors.New("rate-limited")
+
+	promRequestInternalError = promauto.NewCounterVec(prometheus.CounterOpts{


We should use beholder rather than prometheus :)

cedric-cordenier · 2025-07-09T16:51:46Z

core/services/gateway/handlers/vault/handler.go

+
+	userRateLimiter   *ratelimit.RateLimiter
+	nodeRateLimiter   *ratelimit.RateLimiter
+	requestTimeoutSec int


Why not use a time.Duration here directly?

I don't see the timeout being used or enforced in our logic.
The gateway has a separate timeout for user request anyway.
If we want to use this timeout to just keep our pendingRequests queue bounded, we will need to implement a cleanup on timeout.
Currently the timeout is just used on a context when calling the don.SendToNode() method, but that doesn't affect our queue here.

Updated the timeout to affect the queue instead of just the don.SendToNode. This also effectively prunes the queue.

github-actions · 2025-07-10T11:41:35Z

Flakeguard Summary

Ran new or updated tests between develop and 50b9b60 (PRIV-96-e-2-e-gateway-vault-flow-with-mocks).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

2 Results

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestVaultHandler_HandleJSONRPCUserMessage	0%	true	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/v2/core/services/gateway/handlers/vault	true	0s	Unknown
TestVaultHandler_HandleJSONRPCUserMessage/happy_path	0%	true	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/v2/core/services/gateway/handlers/vault	true	0s	Unknown

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

…way-vault-flow-with-mocks

core/services/gateway/handlers/vault/handler.go

cl-sonarqube-production · 2025-07-10T16:58:10Z

Quality Gate passed

Issues
0 New issues
1 Fixed issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

cedric-cordenier · 2025-07-11T07:44:49Z

core/services/gateway/handlers/vault/handler.go

+func (h *handler) Start(ctx context.Context) error {
+	return h.StartOnce("VaultHandler", func() error {
+		h.lggr.Info("starting vault handler")
+		ctx, _ := h.stopCh.NewCtx()


You should call the cancel func by deferring it inside the go func on line 145

(And maybe calling <-ctx.Done() on line 151 for consistency

I'll address this in a follow up PR

DeividasK added 2 commits July 1, 2025 18:27

--wip-- [skip CI]

8ec4217

E2E flow with the request going to Vault and Vault sending a response

a1d6222

prashantkumar1982 reviewed Jul 4, 2025

View reviewed changes

DeividasK added 3 commits July 4, 2025 17:40

Remove comments

05cca45

Merge branch 'develop' into PRIV-96-e-2-e-gateway-vault-flow-with-mocks

741d52b

Fix running tests in CI + changeset

c80c729

DeividasK changed the title ~~Gateway Node => Vault Node implementation~~ Gateway Node proxy user requests to Vault Node Jul 8, 2025

DeividasK changed the title ~~Gateway Node proxy user requests to Vault Node~~ Gateway Node proxies user requests to Vault Node Jul 8, 2025

DeividasK force-pushed the PRIV-96-e-2-e-gateway-vault-flow-with-mocks branch from 9b22a5d to c80c729 Compare July 8, 2025 15:31

DeividasK added 10 commits July 8, 2025 18:32

Merge branch 'develop' into PRIV-96-e-2-e-gateway-vault-flow-with-mocks

41c120a

Fix prom counters

5a8f8dd

Fix vault_test lint errors

d7ced17

More lint fixes

e7fbd06

More cleanup

84e883c

Additional lint fixes

93fcb16

Merge branch 'develop' into PRIV-96-e-2-e-gateway-vault-flow-with-mocks

7675d17

Further refactor to HandleGatewayMessage

e45197c

Merge remote-tracking branch 'origin/develop' into PRIV-96-e-2-e-gate…

69614ac

…way-vault-flow-with-mocks

Update Vault service

0a84555

DeividasK force-pushed the PRIV-96-e-2-e-gateway-vault-flow-with-mocks branch from 100cde6 to 188b76b Compare July 9, 2025 16:21

Refactoring + tests

b4437b5

DeividasK force-pushed the PRIV-96-e-2-e-gateway-vault-flow-with-mocks branch from 188b76b to b4437b5 Compare July 9, 2025 16:28

DeividasK marked this pull request as ready for review July 9, 2025 16:39

DeividasK requested review from a team as code owners July 9, 2025 16:39

DeividasK requested a review from cedric-cordenier July 9, 2025 16:39

cedric-cordenier reviewed Jul 9, 2025

View reviewed changes

DeividasK force-pushed the PRIV-96-e-2-e-gateway-vault-flow-with-mocks branch from e0eabae to 50b9b60 Compare July 10, 2025 11:28

DeividasK added 5 commits July 10, 2025 15:12

Remove trailing comma

e85d634

Merge remote-tracking branch 'origin/develop' into PRIV-96-e-2-e-gate…

7c7346c

…way-vault-flow-with-mocks

Fix tests

4af49b9

Fix lint issues

7a880f0

Use Beholder for metrics

61292e7

prashantkumar1982 previously approved these changes Jul 10, 2025

View reviewed changes

core/services/gateway/handlers/vault/handler.go Outdated Show resolved Hide resolved

Remove a leftover mutex

d027748

DeividasK dismissed prashantkumar1982’s stale review via d027748 July 10, 2025 16:41

prashantkumar1982 approved these changes Jul 10, 2025

View reviewed changes

cedric-cordenier approved these changes Jul 11, 2025

View reviewed changes

cedric-cordenier reviewed Jul 11, 2025

View reviewed changes

cedric-cordenier approved these changes Jul 11, 2025

View reviewed changes

DeividasK added this pull request to the merge queue Jul 11, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 11, 2025

cedric-cordenier added this pull request to the merge queue Jul 11, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 11, 2025

DeividasK added this pull request to the merge queue Jul 11, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 11, 2025

DeividasK added this pull request to the merge queue Jul 11, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 11, 2025

secure-michelemin merged commit 10d02c8 into develop Jul 11, 2025
158 checks passed

secure-michelemin deleted the PRIV-96-e-2-e-gateway-vault-flow-with-mocks branch July 11, 2025 15:34

This was referenced Jul 11, 2025

[DO NOT MERGE] Changeset Release Preview - v2.33.0 #13148

Draft

[DO NOT MERGE] Changeset Release Preview - v2.33.0 mikeyhodl/chainlink#830

Draft

[DO NOT MERGE] Changeset Release Preview - v2.33.0 MarcelRaschke/chainlink#6

Draft

github-actions bot mentioned this pull request Jul 25, 2025

[DO NOT MERGE] Changeset Release Preview - v2.29.0 JSTONE1111/chainlink#1

Merged

github-actions bot mentioned this pull request Aug 21, 2025

[DO NOT MERGE] Changeset Release Preview - v2.27.0 philipjonsen/chainlink#2

Draft

Gateway Node proxies user requests to Vault Node #18420

Gateway Node proxies user requests to Vault Node #18420

Uh oh!

Conversation

DeividasK commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODOs

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cedric-cordenier Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 10, 2025

Flakeguard Summary

Found Flaky Tests ❌

Artifacts

Uh oh!

Uh oh!

cl-sonarqube-production bot commented Jul 10, 2025

Quality Gate passed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

DeividasK commented Jul 1, 2025 •

edited

Loading

cedric-cordenier Jul 9, 2025 •

edited

Loading