Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(config)_: rpc providers configuration #6151

Merged
merged 5 commits into from
Jan 10, 2025

Conversation

friofry
Copy link
Contributor

@friofry friofry commented Dec 1, 2024

Purpose to add some explicitness for:

  • provider order
  • provider type (user/embedded)
  • authentication type
  • Updating auth parameters for embedded providers (mostly covered by the 2nd PR)
  • RPS limiter enabling for some providers
  • Enable providers (probably needed if user doesn't want to use status' endpoints)
  • toggling on/off user providers
	DefaultRPCURL       string `json:"defaultRpcUrl" validate:"omitempty,url"`       // Deprecated: proxy rpc url
	DefaultFallbackURL  string `json:"defaultFallbackURL" validate:"omitempty,url"`  // Deprecated: proxy fallback url
	DefaultFallbackURL2 string `json:"defaultFallbackURL2" validate:"omitempty,url"` // Deprecated: second proxy fallback url
	RPCURL              string `json:"rpcUrl" validate:"omitempty,url"`              // Deprecated: direct rpc url
	OriginalRPCURL      string `json:"originalRpcUrl" validate:"omitempty,url"`      // Deprecated: direct rpc url if user overrides RPCURL
	FallbackURL         string `json:"fallbackURL" validate:"omitempty,url"`         // Deprecated
	OriginalFallbackURL string `json:"originalFallbackURL" validate:"omitempty,url"` // Deprecated

->

        RpcProviders []RpcProvider `json:"rpcProviders" validate:"dive,required"` // List of RPC providers, in the order in which they are accessed

This PR extracts rpc_provider_persistence

  • Add rpc_providers table, migration
  • add RpcProvider type + validation
  • deprecate old rpc fields in networks, add RpcProviders list
  • add persistence packages for rpc_providers, networks
  • Tests

This PR does not change any existing functionality. The next PR will include integration of the new data structure: #6178

Closes #5997

@status-im-auto
Copy link
Member

status-im-auto commented Dec 1, 2024

Jenkins Builds

Click to see older builds (88)
Commit #️⃣ Finished (UTC) Duration Platform Result
✖️ b75b86c #1 2024-12-01 11:12:36 ~1 min tests 📄log
✖️ b75b86c #1 2024-12-01 11:15:32 ~4 min tests-rpc 📄log
✔️ b75b86c #1 2024-12-01 11:15:44 ~4 min windows 📦zip
✔️ b75b86c #1 2024-12-01 11:16:45 ~5 min macos 📦zip
✔️ b75b86c #1 2024-12-01 11:16:58 ~5 min ios 📦zip
✔️ b75b86c #1 2024-12-01 11:18:07 ~7 min linux 📦zip
✔️ b75b86c #1 2024-12-01 11:18:32 ~7 min android 📦aar
✔️ b75b86c #1 2024-12-01 11:19:15 ~8 min macos 📦zip
✖️ 8195c81 #2 2024-12-05 10:20:47 ~1 min tests 📄log
✔️ 8195c81 #2 2024-12-05 10:23:36 ~4 min windows 📦zip
✔️ 8195c81 #2 2024-12-05 10:25:00 ~5 min linux 📦zip
✔️ 8195c81 #2 2024-12-05 10:25:14 ~6 min ios 📦zip
✖️ 8195c81 #2 2024-12-05 10:25:16 ~6 min tests-rpc 📄log
✔️ 8195c81 #2 2024-12-05 10:25:32 ~6 min android 📦aar
✔️ 8195c81 #2 2024-12-05 10:26:43 ~7 min macos 📦zip
✔️ 8195c81 #2 2024-12-05 10:26:52 ~7 min macos 📦zip
✖️ 099ebdf #3 2024-12-05 11:56:10 ~1 min tests 📄log
✔️ 099ebdf #3 2024-12-05 11:57:47 ~3 min windows 📦zip
✔️ 099ebdf #3 2024-12-05 11:59:21 ~5 min macos 📦zip
✔️ 099ebdf #3 2024-12-05 11:59:52 ~5 min ios 📦zip
✔️ 099ebdf #3 2024-12-05 12:00:30 ~6 min linux 📦zip
✔️ 099ebdf #3 2024-12-05 12:00:58 ~6 min android 📦aar
✔️ 099ebdf #3 2024-12-05 12:01:07 ~6 min tests-rpc 📄log
✔️ 099ebdf #3 2024-12-05 12:02:20 ~7 min macos 📦zip
✔️ 63993fb #4 2024-12-05 12:13:42 ~3 min windows 📦zip
✖️ 63993fb #4 2024-12-05 12:14:34 ~3 min tests 📄log
✔️ 63993fb #4 2024-12-05 12:14:57 ~4 min ios 📦zip
✔️ 63993fb #4 2024-12-05 12:15:03 ~4 min macos 📦zip
✔️ 63993fb #4 2024-12-05 12:16:28 ~5 min linux 📦zip
✔️ 63993fb #4 2024-12-05 12:16:55 ~6 min tests-rpc 📄log
✔️ 63993fb #4 2024-12-05 12:17:04 ~6 min android 📦aar
✔️ 63993fb #4 2024-12-05 12:18:22 ~7 min macos 📦zip
✔️ 49c6278 #5 2024-12-05 13:18:52 ~3 min windows 📦zip
✔️ 49c6278 #5 2024-12-05 13:21:24 ~5 min linux 📦zip
✔️ 49c6278 #5 2024-12-05 13:21:53 ~6 min android 📦aar
✔️ 49c6278 #5 2024-12-05 13:21:55 ~6 min tests-rpc 📄log
✔️ 49c6278 #5 2024-12-05 13:22:00 ~6 min ios 📦zip
✔️ 49c6278 #5 2024-12-05 13:22:50 ~7 min macos 📦zip
✔️ 49c6278 #5 2024-12-05 13:25:40 ~10 min macos 📦zip
✔️ 49c6278 #5 2024-12-05 13:44:27 ~28 min tests 📄log
✔️ 0588310 #6 2024-12-05 13:30:12 ~2 min windows 📦zip
✔️ 0588310 #6 2024-12-05 13:33:26 ~6 min linux 📦zip
✔️ 0588310 #6 2024-12-05 13:33:45 ~6 min ios 📦zip
✔️ 0588310 #6 2024-12-05 13:33:54 ~6 min android 📦aar
✔️ 0588310 #6 2024-12-05 13:34:08 ~6 min tests-rpc 📄log
✔️ 0588310 #6 2024-12-05 13:36:39 ~9 min macos 📦zip
✔️ 0588310 #6 2024-12-05 13:39:41 ~12 min macos 📦zip
✔️ 0588310 #6 2024-12-05 14:15:04 ~30 min tests 📄log
✔️ 50ee868 #7 2024-12-06 13:11:50 ~3 min windows 📦zip
✔️ 50ee868 #7 2024-12-06 13:14:32 ~6 min linux 📦zip
✔️ 50ee868 #7 2024-12-06 13:14:36 ~6 min android 📦aar
✔️ 50ee868 #7 2024-12-06 13:15:03 ~6 min tests-rpc 📄log
✔️ 50ee868 #7 2024-12-06 13:16:40 ~8 min ios 📦zip
✔️ 50ee868 #7 2024-12-06 13:18:59 ~10 min macos 📦zip
✔️ 50ee868 #7 2024-12-06 13:19:31 ~11 min macos 📦zip
✔️ 50ee868 #7 2024-12-06 13:39:43 ~31 min tests 📄log
1ecae6e #8 2025-01-06 18:13:25 ~25 sec windows 📄log
✔️ 1ecae6e #8 2025-01-06 18:17:22 ~4 min macos 📦zip
✔️ 1ecae6e #8 2025-01-06 18:17:47 ~4 min ios 📦zip
✔️ 1ecae6e #8 2025-01-06 18:18:27 ~5 min linux 📦zip
✔️ 1ecae6e #8 2025-01-06 18:18:42 ~5 min android 📦aar
✔️ 1ecae6e #8 2025-01-06 18:19:53 ~6 min tests-rpc 📄log
✔️ 1ecae6e #8 2025-01-06 18:22:03 ~9 min macos 📦zip
✔️ 1ecae6e #8 2025-01-06 18:43:33 ~30 min tests 📄log
afb9730 #9 2025-01-06 18:16:57 ~26 sec windows 📄log
✔️ afb9730 #9 2025-01-06 18:21:47 ~4 min macos 📦zip
✔️ afb9730 #9 2025-01-06 18:22:50 ~4 min ios 📦zip
✔️ afb9730 #9 2025-01-06 18:24:00 ~5 min linux 📦zip
✔️ afb9730 #9 2025-01-06 18:24:40 ~5 min android 📦aar
✔️ afb9730 #9 2025-01-06 18:26:43 ~6 min tests-rpc 📄log
✔️ afb9730 #9 2025-01-06 18:30:32 ~8 min macos 📦zip
✔️ afb9730 #9 2025-01-06 19:14:03 ~30 min tests 📄log
5df7c7e #10 2025-01-07 12:13:22 ~25 sec windows 📄log
✔️ 5df7c7e #10 2025-01-07 12:17:16 ~4 min macos 📦zip
✔️ 5df7c7e #10 2025-01-07 12:17:27 ~4 min ios 📦zip
✔️ 5df7c7e #10 2025-01-07 12:18:25 ~5 min linux 📦zip
✔️ 5df7c7e #10 2025-01-07 12:18:35 ~5 min android 📦aar
✔️ 5df7c7e #10 2025-01-07 12:19:28 ~6 min tests-rpc 📄log
✔️ 5df7c7e #10 2025-01-07 12:24:23 ~11 min macos 📦zip
✔️ 5df7c7e #10 2025-01-07 12:43:50 ~30 min tests 📄log
e04001a #11 2025-01-07 12:40:16 ~26 sec windows 📄log
✔️ e04001a #11 2025-01-07 12:44:11 ~4 min macos 📦zip
✔️ e04001a #11 2025-01-07 12:44:34 ~4 min ios 📦zip
✔️ e04001a #11 2025-01-07 12:44:48 ~5 min linux 📦zip
✔️ e04001a #11 2025-01-07 12:45:18 ~5 min android 📦aar
✔️ e04001a #11 2025-01-07 12:45:50 ~6 min tests-rpc 📄log
✖️ e04001a #11 2025-01-07 12:46:35 ~2 min tests 📄log
✔️ e04001a #11 2025-01-07 12:51:27 ~11 min macos 📦zip
Commit #️⃣ Finished (UTC) Duration Platform Result
1c51d12 #12 2025-01-10 15:48:16 ~25 sec windows 📄log
✔️ 1c51d12 #12 2025-01-10 15:52:16 ~4 min macos 📦zip
✔️ 1c51d12 #12 2025-01-10 15:52:38 ~4 min ios 📦zip
✔️ 1c51d12 #12 2025-01-10 15:53:25 ~5 min linux 📦zip
✔️ 1c51d12 #12 2025-01-10 15:53:37 ~5 min android 📦aar
✔️ 1c51d12 #12 2025-01-10 15:54:38 ~6 min tests-rpc 📄log
✔️ 1c51d12 #12 2025-01-10 15:56:21 ~8 min macos 📦zip
✔️ 1c51d12 #12 2025-01-10 16:16:59 ~29 min tests 📄log
c1b756f #13 2025-01-10 15:51:28 ~25 sec windows 📄log
c1b756f #14 2025-01-10 15:52:56 ~24 sec windows 📄log
✔️ c1b756f #13 2025-01-10 15:56:42 ~4 min macos 📦zip
✔️ c1b756f #13 2025-01-10 15:57:54 ~5 min ios 📦zip
✔️ c1b756f #13 2025-01-10 15:59:01 ~5 min linux 📦zip
✔️ c1b756f #13 2025-01-10 15:59:35 ~5 min android 📦aar
✔️ c1b756f #13 2025-01-10 16:01:54 ~7 min tests-rpc 📄log
✔️ c1b756f #13 2025-01-10 16:05:23 ~8 min macos 📦zip
✔️ c1b756f #13 2025-01-10 16:46:55 ~29 min tests 📄log

@friofry friofry force-pushed the ab/issue-5997-rpc-providers-configuration branch 2 times, most recently from 8195c81 to 099ebdf Compare December 5, 2024 11:54
@friofry friofry changed the title Rpc providers configuration fix_: rpc providers configuration Dec 5, 2024
@friofry friofry changed the title fix_: rpc providers configuration chore_: rpc providers configuration Dec 5, 2024
@friofry friofry changed the title chore_: rpc providers configuration chore(config)_: rpc providers configuration Dec 5, 2024
@friofry friofry force-pushed the ab/issue-5997-rpc-providers-configuration branch 3 times, most recently from 49c6278 to 0588310 Compare December 5, 2024 13:26
Copy link

codecov bot commented Dec 5, 2024

Codecov Report

Attention: Patch coverage is 76.10790% with 124 lines in your changes missing coverage. Please review.

Project coverage is 61.47%. Comparing base (4faaa3e) to head (c1b756f).
Report is 3 commits behind head on develop.

Files with missing lines Patch % Lines
rpc/network/db/network_db.go 68.85% 20 Missing and 18 partials ⚠️
params/networkhelper/provider_utils.go 64.21% 31 Missing and 3 partials ⚠️
rpc/network/db/rpc_provider_db.go 77.46% 16 Missing and 16 partials ⚠️
rpc/network/db/utils.go 48.27% 11 Missing and 4 partials ⚠️
params/networkhelper/validate.go 81.48% 3 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #6151      +/-   ##
===========================================
+ Coverage    61.44%   61.47%   +0.02%     
===========================================
  Files          835      842       +7     
  Lines       110095   110595     +500     
===========================================
+ Hits         67651    67991     +340     
- Misses       34516    34623     +107     
- Partials      7928     7981      +53     
Flag Coverage Δ
functional 21.49% <35.29%> (-0.04%) ⬇️
unit 60.01% <76.10%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
api/default_networks.go 100.00% <100.00%> (ø)
api/defaults.go 81.25% <100.00%> (ø)
params/config.go 69.02% <ø> (ø)
params/network_config.go 100.00% <100.00%> (ø)
rpc/network/testutil/testutil.go 100.00% <100.00%> (ø)
params/networkhelper/validate.go 81.48% <81.48%> (ø)
rpc/network/db/utils.go 48.27% <48.27%> (ø)
rpc/network/db/rpc_provider_db.go 77.46% <77.46%> (ø)
params/networkhelper/provider_utils.go 64.21% <64.21%> (ø)
rpc/network/db/network_db.go 68.85% <68.85%> (ø)

... and 30 files with indirect coverage changes

@friofry friofry force-pushed the ab/issue-5997-rpc-providers-configuration branch from 0588310 to 50ee868 Compare December 6, 2024 13:08
@igor-sirotin igor-sirotin self-requested a review December 16, 2024 15:31
params/config.go Show resolved Hide resolved
params/network_config.go Outdated Show resolved Hide resolved
params/networkhelper/provider_utils.go Outdated Show resolved Hide resolved
Comment on lines +43 to +47
if providers[i].Type == params.UserProviderType {
providers[i].Enabled = enabled
} else {
providers[i].Enabled = !enabled
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if providers[i].Type == params.UserProviderType {
providers[i].Enabled = enabled
} else {
providers[i].Enabled = !enabled
}
providers[i].Enabled = enabled && (providers[i].Type == params.UserProviderType)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the equivalent form:
but I'm not sure it's any easier to understand

providers[i].Enabled = enabled == (providers[i].Type == params.UserProviderType)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, my bad.
The == id definitely worse readable, so better to keep as you did then 👍

params/networkhelper/provider_utils.go Outdated Show resolved Hide resolved
rpc/network/db/network_db_test.go Outdated Show resolved Hide resolved
rpc/network/db/rpc_provider_db_test.go Outdated Show resolved Hide resolved
rpc/network/db/utils.go Outdated Show resolved Hide resolved
rpc/network/db/utils.go Outdated Show resolved Hide resolved
rpc/network/testutil/testutil.go Outdated Show resolved Hide resolved
@igor-sirotin
Copy link
Collaborator

@friofry forgot to say: by the end of the review I was so annoyed of the size of the PR that I was tempting to reject it and ask to split in multiple... Looks like it was pretty possible to split it, there's good independency in the new code.

@friofry friofry force-pushed the ab/issue-5997-rpc-providers-configuration branch from 50ee868 to 1ecae6e Compare January 6, 2025 18:12
@friofry
Copy link
Contributor Author

friofry commented Jan 6, 2025

@friofry forgot to say: by the end of the review I was so annoyed of the size of the PR that I was tempting to reject it and ask to split in multiple... Looks like it was pretty possible to split it, there's good independency in the new code.

Sorry for the huge PR, I split it into 2 PRs (configuration + integration). Probably should have split it into more parts.
And thank you for your comments and time!

@friofry friofry force-pushed the ab/issue-5997-rpc-providers-configuration branch 2 times, most recently from 5df7c7e to e04001a Compare January 7, 2025 12:39
Copy link
Collaborator

@igor-sirotin igor-sirotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixes 👍

@friofry friofry force-pushed the ab/issue-5997-rpc-providers-configuration branch from e04001a to 1c51d12 Compare January 10, 2025 15:47
* Add rpc_providers table, migration
* add RpcProvider type
* deprecate old rpc fields in networks, add RpcProviders list
* add persistence packages for rpc_providers, networks
* Tests

(without integration)
@friofry friofry force-pushed the ab/issue-5997-rpc-providers-configuration branch from 1c51d12 to c1b756f Compare January 10, 2025 15:50
@friofry friofry requested review from dlipicar, Khushboo-dev-cpp and saledjenic and removed request for dlipicar, Khushboo-dev-cpp and saledjenic January 10, 2025 21:59
@friofry friofry merged commit e9abf16 into develop Jan 10, 2025
18 of 19 checks passed
@friofry friofry deleted the ab/issue-5997-rpc-providers-configuration branch January 10, 2025 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Enable RPC limiters for some providers
4 participants