Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sui-types] Update bounded visitor to take an environment variable override at startup #18175

Merged
merged 1 commit into from
Jun 10, 2024

Conversation

tzakian
Copy link
Contributor

@tzakian tzakian commented Jun 10, 2024

Description

This adds the ability to start the Sui process (/indexers/fullnodes etc) with different MAX_BOUNDs for annotated values -- this is particularly useful for fullnodes and indexers.

To use a different bound from the default, you simply set MAX_ANNOTATED_VALUE_SIZE=N in your environment before starting the process. Where N is the number of bytes "in addition" that you're willing to tolerate when creating the annotated value from the un-annotated value (i.e., on types, variant names, and field names).

Test plan

Added new test to make sure:

  1. We respect this variable if set; and
  2. Do not update the value if changed during runtime;

Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

  • Protocol:
  • Nodes (Validators and Full nodes): Added MAX_ANNOTATED_VALUE_SIZE environment variable which can be used to override the MAX_BOUND for annotated values at startup (useful if encountering "Deserialized value too large" errors). If the MAX_ANNOTATED_VALUE_SIZE is not set, MAX_BOUND will default to the existing max bound of ~1MiB.
  • Indexer: Added MAX_ANNOTATED_VALUE_SIZE environment variable which can be used to override the MAX_BOUND for annotated values at startup (useful if encountering "Deserialized value too large" errors). If the MAX_ANNOTATED_VALUE_SIZE is not set, MAX_BOUND will default to the existing max bound of ~1MiB.
  • JSON-RPC: Added MAX_ANNOTATED_VALUE_SIZE environment variable which can be used to override the MAX_BOUND for annotated values at startup (useful if encountering "Deserialized value too large" errors). If the MAX_ANNOTATED_VALUE_SIZE is not set, MAX_BOUND will default to the existing max bound of ~1MiB.
  • GraphQL: Added MAX_ANNOTATED_VALUE_SIZE environment variable which can be used to override the MAX_BOUND for annotated values at startup (useful if encountering "Deserialized value too large" errors). If the MAX_ANNOTATED_VALUE_SIZE is not set, MAX_BOUND will default to the existing max bound of ~1MiB.
  • CLI:
  • Rust SDK:

Copy link

vercel bot commented Jun 10, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
sui-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 10, 2024 10:41pm
3 Ignored Deployments
Name Status Preview Comments Updated (UTC)
multisig-toolkit ⬜️ Ignored (Inspect) Visit Preview Jun 10, 2024 10:41pm
sui-kiosk ⬜️ Ignored (Inspect) Visit Preview Jun 10, 2024 10:41pm
sui-typescript-docs ⬜️ Ignored (Inspect) Visit Preview Jun 10, 2024 10:41pm

Copy link
Contributor

@amnn amnn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Could you also add release notes for the GraphQL and JSON-RPC sections? I think they will also be affected by (or benefit from) this change. Thanks @tzakian !

crates/sui-types/src/object/bounded_visitor.rs Outdated Show resolved Hide resolved
@tzakian tzakian force-pushed the tzakian/bounded-visitor-env-override-runtime branch from 6669f4b to 1db74a2 Compare June 10, 2024 22:38
@tzakian tzakian enabled auto-merge (squash) June 10, 2024 22:48
@tzakian tzakian merged commit 900c6bf into main Jun 10, 2024
45 of 46 checks passed
@tzakian tzakian deleted the tzakian/bounded-visitor-env-override-runtime branch June 10, 2024 23:01
tx-tomcat pushed a commit to tx-tomcat/sui-network that referenced this pull request Jul 29, 2024
…erride at startup (MystenLabs#18175)

## Description 

This adds the ability to start the Sui process (/indexers/fullnodes etc)
with different `MAX_BOUNDs` for annotated values -- this is particularly
useful for fullnodes and indexers.

To use a different bound from the default, you simply set
`MAX_ANNOTATED_VALUE_SIZE=N` in your environment before starting the
process. Where `N` is the number of bytes "in addition" that you're
willing to tolerate when creating the annotated value from the
un-annotated value (i.e., on types, variant names, and field names).

## Test plan 

Added new test to make sure:
1. We respect this variable if set; and
2. Do not update the value if changed during runtime;

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [X] Nodes (Validators and Full nodes): Added
`MAX_ANNOTATED_VALUE_SIZE` environment variable which can be used to
override the `MAX_BOUND` for annotated values at startup (useful if
encountering `"Deserialized value too large"` errors). If the
`MAX_ANNOTATED_VALUE_SIZE` is not set, `MAX_BOUND` will default to the
existing max bound of ~1MiB.
- [X] Indexer: Added `MAX_ANNOTATED_VALUE_SIZE` environment variable
which can be used to override the `MAX_BOUND` for annotated values at
startup (useful if encountering `"Deserialized value too large"`
errors). If the `MAX_ANNOTATED_VALUE_SIZE` is not set, `MAX_BOUND` will
default to the existing max bound of ~1MiB.
- [X] JSON-RPC: Added `MAX_ANNOTATED_VALUE_SIZE` environment variable
which can be used to override the `MAX_BOUND` for annotated values at
startup (useful if encountering `"Deserialized value too large"`
errors). If the `MAX_ANNOTATED_VALUE_SIZE` is not set, `MAX_BOUND` will
default to the existing max bound of ~1MiB.
- [X] GraphQL: Added `MAX_ANNOTATED_VALUE_SIZE` environment variable
which can be used to override the `MAX_BOUND` for annotated values at
startup (useful if encountering `"Deserialized value too large"`
errors). If the `MAX_ANNOTATED_VALUE_SIZE` is not set, `MAX_BOUND` will
default to the existing max bound of ~1MiB.
- [ ] CLI: 
- [ ] Rust SDK:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants