Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid etcd request too large #7728

Closed
lmatz opened this issue Feb 7, 2023 · 0 comments · Fixed by #9641
Closed

Avoid etcd request too large #7728

lmatz opened this issue Feb 7, 2023 · 0 comments · Fixed by #9641
Assignees

Comments

@lmatz
Copy link
Contributor

lmatz commented Feb 7, 2023

etcd request too large error appeared in #6726 and #7315

Correct me if I am wrong,
the etcd request size can be linearly related to

  1. the number of queries (during recovery, migration only)
  2. the parallelism of each query
  3. the size of the metadata of each query

This error now also appears in one POC case: only one query, but the query is particularly long and complicated

Of course, we can increase the --max-request-size of etcd to avoid the error as usual.
Although the doc of etcd said that a request of larger size may cause longer latency(I suppose it can be superlinear otherwise it is trivial), the etcd latency should be negligible compared to the total e2e latency of creating MV or recovery.

But in long term, we probably need to

  1. compress the part that can be compressed,
  2. or come up with a new and more concise representation.

Not particularly urgent as --max-request-size can be adjusted, but open this issue as a record

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants