Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[INLONG-10463][SDK] Optimization of ultra-long field processing in InlongSDK (Temp) #11213

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

qy-liuhuo
Copy link
Contributor

Fixes #10463
Last Pr: #11119

Motivation

In #11119 We discussed how to get the max message size configuration. Everyone had different ideas. Finally, we decided to temporarily configure it locally in the DataProxy SDK.

If the message is longer than the ConfigConstants.MAX_MESSAGE_LENGTH, it will directly return BODY_EXCEED_MAX_LEN error

Modifications

  • Added ConfigConstants.MAX_MESSAGE_LENGTH constant in the ConfigConstants file
  • Added the DataBodyUtil to get the msg size of body or bodyList
  • Added logic for handling body length for the DefaultMessageSender

Verifying this change

(Please pick either of the following options)

  • This change is a trivial rework/code cleanup without any test coverage.

  • This change is already covered by existing tests, such as:
    (please describe tests)

  • This change added tests and can be verified as follows:

    (example:)

    • Added integration tests for end-to-end deployment with large payloads (10MB)
    • Extended integration test for recovery after broker failure

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Improve][SDK] Optimization of ultra-long field processing in InlongSDK
1 participant