Disable flush after every DefaultObjectMapper write #3748
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR disables the FLUSH_AFTER_WRITE_VALUE setting on DefaultObjectMapper:
http://wiki.fasterxml.com/JacksonFeaturesSerialization
https://fasterxml.github.io/jackson-databind/javadoc/2.3.0/com/fasterxml/jackson/databind/SerializationFeature.html#FLUSH_AFTER_WRITE_VALUE
Disabling this setting resulted in better network performance between the historical and broker when transferring query results. When serializing results from a yielder, a flush was occurring for every yielder result. This had significant impact on a test groupby query with compression disabled at the HttpClient level.
Using
iftop
, a transfer rate of ~350Mb/s was observed between the historical (i2.8xlarge node) and broker (r3.8xlarge node) with flush after write disabled, compared to a rate of ~170Mb/s with the setting enabled.