Snap 2358 #395
Open
Snap 2358 #395
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- force decompress the underlying buffer in BlobChunk if remote node is on same host - increment reference in BlobChunk only when writing and decrement at the end of write; earlier it was incrementing in constructor which would lead to incorrect ref count if the BlobChunk was never written (e.g. filtered out in the query) - added free calls to BlobChunk in all relevant places when assigning new buffers - add IP address to the server host names (host/address[port]) in GET_TABLE_METADATA - added "decompressedReplaced" gfs stat to note the decompressions that were able to replace the underlying buffer ColumnFormatValue - allow for null cache for LowMemoryException (in case of connector mode)
Includes the changes for the two issues and a bunch of other fixes found in testing. - Added "COLUMN_TABLE_SCAN" procedure to be invoked from smart connector and a corresponding "columnTableScan" method to StoreCallbacks. - Implementation of columnTableScan returns iterator of "ColumnTableEntry" that encapsulates a single column of a batch that is converted to a ResultSet by procedure implementation. - Send back Blob value as is from DVDStoreResultSet that is used by above procedure. - Reference increment is done by columnTableScan, handed over to BlobChunk by ClientBlob, and released after thrift write by BlobChunk. - BlobChunk now will release the previously held reference then get the buffer again with DECOMPRESS_IF_IN_MEMORY flag when target node is same host, else COMPRESS for remote sends. - Fixed index column fetch in GET_TABLE_METADATA that would append indexes to "null" - Added a new type of fetch from ColumnFormatValue: DECOMPRESS_IF_IN_MEMORY. This will decompress buffer only if it can be replaced in memory else return as is (or null). This allows smart connector to avoid decompression on server side if unable to store it on server. Changed boolean flags to a "FetchRequest" enumeration. - Close bucket entries iterator before making it null else it leaks reference count of last batch. - Set the disk RegionEntry itself in the ColumnFormatValue to read from disk in case entry is deleted from region (and some reader is holding a handle). Earlier setting of DiskId was not proper since it may change due to compaction. - Zero the limit in case of full release of buffers to fail any further reads. - Corrected parenthesis in FunctionException checks in SnappyTableStatsVTI
added 14 commits
May 8, 2018 13:12
…rted when user has given option while create table.
sumwale
force-pushed
the
snappy/master
branch
2 times, most recently
from
337768c to
ec453a6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes proposed in this pull request
Now user can create sorted Column Batches on partitioning keys using DDL mentioned below. This will keep a column batch in sorted manner that can be leveraged for better performance of point queries, range queries and colocated join queries on partitioning columns. For more details please refer https://jira.snappydata.io/browse/SNAP-2358
Patch testing
Unit test
ReleaseNotes changes
See Other PR link below
Other PRs
TIBCOSoftware/snappydata#1054