change critical-heap-percentage to leave out max 200MB (upper limit of 99%) #507
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…f 99% for critical-heap)
Tests checking for native timer being loaded occasionally fail due to some previous tests initializing system in a different way. Now re-initialize the native timer for such a case.
- Track off-heap size in BucketRegion and add to getSizeInMemory() and getTotalBytes(). This fixes the callers of getTotalBytes() including rebalancing and determination of smallest bucket in SD layer. - Update SnappyRegionStatsCollectorFunction to avoid separate collection of off-heap size.
dshirish
approved these changes
Aug 1, 2019
| @@ -300,21 +300,22 @@ protected void setDefaultVMArgs(Map<String, Object> map, boolean hostData, | |||
| if (maxHeapStr != null && maxHeapStr.equals(this.initialHeapSize)) { | |||
| String criticalHeapStr = (String)map.get(CRITICAL_HEAP_PERCENTAGE); | |||
| if (criticalHeapStr == null) { | |||
| // for larger heaps, keep critical as 95% and 90% for smaller ones; | |||
| // for larger heaps, keep critical as 95-99% and 90% for smaller ones; | |||
| // also limit memory remaining beyond critical to 4GB | |||
There was a problem hiding this comment.
Is this comment now no more applicable ("limit memory remaining beyond critical to 4GB")?
|
@sumwale the change looks good. From comment in Jira, looks like for cases other than ingestion 200 MB headroom is sufficient to avoid going OOME in most cases and for ingestion case this will likely avoid LowMemoryException when there is some space still available? |
Yes with these limits the GC does hit before CRITICAL_UP and free up space. |
sumwale
added a commit
to sumwale/snappy-store
that referenced
this pull request
Apr 30, 2021
…per limit of 99%) (TIBCOSoftware#507) - For large heaps >40GB leave out max 1GB, for <2GB heaps leave 90% like before while for rest leave 200MB for critical-heap-percentage. This translates to 97.5% for 8GB & 98.8% for 16GB. An upper limit of 99% percent is still applied to be a bit on the safe side. - Track off-heap size in BucketRegion and add to getSizeInMemory() and getTotalBytes(). This fixes the callers of getTotalBytes() including rebalancing and determination of smallest bucket in snappydata. - Update SnappyRegionStatsCollectorFunction to avoid separate collection of off-heap size. - Tests checking for native timer being loaded occasionally fail due to some previous tests initializing system in a different way. Now re-initialize the native timer for such a case.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes proposed in this pull request
leave 200MB for critical-heap-percentage. This translates to 97.5% with 8GB and 98.8% for 16GB.
An upper limit of 99% percent is still applied to be a bit on the safe side.
the callers of getTotalBytes() including rebalancing and determination of smallest bucket in SD.
initializing system in a different way. Now re-initialize the native timer for such a case.
Patch testing
precheckin
Is precheckin with -Pstore clean?
in progress
ReleaseNotes changes
Document the default critical-heap-percentage and conditions under which user may want to tweak.
Other PRs
TIBCOSoftware/snappydata#1403