Fix for SparkML memory pressure issues #344
Conversation
rather than methods.
…ory pressure. Reworked TileExplodeBench
| @@ -293,3 +296,12 @@ trait StandardSerializers { | |||
| implicit val spaceTimeKeyTLMSerializer = tileLayerMetadataSerializer[SpaceTimeKey] | |||
|
|
|||
| } | |||
|
|
|||
| object StandardSerializers { | |||
| private val s2ctCache = Scaffeine().build[String, CellType]( | |||
There was a problem hiding this comment.
Oh no... :D Have you figured out a proper fix for geotrellis-core? we could back port a proper fix into the 2.x branch and release it.
There was a problem hiding this comment.
@pomadchin This is just temporary for now. I'll submit a PR to GeoTrellis as soon as I can.... but our team needed a quick fix.
There was a problem hiding this comment.
Looks good to me.
Do you have jmh results from running TileExplodeBench before/after? Would be interested to see those on the ticket #343 or on this PR just for reference. Curious how much difference from the changes in ExplodeTiles ...
|
@vpipkt I'm really sorry but I accidentally did a |
Fixed SparkML memory pressure issue caused by unnecessary reevaluation, overallocation, and primitive boxing.
Closes #343