You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
S3PropertiesReloader already applies some configuration for whether to reload properties. We can add an instance property to reload periodically and default that to e.g. every 5 minutes.
We'll need to add PropertiesReloader into StateStoreCommitter, and we could avoid letting you force reload the properties there. We could have a separate static constructor for S3PropertiesReloader for the committer, similar to in StateStoreFactory.forCommitterProcess.
We could also consider how to apply changes to the table properties, but there's a problem as it is in that the state store objects hold the state of the Sleeper tables in memory and don't retain a reference to the table properties. The table properties are looked up once when the state store is created, and then not checked again. We could deal with this as a separate issue.
The text was updated successfully, but these errors were encountered:
Analysis
S3PropertiesReloader already applies some configuration for whether to reload properties. We can add an instance property to reload periodically and default that to e.g. every 5 minutes.
We'll need to add PropertiesReloader into StateStoreCommitter, and we could avoid letting you force reload the properties there. We could have a separate static constructor for S3PropertiesReloader for the committer, similar to in StateStoreFactory.forCommitterProcess.
We could also consider how to apply changes to the table properties, but there's a problem as it is in that the state store objects hold the state of the Sleeper tables in memory and don't retain a reference to the table properties. The table properties are looked up once when the state store is created, and then not checked again. We could deal with this as a separate issue.
The text was updated successfully, but these errors were encountered: