-
Notifications
You must be signed in to change notification settings - Fork 231
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Resolving imported policies in Ditto's "things-search" does not use a cache to load same policy #1869
Comments
thjaeckle
added a commit
that referenced
this issue
Jan 23, 2024
…loaded after invalidation in search * cache is used in ResolvedPolicyCacheLoader * it is now added that an invalidation caused by a policy contains the "causingPolicyTag" - which is then also invalidated * however, the "causingPolicyTag" is only invalidated once (per search cluster node) to not overwhelm the cluster again due to too many invalidations Signed-off-by: Thomas Jäckle <thomas.jaeckle@beyonnex.io>
thjaeckle
added a commit
that referenced
this issue
Jan 24, 2024
…loaded after invalidation in search * cache is used in ResolvedPolicyCacheLoader * it is now added that an invalidation caused by a policy contains the "causingPolicyTag" - which is then also invalidated * however, the "causingPolicyTag" is only invalidated once (per search cluster node) to not overwhelm the cluster again due to too many invalidations Signed-off-by: Thomas Jäckle <thomas.jaeckle@beyonnex.io>
thjaeckle
added a commit
that referenced
this issue
Jan 24, 2024
…loaded after invalidation in search * cache is used in ResolvedPolicyCacheLoader * it is now added that an invalidation caused by a policy contains the "causingPolicyTag" - which is then also invalidated * however, the "causingPolicyTag" is only invalidated once (per search cluster node) to not overwhelm the cluster again due to too many invalidations Signed-off-by: Thomas Jäckle <thomas.jaeckle@beyonnex.io>
thjaeckle
added a commit
that referenced
this issue
Jan 24, 2024
Signed-off-by: Thomas Jäckle <thomas.jaeckle@beyonnex.io>
thjaeckle
added a commit
that referenced
this issue
Jan 25, 2024
…orted-policies-in-ditto-search #1869 use cache in order to load imported policies when policies are …
Resolved by: #1872 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
During updating a single policy which is imported in ~5.000 other Policies, I noticed many "ask timeout exceptions" being thrown in Ditto's "things-search" service.
It is intended that - when a imported policy is changed - all things using policies importing from that single policy are updated in the search index (to reflect the changed permissions).
It is however quite unexpected that e.g. the same policy
org.eclipse.ditto:my-root-policy
is loaded for all policies which use that exact same policy. But this seems to be the case:ditto/thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/persistence/write/streaming/ResolvedPolicyCacheLoader.java
Lines 49 to 50 in aba53ad
This will of course cause a huge amount of
SudoRetrievePolicy
commands issued at the exact same time against thepolicies
shard region, leading to many timeouts.This place should make use of the Policy cache, so that for each cluster instance of "things-search" a heavily "imported" policy is only loaded once.
The text was updated successfully, but these errors were encountered: