-
Notifications
You must be signed in to change notification settings - Fork 78
Description
Add support for eviction with queries in cache_evict decorator
Summary
Add the ability to evict cache entries using adapter-specific queries in the cache_evict decorator. This enables bulk eviction of entries that match specific criteria without needing to know all the keys in advance.
Motivation
Currently, the cache_evict decorator requires you to specify explicit keys or a list of keys to evict. However, there are scenarios where you need to evict multiple entries based on specific criteria rather than by their keys. For example:
- Deleting all cache entries that have a specific tag or category
- Evicting entries that match certain metadata or attributes
- Bulk eviction based on complex filtering conditions
- Removing entries that share common characteristics
Without query-based eviction, users must either:
- Keep track of all keys that need to be evicted (memory overhead and complexity)
- Manually iterate and delete entries (inefficient and error-prone)
- Clear the entire cache (too aggressive and impacts unrelated data)
Proposed Solution
Add a new :query option to the cache_evict decorator that accepts a query (or a function that returns a query at runtime). The query must be supported by the cache adapter.
Important: When the :query option is present, it overrides the :key option. This allows you to switch between key-based eviction and query-based eviction without needing to remove the :key option.
Example
@decorate cache_evict(query: &__MODULE__.query_for_tag/1)
def delete_objects_by_tag(tag) do
# your logic to delete data from the SoR
end
def query_for_tag(%{args: [tag]} = _context) do
# Assuming we are using the `Nebulex.Adapters.Local` adapter and the
# cached entry value is a map with a `tag` field, the match spec
# would look like this:
[
{
{:entry, :"$1", %{tag: :"$2"}, :_, :_},
[{:"=:=", :"$2", tag}],
[true]
}
]
endIn this example:
- The decorator calls the
query_for_tag/1function with the context - The function builds an adapter-specific query (match spec for Local adapter)
- The decorator uses the query to evict all matching entries from the cache
Benefits
- Bulk eviction without key tracking - Evict multiple entries based on criteria without maintaining key lists
- Adapter flexibility - Leverage adapter-specific query capabilities for efficient bulk operations
- Clean separation of concerns - Keep complex query logic separate from decorator annotations
- Performance optimization - Use native adapter query features for efficient filtering and deletion
- Reduced memory overhead - No need to store all keys that might need eviction
- Dynamic eviction criteria - Build queries dynamically based on runtime conditions
- Flexible eviction strategy - Use
:queryfor criteria-based eviction or:keyfor specific key eviction;:queryoverrides:keywhen both are present
Use Cases
This feature is particularly useful in scenarios where:
1. Tag-based eviction
# Evict all products in a specific category
@decorate cache_evict(query: &build_category_query/1)
def delete_products_by_category(category) do
# delete from database
end2. User-specific data cleanup
# Evict all cache entries for a specific user
@decorate cache_evict(query: &build_user_query/1)
def delete_user_data(user_id) do
# delete user data from database
end3. Time-based eviction
# Evict all entries created before a certain date
@decorate cache_evict(query: &build_date_range_query/1)
def cleanup_old_data(before_date) do
# cleanup from database
end4. Multi-attribute filtering
# Evict entries matching complex criteria
@decorate cache_evict(query: &build_complex_query/1)
def delete_items(filter_params) do
# delete from database based on multiple conditions
endImplementation Notes
API Design
The implementation should:
- Add a
:queryoption to thecache_evictdecorator - Support function-based queries that optionally receive the decorator context
- Support direct query values for static queries
- Override the
:keyoption when present - If both:queryand:keyare provided,:querytakes precedence - Delegate to adapter's query capabilities using the adapter's native query interface via
delete_all/2
Query Function Contract
Query functions optionally receive a context map with the following structure:
%{
args: [arg1, arg2, ...], # Function arguments
opts: [...], # Decorator options
cache: CacheModule, # Cache module
# ... other metadata
}See
[Nebulex.Caching.Decorators.context()](https://hexdocs.pm/nebulex/Nebulex.Caching.Decorators.html#t:context/0)for the complete structure.
And should return a query supported by the cache adapter (the specific format depends on the adapter being used).
Adapter Compatibility
Different adapters support different query formats:
- Local adapter: Erlang match specifications
- Partitioned adapter: Depends on the configured local adapter
- Replicated adapter: Depends on the configured local adapter
- Multilevel adapter: Depends on the configured levels
- Other adapters: Adapter-specific query formats
The implementation should document which adapters support query-based eviction and provide examples for the most common adapters.
Error Handling
Error handling is delegated to the adapter:
- Invalid query formats - Adapter should raise or return error
- Unsupported adapter operations - Adapter should raise or return error
- Query execution failures - Handled by adapter and decorator's
:on_erroroption - Empty result sets - Not an error, just a no-op
Example with Direct Query
For simple cases, queries can be provided directly:
@decorate cache_evict(
query: [
{
{:entry, :"$1", %{archived: true}, :_, :_},
[],
[true]
}
]
)
def cleanup_archived_entries do
# delete all archived entries from database
endHowever, using a function is recommended for:
- Complex queries
- Dynamic query building based on function arguments
- Better readability and maintainability
- Reusability across multiple decorators
Documentation Updates
The following documentation should be updated:
- Add a new section "Eviction with a query" to the
cache_evictdecorator docs - Update the
:queryoption documentation with comprehensive examples - Provide examples for the most common adapters (Local adapter with match specs)
- Include best practices for building and organizing query functions
- Document the interaction between
:queryand:keyoptions (:queryoverrides:key)
Migration Path
For existing users:
- This is a new feature, no breaking changes
- Existing
:keyoptions continue to work as before - The
:queryoption overrides:keywhen both are present - Users can gradually adopt query-based eviction where beneficial
Related
- Adds new
:queryoption to complement existing:keyoption incache_evictdecorator - Leverages adapter query capabilities already present in Nebulex via
delete_all/2 - Similar to query-based operations in other Nebulex functions like
get_all/2,count_all/2, etc.
Future Enhancements
Potential future improvements:
- Query builder helpers for common patterns (e.g., tag matching, user filtering)