Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(swiftide): documented file swiftide/src/integrations/redis/node_cache.rs #29

Merged
merged 1 commit into from
Jun 13, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 69 additions & 4 deletions swiftide/src/integrations/redis/node_cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,35 @@ use async_trait::async_trait;

use crate::{ingestion::IngestionNode, traits::NodeCache};

/// `RedisNodeCache` provides a caching mechanism for nodes using Redis.
/// It helps in optimizing the ingestion process by skipping nodes that have already been processed.
///
/// # Fields
///
/// * `client` - The Redis client used to interact with the Redis server.
/// * `connection_manager` - Manages the Redis connections asynchronously.
/// * `key_prefix` - A prefix used for keys stored in Redis to avoid collisions.
pub struct RedisNodeCache {
client: redis::Client,
connection_manager: RwLock<Option<redis::aio::ConnectionManager>>,
key_prefix: String,
}

impl RedisNodeCache {
/// Creates a new `RedisNodeCache` instance from a given Redis URL and key prefix.
///
/// # Parameters
///
/// * `url` - The URL of the Redis server.
/// * `prefix` - The prefix to be used for keys stored in Redis.
///
/// # Returns
///
/// A `Result` containing the `RedisNodeCache` instance or an error if the client could not be created.
///
/// # Errors
///
/// Returns an error if the Redis client cannot be opened.
pub fn try_from_url(url: impl AsRef<str>, prefix: impl AsRef<str>) -> Result<Self> {
let client = redis::Client::open(url.as_ref()).context("Failed to open redis client")?;
Ok(Self {
Expand All @@ -22,7 +44,15 @@ impl RedisNodeCache {
})
}

// Connectionmanager is meant to be cloned
/// Lazily connects to the Redis server and returns the connection manager.
///
/// # Returns
///
/// An `Option` containing the `ConnectionManager` if the connection is successful, or `None` if it fails.
///
/// # Errors
///
/// Logs an error and returns `None` if the connection manager cannot be obtained.
async fn lazy_connect(&self) -> Option<redis::aio::ConnectionManager> {
if self.connection_manager.read().await.is_none() {
let result = self.client.get_connection_manager().await;
Expand All @@ -37,12 +67,26 @@ impl RedisNodeCache {
self.connection_manager.read().await.clone()
}

/// Generates a Redis key for a given node using the key prefix and the node's hash.
///
/// # Parameters
///
/// * `node` - The node for which the key is to be generated.
///
/// # Returns
///
/// A `String` representing the Redis key for the node.
fn key_for_node(&self, node: &IngestionNode) -> String {
format!("{}:{}", self.key_prefix, node.calculate_hash())
}

/// Resets the cache by deleting all keys with the specified prefix.
/// This function is intended for testing purposes and is inefficient for production use.
///
/// # Errors
///
/// Panics if the keys cannot be retrieved or deleted.
#[allow(dead_code)]
// Testing only, super inefficient
async fn reset_cache(&self) {
if let Some(mut cm) = self.lazy_connect().await {
let keys: Vec<String> = redis::cmd("KEYS")
Expand Down Expand Up @@ -73,8 +117,19 @@ impl Debug for RedisNodeCache {

#[async_trait]
impl NodeCache for RedisNodeCache {
// false -> not cached, expect node to be processed
// true -> cached, expect node to be skipped
/// Checks if a node is present in the cache.
///
/// # Parameters
///
/// * `node` - The node to be checked in the cache.
///
/// # Returns
///
/// `true` if the node is present in the cache, `false` otherwise.
///
/// # Errors
///
/// Logs an error and returns `false` if the cache check fails.
#[tracing::instrument(skip_all, name = "node_cache.redis.get", fields(hit))]
async fn get(&self, node: &IngestionNode) -> bool {
let cache_result = if let Some(mut cm) = self.lazy_connect().await {
Expand Down Expand Up @@ -104,6 +159,15 @@ impl NodeCache for RedisNodeCache {
cache_result
}

/// Sets a node in the cache.
///
/// # Parameters
///
/// * `node` - The node to be set in the cache.
///
/// # Errors
///
/// Logs an error if the node cannot be set in the cache.
#[tracing::instrument(skip_all, name = "node_cache.redis.get")]
async fn set(&self, node: &IngestionNode) {
if let Some(mut cm) = self.lazy_connect().await {
Expand All @@ -126,6 +190,7 @@ mod tests {
use std::collections::HashMap;
use testcontainers::runners::AsyncRunner;

/// Tests the `RedisNodeCache` implementation.
#[test_log::test(tokio::test)]
async fn test_redis_cache() {
let redis = testcontainers::GenericImage::new("redis", "7.2.4")
Expand Down
Loading