Skip to content

Comments

fix(redis): Redis Pub/Sub 跨进程缓存失效通知#493

Merged
ding113 merged 3 commits intodevfrom
fix/issue-492-redis-pubsub-invalidation
Jan 1, 2026
Merged

fix(redis): Redis Pub/Sub 跨进程缓存失效通知#493
ding113 merged 3 commits intodevfrom
fix/issue-492-redis-pubsub-invalidation

Conversation

@ding113
Copy link
Owner

@ding113 ding113 commented Jan 1, 2026

Summary

Implements Redis Pub/Sub-based cross-process cache invalidation for error rules and request filters in multi-worker/multi-instance Next.js deployments.

Related Issues:

Problem

In Next.js multi-worker or multi-instance deployments, in-memory caches for error rules and request filters are only refreshed within the current process. When a new rule/filter is created or updated:

  • Database is updated successfully
  • Only the worker handling the request reloads its cache
  • Other workers continue using stale cache until manual sync or restart
  • Users experience inconsistent behavior across requests

问题背景(中文):

  • Next.js 多 worker/多实例环境下,错误规则与请求过滤器的内存缓存仅在当前进程刷新,导致新建/更新后不立即生效。

Solution

Implemented Redis Pub/Sub pattern for broadcasting cache invalidation events across all workers/instances:

  1. New Redis Pub/Sub Module (src/lib/redis/pubsub.ts):

    • publishCacheInvalidation() - Publishes invalidation messages
    • subscribeCacheInvalidation() - Subscribes to invalidation events
    • Two channels: CHANNEL_ERROR_RULES_UPDATED, CHANNEL_REQUEST_FILTERS_UPDATED
    • Graceful degradation when Redis unavailable
  2. Event Broadcasting (src/lib/emit-event.ts):

    • Dual notification: local EventEmitter + Redis Pub/Sub
    • Ensures both single-worker (immediate) and multi-worker (eventual) consistency
  3. Subscriber Integration:

    • ErrorRuleDetector subscribes to CHANNEL_ERROR_RULES_UPDATED
    • RequestFilterEngine subscribes to CHANNEL_REQUEST_FILTERS_UPDATED
    • Both trigger reload() on notification
  4. Publisher Integration (src/actions/error-rules.ts):

    • Changed from direct reload() calls to emitErrorRulesUpdated() broadcasts
    • Supports cross-process synchronization

Changes

Core Changes

  • src/lib/redis/pubsub.ts (+97) - New Redis Pub/Sub utility module
  • src/lib/emit-event.ts (+18) - Added Redis Pub/Sub publishing to event emitters
  • src/lib/error-rule-detector.ts (+11/-2) - Subscribe to Redis invalidation events
  • src/lib/request-filter-engine.ts (+18) - Subscribe to Redis invalidation events with cleanup
  • src/actions/error-rules.ts (+7/-6) - Changed from direct reload to event broadcast

Testing

  • src/lib/redis/__tests__/pubsub.test.ts (+91) - Comprehensive test suite:
    • Message publishing
    • Subscription and message reception
    • Graceful degradation when Redis unavailable
    • Cleanup/unsubscription logic

Technical Details

Architecture Pattern:

Action (create/update/delete)
  ↓
emitEvent() → Local EventEmitter + Redis Pub/Sub
  ↓
All workers receive notification
  ↓
Each worker calls reload() → Fresh data from DB

Subscriber Lifecycle:

  • Subscriber client created via baseClient.duplicate() (Redis Pub/Sub requires dedicated connection)
  • Multiple callbacks per channel supported (Set-based storage)
  • Auto-unsubscribe when last callback removed
  • Cleanup functions returned for memory leak prevention

Graceful Degradation:

  • Publish/Subscribe failures logged but don't throw errors
  • System continues working in single-worker mode if Redis unavailable
  • EventEmitter fallback for same-process notifications

Testing

Automated Tests

  • Unit tests added for Redis Pub/Sub module (pubsub.test.ts)
  • Coverage: publish, subscribe, unsubscribe, error handling
  • Tested graceful degradation scenarios

Verification Steps

bun run test          # All tests pass
bun run typecheck     # Type checks pass
bun run build         # Production build succeeds

Manual Testing Checklist

For multi-worker deployments:

  1. Start application with multiple workers/instances
  2. Create a new error rule via UI
  3. Verify rule takes effect immediately across all workers
  4. Update an existing error rule
  5. Verify update propagates to all workers
  6. Repeat for request filters

Breaking Changes

None - This is a backward-compatible enhancement.

Migration Guide

No migration required. If Redis is not configured, the system gracefully falls back to single-process EventEmitter (existing behavior).


Description enhanced by Claude AI

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @ding113, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

此拉取请求旨在解决 Next.js 多 worker/多实例环境下内存缓存(特别是错误规则和请求过滤器)无法跨进程同步刷新的问题。通过引入 Redis Pub/Sub 机制,当缓存数据发生变化时,系统能够发布通知,并由其他进程订阅这些通知来触发各自缓存的重新加载,从而确保所有实例的数据一致性。

Highlights

  • 新增 Redis Pub/Sub 模块: 引入了一个新的 Redis Pub/Sub 工具模块,提供了发布和订阅缓存失效通知的接口,以支持跨进程通信。
  • 跨进程缓存失效通知: emit-event 模块现在除了本地事件外,还会同步发布 Redis 通知,确保在多 worker/多实例环境下缓存能够及时刷新。
  • 错误规则与请求过滤器更新: ErrorRuleDetector 和 RequestFilterEngine 现在订阅 Redis 通知,以便在接收到跨进程通知时触发缓存的重新加载。
  • 错误规则操作变更: error-rules 相关的 actions (创建、更新、删除) 不再直接调用 reload 方法,而是改为广播事件,通过 Redis Pub/Sub 机制实现缓存的分布式刷新。
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions github-actions bot added bug Something isn't working area:core size/M Medium PR (< 500 lines) labels Jan 1, 2026
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

本次 PR 通过引入 Redis Pub/Sub 机制,解决了在 Next.js 多 worker/多实例环境下内存缓存同步的问题,这是一个很好的改进。代码实现整体上是清晰和健壮的,特别是对 Redis 不可用时的优雅降级处理,以及为新功能增加了完整的单元测试。

我的反馈主要集中在一些可以进一步提升代码健壮性和可维护性的细节上:

  1. 在几个地方,错误被静默地捕获,建议增加警告日志,以便于在问题发生时进行调试。
  2. pubsub.ts 模块中存在一些潜在的并发问题和未处理的 Promise,修复后可以使模块更加通用和安全。
  3. emit-event.ts 中存在一些代码重复,可以考虑在未来进行重构。

总的来说,这是一个高质量的 PR,上述建议旨在让代码在边缘情况下表现得更好。

Comment on lines +25 to +27
} catch {
// 忽略导入错误
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

空的 catch 块会静默地忽略导入 @/lib/redis/pubsub 模块时可能发生的任何错误。虽然注释说明了这是有意为之,但这可能会导致在出现问题(例如,模块路径错误、依赖问题)时难以调试,因为缓存失效将无声地失败。建议至少在这里添加一条警告日志,以便在开发或生产环境中观察到此类问题。

此外,此 try-catch 块与 emitRequestFiltersUpdated 函数中的逻辑完全相同,存在代码重复。可以考虑将此逻辑提取到辅助函数中以提高代码的可维护性。

    } catch (error) {
      // 忽略导入错误,但记录警告以便调试
      const { logger } = await import("@/lib/logger");
      logger.warn("[EmitEvent] Failed to publish error rules update notification", { error });
    }

Comment on lines +62 to +64
} catch {
// 忽略导入错误
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

emitErrorRulesUpdated 函数中的情况类似,空的 catch 块会静默地忽略错误,这不利于调试。建议添加警告日志来记录潜在的导入或发布失败。

    } catch (error) {
      // 忽略导入错误,但记录警告以便调试
      const { logger } = await import("@/lib/logger");
      logger.warn("[EmitEvent] Failed to publish request filters update notification", { error });
    }

Comment on lines +15 to +39
function ensureSubscriber(baseClient: Redis): Redis {
if (subscriberClient) return subscriberClient;

// 订阅必须使用独立连接(Pub/Sub 模式下连接不能再执行普通命令)
subscriberClient = baseClient.duplicate();

subscriberClient.on("message", (channel: string) => {
const callbacks = subscriptions.get(channel);
if (!callbacks || callbacks.size === 0) return;

for (const cb of callbacks) {
try {
cb();
} catch (error) {
logger.error("[RedisPubSub] Callback error", { channel, error });
}
}
});

subscriberClient.on("error", (error) => {
logger.warn("[RedisPubSub] Subscriber connection error", { error });
});

return subscriberClient;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

ensureSubscriber 函数不是可重入的(re-entrant)。如果在 subscriberClient 被赋值之前并发调用 subscribeCacheInvalidation,可能会创建多个重复的 Redis 订阅客户端。虽然在当前的使用场景下(在单例的构造函数中调用)可能不会发生并发问题,但这使得该模块在未来重用时存在潜在风险。建议通过 Promise 来确保只创建一个订阅客户端实例,以增强模块的健壮性。

if (cbs.size === 0) {
subscriptions.delete(channel);
if (subscriberClient) {
void subscriberClient.unsubscribe(channel);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

使用 void 关键字会忽略 unsubscribe 方法返回的 Promise。如果取消订阅失败,错误将被静默忽略,可能导致订阅关系在 Redis 服务器上残留,造成不必要的资源消耗。建议处理这个 Promise,至少捕获并记录潜在的错误,以便于问题排查。

Suggested change
void subscriberClient.unsubscribe(channel);
subscriberClient.unsubscribe(channel).catch(error => logger.warn("[RedisPubSub] Failed to unsubscribe from channel", { channel, error }));

Comment on lines +158 to +160
} catch {
// 忽略导入错误
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

空的 catch 块会静默地忽略导入或订阅 Redis Pub/Sub 时可能发生的错误。这使得在出现问题时难以调试,因为跨进程的缓存失效将无声地失败。建议至少在这里添加一条警告日志,以便于追踪潜在的配置或连接问题。

        } catch (error) {
          // 忽略导入错误
          logger.warn("[RequestFilterEngine] Failed to subscribe to Redis cache invalidation", { error });
        }

@github-actions
Copy link
Contributor

github-actions bot commented Jan 1, 2026

🧪 测试结果

测试类型 状态
代码质量
单元测试
集成测试
API 测试

总体结果: ✅ 所有测试通过

"@/lib/redis/pubsub"
);
await publishCacheInvalidation(CHANNEL_ERROR_RULES_UPDATED);
} catch {
Copy link
Contributor

@github-actions github-actions bot Jan 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] [ERROR-SILENT] Swallowing Redis invalidation failures hides stale-cache bugs

Why this is a problem: At src/lib/emit-event.ts:25, the new Redis Pub/Sub broadcast is wrapped in an empty catch:

} catch {
  // 忽略导入错误
}

Violated guideline: "CRITICAL: Empty catch blocks are FORBIDDEN"

This can reintroduce the cross-worker cache staleness this PR is meant to fix when the dynamic import/broadcast fails, with no diagnostics.

Suggested fix:

} catch (error) {
  const { logger } = await import("@/lib/logger");
  logger.warn(
    "[emit-event] Failed to publish Redis cache invalidation; cross-worker reload disabled",
    { error }
  );
}

Apply the same pattern to the emitRequestFiltersUpdated block as well.

CHANNEL_REQUEST_FILTERS_UPDATED,
handler
);
} catch {
Copy link
Contributor

@github-actions github-actions bot Jan 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] [ERROR-SILENT] Redis Pub/Sub subscription setup is silently ignored

Why this is a problem: At src/lib/request-filter-engine.ts:158, the cross-worker reload subscription is wrapped in an empty catch:

} catch {
  // 忽略导入错误
}

Violated guideline: "CRITICAL: Empty catch blocks are FORBIDDEN"

If the dynamic import or subscription fails, other workers won't reload request filters and there is no log to diagnose stale behavior.

Suggested fix:

} catch (error) {
  logger.warn("[RequestFilterEngine] Failed to enable Redis cache invalidation subscription", {
    error,
  });
}

if (cbs.size === 0) {
subscriptions.delete(channel);
if (subscriberClient) {
void subscriberClient.unsubscribe(channel);
Copy link
Contributor

@github-actions github-actions bot Jan 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[High] [ERROR-SILENT] unsubscribe() promise is discarded (possible unhandled rejection)

Why this is a problem: At src/lib/redis/pubsub.ts:89, the Promise from unsubscribe is ignored:

void subscriberClient.unsubscribe(channel);

Violated guideline: "No Silent Failures: Any error caught without logging or user feedback is a CRITICAL defect."

If unsubscribe rejects (e.g. connection error), this can surface as an unhandled promise rejection and also hides failures to clean up channel subscriptions.

Suggested fix:

if (subscriberClient) {
  void subscriberClient.unsubscribe(channel).catch((error) => {
    logger.warn("[RedisPubSub] Failed to unsubscribe channel", { channel, error });
  });
}

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

This PR adds Redis Pub/Sub-based cache invalidation so error rules and request filters reload across workers/instances. Several new catch {} blocks silently swallow failures, which can make cross-process invalidation stop working with no diagnostics.

PR Size: M

  • Lines changed: 250
  • Files changed: 6

Issues Found

Category Critical High Medium Low
Logic/Bugs 0 0 0 0
Security 0 0 0 0
Error Handling 2 1 0 0
Types 0 0 0 0
Comments/Docs 0 0 0 0
Tests 0 0 0 0
Simplification 0 0 0 0

Critical Issues (Must Fix)

  • src/lib/emit-event.ts:25 [ERROR-SILENT] Empty catch {} hides Redis invalidation broadcast failures (Confidence: 100)
  • src/lib/request-filter-engine.ts:158 [ERROR-SILENT] Empty catch {} hides Pub/Sub subscription failures (Confidence: 100)

High Priority Issues (Should Fix)

  • src/lib/redis/pubsub.ts:89 [ERROR-SILENT] Ignored unsubscribe() promise may cause unhandled rejection (Confidence: 85)

Review Coverage

  • Logic and correctness
  • Security (OWASP Top 10)
  • Error handling
  • Type safety
  • Documentation accuracy
  • Test coverage
  • Code clarity

Automated review by Codex AI

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Reviewed PR #493 and applied label size/M (250 lines, 6 files).
  • Posted 3 inline review comments on NEW/changed lines:
    • src/lib/emit-event.ts:25Critical [ERROR-SILENT] new catch {} silently swallows Redis invalidation broadcast failures.
    • src/lib/request-filter-engine.ts:158Critical [ERROR-SILENT] new catch {} silently swallows Pub/Sub subscription failures.
    • src/lib/redis/pubsub.ts:89High [ERROR-SILENT] ignored unsubscribe() promise can become an unhandled rejection.
  • Submitted the required PR review summary via gh pr review --comment.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

This PR implements Redis Pub/Sub for cross-process cache invalidation. After comprehensive review across 6 perspectives (Comments, Tests, Error Handling, Types, General Code, and Simplification), no significant issues were identified.

PR Size: M (Medium)

  • Lines changed: 250 (242 additions + 8 deletions)
  • Files changed: 6

Review Coverage

  • Logic and correctness - Clean
  • Security (OWASP Top 10) - Clean
  • Error handling - Clean (consistent with codebase fail-open pattern)
  • Type safety - Clean
  • Documentation accuracy - Clean (comments updated correctly)
  • Test coverage - Adequate (happy path + error cases covered)
  • Code clarity - Good

Key Observations

Architecture Pattern:
The implementation follows the established "fail-open" pattern used consistently throughout the codebase:

  • Graceful degradation when Redis unavailable
  • All errors logged but not thrown
  • System continues functioning in single-worker mode if Redis fails
  • Consistent with 15+ similar patterns in existing code

Error Handling:
All Redis operations properly catch errors and log warnings without throwing. This matches the existing patterns in:

  • src/lib/rate-limit/service.ts (fail-open on Redis errors)
  • src/lib/session-manager.ts (graceful degradation)
  • src/lib/redis/circuit-breaker-state.ts (fail-open strategy)

Testing:
Test coverage includes:

  • Message publishing
  • Subscription and callback invocation
  • Graceful degradation when Redis unavailable
  • Cleanup/unsubscription logic

Memory Management:
Proper cleanup functions implemented in both ErrorRuleDetector and RequestFilterEngine to prevent memory leaks from event listeners.


Automated review by Claude AI

@ding113 ding113 merged commit 1e4c59d into dev Jan 1, 2026
8 checks passed
@github-project-automation github-project-automation bot moved this from Backlog to Done in Claude Code Hub Roadmap Jan 1, 2026
@github-actions github-actions bot mentioned this pull request Jan 1, 2026
10 tasks
@ding113 ding113 deleted the fix/issue-492-redis-pubsub-invalidation branch January 27, 2026 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core bug Something isn't working size/M Medium PR (< 500 lines)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant