[NV-3015] Idempotent API Requests Implementation

[Cliftonz]

Enhance API reliability by introducing Idempotency headers, especially for POST and possibly PATCH HTTP methods. This will ensure requests are processed only once, even in scenarios of resubmissions due to perceived failures.

Benefits

• Improves API reliability and trustworthiness.
• Acts as a precursor to broader system resilience.
• Reduces potential conflicts, data inconsistencies, and duplicate messages.

🚀 Implementation Steps

1. Leveraging Redis for Caching

   • Store idempotency keys, prefixed with organization Id and Environment Id.
   • Set TTL for keys to 24 hours.

2. Interceptor Logic

   • Check if request is of type POST or PATCH.
   • Inspect for Idempotency Header.
   • Interact with Redis cache.
   • Handle potential Redis errors.
   • On return, cache responce.

3. Documentation and Developer Guidelines Update

   • Update API documentation regarding idempotency headers.
   • Provide guidelines on key generation mechanisms.

4. Testing

   1. Idempotency Key Absence:

      • Submit a POST or PATCH request without an Idempotency header.
      • Ensure the request is processed normally without any idempotency guarantees.

   2. Idempotency Key Presence & Cache Miss:

      • Submit a POST request with a unique Idempotency header that hasn't been used.
      • Ensure the request is processed, and the response is cached with the correct TTL in Redis.

   3. Idempotency Key Presence & Cache Hit:

      • Resubmit the POST request from the previous scenario using the same Idempotency header.
      • Confirm the Interceptor fetches the response directly from Redis, bypassing any further processing or database interactions.

   4. Redis Error Handling:

      • Simulate a Redis outage or error.
      • Submit a POST request with an Idempotency header.
      • Ensure the request is processed normally without any idempotency guarantees, and appropriate error handling is in place.

   5. Non-POST Request Processing:

      • Submit a non-POST request (e.g., GET) with an Idempotency header.
      • Confirm the Interceptor does not enforce idempotency and the request is processed as usual.

   6. Testing Specific Endpoints:

      • For all mentioned PATCH endpoints, test the behavior with and without the Idempotency header.
      • Ensure the system behaves correctly, either processing normally or fetching from cache as expected.

_NV-3015

[github-actions]

Hi, I'm Jarvis 🤖%0A%0AI'm a bot built to help you with your contribution to Novu. %0AI will add instructions and guides on how to run the subset of the Novu platform associated to this issue and make your first contribution.%0A%0AThis issue was tagged as related to @novu/api and the related code is located at the apps/api folder, here is how I can help you:%0A%0A

%0A

First time contributing to Novu?

%0A%0A If that's the first time you want to contribute to Novu here are a few simple steps to get you started:%0A 1. Fork the repository and clone your fork to your local machine.%0A 2. Install the dependencies using npm run setup:project.%0A 3. Create a new branch with the number of the issue, for example: 1454-fix-something-cool and start contributing based on the Contributing Guide or the short guide in the section below.%0A 4. Create a Pull request and follow the template of creation %0A

%0A%0A

%0A

Run and test @novu/api locally

%0A%0A ### Run API in watch mode%0A The easiest way to start the API is to run npm run start:api from the root of the repository%0A%0A ### Run API integration tests%0A To validate your changes or simply to run the e2e tests run npm run start:e2e:api. All the e2e tests have the .e2e.ts suffix and usually are located near the controller files of each module.%0A

[mahendraHegde]

@Cliftonz plz assign this to me