Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add permissions for ratelimit overrides #2126

Merged
merged 79 commits into from
Nov 18, 2024

Conversation

MichaelUnkey
Copy link
Collaborator

@MichaelUnkey MichaelUnkey commented Sep 23, 2024

What does this PR do?

Fixes # (issue)
eng-1336

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Chore (refactoring code, technical debt, workflow improvements)
  • Enhancement (small improvements)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How should this be tested?

Check if I understood the assignment properly.

Checklist

Required

  • Filled out the "How to test" section in this PR
  • Read Contributing Guide
  • Self-reviewed my own code
  • Commented on my code in hard-to-understand areas
  • Ran pnpm build
  • Ran pnpm fmt
  • Checked for warnings, there are none
  • Removed all console.logs
  • Merged the latest changes from main onto my branch with git pull origin main
  • My changes don't cause any responsiveness issues

Appreciated

  • If a UI change was made: Added a screen recording or screenshots to this PR
  • Updated the Unkey Docs if changes were necessary

Summary by CodeRabbit

  • New Features

    • Introduced new API routes for managing rate limit overrides, including creation, deletion, and retrieval of overrides.
    • Added permissions for managing rate limit overrides in the workspace settings.
    • Enhanced the CreateKey component to improve form handling and UI interactions.
    • Improved logging and error handling for fetching verification data from keys.
  • Bug Fixes

    • Improved error handling for missing parameters in API requests related to rate limits.
  • Tests

    • Added new test cases for the rate limit override functionality, covering both success and error scenarios.
  • Chores

    • Updated workflow configurations for improved job dependencies and execution sequences.
    • Modified package scripts to enhance testing processes.

Copy link

linear bot commented Sep 23, 2024

Copy link

changeset-bot bot commented Sep 23, 2024

⚠️ No Changeset found

Latest commit: 8d82f65

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link

vercel bot commented Sep 23, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
dashboard ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 18, 2024 1:52pm
engineering ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 18, 2024 1:52pm
planetfall ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 18, 2024 1:52pm
play ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 18, 2024 1:52pm
unkey-engineering ❌ Failed (Inspect) Nov 18, 2024 1:52pm
workflows ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 18, 2024 1:52pm
www ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 18, 2024 1:52pm

Copy link
Contributor

coderabbitai bot commented Sep 23, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The changes in this pull request primarily enhance the rate limiting functionality across various components of the application. Key modifications include the addition of new route handlers for managing rate limit overrides, updates to permissions related to rate limits, and adjustments to logging events. The import paths for certain modules have been corrected, and several tests have been added or modified to ensure proper functionality and error handling for the new rate limit features.

Changes

File Change Summary
apps/api/src/worker.ts Updated import path for registerV1RatelimitLimit and added four new rate limit related functions.
apps/dashboard/app/(app)/settings/root-keys/[keyId]/permissions/permissions.ts Added three new permissions (set_override, read_override, delete_override) under the Ratelimit category.
internal/schema/src/auditlog.ts Updated unkeyAuditLogEvents enum by removing three event types and adding three new ones related to rate limits.
packages/api/src/client.ts Commented out four methods related to rate limits in the Unkey class.
apps/dashboard/lib/trpc/routers/ratelimit/createOverride.ts Renamed audit log event for creating a rate limit override.
apps/dashboard/lib/trpc/routers/ratelimit/deleteNamespace.ts Renamed audit log event for deleted rate limit overrides and standardized error handling.
apps/dashboard/lib/trpc/routers/ratelimit/deleteOverride.ts Renamed audit log event for deletion of rate limits.
apps/dashboard/lib/trpc/routers/ratelimit/updateOverride.ts Renamed audit log event for updating rate limits while maintaining existing logic.
apps/api/src/pkg/keys/service.ts Enhanced error handling and logging in the _verifyKey method, particularly for workspace and rate limit management.
apps/api/src/routes/v1_ratelimits_deleteOverride.error.test.ts Introduced a test case for handling requests without namespaceId or namespaceName.
apps/api/src/routes/v1_ratelimits_deleteOverride.happy.test.ts Added a test for successful deletion of rate limit overrides.
apps/api/src/routes/v1_ratelimits_deleteOverride.security.test.ts Introduced tests for role-based access control on the deleteOverride endpoint.
apps/api/src/routes/v1_ratelimits_deleteOverride.ts Added a new route for deleting rate limit overrides with appropriate request validation and error handling.
apps/api/src/routes/v1_ratelimits_getOverride.error.test.ts Introduced a test case for missing namespaceId in the getOverride endpoint.
apps/api/src/routes/v1_ratelimits_getOverride.happy.test.ts Added tests for retrieving rate limit overrides using namespaceId and namespaceName.
apps/api/src/routes/v1_ratelimits_getOverride.security.test.ts Created a test suite for role permissions in the getOverride endpoint.
apps/api/src/routes/v1_ratelimits_getOverride.ts Introduced a new API route for retrieving rate limit overrides with appropriate validation and error handling.
apps/api/src/routes/v1_ratelimits_limit.accuracy.test.ts Updated import paths for rate limit request and response types.
apps/api/src/routes/v1_ratelimits_limit.consistency.test.ts.skipped Updated import paths for rate limit request and response types.
apps/api/src/routes/v1_ratelimits_limit.happy.test.ts Updated import paths for rate limit request and response types.
apps/api/src/routes/v1_ratelimits_limit.overrides.test.ts Updated import paths for rate limit request and response types.
apps/api/src/routes/v1_ratelimits_listOverrides.error.test.ts Introduced a test case for missing namespaceId or namespaceName in the listOverrides endpoint.
apps/api/src/routes/v1_ratelimits_listOverrides.happy.test.ts Added tests for listing rate limit overrides with various scenarios.
apps/api/src/routes/v1_ratelimits_listOverrides.security.test.ts Created a test suite for role permissions in the listOverrides endpoint.
apps/api/src/routes/v1_ratelimits_listOverrides.ts Introduced a new API route for listing rate limit overrides with appropriate validation and error handling.
apps/api/src/routes/v1_ratelimits_setOverride.error.test.ts Added tests for error scenarios in the setOverride endpoint.
apps/api/src/routes/v1_ratelimits_setOverride.happy.test.ts Implemented tests for setting and updating rate limit overrides.
apps/api/src/routes/v1_ratelimits_setOverride.security.test.ts Created a test suite for role permissions in the setOverride endpoint.
apps/api/src/routes/v1_ratelimits_setOverride.ts Introduced a new API route for setting rate limit overrides with appropriate validation and error handling.
apps/dashboard/app/(app)/apis/[apiId]/keys/[keyAuthId]/new/client.tsx Added a useEffect hook to reset form fields related to rate limiting upon component mount.
apps/dashboard/app/(app)/settings/root-keys/new/client.tsx Introduced a useEffect hook to optimize the initialization of selected API permissions.
apps/dashboard/app/auth/sign-in/email-signin.tsx Corrected a typo in a comment related to linting.
apps/dashboard/app/auth/sign-up/email-signup.tsx Enhanced sign-up functionality to process URL parameters and improved error handling.
apps/www/components/shiny-card.tsx Improved formatting of className properties and added mouse position tracking functionality.
internal/clickhouse/src/insert_verifications.test.ts Updated test case to handle new result structure from ClickHouse database.
package.json Modified the test script to enforce single-threaded execution during testing.
packages/api/package.json Added a new pull script for Docker image updates and modified the test script to run the pull script before executing tests.
.github/workflows/deploy.yaml Adjusted job dependencies to streamline the workflow execution sequence.
.github/workflows/build.yaml Modified the Run worker step and added a new Build step to the workflow.
.github/workflows/job_test_api_local.yaml Added a step to delete a specific directory before running tests.
deployment/docker-compose.yaml Commented out configurations for prometheus and grafana services in the deployment setup.
.github/workflows/pr.yaml Updated the test_packages job to use a different workflow file for testing.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant API
    participant Database
    participant Logger

    User->>API: Request to set rate limit override
    API->>Database: Validate namespace and identifier
    Database-->>API: Return validation result
    alt If valid
        API->>Database: Create or update override
        Database-->>API: Return override details
        API->>Logger: Log override action
        Logger-->>API: Confirm log entry
        API-->>User: Respond with success
    else If invalid
        API-->>User: Respond with error
    end
Loading

Possibly related PRs

Suggested labels

Feature, 🕹️ oss.gg, :joystick: 300 points, hacktoberfest

Suggested reviewers

  • mcstepp
  • perkinsjr

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

github-actions bot commented Sep 23, 2024

Thank you for following the naming conventions for pull request titles! 🙏

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (2)
.github/workflows/pr.yaml (2)

11-26: LGTM with suggestions for robustness.

The new test_unit job structure is clean and follows good practices. However, consider these enhancements for better reliability and visibility:

 test_unit:
   name: Unit Test
   runs-on: ubuntu-latest
+  timeout-minutes: 15
   steps:
     - uses: actions/checkout@v4

     - name: Install
       uses: ./.github/actions/install
       with:
         ts: true

     - name: Build
       uses: ./.github/workflows/build.yaml

     - name: Test
       run: pnpm test
+      env:
+        CI: true
+    
+    - name: Upload test results
+      if: always()
+      uses: actions/upload-artifact@v3
+      with:
+        name: test-results
+        path: |
+          coverage/
+          junit.xml

23-23: Consider using an action instead of a workflow for the build step.

Using a workflow file (.github/workflows/build.yaml) as a dependency might cause maintenance issues. Workflows are typically triggered by events, while actions are designed for reuse as steps.

Consider converting the build workflow into a composite action at .github/actions/build for better maintainability and clearer dependency management.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 35bde28 and 81999a7.

📒 Files selected for processing (2)
  • .github/workflows/pr.yaml (1 hunks)
  • .github/workflows/unit_test.yaml (0 hunks)
💤 Files with no reviewable changes (1)
  • .github/workflows/unit_test.yaml
🔇 Additional comments (1)
.github/workflows/pr.yaml (1)

25-26: Verify test coverage for rate limit permissions.

Since this PR introduces new permissions for rate limit overrides, ensure that the test command provides adequate coverage for permission-related functionality.

✅ Verification successful

Let me gather more information about the test coverage and implementation details.


Let me check the actual test implementations for rate limit and permission-related tests.


Based on the test files analysis, I can now provide a final response:

Test coverage for rate limit permissions is comprehensive

The test suite includes extensive coverage for rate limit permissions and functionality:

  • Security tests for rate limit permissions (create, read, update, delete)
  • Happy path tests for rate limit operations
  • Error handling tests for rate limit operations
  • Accuracy tests for rate limit enforcement
  • Integration tests for rate limit verification
  • Tests for rate limit overrides and configurations

Key test files:

  • v1_ratelimits_*.test.ts files cover all CRUD operations
  • v1_keys_verifyKey.ratelimit_accuracy.test.ts verifies rate limiting accuracy
  • v1_ratelimits_limit.accuracy.test.ts tests limit enforcement
  • v1_ratelimits_limit.overrides.test.ts validates override functionality

The test command pnpm test runs all these tests with proper configuration.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for permission-related test files and their coverage

# Look for test files related to permissions and rate limits
echo "=== Searching for relevant test files ==="
rg -l "permission.*test|ratelimit.*test" -g "*.{ts,tsx,js,jsx}"

# Check if there are any permission-related tests in the test command configuration
echo -e "\n=== Checking test configuration ==="
fd -e json "jest.config|vitest.config" -x cat {}

Length of output: 886


Script:

#!/bin/bash
# Check the actual test implementations for rate limit permissions
echo "=== Checking rate limit permission test implementations ==="
rg "rate.*limit.*permission" -A 5 -B 5

# Check for rate limit permission-related code
echo -e "\n=== Checking rate limit permission implementations ==="
ast-grep --pattern 'function $FUNC($$$) {
  $$$
  rateLimit
  $$$
}'

# Check test scripts in package.json
echo -e "\n=== Checking test scripts in package.json ==="
fd "package.json" -x cat {}

Length of output: 36268


Script:

#!/bin/bash
# Check the content of permission and rate limit test files
echo "=== Checking permission test implementations ==="
rg -A 10 "describe|it\(" apps/api/src/routes/v1_permissions_*.test.ts

echo -e "\n=== Checking rate limit test files ==="
rg -l "ratelimit" -g "*.test.{ts,tsx,js,jsx}"

echo -e "\n=== Checking rate limit test implementations ==="
rg -A 10 "describe|it\(" $(rg -l "ratelimit" -g "*.test.{ts,tsx,js,jsx}")

Length of output: 191820

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (1)
apps/api/src/routes/v1_keys_getVerifications.ts (1)

243-246: Consider enriching the response with error information

While the error handling improvement is good, consider enriching the response to include information about which keys failed to retrieve data. This would help clients understand if they're receiving partial data.

Here's a suggested implementation:

export type V1KeysGetVerificationsResponse = z.infer<
  (typeof route.responses)[200]["content"]["application/json"]["schema"]
>;
+type VerificationError = {
+  keyId: string;
+  error: string;
+};

// Update the response schema
const route = createRoute({
  // ... other config
  responses: {
      content: {
        "application/json": {
          schema: z.object({
            verifications: z.array(/* existing schema */),
+           errors: z.array(z.object({
+             keyId: z.string(),
+             error: z.string(),
+           })).optional(),
          }),
        },
      },
    },
  },
});

// In the route handler
const errors: VerificationError[] = [];
const verificationsFromAllKeys = await Promise.all(
  ids.map(({ keyId, keySpaceId }) => {
    // ... existing code
  }),
);

for (const [index, dataPoint] of verificationsFromAllKeys.entries()) {
  if (dataPoint.err) {
    logger.error(dataPoint.err.message);
+   errors.push({
+     keyId: ids[index].keyId,
+     error: dataPoint.err.message,
+   });
    continue;
  }
  // ... existing code
}

return c.json({
  verifications: Object.entries(verifications).map(/* existing code */),
+ errors: errors.length > 0 ? errors : undefined,
});
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between bb27975 and 8d82f65.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (1)
  • apps/api/src/routes/v1_keys_getVerifications.ts (2 hunks)
🔇 Additional comments (1)
apps/api/src/routes/v1_keys_getVerifications.ts (1)

83-83: LGTM: Logger service extraction

The addition of the logger service follows the established pattern of service extraction.

@chronark chronark merged commit e97c730 into main Nov 18, 2024
27 of 28 checks passed
@chronark chronark deleted the eng-1336-custom-override-permissions branch November 18, 2024 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants