feat(execution-queuing): async api mode + ratelimiting by subscription tier #702
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
1 Skipped Deployment
|
|
✅ No security or compliance issues detected. Reviewed everything up to d0bbad5. Security Overview
Detected Code ChangesThe diff is too large to display a summary of code changes. Reply to this PR with |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Greptile Summary
This PR implements a significant infrastructure change to support asynchronous workflow execution with tiered rate limiting. The key components are:
- A job queuing system that allows workflows to be executed either synchronously (immediate results) or asynchronously (background processing)
- Rate limiting based on subscription tiers with different limits for sync/async execution
- New APIs for job status monitoring and rate limit checking
- Refactored instrumentation to properly handle different runtime environments
The implementation includes proper database migrations, atomic operations for job state changes, comprehensive error handling, and extensive test coverage. The system is designed to prevent resource exhaustion while providing differentiated service levels based on subscription tiers.
Confidence score: 5/5
- This PR is very safe to merge due to its comprehensive test coverage and careful handling of edge cases
- The code shows excellent attention to detail in areas like atomic operations, error handling, and proper resource cleanup
- The migration files and queue management code deserve extra attention during review due to their critical nature
23 files reviewed, 26 comments
Edit PR Review Bot Settings | Greptile
apps/sim/instrumentation-edge.ts
Outdated
| const logger = createLogger('EdgeInstrumentation') | ||
|
|
||
| export async function register() { | ||
| console.log('[Edge Instrumentation] register() called') |
There was a problem hiding this comment.
style: using console.log here while logger.info is used below - should be consistent
| } | ||
| } | ||
|
|
||
| // TODO: Implement task cancellation via Trigger.dev API if needed |
| @@ -1,9 +1,32 @@ | |||
| export async function register() { | |||
| console.log('[Main Instrumentation] register() called, environment:', { | |||
There was a problem hiding this comment.
remove the logging here since it'll crowd the logs and is only useful for our debugging
…n tier (simstudioai#702) * v1 queuing system * working async queue * working impl of sync + async request formats * fix tests * fix rate limit calc * fix rate limiting issues * regen migration * fix test * fix instrumentation script issues * remove use workflow queue env var * make modal have async examples * Remove conflicting 54th migration before merging staging * new migration files * remove console log * update modal correctly * working sync executor * works for sync * remove useless stats endpoint * fix tests * add sync exec timeout * working impl with cron job * migrate to trigger.dev * remove migration * remove unused code * update readme * restructure jobs API response * add logging for async execs * improvement: example ui/ux * use getBaseUrl() func --------- Co-authored-by: Waleed Latif <walif6@gmail.com> Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu>
Description
We will have the following Rate Limits
Features Added
Notes:
Synchronous Execution: Direct execution in API route for immediate results. System limits only protected by rate limits.
Rate Limiting: Applied to all non-manual trigger types (API, webhook, schedule)
Type of change
How Has This Been Tested?
Test curling deployed APIs with async + sync. Write stress tests for different kinds of workflows.
Checklist:
bun run test)Security Considerations: