fix: efficient task trigger queue updates #1489
Conversation
…e if the concurrency limit changes, instead of on every single trigger task call
WalkthroughThe pull request introduces significant updates to the Changes
Possibly related PRs
Suggested reviewers
Warning There were issues while running some tools. Please review the errors and either fix the tool’s configuration or disable the tool if it’s a critical failure. 🔧 eslint (1.23.1)
apps/webapp/app/v3/services/triggerTask.server.ts(node:9963) ESLintIgnoreWarning: The ".eslintignore" file is no longer supported. Switch to using the "ignores" property in "eslint.config.js": https://eslint.org/docs/latest/use/configure/migration-guide#ignoring-files Oops! Something went wrong! :( ESLint: 9.15.0 ESLint couldn't find an eslint.config.(js|mjs|cjs) file. From ESLint v9.0.0, the default configuration file is now eslint.config.js. https://eslint.org/docs/latest/use/configure/migration-guide If you still have problems after following the migration guide, please stop by 📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Outside diff range and nitpick comments (2)
references/v3-catalog/src/trigger/batch.ts (1)
Line range hint
31-54: Several improvements needed inbatchParentWitFailsTask
- There's a typo in the variable name 'respone2'
- Missing return type
- Missing error handling for potential failures
- Consider adding comments explaining the test scenarios
export const batchParentWitFailsTask = task({ id: "batch-parent-with-fails-task", retry: { maxAttempts: 1, }, - run: async () => { + run: async (): Promise<void> => { + // Test scenario 1: Mix of failing and succeeding tasks const response = await taskThatFails.batchTriggerAndWait([ { payload: false }, { payload: true }, { payload: false }, - ]); + ]).catch((error) => { + logger.error("Batch operation failed", { error }); + throw error; + }); logger.info("Batch response", { response }); + // Test scenario 2: Different combination of failing and succeeding tasks - const respone2 = await taskThatFails.batchTriggerAndWait([ + const response2 = await taskThatFails.batchTriggerAndWait([ { payload: true }, { payload: false }, { payload: true }, - ]); + ]).catch((error) => { + logger.error("Second batch operation failed", { error }); + throw error; + }); - logger.info("Batch response2", { respone2 }); + logger.info("Batch response2", { response2 }); }, });apps/webapp/app/v3/services/triggerTask.server.ts (1)
Line range hint
446-466: Enhance error handling and code organization in queue creationWhile the addition of friendly IDs improves queue tracking, there are opportunities to enhance the code:
Consider these improvements:
const queueId = generateFriendlyId("queue"); +try { taskQueue = await tx.taskQueue.create({ data: { friendlyId: queueId, name: queueName, concurrencyLimit, runtimeEnvironmentId: environment.id, projectId: environment.projectId, rateLimit: body.options.queue.rateLimit, type: "NAMED", }, }); if (typeof taskQueue.concurrencyLimit === "number") { await marqs?.updateQueueConcurrencyLimits( environment, taskQueue.name, taskQueue.concurrencyLimit ); } +} catch (error) { + logger.error("Failed to create task queue", { + queueId, + queueName, + error, + }); + throw error; +}
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (3)
apps/webapp/app/v3/services/completeAttempt.server.ts(1 hunks)apps/webapp/app/v3/services/triggerTask.server.ts(3 hunks)references/v3-catalog/src/trigger/batch.ts(1 hunks)
🔇 Additional comments (2)
references/v3-catalog/src/trigger/batch.ts (1)
Line range hint 31-54: Verify the retry configuration
The task has a single retry attempt, but it's triggering multiple failures intentionally. Ensure this aligns with the expected behavior and error handling strategy.
✅ Verification successful
The retry configuration is intentionally set for testing batch failure scenarios
The task's single retry attempt is intentional as it's part of a test suite demonstrating batch task failure handling. This is evidenced by:
- The task
batchParentWitFailsTaskis specifically designed to test failure scenarios, as indicated by its name and the fact that it callstaskThatFailswith intentionally failing payloads. - The child task
taskThatFailshas 2 retry attempts, while the parent task intentionally has 1 attempt to demonstrate how batch operations handle failures. - The test includes both successful and failing scenarios in the batch operations to verify the behavior:
- First batch:
[false, true, false] - Second batch:
[true, false, true]
- First batch:
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for retry configurations across the codebase to verify consistency
rg --type typescript "maxAttempts:" -B 2 -A 2
Length of output: 80
Script:
#!/bin/bash
# Search for retry configurations using ripgrep without file type filter
rg "maxAttempts:" -B 2 -A 2
Length of output: 49370
Script:
#!/bin/bash
# Search for task configurations with retry patterns
ast-grep --pattern 'task({
$$$
retry: {
$$$
},
$$$
})'
Length of output: 19119
Script:
#!/bin/bash
# Look for the specific test file content
cat references/v3-catalog/src/trigger/batch.ts
Length of output: 1884
apps/webapp/app/v3/services/triggerTask.server.ts (1)
417-444: Consider potential race conditions in queue updates
While the optimization to update the queue only when the concurrency limit changes is good, there's a potential race condition when multiple concurrent updates occur to the same queue. The marqs service operations are outside the transaction scope.
Let's verify if there are other places in the codebase where queue updates occur:
Consider these improvements:
- Add optimistic locking to prevent concurrent updates to the same queue
- Consider implementing a queue update queue to serialize updates
- Add logging around queue updates to track potential issues
| @@ -599,7 +617,7 @@ export class TriggerTaskService extends BaseService { | |||
|
|
|||
| const filename = `${pathPrefix}/payload.json`; | |||
|
|
|||
| await uploadToObjectStore(filename, packet.data, packet.dataType, environment); | |||
| await uploadPacketToObjectStore(filename, packet.data, packet.dataType, environment); | |||
There was a problem hiding this comment.
💡 Codebase verification
Inconsistency found in method renaming
The method is still named uploadToObjectStore in both files, but it's being called as uploadPacketToObjectStore in the code snippet. This indicates a mismatch that needs to be corrected.
apps/webapp/app/v3/r2.server.ts: Method is defined asuploadToObjectStoreapps/webapp/app/v3/services/triggerTask.server.ts: Imports and attempts to useuploadPacketToObjectStorebut importsuploadToObjectStore
🔗 Analysis chain
Verify all call sites for renamed upload method
The rename from uploadToObjectStore to uploadPacketToObjectStore improves clarity, but we should verify all call sites have been updated.
Let's check for any remaining references to the old method name:
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for any remaining references to the old method name
rg "uploadToObjectStore"
# Search for any imports of the old method name
ast-grep --pattern 'import { $$$, uploadToObjectStore, $$$ } from'
Length of output: 376
Script:
#!/bin/bash
# Let's check the implementation and usage in both files
echo "=== Content of r2.server.ts ==="
cat apps/webapp/app/v3/r2.server.ts
echo -e "\n=== Content of triggerTask.server.ts ==="
cat apps/webapp/app/v3/services/triggerTask.server.ts
Length of output: 26530
|
@trigger.dev/react-hooks
@trigger.dev/rsc
@trigger.dev/sdk
trigger.dev
@trigger.dev/build
@trigger.dev/core
commit: |
Summary by CodeRabbit
New Features
Improvements
Method Updates