Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(worker): fetch next job on failure #2342

Merged
merged 10 commits into from
Jul 19, 2024
Merged

Conversation

roggervalf
Copy link
Collaborator

No description provided.

Copy link
Contributor

@manast manast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a small but a breaking change 😅

this.emit('failed', job, err, 'active');

if (failed) {
const [jobData, jobId, limitUntil, delayUntil] = failed || [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

|| [] is redundant as it is already checking if failed is truthy

Copy link
Collaborator Author

@roggervalf roggervalf Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -766,8 +766,14 @@ will never work with more accuracy than 1ms. */
return;
}

await job.moveToFailed(err, token);
const failed = await job.moveToFailed(err, token, true);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "failed" here is misleading, as this includes the next job to process. Should be called probably "result" or similar.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good to me

@@ -786,7 +792,7 @@ will never work with more accuracy than 1ms. */
const result = await this.callProcessJob(job, token);
return await handleCompleted(result);
} catch (err) {
return handleFailed(<Error>err);
return await handleFailed(<Error>err);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

await here will imply an unhandled exception if handleFailed throws an exception, is this the intent?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was placed when testing, removing it now

Copy link
Contributor

@manast manast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I see this PR more from a consistency side, as failed jobs are a deviation of the normal functioning of the queue and therefore it is not so important to optimize for this path, however having the same code for both completed and failed jobs is beneficial in my opinion.

@roggervalf roggervalf merged commit f917b80 into master Jul 19, 2024
9 of 10 checks passed
@roggervalf roggervalf deleted the fetch-next-on-failed branch July 19, 2024 13:07
github-actions bot pushed a commit that referenced this pull request Jul 19, 2024
## [5.10.2](v5.10.1...v5.10.2) (2024-07-19)

### Performance Improvements

* **worker:** fetch next job on failure ([#2342](#2342)) ([f917b80](f917b80))
@@ -634,7 +634,7 @@ export class Job<
err: E,
token: string,
fetchNext = false,
): Promise<void> {
): Promise<void | any[]> {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀 The @returns comment still says only void is returned.

github-actions bot pushed a commit that referenced this pull request Aug 14, 2024
## [5.12.6](v5.12.5...v5.12.6) (2024-08-14)

### Bug Fixes

* **job:** change moveToFinished return type to reflect jobData ([#2706](#2706)) ref [#2342](#2342) ([de094a3](de094a3))
* **stalled:** support removeDependencyOnFailure option when job is stalled ([#2708](#2708)) ([e0d3790](e0d3790))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants