-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tajul/keyword async processing #14
Tajul/keyword async processing #14
Conversation
…and isProcessing added boolean field isProcessing made all fields optional other than keyword and isProcessing
transferred search functionality to queue
…y for failed jobs
…evaluate and used tags as constants
cachedResponse: string; | ||
|
||
@Column({ type: 'bool', default: false }) | ||
isProcessed: boolean; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This does not account for a failed status. How about instead using an enum:
enum SearchStatus {
Pending = 'pending',
Completed = 'completed',
Failed = 'failed'
}
@Entity()
export class SearchStat {
...
@Column({ type: 'text' })
status: SearchStatus;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did not consider the failed status, as in the queue used max attempts and exponential backoff delay to make sure the failed job retries continue with exponential backoff. But there can be cases, the job exhausts all the retries and unable to process the search. Should have considered this scenerio
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In that case, when the queued job exhausts all its attempts and is unable to process the search operation, I guess in that scenario I need to add a cron job scheduler to pick all failed jobs and process them again?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In a production application, taking care of the failed jobs could take many forms (retrying them, or sending a report/error), but it is outside of the scope of this exercise.
As for the exponential backoff, still, the status could be marked as failed in the mean time to differentiate it from other jobs 💡
In the above commit
|
…saajan/google-searcher into tajul/keyword-async-processing # Conflicts: # src/search/search.controller.ts # src/search/search.module.ts
…jan/google-searcher into tajul/improve-google-searcher # Conflicts: # package-lock.json # package.json
…an/google-searcher into tajul/consolidate-ui-layouts
…n/google-searcher into tajul/test-coverage
Tajul/test coverage
Tajul/consolidate UI layouts
Tajul/improve google searcher
resolves issue #8