-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Labels
enhancementNew feature or requestNew feature or request
Description
with the changes in how dorado-server processes batches, it is causing performance issues.
I need to change the current static pipeline of
1. get batch
2. mutate batch into compatible structure
3. send that to basecaller
4. wait for it to be basecalled
5. get the reads back from the basecaller and process them
6. send to the writter queue.
to something like
1. get batch
2. mutate to compatible structure
3. send to basecaller
4. check if reads are ready, if so, process them, otherwise repeat 1-3
5. if a batch comes back, process it and send to writter
6. have a max number of batches "in flight" limit for 1-3
This should fix the batch waiting shinanigans caused by the new "optimisations" caused by ONT.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request