Fix 32-bit revcomp failures #8116
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 tasks
mppf
added a commit
that referenced
this pull request
Feb 1, 2018
Add channel.advancePastByte, use it to improve revcomp PR #8045 added some messy rev-comp studies that improve performance by improving the I/O pattern. What the performance comes down to is two things: 1. Copying large chunks of the input from the channel buffer in to the array to be used 2. Using memchr to identify the relevant chunks of the input I experimented with a version that used regexp format strings to replace the memchr call but that had unsatisfying performance. This PR adds channel.advancePastByte in order to enable the expression of the fast I/O pattern in revcomp easily. Now the revcomp version does the following: * "mark" (indicate to the I/O system not to drop the buffer as we might return) * identify the offset of the newline (end of the sequence description) * identify the offset of the > (start of the next sequence) * "revert" (go back to where we "marked") * read the data again in one go with readBytes I'm seeing a 10% speedup for this version beyond revcomp-buf.chpl, and it is much simpler. While there, I noticed that qio_channel_advance might not set up the buffer in some cases, so added code to do that. - [x] full local testing - [x] docs for advancePastByte - [x] update catch statement to specify type - [x] check for 32-bit issues ala #8116 Closes #8105. Reviewed by @benharsh - thanks!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix 32-bit failures by casting to ssize_t and c_intptr.
Tested on linux64, linux32, and OS X.