-
Notifications
You must be signed in to change notification settings - Fork 136
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rework #146. Fixes various errors relating to input length. #150
Merged
ehsandeep
merged 7 commits into
projectdiscovery:dev
from
justinsteven:bugfix/rework-146
Jun 13, 2022
Merged
Rework #146. Fixes various errors relating to input length. #150
ehsandeep
merged 7 commits into
projectdiscovery:dev
from
justinsteven:bugfix/rework-146
Jun 13, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 173f914.
Fixes projectdiscovery#134 (this was previously fixed but got backed out in landing projectdiscovery#130)
This surfaces the error that's causing projectdiscovery#138: 'bufio.Scanner: token too long'
We're using '...' to indicate that a line has been truncated. If -char-limit was less than the length of this ellipsis string, the scanner would never terminate. Raise an error if the charLimit given to a splitter is <= len('...')
We know we never need more than CharLimit chars from the Scanner in one go First of all, we ensure that the Scanner has a buffer which can hold at least CharLimit chars. Then we handle cases where the Scanner wants more data but we don't need it to get more data. Thus it should never end up in a place where its internal buffer is filled, and it should never return bufio.ErrToLong Fixes projectdiscovery#138
More demos:
|
Mzack9999
added
the
Type: Bug
Inconsistencies or issues which will cause an issue or problem for users or implementors.
label
Jun 12, 2022
Closed
parrasajad
approved these changes
Jun 13, 2022
This was
linked to
issues
Jun 13, 2022
ehsandeep
approved these changes
Jun 13, 2022
Kudos, SonarCloud Quality Gate passed! 0 Bugs No Coverage information |
Mzack9999
approved these changes
Jun 13, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Type: Bug
Inconsistencies or issues which will cause an issue or problem for users or implementors.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reverts/reworks #146 with IMO a more correct approach that doesn't involve resizing the Scanner's buffer based on the size of the input (which fails if input is stdin)
Fixes/obsoletes #148 and #149
Fixes #134 by emitting logged lines to stdout instead of stderr. I had tried to fix this in #130 but it got reverted during the merge. If #134 isn't a bug, or this isn't the right way to fix the bug, please revert the commit.
Fixes a crash or infinite loop where -char-limit is less than the length of the ellipsis we use to indicated truncated lines (thanks to #149 (review) for catching this). There's no open issue for this, but here's a repro:
Fixes #137 by adding a
bufio.SplitFunc
to split long lines in non-bulk modeFinally, fixes #146 by never allowing the Scanner's buffer to fill.
Testing
Test cases
Before #146
After #146
After this patch