chore(performance): file-source testing and benchmarks
#6742
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
binarylogic
changed the title
file-sourcefile source
lib/file-source/src/file_watcher.rs
Outdated
Comment on lines
261
to
267
| /// The return is unusual. In the Err case this function has not written into | ||
| /// `buf` and the caller should not examine its contents. In the Ok case if the | ||
| /// inner value is None the caller should retry the call as the buffering read | ||
| /// hit the end of the buffer but did not find a `delim` yet, indicating that | ||
| /// we've sheered a write or that there were no bytes available in the `reader` | ||
| /// and the `reader` was very sure about it. If the inner value is Some the | ||
| /// interior `usize` is the number of bytes written into `buf`. |
There was a problem hiding this comment.
There's more context on why it's in this unusual state in the description of #4089. Very much a classic "we'll clean this up soon" 🙂
There was a problem hiding this comment.
A good friend of mine was prone to say "there's nothing permanent like temporary". 😂
blt
changed the title
file sourcefile-source testing and benchmarks
blt
added
source: file
blt
requested review from
a team,
bruceg and
ktff
and removed request for
a team
ktff
reviewed
Mar 13, 2021
ktff
approved these changes
Mar 15, 2021
| @@ -0,0 +1 @@ | |||
|
|
|||
There was a problem hiding this comment.
We sure do not. Just junk I accidentally left in-tree.
This commit is the start of a process to address #6730. The major changes introduced in this commit so far are application of clippy suggestions and model checks of the `read_until_with_max_size` test. I have a pretty good idea of how that function works now and will be introducing benchmarks. Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
In this commit I have introduce a benchmark for the read_until etc function. To do this I've had to make it part of the public API of the crate, but since the crate sits inside a larger project I'm less chuffed about this. I have fiddled with the test layout some as well. Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
This crate looks to be challenging to upgrade. Best to do once tokio is updated in this project. Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
|
A merge conflict has summoned me, just came to confirm: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR commit is the start of a process to address #6730 by way of implementation of RFC 3480. The major changes introduced here are criterion benchmarks for an internal component of the
file-sourcecrate and modification of the testing regime to make use of Quickcheck. The results are nothing ground shaking: the new model tests do not turn up any flaw in theread_until...function and the benchmarks show it to be reasonably fast. That's good. Now we know. The more recent indications from vector-test-harness are that our issue centers not on any individually expensive operation but on the cooperative, fair scheduling of this crate. Approaches to that are beyond the scope of this PR.Signed-off-by: Brian L. Troutwine brian@troutwine.us