[Benchmark] Remove ingest results collection #272
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Removes collecting ingest results from the benchmarking tool. On big data sets, this will prevent the process from going out of memory. Signed-off-by: John Mazanec <jmazane@amazon.com>
jmazanec15
added
the
Infrastructure
Codecov Report
@@ Coverage Diff @@
## main #272 +/- ##
=========================================
Coverage 83.38% 83.38%
Complexity 884 884
=========================================
Files 127 127
Lines 3833 3833
Branches 361 361
=========================================
Hits 3196 3196
Misses 475 475
Partials 162 162 Continue to review full report at Codecov.
|
|
Nice finding. Do you know why those responses were introduced in a first place - was it some un-implemented idea or it's a leftover code after some changes in a past? |
| for i in range(0, self.doc_count, self.bulk_size): | ||
| partition = self.dataset.read(self.bulk_size) | ||
| if partition is None: | ||
| break | ||
| body = bulk_transform(partition, self.field_name, action, i) | ||
| result = bulk_index(self.opensearch, self.index_name, body) | ||
| index_responses.append(result) | ||
| bulk_index(self.opensearch, self.index_name, body) |
There was a problem hiding this comment.
Do you think it makes sense to document our decision with a code comment?
|
@martin-gaievski Initially, we collected the responses and parse the took time per request after the loop. If we wanted to do this again, we would need to do the parsing in the loop |
Signed-off-by: John Mazanec <jmazane@amazon.com>
VijayanB
approved these changes
Jan 28, 2022
martin-gaievski
pushed a commit
to martin-gaievski/k-NN
that referenced
this pull request
Mar 7, 2022
Removes collecting ingest results from the benchmarking tool. On big data sets, this will prevent the process from going out of memory. Signed-off-by: John Mazanec <jmazane@amazon.com>
martin-gaievski
pushed a commit
to martin-gaievski/k-NN
that referenced
this pull request
Mar 7, 2022
Removes collecting ingest results from the benchmarking tool. On big data sets, this will prevent the process from going out of memory. Signed-off-by: John Mazanec <jmazane@amazon.com> Signed-off-by: Martin Gaievski <gaievski@amazon.com>
martin-gaievski
pushed a commit
to martin-gaievski/k-NN
that referenced
this pull request
Mar 30, 2022
Removes collecting ingest results from the benchmarking tool. On big data sets, this will prevent the process from going out of memory. Signed-off-by: John Mazanec <jmazane@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: John Mazanec jmazane@amazon.com
Description
When running a benchmark with one of the BIGANN data sets, the tool was killed due to OOM during ingest step:
Looking into the ingest step, I realized the problem was most likely that we were collecting the responses to the index requests. Because we dont use these responses (we just track total time), we can just get rid of this collection.
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.