Refactoring in Pipeline, old Executor + name lookup improvement in old OpGraph. #5495
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Michał Zientkiewicz <mzient@gmail.com>
Signed-off-by: Michał Zientkiewicz <mzient@gmail.com>
Signed-off-by: Michał Zientkiewicz <mzient@gmail.com>
|
CI MESSAGE: [15531112]: BUILD STARTED |
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
mzient
changed the title
|
CI MESSAGE: [15531112]: BUILD FAILED |
mzient
commented
Jun 3, 2024
| @@ -113,7 +114,7 @@ TensorNode& OpGraph::PlaceNewTensor() { | |||
| } | |||
|
|
|||
|
|
|||
| void OpGraph::AddOp(const OpSpec &spec, const std::string &op_name) { | |||
| OpNode &OpGraph::AddOp(const OpSpec &spec, const std::string &op_name) { | |||
There was a problem hiding this comment.
This is necessary in a follow-up PR.
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
|
CI MESSAGE: [15531540]: BUILD STARTED |
mzient
commented
Jun 4, 2024
| for (auto &node : op_nodes_) { | ||
| if (node.instance_name == name) { | ||
| return node; | ||
| } | ||
| } | ||
| DALI_FAIL("Operator node with name " + name + " not found."); | ||
| DALI_FAIL(make_string("Operator node with name ", name, " not found.")); |
There was a problem hiding this comment.
For very dubious reasons, C++ doesn't define operator+ for string_view.
mzient
commented
Jun 4, 2024
| @@ -467,8 +469,9 @@ class DLL_PUBLIC OpGraph { | |||
| */ | |||
| void RemoveOpNode(OpNodeId id); | |||
|
|
|||
| std::map<std::string, TensorNodeId> tensor_name_to_id_; | |||
| std::map<std::string, TensorNodeId, std::less<>> tensor_name_to_id_; | |||
There was a problem hiding this comment.
Passing std::less<> allows use of any comparable key for find and operator[] - in this case, a string_view.
See example - look for "transparent comparison".
mzient
commented
Jun 4, 2024
Comment on lines
-86
to
-90
| template <typename WorkspacePolicy, typename QueuePolicy> | ||
| void PipelinedExecutorImpl<WorkspacePolicy, QueuePolicy>::Build(OpGraph *graph, | ||
| vector<string> output_names) { | ||
| Executor<WorkspacePolicy, QueuePolicy>::Build(graph, output_names); | ||
| } |
There was a problem hiding this comment.
Useless override removed.
mzient
commented
Jun 4, 2024
| @@ -571,8 +571,6 @@ void Pipeline::Build(std::vector<PipelineOutputDesc> output_descs) { | |||
| } | |||
| } | |||
|
|
|||
| graph_.InstantiateOperators(); | |||
There was a problem hiding this comment.
That's the 1st thing that Executor::Build does.
|
CI MESSAGE: [15531540]: BUILD PASSED |
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
|
CI MESSAGE: [15556390]: BUILD STARTED |
stiepan
reviewed
Jun 4, 2024
szkarpinski
reviewed
Jun 4, 2024
stiepan
reviewed
Jun 4, 2024
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
szkarpinski
approved these changes
Jun 4, 2024
stiepan
approved these changes
Jun 4, 2024
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
|
CI MESSAGE: [15559469]: BUILD STARTED |
|
CI MESSAGE: [15559469]: BUILD PASSED |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Category:
Refactoring (Redesign of existing code that doesn't affect functionality)
Description:
This PR does some API adjustments that will work better with the new graph.
There are some name lookup improvements in (old) OpGraph and Pipeline and there's a transition to querying by string_view as opposed to
const string &- should work better with new OpGraph and, as a bonus, avoid creating temporary strings in C API.Additional information:
Affected modules and functionalities:
Key points relevant for the review:
Tests:
Checklist
Documentation
DALI team only
Requirements
REQ IDs: N/A
JIRA TASK: DALI-3974