forked from openvinotoolkit/openvino
-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add unsqueeze and transpose patterns #178
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
ca5745d
Add floor and squeeze patterns
kurapov-peter 14850dc
Add shape_of conversion
kurapov-peter 1f2099c
fixup! Add shape_of conversion
kurapov-peter de62f71
Add gather pattern
kurapov-peter 1cde2e6
Add slice pattern
kurapov-peter 45737b9
Enable slice and gather patterns
kurapov-peter d1eb382
Add concat pattern
kurapov-peter 80ea073
Add unsqueeze pattern
kurapov-peter fcd72be
Add transpose pattern
kurapov-peter 5133aca
fixup! Add transpose pattern
kurapov-peter 2b2d34f
Merge branch 'mlir' into pakurapo/unsqueeze-transpose
slyalin File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
54 changes: 54 additions & 0 deletions
54
src/common/transformations/src/transformations/mlir/op/transpose.cpp
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
// Copyright (C) 2018-2024 Intel Corporation | ||
// SPDX-License-Identifier: Apache-2.0 | ||
// | ||
|
||
#include "mlir/Dialect/Linalg/Passes.h" | ||
#include "mlir/Dialect/Tensor/IR/Tensor.h" | ||
|
||
#include <openvino/op/transpose.hpp> | ||
#include "openvino/pass/pattern/op/wrap_type.hpp" | ||
|
||
#include "transpose.hpp" | ||
#include "../convert_common.hpp" | ||
|
||
namespace { | ||
|
||
using namespace ov::mlir; | ||
|
||
struct ConvertTranspose { | ||
void operator()(ConversionContext& context, NodePtr node) { | ||
auto loc = createLocation(context.context, node); | ||
auto& builder = context.builder(); | ||
const auto input = context.getInputs(node)[0]; | ||
// TODO: support dynamic inputs | ||
// const auto order = context.getInputs(node)[1]; | ||
|
||
const auto ov_output_element_type = node->get_output_element_type(0); | ||
const auto ov_output_shape = node->get_output_partial_shape(0); | ||
auto out_type = importTensor(context.context, ov_output_shape, ov_output_element_type); | ||
auto dynamic_dimensions = context.get_dynamic_dimension_values(ov_output_shape); | ||
|
||
auto const_order = dynamic_cast<ov::op::v0::Constant*>(node->get_input_node_ptr(1)); | ||
assert(const_order && "non-const order not supported"); | ||
ov::Coordinate coords = const_order->get_coordinate_val(); | ||
SmallVector<int64_t> order(coords.begin(), coords.end()); | ||
|
||
auto empty = builder.create<tensor::EmptyOp>(loc, out_type, dynamic_dimensions); | ||
auto transpose = builder.create<linalg::TransposeOp>(loc, input, empty, order); | ||
context.addOutputs(node, transpose); | ||
} | ||
}; | ||
|
||
} // namespace | ||
|
||
namespace ov { | ||
namespace mlir { | ||
|
||
using namespace ov::pass::pattern; | ||
using namespace ov::op; | ||
|
||
TransposePattern::TransposePattern() : MarkPattern(wrap_type<v1::Transpose>({any_input(), any_input()}), ConvertTranspose()) {} | ||
|
||
} // namespace mlir | ||
} // namespace ov | ||
|
23 changes: 23 additions & 0 deletions
23
src/common/transformations/src/transformations/mlir/op/transpose.hpp
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
// Copyright (C) 2018-2024 Intel Corporation | ||
// SPDX-License-Identifier: Apache-2.0 | ||
// | ||
|
||
#pragma once | ||
|
||
#include "mlir/IR/Builders.h" | ||
#include "mlir/IR/MLIRContext.h" | ||
#include "mlir/IR/Value.h" | ||
|
||
#include "../conversion_context.hpp" | ||
|
||
namespace ov { | ||
namespace mlir { | ||
|
||
class TransposePattern : public MarkPattern { | ||
public: | ||
OPENVINO_RTTI("TransposePattern", "0"); | ||
TransposePattern(); | ||
}; | ||
|
||
} // namespace mlir | ||
} // namespace ov |
75 changes: 75 additions & 0 deletions
75
src/common/transformations/src/transformations/mlir/op/unsqueeze.cpp
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
// Copyright (C) 2018-2024 Intel Corporation | ||
// SPDX-License-Identifier: Apache-2.0 | ||
// | ||
|
||
#include "mlir/Dialect/Shape/IR/Shape.h" | ||
#include "mlir/Dialect/Linalg/Passes.h" | ||
#include "mlir/Dialect/Tensor/IR/Tensor.h" | ||
|
||
#include <openvino/op/constant.hpp> | ||
#include <openvino/op/unsqueeze.hpp> | ||
#include "openvino/pass/pattern/op/wrap_type.hpp" | ||
|
||
#include "unsqueeze.hpp" | ||
#include "../convert_common.hpp" | ||
|
||
|
||
namespace { | ||
|
||
using namespace ov::mlir; | ||
|
||
struct ConvertUnsqueeze { | ||
void operator()(ConversionContext& context, NodePtr node) { | ||
auto loc = createLocation(context.context, node); | ||
auto& builder = context.builder(); | ||
const auto input = context.getInputs(node)[0]; | ||
// TODO: support dynamic inputs | ||
// const auto axes = context.getInputs(node)[1]; | ||
|
||
const auto ov_output_element_type = node->get_output_element_type(0); | ||
const auto ov_input_shape = node->get_input_partial_shape(0); | ||
|
||
assert(ov_input_shape.rank().is_static() && "expecting static output shape"); | ||
|
||
auto const_axes = dynamic_cast<ov::op::v0::Constant*>(node->get_input_node_ptr(1)); | ||
assert(const_axes && "non-const axes not supported"); | ||
ov::Coordinate coords = const_axes->get_coordinate_val(); | ||
|
||
// Calculate the resulting shape. | ||
// E.g., for an input tensor<4x2xf32> and axes [0, 2] (tensor<2xi64>) need to build a shape 1x4x1x2 | ||
SmallVector<ReassociationIndices> expand_groups; | ||
ReassociationIndices group = ReassociationIndices(); | ||
SmallVector<int64_t> shape(coords.size() + ov_input_shape.rank().get_length()); | ||
for (size_t input_idx = 0, coord_idx = 0, i = 0; i < shape.size(); ++i) { | ||
group.push_back(i); | ||
if (coord_idx < coords.size() && i == coords[coord_idx]) { | ||
shape[i] = 1; | ||
coord_idx++; | ||
} else { | ||
const auto& dim = ov_input_shape[input_idx]; | ||
shape[i] = dim.is_dynamic() ? ShapedType::kDynamic : dim.get_length(); | ||
input_idx++; | ||
expand_groups.push_back(group); | ||
group = ReassociationIndices(); | ||
} | ||
} | ||
|
||
auto result_type = RankedTensorType::get(shape, importPrecision(context.context, ov_output_element_type)); | ||
auto expand_shape = builder.create<tensor::ExpandShapeOp>(loc, result_type, input, expand_groups); | ||
context.addOutputs(node, expand_shape); | ||
} | ||
}; | ||
|
||
} // namespace | ||
|
||
namespace ov { | ||
namespace mlir { | ||
|
||
using namespace ov::pass::pattern; | ||
using namespace ov::op; | ||
|
||
UnsqueezePattern::UnsqueezePattern() : MarkPattern(wrap_type<v0::Unsqueeze>({any_input(), any_input()}), ConvertUnsqueeze()) {} | ||
|
||
} // namespace mlir | ||
} // namespace ov | ||
|
24 changes: 24 additions & 0 deletions
24
src/common/transformations/src/transformations/mlir/op/unsqueeze.hpp
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
// Copyright (C) 2018-2024 Intel Corporation | ||
// SPDX-License-Identifier: Apache-2.0 | ||
// | ||
|
||
#pragma once | ||
|
||
#include "mlir/IR/Builders.h" | ||
#include "mlir/IR/MLIRContext.h" | ||
#include "mlir/IR/Value.h" | ||
|
||
#include "../conversion_context.hpp" | ||
|
||
namespace ov { | ||
namespace mlir { | ||
|
||
class UnsqueezePattern : public MarkPattern { | ||
public: | ||
OPENVINO_RTTI("UnsqueezePattern", "0"); | ||
UnsqueezePattern(); | ||
}; | ||
|
||
} // namespace mlir | ||
} // namespace ov | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why cannot we just take it as
node->get_output_partial_shape(0)
? I believe the shape in MLIR should match the shape from OV.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be always available? (dynamic shapes?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd still need to generate the reassociation groups anyway though
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd still need to generate the reassociation groups anyway though
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
get_output_partial_shape
always returns shape, including dynamic shapes, which means a part of the dimensions can be dynamic and you already know how to detect them --dim.is_dynamic()
. But I agree that you still need to build reassociation groups.