Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass XSMM runner lib to MLIR execution engine #147

Merged
merged 1 commit into from
Jul 25, 2024

Conversation

adam-smnk
Copy link
Collaborator

Manually finds and passes XSMM runner library to the MLIR JIT engine to resolve missing TPP xsmm_* symbols when executing from Python.

Works only for Linux currently.

Tested with:

import torch
import torch.nn as nn
import openvino as ov

# Define a synthetic model
class LinearModel(nn.Module):
    def __init__(self, input_size, output_size):
        super(LinearModel, self).__init__()
        self.model = nn.Sequential(
            nn.Linear(input_size, output_size),
            nn.ReLU(),
        )
    def forward(self, a):
        return self.model(a)

# Create an instance of the model
input_size = 1024
output_size = 128
model = LinearModel(input_size, output_size)
# Generate random weights
model.model[0].weight.data.normal_(0, 0.01)
model.model[0].bias.data.fill_(0.01)

input_data = torch.tensor(range(1, input_size*output_size+1)).to(torch.float32)\
    .view(output_size, input_size)

with torch.no_grad():
    reference = model(input_data)
    print('Reference:\n', reference)

# Dynamic shapes
# ov_model = ov.convert_model(model, example_input=input_data)
# Static shapes
ov_model = ov.convert_model(model, input=(ov.Shape([output_size, input_size]), ov.Type.f32))

# expect lots of debug output with MLIR fragments before and after lowering in this call:
ov_compiled = ov.compile_model(ov_model, 'CPU')

tested = ov_compiled(input_data)[0]
print('Tested:\n', tested)
diff = reference - tested
print('Reference - Tested:\n', diff)

Manually finds and passes XSMM runner library to the MLIR JIT engine
to resolve missing TPP xsmm_* symbols when executing from Python.

Works only for Linux currently.
@adam-smnk
Copy link
Collaborator Author

Still a bit hacky but the same lib finding mechanism seems to be used for OV plugins.
Long term there must be a better solution but it should work "out of the box" with Python for now.

Copy link
Collaborator

@rengolin rengolin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Later this could be moved to some sys library, but it should be fine for now, thanks!

@adam-smnk adam-smnk merged commit a510e01 into slyalin:mlir Jul 25, 2024
13 of 29 checks passed
@slyalin
Copy link
Owner

slyalin commented Jul 25, 2024

Fails in both C++ and Python with:

LLVM ERROR: out of memory
Allocation failed
Aborted (core dumped)

static std::string get_xsmm_runner_path() {
#if defined(__APPLE__) || defined(__linux__) || defined(__EMSCRIPTEN__)
// TODO: Add multiplatform shared library search support.
Dl_info info;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adam-smnk, it fails to compile without TPP_MLIR. Should be under #ifdef TPP_MLIR as well.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants