Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix the crashing when there are zero-size inputs #120

Merged

Conversation

yongbinfeng
Copy link
Contributor

PR to fix the crashing when some of the PyTorch inputs have zero size. One example here:

triton-inference-server/server#5972

The error seems tbe be because of Torch::from_blob function for copying inputs not working when the tensor is zero-size. The fix is to add the special handle in this case, directly creating a zero-size tensor, instead of calling the PyTorch function.

Tested and it seems to have fixed the crashing.

Tabrizian
Tabrizian previously approved these changes Dec 15, 2023
Copy link
Member

@Tabrizian Tabrizian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! I think I just need to add some testing and run it through the CI otherwise it is looking good.

@Tabrizian Tabrizian merged commit d900538 into triton-inference-server:main Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants