[BUG] tritonserver not found in Merlin-inference 22.03 image

[IamGianluca]

Hi,

It seems that tritonserver is not installed in the latest stable merlin-inference image (22.03).

==========
== CUDA ==
==========

NVIDIA Release  (build )
CUDA Version 11.6.0.021

Container image Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

NOTE: MOFED driver for multi-node communication was not detected.
      Multi-node communication performance may be reduced.

root@746a74b9ea7b:/# tritonserver --model-repository=/root/models/ --model-control-mode=explicit
bash: tritonserver: command not found

[albert17]

HI @IamGianluca looking into this right now

[albert17]

Found the issue. Working on fixing this issue

[IamGianluca]

Thank you for looking into that @albert17 🙏

[albert17]

Created a PR: NVIDIA-Merlin/Merlin#135

[rnyak]

@IamGianluca you can pull nvcr.io/nvidia/merlin/merlin-pytorch-inference:22.03 image as well. You might need to install torch and torchmetrics in the container.

[IamGianluca]

Good point @rnyak. I should probably use nvcr.io/nvidia/merlin/merlin-pytorch-inference:22.03 since I'm developing in pytorch. I'll try that. What is the reasoning behind not installing pytorch and torchmetrics in that image?

[EvenOldridge]

Hi @IamGianluc. Our inference container uses Triton inference server which can serve a pytorch model without requiring the library. We're going to release unified training and inference containers in the next release which should make this unnecessary.