easy tutorial on linux

[AbhinavJangra29]

Installation Guide

For those running into issues with installation, here’s a streamlined guide! This has been tested on the following container:

• Container: runpod/pytorch:2.0.1-py3.10-cuda11.8.0-devel-ubuntu22.04

Steps

1. Clone the Repository

   git clone <this-repo-url>
   cd <repo-directory>

2. Install Requirements

   pip install -r requirements.txt

3. Install FFmpeg

   sudo apt-get install ffmpeg

4. Install ONNX Runtime

   pip install onnxruntime

Skip the TensorRT Conversion

Note: To avoid building and converting to TensorRT, you can directly download pre-trained weights!

Download the pre-built TensorRT weights using the Hugging Face CLI:

huggingface-cli download mlgawd/trt-weights --local-dir ./checkpoints

Run the Application

Now, you’re ready to run the app in TensorRT mode:

python app.py --mode trt

No need to modify paths or get confused! Just follow these steps and you’re all set. 😎

[warmshao]

Thank you, but the TRT weight is associated with the cuda version and NVDIA GPU, so it cannot be used directly like this.

[AbhinavJangra29]

yes for rtx3090 it works

[Rizwanali324]

yes for rtx3090 it works

have you deploy it as inference API at runpod ?
what was speed at A10 GPu