Introducing TensorRT lazy export and caching option with trt_compile()

[borisfom]

What does this PR do ?

Introducing TensorRT lazy export and caching option with trt_compile():

• TRT engine build happens on first forward() call, using either ONNX->TRT or Torch-TensorRT and the inputs passed to forward()
• Dynamic dimensions can be specified either via 'input_profiles' argument (full-blowt TRT optimization profile values) or 'dynamic_batchsize' argument (batch dimension min/opt/max only).
• The engine is serialized to the disk for reuse

See inline documentation in the source for more details.

Usage

            args = {"method": "torch_trt"}
            model = trt_compile(
                model,
                "path_to_save_engine",
                args=args,
            )

[meatybobby]

@borisfom could you resolve the alert from CodeQL? If it's intended, you can dismiss it.

[github-actions]

beep boop 🤖: 🚨 The following files must be fixed before merge!

---

Your code was analyzed with PyLint. The following annotations have been identified:

------------------------------------
Your code has been rated at 10.00/10

---

Thank you for improving NeMo's documentation!

[github-actions]

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

------------------------------------
Your code has been rated at 10.00/10

---

Thank you for improving NeMo's documentation!