❓ [Question] The stable diffusion example doesn't work

[0-chan-kor]

❓ Question

What you have already tried

https://github.com/pytorch/TensorRT/blob/main/examples/dynamo/torch_compile_stable_diffusion.py

I tried executing the above Python code, but conversion to TensorRT failed as shown below.

WARNING:torch_tensorrt.dynamo.backend.backends:TRT conversion failed on the subgraph. See trace above. Returning GraphModule forward instead.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/torch_tensorrt/dynamo/backend/backends.py", line 93, in _pretraced_backend
    trt_compiled = compile_module(
  File "/usr/local/lib/python3.10/dist-packages/torch_tensorrt/dynamo/compile.py", line 244, in compile_module
    trt_module = convert_module(
  File "/usr/local/lib/python3.10/dist-packages/torch_tensorrt/dynamo/conversion/conversion.py", line 33, in convert_module
    module_outputs = module(*torch_inputs)
  File "/usr/local/lib/python3.10/dist-packages/torch/fx/graph_module.py", line 726, in call_wrapped
    return self._wrapped_call(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/fx/graph_module.py", line 305, in __call__
    raise e
  File "/usr/local/lib/python3.10/dist-packages/torch/fx/graph_module.py", line 292, in __call__
    return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1519, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1528, in _call_impl
    return forward_call(*args, **kwargs)
  File "<eval_with_key>.14", line 6, in forward
    view_10 = torch.ops.aten.view.default(permute_10, [2, -1, 320]);  permute_10 = None
  File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 499, in __call__
    return self._op(*args, **kwargs or {})
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_stats.py", line 20, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_subclasses/fake_tensor.py", line 1323, in __torch_dispatch__
    return self.dispatch(func, types, args, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_subclasses/fake_tensor.py", line 1621, in dispatch
    r = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 499, in __call__
    return self._op(*args, **kwargs or {})
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

Is this an example python that actually passes? Or is there an environment version that needs to be set for this example?

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

I used the latest version of the pytorch container, nvcr.io/nvidia/pytorch:23.11-py3, and pip installed the latest versions of diffusers and transformers.

Additional context

None

[gs-olive]

Hello - this issue was fixed in #2464, which likely did not make it into the 23.11 container. I believe the issue shows up in newer transformers/diffusers versions and may not appear with transformers==4.33.2, diffusers==0.21.4.

For the newest distribution of Torch-TRT which includes this fix, the main-branch Docker container (ghcr.io/pytorch/tensorrt/torch_tensorrt:main) or nightly
distribution (pip install --pre torch torch_tensorrt --extra-index-url https://download.pytorch.org/whl/nightly/cu121) can be used.

[VlaDanilov]

I have the same problem with the night distribution