Stable Diffusion V2 with torch FX

[anzr299]

Created a new example notebook which Converts the unet model to an Fx graph and quantized it.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[daniil-lyakhov]

@anzr299, let's finish this notebook up!
As it was found by a performance check, unet should work ok in following conditions:

• Use torch.compile(..., backend="openvino") directly on the torch module (do not use torch.torch.export)
• Use ignored scope https://github.com/openvinotoolkit/nncf/pull/2899/files#diff-f5b33dbdaa67dba685ea98a6e1e0d832b09d6fa993aed4c0aefff18a1e1530edR63-R66 because some linear layers cause a error
• Most important - disable nncf patching, as it ruins model performance significantly. To do so you can call unpatch_torch_operators https://github.com/openvinotoolkit/nncf/blob/develop/nncf/torch/dynamic_graph/patch_pytorch.py#L434-L440 at the beginning of the notebook

As my tests shows, int8 version of unet is indeed faster, we could showcase that

[anzr299]

@anzr299, let's finish this notebook up! As it was found by a performance check, unet should work ok in following conditions:

• Use torch.compile(..., backend="openvino") directly on the torch module (do not use torch.torch.export)
• Use ignored scope https://github.com/openvinotoolkit/nncf/pull/2899/files#diff-f5b33dbdaa67dba685ea98a6e1e0d832b09d6fa993aed4c0aefff18a1e1530edR63-R66 because some linear layers cause a error
• Most important - disable nncf patching, as it ruins model performance significantly. To do so you can call unpatch_torch_operators https://github.com/openvinotoolkit/nncf/blob/develop/nncf/torch/dynamic_graph/patch_pytorch.py#L434-L440 at the beginning of the notebook

As my tests shows, int8 version of unet is indeed faster, we could showcase that

Sounds good! and yeah I did suspect point #2 since it was always the linear layers which caused the issue but could not find out why it does that. About point #1, are we not supposed to use torch.compile with torch fx models as well?

[daniil-lyakhov]

Please do

• capture_pre_autograd_graph to capture the GraphModule
• Quantize model with the ignored scope
• Compile it the same way it done for fp32 model (torch.compile(..., backend="openvino"))

[anzr299]

Please do

• capture_pre_autograd_graph to capture the GraphModule
• Quantize model with the ignored scope
• Compile it the same way it done for fp32 model (torch.compile(..., backend="openvino"))

Alright

[github-actions]

This PR will be closed in a week because of 2 weeks of no activity.

[github-actions]

This PR will be closed in a week because of 2 weeks of no activity.

[github-actions]

This PR was closed because it has been stalled for 2 week with no activity.