Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PT2] MinMax #3166

Open
wants to merge 16 commits into
base: develop
Choose a base branch
from

Conversation

AlexanderDokuchaev
Copy link
Collaborator

@AlexanderDokuchaev AlexanderDokuchaev commented Dec 23, 2024

Changes

Introduce TORCH2 backend
MinMax algorithms for torch2 backend
Add handle_torch_function for quantization function to trace it by torch_function

Related tickets

152996

@github-actions github-actions bot added NNCF PT Pull requests that updates NNCF PyTorch NNCF Common Pull request that updates NNCF Common experimental NNCF PTQ Pull requests that updates NNCF PTQ API Public API-impacting changes labels Dec 23, 2024
@AlexanderDokuchaev AlexanderDokuchaev marked this pull request as ready for review December 25, 2024 09:58
@AlexanderDokuchaev AlexanderDokuchaev requested a review from a team as a code owner December 25, 2024 09:58
Copy link
Collaborator

@daniil-lyakhov daniil-lyakhov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job!
It would be great if you would separate the small typo fixes from the min-max implementation, it's too much small changes for me (and probably for others as well)

from nncf.experimental.torch2.function_hook.nncf_graph.nncf_graph_builder import GraphModelWrapper


class PT2Engine(Engine):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we reuse PTEngine as it done for the TorchFX backend?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can , but what a reason?
I have try to keep one way dependence torch2 from torch

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was asked to remove the PTFXEngine by @alexsu52, await the same comment from him

Comment on lines +65 to +66
def _apply_insertion_transformation(
self, model: nn.Module, transformations: List[PT2InsertionCommand]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about introduction of transformations as it done for torchFX? https://github.com/openvinotoolkit/nncf/blob/develop/nncf/experimental/torch/fx/model_transformer.py#L195-L208

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't see any advantages in this approach (using function instead of commands), but it makes development and debugging much more difficult.
This discuss should be part of refactoring of ModelTransformer,

Copy link
Collaborator

@daniil-lyakhov daniil-lyakhov Dec 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I have to ask so be sure PT2 and FX backends are aligned. I'll approve it, but what're the difficulties with transformations? Why closures worse for the debug?

tests/torch2/utils.py Show resolved Hide resolved
@daniil-lyakhov
Copy link
Collaborator

Are conformance tests numbers available?

@AlexanderDokuchaev
Copy link
Collaborator Author

Model metric FQ int8 metric (N) FQ (N) int8 (N)
torchvision/resnet18 0.6945 30 21 0.69488 30 21
timm/mobilenetv3_small_050 0.4261 62 36 0.4192 62 36
timm/deit3_small_patch16_224 0.8126 74 50 0.8126 74 50
timm/crossvit_9_240 0.7275 112 88 0.72746 112 88
hf/bert-base-uncased   74 77   74 77

(N) - experimental tracing without F/BC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Public API-impacting changes experimental NNCF Common Pull request that updates NNCF Common NNCF PT Pull requests that updates NNCF PyTorch NNCF PTQ Pull requests that updates NNCF PTQ
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants