Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[subclasses] Use __slots__ for micro optim of flatten/unflatten #1211

Open
wants to merge 3 commits into
base: gh/IvanKobzarev/3/base
Choose a base branch
from
Open

[subclasses] Use __slots__ for micro optim of flatten/unflatten #1211

wants to merge 3 commits into from

Commits on Nov 1, 2024

  1. [subclasses] Use __slots__ for micro optim of flatten/unflatten 

    [ghstack-poisoned]
    @IvanKobzarev
    IvanKobzarev committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    746c10d View commit details
    Browse the repository at this point in the history

  2. Update on "[subclasses] Use __slots__ for micro optim of flatten/unfl… 

    …atten"


Profiling the case from pytorch/pytorch#129457 found that using __slots__ a bit helps to reduce the cost of flatten (14us -> 11us). As a result 20 fp8 quantized weights flattening gets -40us  (300us -> 260us).



[ghstack-poisoned]
    @IvanKobzarev
    IvanKobzarev committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    c583a12 View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2024

  1. Update on "[subclasses] Use __slots__ for micro optim of flatten/unfl… 

    …atten"


Profiling the case from pytorch/pytorch#129457 found that using __slots__ a bit helps to reduce the cost of flatten (14us -> 11us). As a result 20 fp8 quantized weights flattening gets -40us  (300us -> 260us).



[ghstack-poisoned]
    @IvanKobzarev
    IvanKobzarev committed Nov 4, 2024
    Configuration menu
    Copy the full SHA
    f274127 View commit details
    Browse the repository at this point in the history