Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bugfix] Move observer and g_idx until after module in onloaded #195

Merged
merged 1 commit into from
Oct 23, 2024
Merged

[Bugfix] Move observer and g_idx until after module in onloaded #195

merged 1 commit into from
Oct 23, 2024

Conversation

kylesayrs
Copy link
Contributor

@kylesayrs kylesayrs commented Oct 22, 2024 •
edited
Loading

Purpose

  • Fix calibration of offloaded models

Changes

  • Previously, g_idx was being fetched before the module was onloaded, meaning that it was being fetch as a meta tensor
    • Now g_idx is fetched after the module parameters are onloaded

Testing

  • @anmarques was able to compress an offloaded model using activation ordering

@kylesayrs kylesayrs requested review from dsikka and horheynm October 22, 2024 18:48
@dsikka dsikka mentioned this pull request Oct 22, 2024
Merged
@kylesayrs kylesayrs self-assigned this Oct 23, 2024
@kylesayrs kylesayrs merged commit 07abbf3 into main Oct 23, 2024
1 check passed
@kylesayrs kylesayrs deleted the kylesayrs/hotfix-g-idx-offloaded branch October 23, 2024 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@horheynm horheynm horheynm approved these changes

@rahul-tuli rahul-tuli rahul-tuli approved these changes

@dsikka dsikka dsikka approved these changes

Assignees

@kylesayrs kylesayrs

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@kylesayrs @dsikka @rahul-tuli @horheynm