Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bugfix] Move observer and g_idx until after module in onloaded #195

Merged
merged 1 commit into from
Oct 23, 2024

Conversation

kylesayrs
Copy link
Contributor

@kylesayrs kylesayrs commented Oct 22, 2024

Purpose

  • Fix calibration of offloaded models

Changes

  • Previously, g_idx was being fetched before the module was onloaded, meaning that it was being fetch as a meta tensor
    • Now g_idx is fetched after the module parameters are onloaded

Testing

  • @anmarques was able to compress an offloaded model using activation ordering

@kylesayrs kylesayrs self-assigned this Oct 23, 2024
@kylesayrs kylesayrs merged commit 07abbf3 into main Oct 23, 2024
1 check passed
@kylesayrs kylesayrs deleted the kylesayrs/hotfix-g-idx-offloaded branch October 23, 2024 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants