Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

metric logging on GPTQ path #65

Merged
merged 7 commits into from
Aug 21, 2024
Merged

metric logging on GPTQ path #65

merged 7 commits into from
Aug 21, 2024

Conversation

horheynm
Copy link
Collaborator

@horheynm horheynm commented Aug 7, 2024

SUMMARY:
Metric logging on GPTQ pathway
Includes layer size, GPU usage and timings

TEST PLAN:
Test run of the pathway

Output:

===== Compressing layer 1/22  =====
2024-08-14T21:15:10.224365+0000 | compress_module | INFO - Compressing model.layers.0.model.layers.0.self_attn.q_proj...
2024-08-14T21:15:12.389605+0000 | compress | METRIC - time 2.15
2024-08-14T21:15:12.389919+0000 | compress | METRIC - error 111.42
2024-08-14T21:15:12.390545+0000 | compress | METRIC - GPU 0 | usage: 2.37% | total memory: 80 GB
2024-08-14T21:15:12.390630+0000 | compress | METRIC - GPU 1 | usage: 3.24% | total memory: 80 GB
2024-08-14T21:15:12.390691+0000 | compress | METRIC - GPU 2 | usage: 3.24% | total memory: 80 GB
2024-08-14T21:15:12.390745+0000 | compress | METRIC - GPU 3 | usage: 3.24% | total memory: 80 GB
2024-08-14T21:15:12.390797+0000 | compress | METRIC - GPU 4 | usage: 3.24% | total memory: 80 GB
2024-08-14T21:15:12.390849+0000 | compress | METRIC - GPU 5 | usage: 3.24% | total memory: 80 GB
2024-08-14T21:15:12.390900+0000 | compress | METRIC - GPU 6 | usage: 2.26% | total memory: 80 GB
2024-08-14T21:15:12.390951+0000 | compress | METRIC - GPU 7 | usage: 1.59% | total memory: 80 GB
2024-08-14T21:15:12.391098+0000 | compress | METRIC - Compressed layer size: 8.09375 MB

@horheynm horheynm changed the title init, log metrics on gptq metric logging on GPTQ path Aug 7, 2024
@horheynm horheynm merged commit 9ef530c into main Aug 21, 2024
7 of 12 checks passed
@horheynm horheynm deleted the metric-logging branch August 21, 2024 15:45
markmc pushed a commit to markmc/llm-compressor that referenced this pull request Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants