[BUG][ENV] WIndows requires two runs of setup.py before success

[giradkar26]

Error in quantizing custom model
ValueError: Trying to use the cuda backend, but could not import the C++/CUDA dependencies with the following error: No module named 'gptqmodel_cuda_64'

(gptq_new) D:\amar\qwen2-vl-2>nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Wed_Feb__8_05:53:42_Coordinated_Universal_Time_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0

Software Info

Operation System - Windows
Python Version - 3.10

torch==2.5.1+cu121
torchaudio==2.5.1+cu121
torchvision==0.20.1+cu121

(gptq_f) D:\amar\qwen2-vl-2>pip show gptqmodel torch transformers accelerate triton
WARNING: Package(s) not found: triton
Name: gptqmodel
Version: 1.4.5.dev0+cpu
Summary: A LLM quantization package with user-friendly apis. Based on GPTQ algorithm.
Home-page: https://github.com/ModelCloud/GPTQModel
Author: ModelCloud
Author-email: qubitium@modelcloud.ai
License:
Location: c:\users\amarg\anaconda3\envs\gptq_f\lib\site-packages
Requires: accelerate, datasets, device-smi, numpy, packaging, protobuf, safetensors, sentencepiece, setuptools, threadpoolctl, torch, transformers
Required-by:
---
Name: torch
Version: 2.5.1+cu121
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: packages@pytorch.org
License: BSD-3-Clause
Location: c:\users\amarg\anaconda3\envs\gptq_f\lib\site-packages
Requires: filelock, fsspec, jinja2, networkx, sympy, typing-extensions
Required-by: accelerate, compressed-tensors, gptqmodel, torchaudio, torchvision, vllm
---
Name: transformers
Version: 4.47.1
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: c:\users\amarg\anaconda3\envs\gptq_f\lib\site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by: compressed-tensors, gptqmodel, vllm
---
Name: accelerate
Version: 1.2.1
Summary: Accelerate
Home-page: https://github.com/huggingface/accelerate
Author: The HuggingFace team
Author-email: zach.mueller@huggingface.co
License: Apache
Location: c:\users\amarg\anaconda3\envs\gptq_f\lib\site-packages
Requires: huggingface-hub, numpy, packaging, psutil, pyyaml, safetensors, torch
Required-by: gptqmodel

Installation

# clone repo
git clone https://github.com/ModelCloud/GPTQModel.git && cd GPTQModel
pip install -v . --no-build-isolation

For first attempt of installation, following error occur every time. Where as on 2nd attempt it successfully build.
First attempt log

  Using cached idna-3.10-py3-none-any.whl (70 kB)
  Using cached MarkupSafe-3.0.2-cp310-cp310-win_amd64.whl (15 kB)
  Using cached mpmath-1.3.0-py3-none-any.whl (536 kB)
  Using cached multidict-6.1.0-cp310-cp310-win_amd64.whl (28 kB)
  Using cached propcache-0.2.1-cp310-cp310-win_amd64.whl (44 kB)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
  Using cached pytz-2024.2-py2.py3-none-any.whl (508 kB)
  Using cached tzdata-2024.2-py2.py3-none-any.whl (346 kB)
  Using cached urllib3-2.2.3-py3-none-any.whl (126 kB)
  Using cached yarl-1.18.3-cp310-cp310-win_amd64.whl (90 kB)
  Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
  Using cached six-1.17.0-py2.py3-none-any.whl (11 kB)
  Installing collected packages: sentencepiece, pytz, mpmath, xxhash, urllib3, tzdata, typing-extensions, threadpoolctl, sympy, six, setuptools, safetensors, regex, pyyaml, pyarrow, psutil, protobuf, propcache, packaging, numpy, networkx, MarkupSafe, idna, fsspec, frozenlist, filelock, dill, device-smi, colorama, charset-normalizer, certifi, attrs, async-timeout, aiohappyeyeballs, tqdm, requests, python-dateutil, multiprocess, multidict, jinja2, aiosignal, yarl, torch, pandas, huggingface-hub, tokenizers, aiohttp, accelerate, transformers, datasets
    Attempting uninstall: setuptools
      Found existing installation: setuptools 75.1.0
      Uninstalling setuptools-75.1.0:
        Successfully uninstalled setuptools-75.1.0
  Successfully installed MarkupSafe-3.0.2 accelerate-1.2.1 aiohappyeyeballs-2.4.4 aiohttp-3.11.11 aiosignal-1.3.2 async-timeout-5.0.1 attrs-24.3.0 certifi-2024.12.14 charset-normalizer-3.4.0 colorama-0.4.6 datasets-3.2.0 device-smi-0.3.2 dill-0.3.8 filelock-3.16.1 frozenlist-1.5.0 fsspec-2024.9.0 huggingface-hub-0.27.0 idna-3.10 jinja2-3.1.4 mpmath-1.3.0 multidict-6.1.0 multiprocess-0.70.16 networkx-3.4.2 numpy-2.2.0 packaging-24.2 pandas-2.2.3 propcache-0.2.1 protobuf-5.29.2 psutil-6.1.0 pyarrow-18.1.0 python-dateutil-2.9.0.post0 pytz-2024.2 pyyaml-6.0.2 regex-2024.11.6 requests-2.32.3 safetensors-0.4.5 sentencepiece-0.2.0 setuptools-75.6.0 six-1.17.0 sympy-1.13.1 threadpoolctl-3.5.0 tokenizers-0.21.0 torch-2.5.1 tqdm-4.67.1 transformers-4.47.1 typing-extensions-4.12.2 tzdata-2024.2 urllib3-2.2.3 xxhash-3.5.0 yarl-1.18.3
  C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\dist.py:294: InformationOnly: Normalizing '1.4.5-dev+cpu' to '1.4.5.dev0+cpu'
    'extras_require': dict,
  running egg_info
  creating C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-ga9n7tqa\gptqmodel.egg-info
  writing C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-ga9n7tqa\gptqmodel.egg-info\PKG-INFO
  writing dependency_links to C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-ga9n7tqa\gptqmodel.egg-info\dependency_links.txt
  writing requirements to C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-ga9n7tqa\gptqmodel.egg-info\requires.txt
  writing top-level names to C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-ga9n7tqa\gptqmodel.egg-info\top_level.txt
  writing manifest file 'C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-ga9n7tqa\gptqmodel.egg-info\SOURCES.txt'
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "D:\amar\qwen2-vl-2\GPTQModel\setup.py", line 254, in <module>
      setup(
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\__init__.py", line 117, in setup
      return distutils.core.setup(**attrs)
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\_distutils\core.py", line 183, in setup
      return run_commands(dist)
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\_distutils\core.py", line 199, in run_commands
      dist.run_commands()
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\_distutils\dist.py", line 954, in run_commands
      self.run_command(cmd)
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\dist.py", line 950, in run_command

    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\_distutils\dist.py", line 973, in run_command
      cmd_obj.run()
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\command\egg_info.py", line 311, in run
      self.delete_file(nl)
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\command\egg_info.py", line 319, in find_sources
      mm.ignore_egg_info_dir = self.ignore_egg_info_in_manifest
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\command\egg_info.py", line 540, in run
      def run(self) -> None:
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\command\egg_info.py", line 578, in add_defaults
      """
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\command\sdist.py", line 108, in add_defaults
      def add_defaults(self) -> None:
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\_distutils\command\sdist.py", line 236, in add_defaults
      self._add_defaults_python()
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\command\sdist.py", line 119, in _add_defaults_python
      if self.distribution.has_pure_modules():
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\_distutils\cmd.py", line 302, in get_finalized_command
      cmd_obj = self.distribution.get_command_obj(command, create)
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\_distutils\dist.py", line 846, in get_command_obj
      klass = self.get_command_class(command)
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\dist.py", line 697, in get_command_class
      ep.load()(self, ep.name, value)
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\importlib\metadata\__init__.py", line 171, in load
      module = import_module(match.group('module'))
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\importlib\__init__.py", line 126, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
    File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
    File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 883, in exec_module
    File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
    File "C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\command\build_py.py", line 15, in <module>
      from .._path import StrPath, StrPathT
  ImportError: cannot import name 'StrPathT' from 'setuptools._path' (C:\Users\amarg\anaconda3\envs\gpt_n\lib\site-packages\setuptools\_path.py)
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\Users\amarg\anaconda3\envs\gpt_n\python.exe' -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize

  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)

  __file__ = %r
  sys.argv[0] = __file__

  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"

  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'D:\\amar\\qwen2-vl-2\\GPTQModel\\setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base 'C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-ga9n7tqa'
  cwd: D:\amar\qwen2-vl-2\GPTQModel\\
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Second attempt

  adding 'gptqmodel/utils/vllm.py'
  adding 'gptqmodel/utils/vram.py'
  adding 'gptqmodel-1.4.5.dev0+cpu.dist-info/LICENSE'
  adding 'gptqmodel-1.4.5.dev0+cpu.dist-info/METADATA'
  adding 'gptqmodel-1.4.5.dev0+cpu.dist-info/WHEEL'
  adding 'gptqmodel-1.4.5.dev0+cpu.dist-info/top_level.txt'
  adding 'gptqmodel-1.4.5.dev0+cpu.dist-info/RECORD'
  removing build\bdist.win-amd64\wheel
  Building wheel for gptqmodel (setup.py) ... done
  Created wheel for gptqmodel: filename=gptqmodel-1.4.5.dev0+cpu-py3-none-any.whl size=251810 sha256=8fea3c72b4f68ebd09d8350ee7976b639a8505f1225c843a9157d017df5cd0d8
  Stored in directory: C:\Users\amarg\AppData\Local\Temp\pip-ephem-wheel-cache-j7c23dhf\wheels\84\a7\91\6963c29cba2e993b933992897a5c70ede53834f52291426ed6
Successfully built gptqmodel
Installing collected packages: gptqmodel
Successfully installed gptqmodel-1.4.5.dev0+cpu

Script to quantize

### Qwen2-VL GPTQ 4bit Quantization code

from datasets import load_dataset
from transformers import AutoTokenizer
from gptqmodel import GPTQModel, QuantizeConfig, get_best_device
import ast

# Define paths
model_id = r"D:\amar\qwen2-vl-2\vinplate2-3000-merged"
quant_path = r"D:\amar\qwen2-vl-2\vinplate2new"
txt_file_path = r'D:\amar\qwen2-vl-2\dataset_caliber.txt'

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id)

text_list = []
with open(txt_file_path, 'r') as file:
    for line in file:
        data = ast.literal_eval(line.strip())
        for entry in data:
            if entry.get('role') == 'user':
                for content in entry.get('content', []):
                    if content.get('type') == 'text':
                        text_list.append(content['text'])


# Get calibration dataset
calibration_dataset = [tokenizer(text) for text in text_list]

# Get quantization configuration
quant_config = QuantizeConfig(bits=4, group_size=128)

# Load Custom Qwen2-VL model
model = GPTQModel.load(model_id, quant_config)

# Quantize the model
model.quantize(calibration_dataset)

# Save the quantized model
model.save(quant_path)

Expected behavior

I am trying to quantize the custom trained model but getting following error.
Let me know resolve for this.

Error log

INFO - {'layer': 25, 'module': 'mlp.down_proj', 'loss': '0.23591', 'damp': '0.01000', 'time': '7.703'}
INFO - {'layer': 26, 'module': 'self_attn.k_proj', 'loss': '0.04152', 'damp': '0.01000', 'time': '0.839'}
INFO - {'layer': 26, 'module': 'self_attn.v_proj', 'loss': '0.14001', 'damp': '0.01000', 'time': '0.967'}
INFO - {'layer': 26, 'module': 'self_attn.q_proj', 'loss': '0.27224', 'damp': '0.01000', 'time': '0.900'}
INFO - {'layer': 26, 'module': 'self_attn.o_proj', 'loss': '0.07098', 'damp': '0.01000', 'time': '0.984'}
INFO - {'layer': 26, 'module': 'mlp.up_proj', 'loss': '2.77056', 'damp': '0.01000', 'time': '3.064'}
INFO - {'layer': 26, 'module': 'mlp.gate_proj', 'loss': '2.61525', 'damp': '0.01000', 'time': '1.168'}
INFO - {'layer': 26, 'module': 'mlp.down_proj', 'loss': '7.26243', 'damp': '0.01000', 'time': '7.990'}
INFO - {'layer': 27, 'module': 'self_attn.k_proj', 'loss': '0.04172', 'damp': '0.01000', 'time': '0.840'}
INFO - {'layer': 27, 'module': 'self_attn.v_proj', 'loss': '0.14870', 'damp': '0.01000', 'time': '0.744'}
INFO - {'layer': 27, 'module': 'self_attn.q_proj', 'loss': '0.28783', 'damp': '0.01000', 'time': '0.808'}
INFO - {'layer': 27, 'module': 'self_attn.o_proj', 'loss': '0.33237', 'damp': '0.01000', 'time': '0.807'}
INFO - {'layer': 27, 'module': 'mlp.up_proj', 'loss': '4.53520', 'damp': '0.01000', 'time': '1.233'}
INFO - {'layer': 27, 'module': 'mlp.gate_proj', 'loss': '4.80280', 'damp': '0.01000', 'time': '1.125'}
INFO - {'layer': 27, 'module': 'mlp.down_proj', 'loss': '1.42427', 'damp': '0.01000', 'time': '7.726'}
INFO - Packing model...
Traceback (most recent call last):
  File "D:\amar\qwen2-vl-2\quant.py", line 37, in <module>
    model.quantize(calibration_dataset)
  File "C:\Users\amarg\anaconda3\envs\gptq_f\lib\site-packages\gptqmodel\models\base.py", line 701, in quantize
    self.qlinear_kernel = pack_model(
  File "C:\Users\amarg\anaconda3\envs\gptq_f\lib\site-packages\gptqmodel\utils\model.py", line 372, in pack_model
    make_quant(
  File "C:\Users\amarg\anaconda3\envs\gptq_f\lib\site-packages\gptqmodel\utils\model.py", line 149, in make_quant
    result = create_quant_layer(linear, bits, desc_act, dynamic, group_size, module, names, sym, device)
  File "C:\Users\amarg\anaconda3\envs\gptq_f\lib\site-packages\gptqmodel\utils\model.py", line 204, in create_quant_layer
    new_layer = QuantLinear(
  File "C:\Users\amarg\anaconda3\envs\gptq_f\lib\site-packages\gptqmodel\nn_modules\qlinear\dynamic_cuda.py", line 53, in __init__
    raise ValueError(
ValueError: Trying to use the cuda backend, but could not import the C++/CUDA dependencies with the following error: No module named 'gptqmodel_cuda_64'
 Quantizing mlp.down_proj in layer 27 of 27 |----------------------------------------| 100.0%

[Qubitium]

@giradkar26

You need to check if your windows cuda toolkit is correctly enabled

import torch

print(torch.cuda.is_available())

[Qubitium]

The error is strange because you are missing a cuda kernel that is part of setup. Did you get errors during install?

also your cuda version 12.1, is very old and not compatible with Pytorch 2.5.1. Please use 12.4 or higher with torch 2.5.1. Please download and install a newer version of cuda-toolkit for windows.

[giradkar26]

import torch

DEVICE = "cuda:0" if torch.cuda.is_available() else "cpu"
print('GPU device : ',DEVICE)

which is True

[giradkar26]

The error is strange because you are missing a cuda kernel that is part of setup. Did you get errors during install?

also your cuda version 12.1, is very old and not compatible with Pytorch 2.5.1. Please use 12.4 or higher with torch 2.5.1. Please download and install a newer version of cuda-toolkit for windows.

I got error on first attempt of installation as i mentioned above. On 2nd attempt it built successfully.

• I will try cuda 12.4 and let u know

[Qubitium]

• I will try cuda 12.4 and let u know

The core problem is for whatever reason, the setup.py did not correctly complete the cuda compile leaving your system with a broken pkg that had no cuda kernel compiled.

[giradkar26]

In new environment,
I have installed cuda 12.4 via windows exe.
then torch via : pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
then

git clone https://github.com/ModelCloud/GPTQModel.git && cd GPTQModel
pip install -v . --no-build-isolation

still same errror

(gptq_new) D:\amar\qwen2-vl-2\liby\GPTQModel>nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Feb_27_16:28:36_Pacific_Standard_Time_2024
Cuda compilation tools, release 12.4, V12.4.99
Build cuda_12.4.r12.4/compiler.33961263_0

torch==2.5.1+cu124
torchaudio==2.5.1+cu124
torchvision==0.20.1+cu124

first attempt error logs

(gptq_new) D:\amar\qwen2-vl-2\liby\GPTQModel>pip install -v . --no-build-isolation
Using pip 24.2 from C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\pip (python 3.10)
Processing d:\amar\qwen2-vl-2\liby\gptqmodel
  Running command python setup.py egg_info
  Collecting accelerate>=1.2.1 (from -r requirements.txt (line 1))
    Using cached accelerate-1.2.1-py3-none-any.whl.metadata (19 kB)
  Collecting datasets>=3.1.0 (from -r requirements.txt (line 2))
    Using cached datasets-3.2.0-py3-none-any.whl.metadata (20 kB)
  Collecting numpy>=1.26.4 (from -r requirements.txt (line 3))
    Using cached numpy-2.2.0-cp310-cp310-win_amd64.whl.metadata (60 kB)
  Requirement already satisfied: torch>=2.0.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 4)) (2.5.1+cu124)
  Collecting safetensors>=0.4.5 (from -r requirements.txt (line 5))
    Using cached safetensors-0.4.5-cp310-none-win_amd64.whl.metadata (3.9 kB)
  Collecting transformers>=4.46.3 (from -r requirements.txt (line 6))
    Using cached transformers-4.47.1-py3-none-any.whl.metadata (44 kB)
  Collecting threadpoolctl>=3.5.0 (from -r requirements.txt (line 7))
    Using cached threadpoolctl-3.5.0-py3-none-any.whl.metadata (13 kB)
  Collecting packaging>=24.2 (from -r requirements.txt (line 8))
    Using cached packaging-24.2-py3-none-any.whl.metadata (3.2 kB)
  Collecting setuptools>=75.5.0 (from -r requirements.txt (line 9))
    Using cached setuptools-75.6.0-py3-none-any.whl.metadata (6.7 kB)
  Collecting device-smi==0.3.2 (from -r requirements.txt (line 10))
    Using cached device_smi-0.3.2-py3-none-any.whl
  Collecting sentencepiece>=0.2.0 (from -r requirements.txt (line 11))
    Using cached sentencepiece-0.2.0-cp310-cp310-win_amd64.whl.metadata (8.3 kB)
  Collecting protobuf>=5.29.1 (from -r requirements.txt (line 12))
    Using cached protobuf-5.29.2-cp310-abi3-win_amd64.whl.metadata (592 bytes)
  Collecting psutil (from accelerate>=1.2.1->-r requirements.txt (line 1))
    Using cached psutil-6.1.0-cp37-abi3-win_amd64.whl.metadata (23 kB)
  Collecting pyyaml (from accelerate>=1.2.1->-r requirements.txt (line 1))
    Using cached PyYAML-6.0.2-cp310-cp310-win_amd64.whl.metadata (2.1 kB)
  Collecting huggingface-hub>=0.21.0 (from accelerate>=1.2.1->-r requirements.txt (line 1))
    Using cached huggingface_hub-0.27.0-py3-none-any.whl.metadata (13 kB)
  Requirement already satisfied: filelock in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (3.13.1)
  Collecting pyarrow>=15.0.0 (from datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached pyarrow-18.1.0-cp310-cp310-win_amd64.whl.metadata (3.4 kB)
  Collecting dill<0.3.9,>=0.3.0 (from datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached dill-0.3.8-py3-none-any.whl.metadata (10 kB)
  Collecting pandas (from datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached pandas-2.2.3-cp310-cp310-win_amd64.whl.metadata (19 kB)
  Collecting requests>=2.32.2 (from datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
  Collecting tqdm>=4.66.3 (from datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached tqdm-4.67.1-py3-none-any.whl.metadata (57 kB)
  Collecting xxhash (from datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached xxhash-3.5.0-cp310-cp310-win_amd64.whl.metadata (13 kB)
  Collecting multiprocess<0.70.17 (from datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached multiprocess-0.70.16-py310-none-any.whl.metadata (7.2 kB)
  Requirement already satisfied: fsspec<=2024.9.0,>=2023.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from fsspec[http]<=2024.9.0,>=2023.1.0->datasets>=3.1.0->-r requirements.txt (line 2)) (2024.2.0)
  Collecting aiohttp (from datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached aiohttp-3.11.11-cp310-cp310-win_amd64.whl.metadata (8.0 kB)
  Requirement already satisfied: typing-extensions>=4.8.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (4.9.0)
  Requirement already satisfied: networkx in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (3.2.1)
  Requirement already satisfied: jinja2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (3.1.3)
  Requirement already satisfied: sympy==1.13.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (1.13.1)
  Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from sympy==1.13.1->torch>=2.0.0->-r requirements.txt (line 4)) (1.3.0)
  Collecting regex!=2019.12.17 (from transformers>=4.46.3->-r requirements.txt (line 6))
    Using cached regex-2024.11.6-cp310-cp310-win_amd64.whl.metadata (41 kB)
  Collecting tokenizers<0.22,>=0.21 (from transformers>=4.46.3->-r requirements.txt (line 6))
    Using cached tokenizers-0.21.0-cp39-abi3-win_amd64.whl.metadata (6.9 kB)
  Collecting aiohappyeyeballs>=2.3.0 (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached aiohappyeyeballs-2.4.4-py3-none-any.whl.metadata (6.1 kB)
  Collecting aiosignal>=1.1.2 (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached aiosignal-1.3.2-py2.py3-none-any.whl.metadata (3.8 kB)
  Collecting async-timeout<6.0,>=4.0 (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached async_timeout-5.0.1-py3-none-any.whl.metadata (5.1 kB)
  Collecting attrs>=17.3.0 (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached attrs-24.3.0-py3-none-any.whl.metadata (11 kB)
  Collecting frozenlist>=1.1.1 (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached frozenlist-1.5.0-cp310-cp310-win_amd64.whl.metadata (14 kB)
  Collecting multidict<7.0,>=4.5 (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached multidict-6.1.0-cp310-cp310-win_amd64.whl.metadata (5.1 kB)
  Collecting propcache>=0.2.0 (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached propcache-0.2.1-cp310-cp310-win_amd64.whl.metadata (9.5 kB)
  Collecting yarl<2.0,>=1.17.0 (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached yarl-1.18.3-cp310-cp310-win_amd64.whl.metadata (71 kB)
  Collecting charset-normalizer<4,>=2 (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached charset_normalizer-3.4.0-cp310-cp310-win_amd64.whl.metadata (34 kB)
  Collecting idna<4,>=2.5 (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached idna-3.10-py3-none-any.whl.metadata (10 kB)
  Collecting urllib3<3,>=1.21.1 (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached urllib3-2.2.3-py3-none-any.whl.metadata (6.5 kB)
  Collecting certifi>=2017.4.17 (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached certifi-2024.12.14-py3-none-any.whl.metadata (2.3 kB)
  Collecting colorama (from tqdm>=4.66.3->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached colorama-0.4.6-py2.py3-none-any.whl.metadata (17 kB)
  Requirement already satisfied: MarkupSafe>=2.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from jinja2->torch>=2.0.0->-r requirements.txt (line 4)) (2.1.5)
  Collecting python-dateutil>=2.8.2 (from pandas->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
  Collecting pytz>=2020.1 (from pandas->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached pytz-2024.2-py2.py3-none-any.whl.metadata (22 kB)
  Collecting tzdata>=2022.7 (from pandas->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached tzdata-2024.2-py2.py3-none-any.whl.metadata (1.4 kB)
  Collecting six>=1.5 (from python-dateutil>=2.8.2->pandas->datasets>=3.1.0->-r requirements.txt (line 2))
    Using cached six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB)
  Using cached accelerate-1.2.1-py3-none-any.whl (336 kB)
  Using cached datasets-3.2.0-py3-none-any.whl (480 kB)
  Using cached numpy-2.2.0-cp310-cp310-win_amd64.whl (12.9 MB)
  Using cached safetensors-0.4.5-cp310-none-win_amd64.whl (285 kB)
  Using cached transformers-4.47.1-py3-none-any.whl (10.1 MB)
  Using cached threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
  Using cached packaging-24.2-py3-none-any.whl (65 kB)
  Using cached setuptools-75.6.0-py3-none-any.whl (1.2 MB)
  Using cached sentencepiece-0.2.0-cp310-cp310-win_amd64.whl (991 kB)
  Using cached protobuf-5.29.2-cp310-abi3-win_amd64.whl (434 kB)
  Using cached dill-0.3.8-py3-none-any.whl (116 kB)
  Using cached aiohttp-3.11.11-cp310-cp310-win_amd64.whl (442 kB)
  Using cached huggingface_hub-0.27.0-py3-none-any.whl (450 kB)
  Using cached multiprocess-0.70.16-py310-none-any.whl (134 kB)
  Using cached pyarrow-18.1.0-cp310-cp310-win_amd64.whl (25.1 MB)
  Using cached PyYAML-6.0.2-cp310-cp310-win_amd64.whl (161 kB)
  Using cached regex-2024.11.6-cp310-cp310-win_amd64.whl (274 kB)
  Using cached requests-2.32.3-py3-none-any.whl (64 kB)
  Using cached tokenizers-0.21.0-cp39-abi3-win_amd64.whl (2.4 MB)
  Using cached tqdm-4.67.1-py3-none-any.whl (78 kB)
  Using cached pandas-2.2.3-cp310-cp310-win_amd64.whl (11.6 MB)
  Using cached psutil-6.1.0-cp37-abi3-win_amd64.whl (254 kB)
  Using cached xxhash-3.5.0-cp310-cp310-win_amd64.whl (30 kB)
  Using cached aiohappyeyeballs-2.4.4-py3-none-any.whl (14 kB)
  Using cached aiosignal-1.3.2-py2.py3-none-any.whl (7.6 kB)
  Using cached async_timeout-5.0.1-py3-none-any.whl (6.2 kB)
  Using cached attrs-24.3.0-py3-none-any.whl (63 kB)
  Using cached certifi-2024.12.14-py3-none-any.whl (164 kB)
  Using cached charset_normalizer-3.4.0-cp310-cp310-win_amd64.whl (102 kB)
  Using cached frozenlist-1.5.0-cp310-cp310-win_amd64.whl (51 kB)
  Using cached idna-3.10-py3-none-any.whl (70 kB)
  Using cached multidict-6.1.0-cp310-cp310-win_amd64.whl (28 kB)
  Using cached propcache-0.2.1-cp310-cp310-win_amd64.whl (44 kB)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
  Using cached pytz-2024.2-py2.py3-none-any.whl (508 kB)
  Using cached tzdata-2024.2-py2.py3-none-any.whl (346 kB)
  Using cached urllib3-2.2.3-py3-none-any.whl (126 kB)
  Using cached yarl-1.18.3-cp310-cp310-win_amd64.whl (90 kB)
  Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
  Using cached six-1.17.0-py2.py3-none-any.whl (11 kB)
  Installing collected packages: sentencepiece, pytz, xxhash, urllib3, tzdata, threadpoolctl, six, setuptools, safetensors, regex, pyyaml, pyarrow, psutil, protobuf, propcache, packaging, numpy, multidict, idna, frozenlist, dill, device-smi, colorama, charset-normalizer, certifi, attrs, async-timeout, aiohappyeyeballs, yarl, tqdm, requests, python-dateutil, multiprocess, aiosignal, pandas, huggingface-hub, aiohttp, tokenizers, accelerate, transformers, datasets
    Attempting uninstall: setuptools
      Found existing installation: setuptools 75.1.0
      Uninstalling setuptools-75.1.0:
        Successfully uninstalled setuptools-75.1.0
    Attempting uninstall: numpy
      Found existing installation: numpy 1.26.3
      Uninstalling numpy-1.26.3:
        Successfully uninstalled numpy-1.26.3
  Successfully installed accelerate-1.2.1 aiohappyeyeballs-2.4.4 aiohttp-3.11.11 aiosignal-1.3.2 async-timeout-5.0.1 attrs-24.3.0 certifi-2024.12.14 charset-normalizer-3.4.0 colorama-0.4.6 datasets-3.2.0 device-smi-0.3.2 dill-0.3.8 frozenlist-1.5.0 huggingface-hub-0.27.0 idna-3.10 multidict-6.1.0 multiprocess-0.70.16 numpy-2.2.0 packaging-24.2 pandas-2.2.3 propcache-0.2.1 protobuf-5.29.2 psutil-6.1.0 pyarrow-18.1.0 python-dateutil-2.9.0.post0 pytz-2024.2 pyyaml-6.0.2 regex-2024.11.6 requests-2.32.3 safetensors-0.4.5 sentencepiece-0.2.0 setuptools-75.6.0 six-1.17.0 threadpoolctl-3.5.0 tokenizers-0.21.0 tqdm-4.67.1 transformers-4.47.1 tzdata-2024.2 urllib3-2.2.3 xxhash-3.5.0 yarl-1.18.3
  C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\dist.py:294: InformationOnly: Normalizing '1.4.5-dev' to '1.4.5.dev0'
    'extras_require': dict,
  conda_cuda_include_dir C:\Users\amarg\anaconda3\envs\gptq_new\Lib\site-packages\nvidia/cuda_runtime/include
  running egg_info
  creating C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-hbz2usy3\gptqmodel.egg-info
  writing C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-hbz2usy3\gptqmodel.egg-info\PKG-INFO
  writing dependency_links to C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-hbz2usy3\gptqmodel.egg-info\dependency_links.txt
  writing requirements to C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-hbz2usy3\gptqmodel.egg-info\requires.txt
  writing top-level names to C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-hbz2usy3\gptqmodel.egg-info\top_level.txt
  writing manifest file 'C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-hbz2usy3\gptqmodel.egg-info\SOURCES.txt'
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "D:\amar\qwen2-vl-2\liby\GPTQModel\setup.py", line 254, in <module>
      setup(
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\__init__.py", line 117, in setup
      return distutils.core.setup(**attrs)
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\_distutils\core.py", line 183, in setup
      return run_commands(dist)
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\_distutils\core.py", line 199, in run_commands
      dist.run_commands()
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\_distutils\dist.py", line 954, in run_commands
      self.run_command(cmd)
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\dist.py", line 950, in run_command

    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\_distutils\dist.py", line 973, in run_command
      cmd_obj.run()
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\command\egg_info.py", line 311, in run
      self.delete_file(nl)
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\command\egg_info.py", line 319, in find_sources
      mm.ignore_egg_info_dir = self.ignore_egg_info_in_manifest
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\command\egg_info.py", line 540, in run
      def run(self) -> None:
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\command\egg_info.py", line 578, in add_defaults
      """
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\command\sdist.py", line 108, in add_defaults
      def add_defaults(self) -> None:
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\_distutils\command\sdist.py", line 236, in add_defaults
      self._add_defaults_python()
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\command\sdist.py", line 119, in _add_defaults_python
      if self.distribution.has_pure_modules():
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\_distutils\cmd.py", line 302, in get_finalized_command
      cmd_obj = self.distribution.get_command_obj(command, create)
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\_distutils\dist.py", line 846, in get_command_obj
      klass = self.get_command_class(command)
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\dist.py", line 697, in get_command_class
      ep.load()(self, ep.name, value)
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\importlib\metadata\__init__.py", line 171, in load
      module = import_module(match.group('module'))
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\importlib\__init__.py", line 126, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
    File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
    File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 883, in exec_module
    File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
    File "C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\command\build_py.py", line 15, in <module>
      from .._path import StrPath, StrPathT
  ImportError: cannot import name 'StrPathT' from 'setuptools._path' (C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\_path.py)
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\Users\amarg\anaconda3\envs\gptq_new\python.exe' -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize

  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)

  __file__ = %r
  sys.argv[0] = __file__

  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"

  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'D:\\amar\\qwen2-vl-2\\liby\\GPTQModel\\setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base 'C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-hbz2usy3'
  cwd: D:\amar\qwen2-vl-2\liby\GPTQModel\\
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

2nd attempt log

(gptq_new) D:\amar\qwen2-vl-2\liby\GPTQModel>pip install -v . --no-build-isolation
Using pip 24.2 from C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\pip (python 3.10)
Processing d:\amar\qwen2-vl-2\liby\gptqmodel
  Running command python setup.py egg_info
  Requirement already satisfied: accelerate>=1.2.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 1)) (1.2.1)
  Requirement already satisfied: datasets>=3.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 2)) (3.2.0)
  Requirement already satisfied: numpy>=1.26.4 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 3)) (2.2.0)
  Requirement already satisfied: torch>=2.0.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 4)) (2.5.1+cu124)
  Requirement already satisfied: safetensors>=0.4.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 5)) (0.4.5)
  Requirement already satisfied: transformers>=4.46.3 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 6)) (4.47.1)
  Requirement already satisfied: threadpoolctl>=3.5.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 7)) (3.5.0)
  Requirement already satisfied: packaging>=24.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 8)) (24.2)
  Requirement already satisfied: setuptools>=75.5.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 9)) (75.6.0)
  Requirement already satisfied: device-smi==0.3.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 10)) (0.3.2)
  Requirement already satisfied: sentencepiece>=0.2.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 11)) (0.2.0)
  Requirement already satisfied: protobuf>=5.29.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 12)) (5.29.2)
  Requirement already satisfied: psutil in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from accelerate>=1.2.1->-r requirements.txt (line 1)) (6.1.0)
  Requirement already satisfied: pyyaml in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from accelerate>=1.2.1->-r requirements.txt (line 1)) (6.0.2)
  Requirement already satisfied: huggingface-hub>=0.21.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from accelerate>=1.2.1->-r requirements.txt (line 1)) (0.27.0)
  Requirement already satisfied: filelock in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (3.13.1)
  Requirement already satisfied: pyarrow>=15.0.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (18.1.0)
  Requirement already satisfied: dill<0.3.9,>=0.3.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (0.3.8)
  Requirement already satisfied: pandas in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (2.2.3)
  Requirement already satisfied: requests>=2.32.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (2.32.3)
  Requirement already satisfied: tqdm>=4.66.3 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (4.67.1)
  Requirement already satisfied: xxhash in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (3.5.0)
  Requirement already satisfied: multiprocess<0.70.17 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (0.70.16)
  Requirement already satisfied: fsspec<=2024.9.0,>=2023.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from fsspec[http]<=2024.9.0,>=2023.1.0->datasets>=3.1.0->-r requirements.txt (line 2)) (2024.2.0)
  Requirement already satisfied: aiohttp in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (3.11.11)
  Requirement already satisfied: typing-extensions>=4.8.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (4.9.0)
  Requirement already satisfied: networkx in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (3.2.1)
  Requirement already satisfied: jinja2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (3.1.3)
  Requirement already satisfied: sympy==1.13.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (1.13.1)
  Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from sympy==1.13.1->torch>=2.0.0->-r requirements.txt (line 4)) (1.3.0)
  Requirement already satisfied: regex!=2019.12.17 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from transformers>=4.46.3->-r requirements.txt (line 6)) (2024.11.6)
  Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from transformers>=4.46.3->-r requirements.txt (line 6)) (0.21.0)
  Requirement already satisfied: aiohappyeyeballs>=2.3.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (2.4.4)
  Requirement already satisfied: aiosignal>=1.1.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (1.3.2)
  Requirement already satisfied: async-timeout<6.0,>=4.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (5.0.1)
  Requirement already satisfied: attrs>=17.3.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (24.3.0)
  Requirement already satisfied: frozenlist>=1.1.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (1.5.0)
  Requirement already satisfied: multidict<7.0,>=4.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (6.1.0)
  Requirement already satisfied: propcache>=0.2.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (0.2.1)
  Requirement already satisfied: yarl<2.0,>=1.17.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (1.18.3)
  Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2)) (3.4.0)
  Requirement already satisfied: idna<4,>=2.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2)) (3.10)
  Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2)) (2.2.3)
  Requirement already satisfied: certifi>=2017.4.17 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2)) (2024.12.14)
  Requirement already satisfied: colorama in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from tqdm>=4.66.3->datasets>=3.1.0->-r requirements.txt (line 2)) (0.4.6)
  Requirement already satisfied: MarkupSafe>=2.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from jinja2->torch>=2.0.0->-r requirements.txt (line 4)) (2.1.5)
  Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from pandas->datasets>=3.1.0->-r requirements.txt (line 2)) (2.9.0.post0)
  Requirement already satisfied: pytz>=2020.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from pandas->datasets>=3.1.0->-r requirements.txt (line 2)) (2024.2)
  Requirement already satisfied: tzdata>=2022.7 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from pandas->datasets>=3.1.0->-r requirements.txt (line 2)) (2024.2)
  Requirement already satisfied: six>=1.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from python-dateutil>=2.8.2->pandas->datasets>=3.1.0->-r requirements.txt (line 2)) (1.17.0)
  C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\dist.py:330: InformationOnly: Normalizing '1.4.5-dev' to '1.4.5.dev0'
    self.metadata.version = self._normalize_version(self.metadata.version)
  conda_cuda_include_dir C:\Users\amarg\anaconda3\envs\gptq_new\Lib\site-packages\nvidia/cuda_runtime/include
  running egg_info
  creating C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-symyqxuo\gptqmodel.egg-info
  writing C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-symyqxuo\gptqmodel.egg-info\PKG-INFO
  writing dependency_links to C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-symyqxuo\gptqmodel.egg-info\dependency_links.txt
  writing requirements to C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-symyqxuo\gptqmodel.egg-info\requires.txt
  writing top-level names to C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-symyqxuo\gptqmodel.egg-info\top_level.txt
  writing manifest file 'C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-symyqxuo\gptqmodel.egg-info\SOURCES.txt'
  C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\utils\cpp_extension.py:497: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  reading manifest file 'C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-symyqxuo\gptqmodel.egg-info\SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching 'gptqmodel_ext\**\*.py' anywhere in distribution
  adding license file 'LICENSE'
  writing manifest file 'C:\Users\amarg\AppData\Local\Temp\pip-pip-egg-info-symyqxuo\gptqmodel.egg-info\SOURCES.txt'
  Preparing metadata (setup.py) ... done
Requirement already satisfied: accelerate>=1.2.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (1.2.1)
Requirement already satisfied: datasets>=3.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (3.2.0)
Requirement already satisfied: numpy>=1.26.4 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (2.2.0)
Requirement already satisfied: torch>=2.0.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (2.5.1+cu124)
Requirement already satisfied: safetensors>=0.4.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (0.4.5)
Requirement already satisfied: transformers>=4.46.3 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (4.47.1)
Requirement already satisfied: threadpoolctl>=3.5.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (3.5.0)
Requirement already satisfied: packaging>=24.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (24.2)
Requirement already satisfied: setuptools>=75.5.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (75.6.0)
Requirement already satisfied: device-smi==0.3.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (0.3.2)
Requirement already satisfied: sentencepiece>=0.2.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (0.2.0)
Requirement already satisfied: protobuf>=5.29.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from gptqmodel==1.4.5.dev0) (5.29.2)
Requirement already satisfied: psutil in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from accelerate>=1.2.1->gptqmodel==1.4.5.dev0) (6.1.0)
Requirement already satisfied: pyyaml in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from accelerate>=1.2.1->gptqmodel==1.4.5.dev0) (6.0.2)
Requirement already satisfied: huggingface-hub>=0.21.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from accelerate>=1.2.1->gptqmodel==1.4.5.dev0) (0.27.0)
Requirement already satisfied: filelock in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->gptqmodel==1.4.5.dev0) (3.13.1)
Requirement already satisfied: pyarrow>=15.0.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->gptqmodel==1.4.5.dev0) (18.1.0)
Requirement already satisfied: dill<0.3.9,>=0.3.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->gptqmodel==1.4.5.dev0) (0.3.8)
Requirement already satisfied: pandas in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->gptqmodel==1.4.5.dev0) (2.2.3)
Requirement already satisfied: requests>=2.32.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->gptqmodel==1.4.5.dev0) (2.32.3)
Requirement already satisfied: tqdm>=4.66.3 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->gptqmodel==1.4.5.dev0) (4.67.1)
Requirement already satisfied: xxhash in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->gptqmodel==1.4.5.dev0) (3.5.0)
Requirement already satisfied: multiprocess<0.70.17 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->gptqmodel==1.4.5.dev0) (0.70.16)
Requirement already satisfied: fsspec<=2024.9.0,>=2023.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from fsspec[http]<=2024.9.0,>=2023.1.0->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (2024.2.0)
Requirement already satisfied: aiohttp in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->gptqmodel==1.4.5.dev0) (3.11.11)
Requirement already satisfied: typing-extensions>=4.8.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->gptqmodel==1.4.5.dev0) (4.9.0)
Requirement already satisfied: networkx in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->gptqmodel==1.4.5.dev0) (3.2.1)
Requirement already satisfied: jinja2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->gptqmodel==1.4.5.dev0) (3.1.3)
Requirement already satisfied: sympy==1.13.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->gptqmodel==1.4.5.dev0) (1.13.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from sympy==1.13.1->torch>=2.0.0->gptqmodel==1.4.5.dev0) (1.3.0)
Requirement already satisfied: regex!=2019.12.17 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from transformers>=4.46.3->gptqmodel==1.4.5.dev0) (2024.11.6)
Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from transformers>=4.46.3->gptqmodel==1.4.5.dev0) (0.21.0)
Requirement already satisfied: aiohappyeyeballs>=2.3.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (2.4.4)
Requirement already satisfied: aiosignal>=1.1.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (1.3.2)
Requirement already satisfied: async-timeout<6.0,>=4.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (5.0.1)
Requirement already satisfied: attrs>=17.3.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (24.3.0)
Requirement already satisfied: frozenlist>=1.1.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (1.5.0)
Requirement already satisfied: multidict<7.0,>=4.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (6.1.0)
Requirement already satisfied: propcache>=0.2.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (0.2.1)
Requirement already satisfied: yarl<2.0,>=1.17.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (1.18.3)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (3.4.0)
Requirement already satisfied: idna<4,>=2.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (2.2.3)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (2024.12.14)
Requirement already satisfied: colorama in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from tqdm>=4.66.3->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (0.4.6)
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from jinja2->torch>=2.0.0->gptqmodel==1.4.5.dev0) (2.1.5)
Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from pandas->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from pandas->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (2024.2)
Requirement already satisfied: tzdata>=2022.7 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from pandas->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (2024.2)
Requirement already satisfied: six>=1.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from python-dateutil>=2.8.2->pandas->datasets>=3.1.0->gptqmodel==1.4.5.dev0) (1.17.0)
Building wheels for collected packages: gptqmodel
  Running command python setup.py bdist_wheel
  Requirement already satisfied: accelerate>=1.2.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 1)) (1.2.1)
  Requirement already satisfied: datasets>=3.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 2)) (3.2.0)
  Requirement already satisfied: numpy>=1.26.4 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 3)) (2.2.0)
  Requirement already satisfied: torch>=2.0.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 4)) (2.5.1+cu124)
  Requirement already satisfied: safetensors>=0.4.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 5)) (0.4.5)
  Requirement already satisfied: transformers>=4.46.3 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 6)) (4.47.1)
  Requirement already satisfied: threadpoolctl>=3.5.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 7)) (3.5.0)
  Requirement already satisfied: packaging>=24.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 8)) (24.2)
  Requirement already satisfied: setuptools>=75.5.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 9)) (75.6.0)
  Requirement already satisfied: device-smi==0.3.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 10)) (0.3.2)
  Requirement already satisfied: sentencepiece>=0.2.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 11)) (0.2.0)
  Requirement already satisfied: protobuf>=5.29.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from -r requirements.txt (line 12)) (5.29.2)
  Requirement already satisfied: psutil in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from accelerate>=1.2.1->-r requirements.txt (line 1)) (6.1.0)
  Requirement already satisfied: pyyaml in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from accelerate>=1.2.1->-r requirements.txt (line 1)) (6.0.2)
  Requirement already satisfied: huggingface-hub>=0.21.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from accelerate>=1.2.1->-r requirements.txt (line 1)) (0.27.0)
  Requirement already satisfied: filelock in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (3.13.1)
  Requirement already satisfied: pyarrow>=15.0.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (18.1.0)
  Requirement already satisfied: dill<0.3.9,>=0.3.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (0.3.8)
  Requirement already satisfied: pandas in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (2.2.3)
  Requirement already satisfied: requests>=2.32.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (2.32.3)
  Requirement already satisfied: tqdm>=4.66.3 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (4.67.1)
  Requirement already satisfied: xxhash in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (3.5.0)
  Requirement already satisfied: multiprocess<0.70.17 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (0.70.16)
  Requirement already satisfied: fsspec<=2024.9.0,>=2023.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from fsspec[http]<=2024.9.0,>=2023.1.0->datasets>=3.1.0->-r requirements.txt (line 2)) (2024.2.0)
  Requirement already satisfied: aiohttp in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from datasets>=3.1.0->-r requirements.txt (line 2)) (3.11.11)
  Requirement already satisfied: typing-extensions>=4.8.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (4.9.0)
  Requirement already satisfied: networkx in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (3.2.1)
  Requirement already satisfied: jinja2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (3.1.3)
  Requirement already satisfied: sympy==1.13.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from torch>=2.0.0->-r requirements.txt (line 4)) (1.13.1)
  Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from sympy==1.13.1->torch>=2.0.0->-r requirements.txt (line 4)) (1.3.0)
  Requirement already satisfied: regex!=2019.12.17 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from transformers>=4.46.3->-r requirements.txt (line 6)) (2024.11.6)
  Requirement already satisfied: tokenizers<0.22,>=0.21 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from transformers>=4.46.3->-r requirements.txt (line 6)) (0.21.0)
  Requirement already satisfied: aiohappyeyeballs>=2.3.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (2.4.4)
  Requirement already satisfied: aiosignal>=1.1.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (1.3.2)
  Requirement already satisfied: async-timeout<6.0,>=4.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (5.0.1)
  Requirement already satisfied: attrs>=17.3.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (24.3.0)
  Requirement already satisfied: frozenlist>=1.1.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (1.5.0)
  Requirement already satisfied: multidict<7.0,>=4.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (6.1.0)
  Requirement already satisfied: propcache>=0.2.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (0.2.1)
  Requirement already satisfied: yarl<2.0,>=1.17.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from aiohttp->datasets>=3.1.0->-r requirements.txt (line 2)) (1.18.3)
  Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2)) (3.4.0)
  Requirement already satisfied: idna<4,>=2.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2)) (3.10)
  Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2)) (2.2.3)
  Requirement already satisfied: certifi>=2017.4.17 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from requests>=2.32.2->datasets>=3.1.0->-r requirements.txt (line 2)) (2024.12.14)
  Requirement already satisfied: colorama in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from tqdm>=4.66.3->datasets>=3.1.0->-r requirements.txt (line 2)) (0.4.6)
  Requirement already satisfied: MarkupSafe>=2.0 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from jinja2->torch>=2.0.0->-r requirements.txt (line 4)) (2.1.5)
  Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from pandas->datasets>=3.1.0->-r requirements.txt (line 2)) (2.9.0.post0)
  Requirement already satisfied: pytz>=2020.1 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from pandas->datasets>=3.1.0->-r requirements.txt (line 2)) (2024.2)
  Requirement already satisfied: tzdata>=2022.7 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from pandas->datasets>=3.1.0->-r requirements.txt (line 2)) (2024.2)
  Requirement already satisfied: six>=1.5 in c:\users\amarg\anaconda3\envs\gptq_new\lib\site-packages (from python-dateutil>=2.8.2->pandas->datasets>=3.1.0->-r requirements.txt (line 2)) (1.17.0)
  conda_cuda_include_dir C:\Users\amarg\anaconda3\envs\gptq_new\Lib\site-packages\nvidia/cuda_runtime/include
  C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\dist.py:330: InformationOnly: Normalizing '1.4.5-dev' to '1.4.5.dev0'
    self.metadata.version = self._normalize_version(self.metadata.version)
  running bdist_wheel
  Guessing wheel URL: https://github.com/ModelCloud/GPTQModel/releases/download/v1.4.5-dev/gptqmodel-1.4.5-dev+cu124torch2.5-cp310-cp310-linux_x86_64.whl
  wheel name=gptqmodel-1.4.5-dev+cu124torch2.5-cp310-cp310-linux_x86_64.whl
  Precompiled wheel not found in url: https://github.com/ModelCloud/GPTQModel/releases/download/v1.4.5-dev/gptqmodel-1.4.5-dev+cu124torch2.5-cp310-cp310-linux_x86_64.whl. Building from source...
  C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\utils\cpp_extension.py:497: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  running build
  running build_py
  creating build\lib.win-amd64-cpython-310\gptqmodel
  copying gptqmodel\version.py -> build\lib.win-amd64-cpython-310\gptqmodel
  copying gptqmodel\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration
  copying gptqmodel\integration\integration.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration
  copying gptqmodel\integration\integration_vllm.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration
  copying gptqmodel\integration\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration
  creating build\lib.win-amd64-cpython-310\gptqmodel\models
  copying gptqmodel\models\auto.py -> build\lib.win-amd64-cpython-310\gptqmodel\models
  copying gptqmodel\models\base.py -> build\lib.win-amd64-cpython-310\gptqmodel\models
  copying gptqmodel\models\loader.py -> build\lib.win-amd64-cpython-310\gptqmodel\models
  copying gptqmodel\models\writer.py -> build\lib.win-amd64-cpython-310\gptqmodel\models
  copying gptqmodel\models\_const.py -> build\lib.win-amd64-cpython-310\gptqmodel\models
  copying gptqmodel\models\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\models
  creating build\lib.win-amd64-cpython-310\gptqmodel\nn_modules
  copying gptqmodel\nn_modules\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules
  creating build\lib.win-amd64-cpython-310\gptqmodel\quantization
  copying gptqmodel\quantization\config.py -> build\lib.win-amd64-cpython-310\gptqmodel\quantization
  copying gptqmodel\quantization\gptq.py -> build\lib.win-amd64-cpython-310\gptqmodel\quantization
  copying gptqmodel\quantization\quantizer.py -> build\lib.win-amd64-cpython-310\gptqmodel\quantization
  copying gptqmodel\quantization\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\quantization
  creating build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\backend.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\bitblas.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\data.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\device.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\eval.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\exllama.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\importer.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\logger.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\marlin.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\model.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\perplexity.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\plotly.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\progress.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\sglang.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\torch.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\vllm.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\vram.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  copying gptqmodel\utils\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\utils
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src
  copying gptqmodel\integration\src\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum
  copying gptqmodel\integration\src\optimum\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft
  copying gptqmodel\integration\src\peft\import_utils.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft
  copying gptqmodel\integration\src\peft\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers
  copying gptqmodel\integration\src\transformers\testing_utils.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers
  copying gptqmodel\integration\src\transformers\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\vllm
  copying gptqmodel\integration\src\vllm\gptq_marlin.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\vllm
  copying gptqmodel\integration\src\vllm\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\vllm
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\gptq
  copying gptqmodel\integration\src\optimum\gptq\quantizer.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\gptq
  copying gptqmodel\integration\src\optimum\gptq\utils.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\gptq
  copying gptqmodel\integration\src\optimum\gptq\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\gptq
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\utils
  copying gptqmodel\integration\src\optimum\utils\import_utils.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\utils
  copying gptqmodel\integration\src\optimum\utils\testing_utils.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\utils
  copying gptqmodel\integration\src\optimum\utils\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\utils
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners
  copying gptqmodel\integration\src\peft\tuners\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\utils
  copying gptqmodel\integration\src\peft\utils\other.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\utils
  copying gptqmodel\integration\src\peft\utils\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\utils
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\adalora
  copying gptqmodel\integration\src\peft\tuners\adalora\model.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\adalora
  copying gptqmodel\integration\src\peft\tuners\adalora\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\adalora
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\lora
  copying gptqmodel\integration\src\peft\tuners\lora\gptq.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\lora
  copying gptqmodel\integration\src\peft\tuners\lora\model.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\lora
  copying gptqmodel\integration\src\peft\tuners\lora\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\lora
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\quantizers
  copying gptqmodel\integration\src\transformers\quantizers\quantizer_gptq.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\quantizers
  copying gptqmodel\integration\src\transformers\quantizers\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\quantizers
  creating build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\utils
  copying gptqmodel\integration\src\transformers\utils\import_utils.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\utils
  copying gptqmodel\integration\src\transformers\utils\quantization_config.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\utils
  copying gptqmodel\integration\src\transformers\utils\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\utils
  creating build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\baichuan.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\bloom.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\chatglm.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\codegen.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\cohere.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\cohere2.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\dbrx.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\dbrx_converted.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\decilm.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\deepseek_v2.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\exaone.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\gemma.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\gemma2.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\glm.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\gpt2.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\gptj.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\gpt_bigcode.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\gpt_neox.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\granite.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\grinmoe.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\hymba.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\internlm.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\internlm2.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\llama.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\longllama.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\minicpm.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\minicpm3.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\mistral.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\mixtral.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\mllama.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\mobilellm.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\moss.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\mpt.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\olmo2.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\opt.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\ovis.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\phi.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\phi3.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\qwen.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\qwen2.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\qwen2_moe.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\qwen2_vl.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\rw.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\stablelmepoch.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\starcoder2.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\xverse.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\yi.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  copying gptqmodel\models\definitions\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\models\definitions
  creating build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  copying gptqmodel\nn_modules\qlinear\bitblas.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  copying gptqmodel\nn_modules\qlinear\bitblas_target_detector.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  copying gptqmodel\nn_modules\qlinear\dynamic_cuda.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  copying gptqmodel\nn_modules\qlinear\exllama.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  copying gptqmodel\nn_modules\qlinear\exllamav2.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  copying gptqmodel\nn_modules\qlinear\ipex.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  copying gptqmodel\nn_modules\qlinear\marlin.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  copying gptqmodel\nn_modules\qlinear\torch.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  copying gptqmodel\nn_modules\qlinear\tritonv2.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  copying gptqmodel\nn_modules\qlinear\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear
  creating build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils
  copying gptqmodel\nn_modules\triton_utils\custom_autotune.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils
  copying gptqmodel\nn_modules\triton_utils\dequant.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils
  copying gptqmodel\nn_modules\triton_utils\kernels.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils
  copying gptqmodel\nn_modules\triton_utils\mixin.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils
  copying gptqmodel\nn_modules\triton_utils\__init__.py -> build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils
  running build_ext
  C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\utils\cpp_extension.py:382: UserWarning: Error checking compiler version for cl: [WinError 2] The system cannot find the file specified
    warnings.warn(f'Error checking compiler version for {compiler}: {error}')
  C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\utils\cpp_extension.py:416: UserWarning: The detected CUDA version (12.1) has a minor version mismatch with the version that was used to compile PyTorch (12.4). Most likely this shouldn't be a problem.
    warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
  building 'gptqmodel_cuda_64' extension
  creating build\temp.win-amd64-cpython-310\Release\gptqmodel_ext\cuda_64
  "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\TH -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\include" -Igptqmodel_cuda -IC:\Users\amarg\anaconda3\envs\gptq_new\include -IC:\Users\amarg\anaconda3\envs\gptq_new\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /EHsc /Tpgptqmodel_ext/cuda_64/gptqmodel_cuda_64.cpp /Fobuild\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_64/gptqmodel_cuda_64.obj /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /wd4624 /wd4067 /wd4068 /EHsc -O3 -std=c++17 -fopenmp -lgomp -DENABLE_BF16-Wno-switch-bool -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=gptqmodel_cuda_64 -D_GLIBCXX_USE_CXX11_ABI=0 /std:c++17
  cl : Command line warning D9002 : ignoring unknown option '-O3'
  cl : Command line warning D9002 : ignoring unknown option '-std=c++17'
  cl : Command line warning D9002 : ignoring unknown option '-fopenmp'
  cl : Command line warning D9002 : ignoring unknown option '-lgomp'
  gptqmodel_cuda_64.cpp
  C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\utils\cpp_extension.py:1964: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation.
  If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
    warnings.warn(
  "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin\nvcc" -c gptqmodel_ext/cuda_64/gptqmodel_cuda_kernel_64.cu -o build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_64/gptqmodel_cuda_kernel_64.obj -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\TH -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\include" -Igptqmodel_cuda -IC:\Users\amarg\anaconda3\envs\gptq_new\include -IC:\Users\amarg\anaconda3\envs\gptq_new\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4068 -Xcompiler /wd4067 -Xcompiler /wd4624 -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -O3 -std=c++17 -DENABLE_BF16 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ -U__CUDA_NO_BFLOAT162_OPERATORS__ -U__CUDA_NO_BFLOAT162_CONVERSIONS__ --threads 4 -Xfatbin -compress-all -diag-suppress=179,39,186 --use_fast_math -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=gptqmodel_cuda_64 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_89,code=compute_89 -gencode=arch=compute_89,code=sm_89 -std=c++17 --use-local-env
  gptqmodel_cuda_kernel_64.cu
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_OPERATORS__' with '/U__CUDA_NO_HALF_OPERATORS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_CONVERSIONS__' with '/U__CUDA_NO_HALF_CONVERSIONS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF2_OPERATORS__' with '/U__CUDA_NO_HALF2_OPERATORS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_BFLOAT16_CONVERSIONS__' with '/U__CUDA_NO_BFLOAT16_CONVERSIONS__'
  gptqmodel_cuda_kernel_64.cu
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_OPERATORS__' with '/U__CUDA_NO_HALF_OPERATORS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_CONVERSIONS__' with '/U__CUDA_NO_HALF_CONVERSIONS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF2_OPERATORS__' with '/U__CUDA_NO_HALF2_OPERATORS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_BFLOAT16_CONVERSIONS__' with '/U__CUDA_NO_BFLOAT16_CONVERSIONS__'
  gptqmodel_cuda_kernel_64.cu
  tmpxft_00000230_00000000-7_gptqmodel_cuda_kernel_64.cudafe1.cpp
  "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\lib "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\lib\x64" /LIBPATH:C:\Users\amarg\anaconda3\envs\gptq_new\libs /LIBPATH:C:\Users\amarg\anaconda3\envs\gptq_new /LIBPATH:C:\Users\amarg\anaconda3\envs\gptq_new\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64" c10.lib torch.lib torch_cpu.lib torch_python.lib cudart.lib c10_cuda.lib torch_cuda.lib /EXPORT:PyInit_gptqmodel_cuda_64 build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_64/gptqmodel_cuda_64.obj build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_64/gptqmodel_cuda_kernel_64.obj /OUT:build\lib.win-amd64-cpython-310\gptqmodel_cuda_64.cp310-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_64\gptqmodel_cuda_64.cp310-win_amd64.lib
     Creating library build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_64\gptqmodel_cuda_64.cp310-win_amd64.lib and object build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_64\gptqmodel_cuda_64.cp310-win_amd64.exp
  Generating code
  Finished generating code
  building 'gptqmodel_cuda_256' extension
  creating build\temp.win-amd64-cpython-310\Release\gptqmodel_ext\cuda_256
  "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\TH -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\include" -Igptqmodel_cuda -IC:\Users\amarg\anaconda3\envs\gptq_new\include -IC:\Users\amarg\anaconda3\envs\gptq_new\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /EHsc /Tpgptqmodel_ext/cuda_256/gptqmodel_cuda_256.cpp /Fobuild\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_256/gptqmodel_cuda_256.obj /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /wd4624 /wd4067 /wd4068 /EHsc -O3 -std=c++17 -fopenmp -lgomp -DENABLE_BF16-Wno-switch-bool -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=gptqmodel_cuda_256 -D_GLIBCXX_USE_CXX11_ABI=0 /std:c++17
  cl : Command line warning D9002 : ignoring unknown option '-O3'
  cl : Command line warning D9002 : ignoring unknown option '-std=c++17'
  cl : Command line warning D9002 : ignoring unknown option '-fopenmp'
  cl : Command line warning D9002 : ignoring unknown option '-lgomp'
  gptqmodel_cuda_256.cpp
  "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin\nvcc" -c gptqmodel_ext/cuda_256/gptqmodel_cuda_kernel_256.cu -o build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_256/gptqmodel_cuda_kernel_256.obj -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\TH -IC:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\include" -Igptqmodel_cuda -IC:\Users\amarg\anaconda3\envs\gptq_new\include -IC:\Users\amarg\anaconda3\envs\gptq_new\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4068 -Xcompiler /wd4067 -Xcompiler /wd4624 -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -O3 -std=c++17 -DENABLE_BF16 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -U__CUDA_NO_BFLOAT16_OPERATORS__ -U__CUDA_NO_BFLOAT16_CONVERSIONS__ -U__CUDA_NO_BFLOAT162_OPERATORS__ -U__CUDA_NO_BFLOAT162_CONVERSIONS__ --threads 4 -Xfatbin -compress-all -diag-suppress=179,39,186 --use_fast_math -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=gptqmodel_cuda_256 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_89,code=compute_89 -gencode=arch=compute_89,code=sm_89 -std=c++17 --use-local-env
  gptqmodel_cuda_kernel_256.cu
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_OPERATORS__' with '/U__CUDA_NO_HALF_OPERATORS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_CONVERSIONS__' with '/U__CUDA_NO_HALF_CONVERSIONS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF2_OPERATORS__' with '/U__CUDA_NO_HALF2_OPERATORS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_BFLOAT16_CONVERSIONS__' with '/U__CUDA_NO_BFLOAT16_CONVERSIONS__'
  gptqmodel_cuda_kernel_256.cu
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_OPERATORS__' with '/U__CUDA_NO_HALF_OPERATORS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF_CONVERSIONS__' with '/U__CUDA_NO_HALF_CONVERSIONS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF2_OPERATORS__' with '/U__CUDA_NO_HALF2_OPERATORS__'
  cl : Command line warning D9025 : overriding '/D__CUDA_NO_BFLOAT16_CONVERSIONS__' with '/U__CUDA_NO_BFLOAT16_CONVERSIONS__'
  gptqmodel_cuda_kernel_256.cu
  tmpxft_000020d4_00000000-7_gptqmodel_cuda_kernel_256.cudafe1.cpp
  "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\torch\lib "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\lib\x64" /LIBPATH:C:\Users\amarg\anaconda3\envs\gptq_new\libs /LIBPATH:C:\Users\amarg\anaconda3\envs\gptq_new /LIBPATH:C:\Users\amarg\anaconda3\envs\gptq_new\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64" c10.lib torch.lib torch_cpu.lib torch_python.lib cudart.lib c10_cuda.lib torch_cuda.lib /EXPORT:PyInit_gptqmodel_cuda_256 build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_256/gptqmodel_cuda_256.obj build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_256/gptqmodel_cuda_kernel_256.obj /OUT:build\lib.win-amd64-cpython-310\gptqmodel_cuda_256.cp310-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_256\gptqmodel_cuda_256.cp310-win_amd64.lib
     Creating library build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_256\gptqmodel_cuda_256.cp310-win_amd64.lib and object build\temp.win-amd64-cpython-310\Release\gptqmodel_ext/cuda_256\gptqmodel_cuda_256.cp310-win_amd64.exp
  Generating code
  Finished generating code
  C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\_distutils\cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
  !!

          ********************************************************************************
          Please avoid running ``setup.py`` directly.
          Instead, use pypa/build, pypa/installer or other
          standards-based tools.

          See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
          ********************************************************************************

  !!
    self.initialize_options()
  installing to build\bdist.win-amd64\wheel
  running install
  running install_lib
  creating build\bdist.win-amd64\wheel
  creating build\bdist.win-amd64\wheel\gptqmodel
  creating build\bdist.win-amd64\wheel\gptqmodel\integration
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\integration.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\integration_vllm.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\optimum
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\optimum\gptq
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\gptq\quantizer.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\optimum\gptq
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\gptq\utils.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\optimum\gptq
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\gptq\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\optimum\gptq
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\optimum\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\utils\import_utils.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\optimum\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\utils\testing_utils.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\optimum\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\utils\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\optimum\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\optimum\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\optimum
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\peft
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\import_utils.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\peft
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\peft\tuners
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\peft\tuners\adalora
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\adalora\model.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\peft\tuners\adalora
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\adalora\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\peft\tuners\adalora
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\peft\tuners\lora
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\lora\gptq.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\peft\tuners\lora
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\lora\model.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\peft\tuners\lora
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\lora\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\peft\tuners\lora
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\tuners\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\peft\tuners
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\peft\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\utils\other.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\peft\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\utils\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\peft\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\peft\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\peft
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\transformers
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\transformers\quantizers
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\quantizers\quantizer_gptq.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\transformers\quantizers
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\quantizers\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\transformers\quantizers
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\testing_utils.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\transformers
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\transformers\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\utils\import_utils.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\transformers\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\utils\quantization_config.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\transformers\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\utils\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\transformers\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\transformers\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\transformers
  creating build\bdist.win-amd64\wheel\gptqmodel\integration\src\vllm
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\vllm\gptq_marlin.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\vllm
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\vllm\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src\vllm
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\src\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration\src
  copying build\lib.win-amd64-cpython-310\gptqmodel\integration\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\integration
  creating build\bdist.win-amd64\wheel\gptqmodel\models
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\auto.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\base.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models
  creating build\bdist.win-amd64\wheel\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\baichuan.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\bloom.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\chatglm.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\codegen.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\cohere.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\cohere2.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\dbrx.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\dbrx_converted.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\decilm.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\deepseek_v2.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\exaone.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\gemma.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\gemma2.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\glm.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\gpt2.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\gptj.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\gpt_bigcode.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\gpt_neox.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\granite.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\grinmoe.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\hymba.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\internlm.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\internlm2.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\llama.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\longllama.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\minicpm.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\minicpm3.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\mistral.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\mixtral.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\mllama.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\mobilellm.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\moss.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\mpt.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\olmo2.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\opt.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\ovis.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\phi.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\phi3.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\qwen.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\qwen2.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\qwen2_moe.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\qwen2_vl.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\rw.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\stablelmepoch.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\starcoder2.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\xverse.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\yi.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\definitions\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models\definitions
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\loader.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\writer.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\_const.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models
  copying build\lib.win-amd64-cpython-310\gptqmodel\models\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\models
  creating build\bdist.win-amd64\wheel\gptqmodel\nn_modules
  creating build\bdist.win-amd64\wheel\gptqmodel\nn_modules\qlinear
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear\bitblas.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\qlinear
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear\bitblas_target_detector.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\qlinear
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear\dynamic_cuda.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\qlinear
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear\exllama.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\qlinear
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear\exllamav2.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\qlinear
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear\ipex.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\qlinear
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear\marlin.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\qlinear
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear\torch.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\qlinear
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear\tritonv2.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\qlinear
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\qlinear\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\qlinear
  creating build\bdist.win-amd64\wheel\gptqmodel\nn_modules\triton_utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils\custom_autotune.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\triton_utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils\dequant.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\triton_utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils\kernels.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\triton_utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils\mixin.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\triton_utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\triton_utils\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules\triton_utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\nn_modules\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\nn_modules
  creating build\bdist.win-amd64\wheel\gptqmodel\quantization
  copying build\lib.win-amd64-cpython-310\gptqmodel\quantization\config.py -> build\bdist.win-amd64\wheel\.\gptqmodel\quantization
  copying build\lib.win-amd64-cpython-310\gptqmodel\quantization\gptq.py -> build\bdist.win-amd64\wheel\.\gptqmodel\quantization
  copying build\lib.win-amd64-cpython-310\gptqmodel\quantization\quantizer.py -> build\bdist.win-amd64\wheel\.\gptqmodel\quantization
  copying build\lib.win-amd64-cpython-310\gptqmodel\quantization\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\quantization
  creating build\bdist.win-amd64\wheel\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\backend.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\bitblas.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\data.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\device.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\eval.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\exllama.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\importer.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\logger.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\marlin.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\model.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\perplexity.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\plotly.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\progress.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\sglang.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\torch.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\vllm.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\vram.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\utils\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel\utils
  copying build\lib.win-amd64-cpython-310\gptqmodel\version.py -> build\bdist.win-amd64\wheel\.\gptqmodel
  copying build\lib.win-amd64-cpython-310\gptqmodel\__init__.py -> build\bdist.win-amd64\wheel\.\gptqmodel
  copying build\lib.win-amd64-cpython-310\gptqmodel_cuda_256.cp310-win_amd64.pyd -> build\bdist.win-amd64\wheel\.
  copying build\lib.win-amd64-cpython-310\gptqmodel_cuda_64.cp310-win_amd64.pyd -> build\bdist.win-amd64\wheel\.
  running install_egg_info
  running egg_info
  creating gptqmodel.egg-info
  writing gptqmodel.egg-info\PKG-INFO
  writing dependency_links to gptqmodel.egg-info\dependency_links.txt
  writing requirements to gptqmodel.egg-info\requires.txt
  writing top-level names to gptqmodel.egg-info\top_level.txt
  writing manifest file 'gptqmodel.egg-info\SOURCES.txt'
  reading manifest file 'gptqmodel.egg-info\SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching 'gptqmodel_ext\**\*.py' anywhere in distribution
  adding license file 'LICENSE'
  writing manifest file 'gptqmodel.egg-info\SOURCES.txt'
  Copying gptqmodel.egg-info to build\bdist.win-amd64\wheel\.\gptqmodel-1.4.5.dev0-py3.10.egg-info
  running install_scripts
  C:\Users\amarg\anaconda3\envs\gptq_new\lib\site-packages\setuptools\command\bdist_wheel.py:108: RuntimeWarning: Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
    if get_flag("Py_DEBUG", hasattr(sys, "gettotalrefcount"), warn=(impl == "cp")):
  creating build\bdist.win-amd64\wheel\gptqmodel-1.4.5.dev0.dist-info\WHEEL
  creating 'C:\Users\amarg\AppData\Local\Temp\pip-wheel-m0pnmxl0\gptqmodel-1.4.5.dev0-cp310-cp310-win_amd64.whl' and adding 'build\bdist.win-amd64\wheel' to it
  adding 'gptqmodel_cuda_256.cp310-win_amd64.pyd'
  adding 'gptqmodel_cuda_64.cp310-win_amd64.pyd'
  adding 'gptqmodel/__init__.py'
  adding 'gptqmodel/version.py'
  adding 'gptqmodel/integration/__init__.py'
  adding 'gptqmodel/integration/integration.py'
  adding 'gptqmodel/integration/integration_vllm.py'
  adding 'gptqmodel/integration/src/__init__.py'
  adding 'gptqmodel/integration/src/optimum/__init__.py'
  adding 'gptqmodel/integration/src/optimum/gptq/__init__.py'
  adding 'gptqmodel/integration/src/optimum/gptq/quantizer.py'
  adding 'gptqmodel/integration/src/optimum/gptq/utils.py'
  adding 'gptqmodel/integration/src/optimum/utils/__init__.py'
  adding 'gptqmodel/integration/src/optimum/utils/import_utils.py'
  adding 'gptqmodel/integration/src/optimum/utils/testing_utils.py'
  adding 'gptqmodel/integration/src/peft/__init__.py'
  adding 'gptqmodel/integration/src/peft/import_utils.py'
  adding 'gptqmodel/integration/src/peft/tuners/__init__.py'
  adding 'gptqmodel/integration/src/peft/tuners/adalora/__init__.py'
  adding 'gptqmodel/integration/src/peft/tuners/adalora/model.py'
  adding 'gptqmodel/integration/src/peft/tuners/lora/__init__.py'
  adding 'gptqmodel/integration/src/peft/tuners/lora/gptq.py'
  adding 'gptqmodel/integration/src/peft/tuners/lora/model.py'
  adding 'gptqmodel/integration/src/peft/utils/__init__.py'
  adding 'gptqmodel/integration/src/peft/utils/other.py'
  adding 'gptqmodel/integration/src/transformers/__init__.py'
  adding 'gptqmodel/integration/src/transformers/testing_utils.py'
  adding 'gptqmodel/integration/src/transformers/quantizers/__init__.py'
  adding 'gptqmodel/integration/src/transformers/quantizers/quantizer_gptq.py'
  adding 'gptqmodel/integration/src/transformers/utils/__init__.py'
  adding 'gptqmodel/integration/src/transformers/utils/import_utils.py'
  adding 'gptqmodel/integration/src/transformers/utils/quantization_config.py'
  adding 'gptqmodel/integration/src/vllm/__init__.py'
  adding 'gptqmodel/integration/src/vllm/gptq_marlin.py'
  adding 'gptqmodel/models/__init__.py'
  adding 'gptqmodel/models/_const.py'
  adding 'gptqmodel/models/auto.py'
  adding 'gptqmodel/models/base.py'
  adding 'gptqmodel/models/loader.py'
  adding 'gptqmodel/models/writer.py'
  adding 'gptqmodel/models/definitions/__init__.py'
  adding 'gptqmodel/models/definitions/baichuan.py'
  adding 'gptqmodel/models/definitions/bloom.py'
  adding 'gptqmodel/models/definitions/chatglm.py'
  adding 'gptqmodel/models/definitions/codegen.py'
  adding 'gptqmodel/models/definitions/cohere.py'
  adding 'gptqmodel/models/definitions/cohere2.py'
  adding 'gptqmodel/models/definitions/dbrx.py'
  adding 'gptqmodel/models/definitions/dbrx_converted.py'
  adding 'gptqmodel/models/definitions/decilm.py'
  adding 'gptqmodel/models/definitions/deepseek_v2.py'
  adding 'gptqmodel/models/definitions/exaone.py'
  adding 'gptqmodel/models/definitions/gemma.py'
  adding 'gptqmodel/models/definitions/gemma2.py'
  adding 'gptqmodel/models/definitions/glm.py'
  adding 'gptqmodel/models/definitions/gpt2.py'
  adding 'gptqmodel/models/definitions/gpt_bigcode.py'
  adding 'gptqmodel/models/definitions/gpt_neox.py'
  adding 'gptqmodel/models/definitions/gptj.py'
  adding 'gptqmodel/models/definitions/granite.py'
  adding 'gptqmodel/models/definitions/grinmoe.py'
  adding 'gptqmodel/models/definitions/hymba.py'
  adding 'gptqmodel/models/definitions/internlm.py'
  adding 'gptqmodel/models/definitions/internlm2.py'
  adding 'gptqmodel/models/definitions/llama.py'
  adding 'gptqmodel/models/definitions/longllama.py'
  adding 'gptqmodel/models/definitions/minicpm.py'
  adding 'gptqmodel/models/definitions/minicpm3.py'
  adding 'gptqmodel/models/definitions/mistral.py'
  adding 'gptqmodel/models/definitions/mixtral.py'
  adding 'gptqmodel/models/definitions/mllama.py'
  adding 'gptqmodel/models/definitions/mobilellm.py'
  adding 'gptqmodel/models/definitions/moss.py'
  adding 'gptqmodel/models/definitions/mpt.py'
  adding 'gptqmodel/models/definitions/olmo2.py'
  adding 'gptqmodel/models/definitions/opt.py'
  adding 'gptqmodel/models/definitions/ovis.py'
  adding 'gptqmodel/models/definitions/phi.py'
  adding 'gptqmodel/models/definitions/phi3.py'
  adding 'gptqmodel/models/definitions/qwen.py'
  adding 'gptqmodel/models/definitions/qwen2.py'
  adding 'gptqmodel/models/definitions/qwen2_moe.py'
  adding 'gptqmodel/models/definitions/qwen2_vl.py'
  adding 'gptqmodel/models/definitions/rw.py'
  adding 'gptqmodel/models/definitions/stablelmepoch.py'
  adding 'gptqmodel/models/definitions/starcoder2.py'
  adding 'gptqmodel/models/definitions/xverse.py'
  adding 'gptqmodel/models/definitions/yi.py'
  adding 'gptqmodel/nn_modules/__init__.py'
  adding 'gptqmodel/nn_modules/qlinear/__init__.py'
  adding 'gptqmodel/nn_modules/qlinear/bitblas.py'
  adding 'gptqmodel/nn_modules/qlinear/bitblas_target_detector.py'
  adding 'gptqmodel/nn_modules/qlinear/dynamic_cuda.py'
  adding 'gptqmodel/nn_modules/qlinear/exllama.py'
  adding 'gptqmodel/nn_modules/qlinear/exllamav2.py'
  adding 'gptqmodel/nn_modules/qlinear/ipex.py'
  adding 'gptqmodel/nn_modules/qlinear/marlin.py'
  adding 'gptqmodel/nn_modules/qlinear/torch.py'
  adding 'gptqmodel/nn_modules/qlinear/tritonv2.py'
  adding 'gptqmodel/nn_modules/triton_utils/__init__.py'
  adding 'gptqmodel/nn_modules/triton_utils/custom_autotune.py'
  adding 'gptqmodel/nn_modules/triton_utils/dequant.py'
  adding 'gptqmodel/nn_modules/triton_utils/kernels.py'
  adding 'gptqmodel/nn_modules/triton_utils/mixin.py'
  adding 'gptqmodel/quantization/__init__.py'
  adding 'gptqmodel/quantization/config.py'
  adding 'gptqmodel/quantization/gptq.py'
  adding 'gptqmodel/quantization/quantizer.py'
  adding 'gptqmodel/utils/__init__.py'
  adding 'gptqmodel/utils/backend.py'
  adding 'gptqmodel/utils/bitblas.py'
  adding 'gptqmodel/utils/data.py'
  adding 'gptqmodel/utils/device.py'
  adding 'gptqmodel/utils/eval.py'
  adding 'gptqmodel/utils/exllama.py'
  adding 'gptqmodel/utils/importer.py'
  adding 'gptqmodel/utils/logger.py'
  adding 'gptqmodel/utils/marlin.py'
  adding 'gptqmodel/utils/model.py'
  adding 'gptqmodel/utils/perplexity.py'
  adding 'gptqmodel/utils/plotly.py'
  adding 'gptqmodel/utils/progress.py'
  adding 'gptqmodel/utils/sglang.py'
  adding 'gptqmodel/utils/torch.py'
  adding 'gptqmodel/utils/vllm.py'
  adding 'gptqmodel/utils/vram.py'
  adding 'gptqmodel-1.4.5.dev0.dist-info/LICENSE'
  adding 'gptqmodel-1.4.5.dev0.dist-info/METADATA'
  adding 'gptqmodel-1.4.5.dev0.dist-info/WHEEL'
  adding 'gptqmodel-1.4.5.dev0.dist-info/top_level.txt'
  adding 'gptqmodel-1.4.5.dev0.dist-info/RECORD'
  removing build\bdist.win-amd64\wheel
  Building wheel for gptqmodel (setup.py) ... done
  Created wheel for gptqmodel: filename=gptqmodel-1.4.5.dev0-cp310-cp310-win_amd64.whl size=565532 sha256=51a228b04f3bb9998e3bfeed003337ab6c74f4db9d877396087570d4eb5b335d
  Stored in directory: C:\Users\amarg\AppData\Local\Temp\pip-ephem-wheel-cache-_u31r0p6\wheels\c3\f3\7a\4097f49368a21a23172ab164fdb56e3b4fff0d4192c67b4b45
Successfully built gptqmodel
Installing collected packages: gptqmodel
Successfully installed gptqmodel-1.4.5.dev0

[giradkar26]

@Qubitium Let me know where I am making mistake.

[Qubitium]

@Qubitium Let me know where I am making mistake.

Can you try WSL2 on windows? Debugging windows is a horrible experience. We can take another look at it tomorrow but you can try Ubuntu 24.04 WSL2 container on Windows for now. You will feel freedom and zero issues.

[giradkar26]

As I mentioned above though I got error in first attempt and built on 2nd attempt, I run the quantize script and it run successfully.
now I have folder with following files. Do I have to add or merge some files to infer them or is it wrongly quantized?
Because it is giving error while inferencing

[Qubitium]

@giradkar26 You need to merge ALL .py and .json files from original model to the new quantized model for inference. Also merge all tokenizer model files (since VL model has a special model for image tokenizer which we do not quantize).

Tomorrow we can check why the save() did not correctly copy those files over. Except for config.json. We already modified this file.

[Qubitium]

Windows setup requireing two-runs has been fixed in PR #939

[Qubitium]

We are closing this issue as completed. For windows, users need to download Pytorch from Pytorch/Nvidia repo and not directly from pypi.