TypeError: quantize_dynamic() got an unexpected keyword argument 'activation_type'

[ByUnal]

I've started fresh notebook, try install and use the library. I followed the instructions in the readme file, but it throws an error and doesn't work. Here is the my code and the error.

the code:

from fastT5 import (OnnxT5, get_onnx_runtime_sessions,
                    generate_onnx_representation, quantize)
from transformers import AutoTokenizer

model_or_model_path = 't5-small'

# Step 1. convert huggingfaces t5 model to onnx
onnx_model_paths = generate_onnx_representation(model_or_model_path)

# Step 2. (recommended) quantize the converted model for fast inference and to reduce model size.
quant_model_paths = quantize(onnx_model_paths)

# step 3. setup onnx runtime
model_sessions = get_onnx_runtime_sessions(quant_model_paths)

# step 4. get the onnx model
model = OnnxT5(model_or_model_path, model_sessions)

and there is the error:

Downloading (…)lve/main/config.json: 100%
1.21k/1.21k [00:00<00:00, 52.2kB/s]
Downloading model.safetensors: 100%
242M/242M [00:02<00:00, 127MB/s]
Downloading (…)neration_config.json: 100%
147/147 [00:00<00:00, 9.59kB/s]
Exporting to onnx... |##########                      | 1/3============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

Exporting to onnx... |#####################           | 2/3============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

Exporting to onnx... |################################| 3/3
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-5-36e410e261fe>](https://localhost:8080/#) in <cell line: 11>()
      9 
     10 # Step 2. (recommended) quantize the converted model for fast inference and to reduce model size.
---> 11 quant_model_paths = quantize(onnx_model_paths)
     12 
     13 # step 3. setup onnx runtime

[/content/fastT5/fastT5/onnx_exporter.py](https://localhost:8080/#) in quantize(models_name_or_path)
    278         model_name = model.as_posix()
    279         output_model_name = f"{model_name[:-5]}-quantized.onnx"
--> 280         quantize_dynamic(
    281             model_input=model_name,
    282             model_output=output_model_name,

TypeError: quantize_dynamic() got an unexpected keyword argument 'activation_type'

What is the problem here ? Can you help me on this ?

[sean7601]

I'm having the same problem

[DRXD1000]

Please check your versions installed. fastt5 requiers specific packages like onnxruntime=1.10.0
If you have installed a newer version the code wont work

[Neo-vortex]

onnxruntime

pip install onnxruntime==1.10.0
ERROR: Could not find a version that satisfies the requirement onnxruntime==1.10.0 (from versions: 1.15.0, 1.15.1, 1.16.0, 1.16.1, 1.16.2, 1.16.3)
ERROR: No matching distribution found for onnxruntime==1.10.0

[willwade]

use python 3.8
then install 1.10.0

eg.

conda create --name py38 python=3.8
conda activate py38 
conda install onnxruntime=1.10.0 -c conda-forge

then that code works