Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

error: 'tf.TensorListSetItem' op is neither a custom op nor a flex op while trying to quantize the model into int8 #76

Open
nitolpalak opened this issue Apr 29, 2024 · 4 comments
Open

error: 'tf.TensorListSetItem' op is neither a custom op nor a flex op while trying to quantize the model into int8 #76

nitolpalak opened this issue Apr 29, 2024 · 4 comments

Comments

@nitolpalak
Copy link

Hello,

I am trying to learn about quantization so was playing with a github repo trying to quantize it into int8 format. I have used the following code to quantize the model.

modelClass = DTLN_model()
modelClass.build_DTLN_model(norm_stft=False)
modelClass.model.load_weights(model_path)

converter = tf.lite.TFLiteConverter.from_keras_model(modelClass.model)
converter.experimental_new_converter = True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [
    tf.lite.OpsSet.TFLITE_BUILTINS, 
    tf.lite.OpsSet.SELECT_TF_OPS
]

converter._experimental_lower_tensor_list_ops = False
converter.target_spec.supported_types = [tf.int8]
converter.representative_dataset = lambda: generate_representative_data(num_samples)

tflite_model = converter.convert()

with open('saved_model.tflite', 'wb') as f:
    f.write(tflite_model)

And for the representative data, I have converted the data into numpy, saved them as .npy and then used the following code to use them as representative data.

But after I run the code I get the following error:

error: 'tf.TensorListSetItem' op is neither a custom op nor a flex op

I have tried to follow the doc and some github issues like tensorflow/tensorflow#34350 (comment) and also went through a similar question Issue with tf.ParseExampleV2 when converting to Tensorflow Lite : "op is neither a custom op nor a flex op"

But none of those seemed to be helpful in my case.

Can anyone help me figure out what I am doing wrong? Thanks in advance.

I am adding the full error in my first comment.
@nitolpalak
Copy link
Author

The full error:

error: 'tf.TensorListSetItem' op is neither a custom op nor a flex op
error: failed while converting: 'main': 
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select 
TF Select ops: Angle, Exp, IRFFT, TensorListReserve, TensorListSetItem, TensorListStack
Details:
        tf.Angle(tensor<?x?x257xcomplex<f32>>) -> (tensor<?x?x257xf32>) : {device = ""}
        tf.Exp(tensor<?x?x257xcomplex<f32>>) -> (tensor<?x?x257xcomplex<f32>>) : {device = ""}
        tf.IRFFT(tensor<?x?x257xcomplex<f32>>, tensor<1xi32>) -> (tensor<?x?x512xf32>) : {device = ""}
        tf.TensorListReserve(tensor<2xi32>, tensor<i32>) -> (tensor<!tf_type.variant<tensor<?x128xf32>>>) : {device = ""}
        tf.TensorListSetItem(tensor<!tf_type.variant<tensor<?x128xf32>>>, tensor<i32>, tensor<?x128xf32>) -> (tensor<!tf_type.variant<tensor<?x128xf32>>>) : {device = "", resize_if_index_out_of_bounds = false}
        tf.TensorListStack(tensor<!tf_type.variant<tensor<?x128xf32>>>, tensor<2xi32>) -> (tensor<?x?x128xf32>) : {device = "", num_elements = -1 : i64}

Traceback (most recent call last):
  File "/home/palak/Projects/Personal_Projects/integration/main.py", line 50, in <module>
    tflite_model = converter.convert()
  File "/home/palak/Projects/Personal_Projects/integration/integration_test/lib/python3.10/site-packages/tensorflow/lite/python/lite.py", line 1139, in wrapper
    return self._convert_and_export_metrics(convert_func, *args, **kwargs)
  File "/home/palak/Projects/Personal_Projects/integration/integration_test/lib/python3.10/site-packages/tensorflow/lite/python/lite.py", line 1093, in _convert_and_export_metrics
    result = convert_func(self, *args, **kwargs)
  File "/home/palak/Projects/Personal_Projects/integration/integration_test/lib/python3.10/site-packages/tensorflow/lite/python/lite.py", line 1601, in convert
    saved_model_convert_result = self._convert_as_saved_model()
  File "/home/palak/Projects/Personal_Projects/integration/integration_test/lib/python3.10/site-packages/tensorflow/lite/python/lite.py", line 1582, in _convert_as_saved_model
    return super(TFLiteKerasModelConverterV2, self).convert(
  File "/home/palak/Projects/Personal_Projects/integration/integration_test/lib/python3.10/site-packages/tensorflow/lite/python/lite.py", line 1371, in convert
    result = _convert_graphdef(
  File "/home/palak/Projects/Personal_Projects/integration/integration_test/lib/python3.10/site-packages/tensorflow/lite/python/convert_phase.py", line 212, in wrapper
    raise converter_error from None  # Re-throws the exception.
  File "/home/palak/Projects/Personal_Projects/integration/integration_test/lib/python3.10/site-packages/tensorflow/lite/python/convert_phase.py", line 205, in wrapper
    return func(*args, **kwargs)
  File "/home/palak/Projects/Personal_Projects/integration/integration_test/lib/python3.10/site-packages/tensorflow/lite/python/convert.py", line 984, in convert_graphdef
    data = convert(
  File "/home/palak/Projects/Personal_Projects/integration/integration_test/lib/python3.10/site-packages/tensorflow/lite/python/convert.py", line 366, in convert
    raise converter_error
tensorflow.lite.python.convert_phase.ConverterError: Could not translate MLIR to FlatBuffer. UNKNOWN: /home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.Angle' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.Exp' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListReserve' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListStack' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListReserve' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListStack' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.IRFFT' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListReserve' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListStack' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListReserve' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListStack' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListSetItem' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: called from
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListSetItem' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: called from
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListSetItem' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: called from
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: error: 'tf.TensorListSetItem' op is neither a custom op nor a flex op
tflite_model = converter.convert()
^
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: called from
tflite_model = converter.convert()
^
<unknown>:0: note: loc(fused["StatefulPartitionedCall:", "StatefulPartitionedCall"]): called from
/home/palak/Projects/Personal_Projects/integration/main.py:50:1: note: Error code: ERROR_NEEDS_FLEX_OPS
tflite_model = converter.convert()
^
<unknown>:0: error: failed while converting: 'main': 
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select 
TF Select ops: Angle, Exp, IRFFT, TensorListReserve, TensorListSetItem, TensorListStack
Details:
        tf.Angle(tensor<?x?x257xcomplex<f32>>) -> (tensor<?x?x257xf32>) : {device = ""}
        tf.Exp(tensor<?x?x257xcomplex<f32>>) -> (tensor<?x?x257xcomplex<f32>>) : {device = ""}
        tf.IRFFT(tensor<?x?x257xcomplex<f32>>, tensor<1xi32>) -> (tensor<?x?x512xf32>) : {device = ""}
        tf.TensorListReserve(tensor<2xi32>, tensor<i32>) -> (tensor<!tf_type.variant<tensor<?x128xf32>>>) : {device = ""}
        tf.TensorListSetItem(tensor<!tf_type.variant<tensor<?x128xf32>>>, tensor<i32>, tensor<?x128xf32>) -> (tensor<!tf_type.variant<tensor<?x128xf32>>>) : {device = "", resize_if_index_out_of_bounds = false}
        tf.TensorListStack(tensor<!tf_type.variant<tensor<?x128xf32>>>, tensor<2xi32>) -> (tensor<?x?x128xf32>) : {device = "", num_elements = -1 : i64}

@StuartIanNaylor
Copy link

https://github.com/SaneBow/PiDTLN

@nitolpalak
Copy link
Author

Hello @StuartIanNaylor . Thank you for your reply. But I am not sure how this helps in my case. Will you please explain me if possible? Sorry for the inconvenience.

@StuartIanNaylor
Copy link

Hello @StuartIanNaylor . Thank you for your reply. But I am not sure how this helps in my case. Will you please explain me if possible? Sorry for the inconvenience.

Kudos to the cool tool tflite2tensorflow made by PINTO0309. I used it to convert DTLN-aec tflite models to quantized version.
Quantised models here using pinto's tools as quantising was problematic
https://github.com/SaneBow/PiDTLN/tree/main/models

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@StuartIanNaylor @nitolpalak