LSTM not quantized weights after model_save_quantized_weights

[vkoriukina]

I'm trying to quantize LSTM network from the notebook you have: https://github.com/google/qkeras/blob/eb6e0dc86c43128c6708988d9cb54d1e106685a4/notebook/QRNNTutorial.ipynb.
After seeing this issue I've changed the config file to look like this:

bits = 8
quantizer_config = {
  "bidirectional": {
      'activation' : f"quantized_tanh({bits}, 0, alpha=1)",
      'recurrent_activation' : f"quantized_relu({bits}, 0, alpha=1)",
      'kernel_quantizer' : f"quantized_bits({bits}, 0, alpha=1)",
      'recurrent_quantizer' : f"quantized_bits({bits}, 0, alpha=1)",
      'bias_quantizer' : f"quantized_bits({bits}, 0, alpha=1)",
  },
  "dense": {
      'kernel_quantizer' : f"quantized_bits({bits}, 0, alpha=1)",
      'bias_quantizer' : f"quantized_bits({bits}, 0, alpha=1)",
  },
  "embedding_act": f"quantized_bits({bits}, 0, alpha=1)",
}

I'm training this model and I apply model_save_quantized_weights function. Then when I print weights, they are still in floating point:

model_save_quantized_weights(qmodel, "quant_weights.h5")
for layer in qmodel.layers:
  for i, weights in enumerate(layer.get_weights()):
    print(weights)

The example of printed weights:

[[ 0.08662941 -0.05719738 -0.05291974 ... -0.6543944   0.13776235
   0.39616233]
 [ 0.1125139  -0.09429312  0.16143066 ...  0.12786183  0.1350617
  -0.02886106]
 [ 0.14597955  0.11171963  0.14480615 ...  0.31972137  0.17480904
  -0.15030576]
 ...
 [ 0.03954179 -0.01506722 -0.09103195 ... -0.11322258  0.07701313
  -0.12551346]
 [-0.02650027  0.0823105  -0.01624984 ...  0.2262283   0.08772285
  -0.17474762]
 [-0.11531919 -0.02932754  0.1707585  ...  0.18108878  0.03475188
  -0.16486846]]

Could you, please, guide me, what should I do get int8 weights?

[danielemoro]

Hi. The function model_save_quantized_weights returns a dictionary of the quantized weights. Have you checked if that dictionary has correctly quantized the weights?

See line

qkeras/qkeras/utils.py

Line 122 in 7ba2cb2

Returns: