loading layers with lambda in from .h5 files

[dmagee]

I have a large number of models trained using tensorflow.keras where the first layer is:

s = Lambda(lambda x: x / 255) (inputs)

When I try to load these in keras-core with pytorch backend I get the following error:

    new_model = keras.models.load_model('my_model.h5',compile=False)
  File "d:\Python38\lib\site-packages\keras_core\src\saving\saving_api.py", line 184, in load_model
    return legacy_h5_format.load_model_from_hdf5(filepath)
  File "d:\Python38\lib\site-packages\keras_core\src\legacy\saving\legacy_h5_format.py", line 156, in load_model_from_hdf5
    **saving_utils.compile_args_from_training_config(
  File "d:\Python38\lib\site-packages\keras_core\src\legacy\saving\saving_utils.py", line 145, in compile_args_from_training_config
    loss = _resolve_compile_arguments_compat(loss, loss_config, losses)
  File "d:\Python38\lib\site-packages\keras_core\src\legacy\saving\saving_utils.py", line 247, in _resolve_compile_arguments_compat
    obj = module.get(obj_config["config"]["name"])
TypeError: string indices must be integers

The offending function (in keras_core\src\legacy\saving\saving_utils.py) is:

def _resolve_compile_arguments_compat(obj, obj_config, module):
    """Resolves backwards compatiblity issues with training config arguments.

    This helper function accepts built-in Keras modules such as optimizers,
    losses, and metrics to ensure an object being deserialized is compatible
    with Keras Core built-ins. For legacy H5 files saved within Keras Core,
    this does nothing.
    """
    if isinstance(obj, str) and obj not in module.ALL_OBJECTS_DICT:
        obj = module.get(obj_config["config"]["name"])
    return obj

Debugging obj=<lambda> and obj_config=<lambda>.

I'm not sure why this is even being called, as I use compile=False, and so don't actually specify any optimizer, or loss at this stage. Other (.keras) models load fine. Thanks in advance!

Derek

[sachinprasadhs]

@nkovela1 , Could you please take a look into this. Thanks!

[martin-gorner]

Saving lambas in the ;keras format is not going to work. Lambdas can be arbitrary python code.
Prefer using layers.Rescaling

[dmagee]

I appreciate lambdas are a bad idea, and will definitely change the code for future models, but our customers have a lot of models with these layers in and it has worked fine in tf.keras for a long time loading and saving in h5 format (we even convert these to ONNX using tf2onnx and use them fine). In fact the conversion process to onnx removes these layers, which is fine by us as it's just a normalization layer at the beginning of the network. However surely the current behavior is not optimal ie.

    obj = module.get(obj_config["config"]["name"])
    TypeError: string indices must be integers

At the very least it should print out an informative error:

        if obj== "<lambda>":
        # print("Loading of Lambda layers is no longer supported by ..."

Most usefully it would allow the model to be loaded (perhaps with a warning, perhaps removing these layers) as has always been the behavior in tf.keras. If this is not supported by the new .keras format, fair enough flag an error when trying to save to that format (I guess that is already implemented as the latest Keras still has Lambda layers). Not supporting loading legacy models in h5 format with lambda layers when it has always been supported in tf.keras means these older models cannot be used as the basis of further training in keras-core (which seems to be the future of Keras). A lot of people have put a lot of work (and GPU time) into training these models, and I'm sure there are others out there with such models in h5 format. In our use case the following change in keras allows the model to be loaded and used fine.

from keras_core.layers import Identity

def _resolve_compile_arguments_compat(obj, obj_config, module):
    """Resolves backwards compatiblity issues with training config arguments.

    This helper function accepts built-in Keras modules such as optimizers,
    losses, and metrics to ensure an object being deserialized is compatible
    with Keras Core built-ins. For legacy H5 files saved within Keras Core,
    this does nothing.
    """
    if isinstance(obj, str) and obj not in module.ALL_OBJECTS_DICT:

        if obj== "<lambda>":
        # If obj is a string containing "<lambda>" it breaks this check, do something sensible
            obj = Identity()
        else:
        # Initialise object using obj_config
            obj = module.get(obj_config["config"]["name"])
    return obj

I appreciate if someone has used Lambdas for something more complex than us it may be harder, but at least they would get their models loaded and could fix them. Obviously in an ideal world it would have exactly the same behaviors as tf.keras, which is to load the lambda functions as Lambda layers (assuming no great changes in the python version). This would require changes elsewhere.

[alanwilter]

I'm having the same problem. Is there any solution?

[jmnum]

It looks like the load_model() function calls the legacy function without the compile argument. And compile is true by default.

if str(filepath).endswith((".h5", ".hdf5")):
return legacy_h5_format.load_model_from_hdf5(filepath)

I guess that modifying this would allow the inference to work.

[alanwilter]

@jmnum Many thanks, I just tried that and it worked!
I will create a PR.

.../keras/src/saving/saving_api.py

    if str(filepath).endswith((".h5", ".hdf5")):
        return legacy_h5_format.load_model_from_hdf5(filepath, custom_objects=None, compile=compile)

[alanwilter]

Now I have another model that's failing to load with keras 3.2.0.

It's a similar issue reported here

The patch #19438 above did not solve this one.
Running load_model() from keras 2.15 works fine, but 3.2.0 fails.

import keras_cv_attention_models  # needed for layer: 'beit>MultiHeadRelativePositionalEmbedding'
from keras.models import load_model

load_model("mymodel.h5")

Traceback (most recent call last):
  File "lib/python3.10/site-packages/keras/src/ops/operation.py", line 208, in from_config
    return cls(**config)
  File "lib/python3.10/site-packages/keras/src/layers/convolutional/depthwise_conv2d.py", line 118, in __init__
    super().__init__(
  File "lib/python3.10/site-packages/keras/src/layers/convolutional/base_depthwise_conv.py", line 106, in __init__
    super().__init__(
  File "lib/python3.10/site-packages/keras/src/layers/layer.py", line 263, in __init__
    raise ValueError(
ValueError: Unrecognized keyword arguments passed to DepthwiseConv2D: {'groups': 1}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "debug.py", line 5, in <module>
    load_model("mymodel.h5")
  File "lib/python3.10/site-packages/keras/src/saving/saving_api.py", line 183, in load_model
    return legacy_h5_format.load_model_from_hdf5(filepath, custom_objects=None, compile=compile)
  File "lib/python3.10/site-packages/keras/src/legacy/saving/legacy_h5_format.py", line 133, in load_model_from_hdf5
    model = saving_utils.model_from_config(
  File "lib/python3.10/site-packages/keras/src/legacy/saving/saving_utils.py", line 85, in model_from_config
    return serialization.deserialize_keras_object(
  File "lib/python3.10/site-packages/keras/src/legacy/saving/serialization.py", line 495, in deserialize_keras_object
    deserialized_obj = cls.from_config(
  File "lib/python3.10/site-packages/keras/src/models/model.py", line 512, in from_config
    return functional_from_config(
  File "lib/python3.10/site-packages/keras/src/models/functional.py", line 510, in functional_from_config
    process_layer(layer_data)
  File "lib/python3.10/site-packages/keras/src/models/functional.py", line 490, in process_layer
    layer = saving_utils.model_from_config(
  File "lib/python3.10/site-packages/keras/src/legacy/saving/saving_utils.py", line 85, in model_from_config
    return serialization.deserialize_keras_object(
  File "lib/python3.10/site-packages/keras/src/legacy/saving/serialization.py", line 504, in deserialize_keras_object
    deserialized_obj = cls.from_config(cls_config)
  File "lib/python3.10/site-packages/keras/src/ops/operation.py", line 210, in from_config
    raise TypeError(
TypeError: Error when deserializing class 'DepthwiseConv2D' using config={'name': 'stack_1_block_1_MB_dw_conv', 'trainable': True, 'dtype': 'float32', 'kernel_size': [3, 3], 'strides': [2, 2], 'padding': 'valid', 'data_format': 'channels_last', 'dilation_rate': [1, 1], 'groups': 1, 'activation': 'linear', 'use_bias': False, 'bias_initializer': {'module': 'keras.initializers', 'class_name': 'Zeros', 'config': {}, 'registered_name': None}, 'bias_regularizer': None, 'activity_regularizer': None, 'bias_constraint': None, 'depth_multiplier': 1, 'depthwise_initializer': {'module': 'keras.initializers', 'class_name': 'GlorotUniform', 'config': {'seed': None}, 'registered_name': None}, 'depthwise_regularizer': None, 'depthwise_constraint': None}.

Exception encountered: Unrecognized keyword arguments passed to DepthwiseConv2D: {'groups': 1}

[dmagee]

Alan, check out #19441

[sachinprasadhs]

@dmagee , If you have saved the models using .keras format, could you please try to load it using safe_mode=False some thing like keras.saving.load_model(filepath, custom_objects=None, compile=True, safe_mode=False)

[alanwilter]

The problem I see is that if you look at keras 2.15:
https://github.com/keras-team/keras/blob/r2.15/keras/layers/convolutional/base_conv.py

It has several references to groups but this is gone in the new implementation for keras 3.x: https://github.com/keras-team/keras/blob/master/keras/layers/convolutional/base_depthwise_conv.py

So, if keras 3.x does not need groups for DepthwiseConv2D (and possibly for several other classes), then @dmagee hack to skip it is fine.

I don't know much about what's done in keras, but if I want to keep compatibility with models built in keras 2.x I would use try/catch or a key check to skip the ones that are not needed anymore.

I'm using ONNX in the end so I just build my "convertor" using keras 2.15 as I need to move on.

[dmagee]

@sachinprasadhs the old models are in .h5 format (not .keras). I tried adding safe_mode=False (both with compile=True, and compile=false) and the error is still there.

[dmagee]

I just realised it's not the network that has a lambda layer in it (it seems we actually removed that layer some time ago, I should have checked), but the loss function which is a lambda (it's a custom loss with extra parameters that are passed via the lambda, which is necessary in tensorflow). At least that's where I think the lambda is.

[fchollet]

Do we have a standalone code snippet to reproduce this?

[dmagee]

I provided a model and code in #19441

[fchollet]

The code snippet in that other issue makes no reference to lambdas at all (it is also not reproducible since it refers to a file on disk). Do you have a snippet to reproduce the issue in this thread? Or is this thread purely a duplicate?

[dmagee]

I provided the file further up the thread as a download link. The two threads have become a bit confused as @alanwilter raised the DepthwiseConv2D issue here instead of starting a new thread. I pointed out there was already a thread relating to that issue (started by me). However, the example I provided demonstrates both issues. If I recall it first throws the lambda iissue, then if you fix that it throws the DepthwiseConv2D issue.

[dmagee]

For convenience, the model here too:

https://drive.google.com/file/d/1Jn9C4vTRVWgHCqAXnMfO_i-B3pK4c14I/view?usp=sharing

[ale-dg]

Hi,

Sorry, wasn't sure if I should open a new issue but just bumped into this one looking how to solve the same one. I am being shown this error no matter what I change in the load_model method:

ValueError: Sequential model 'model_4' has already been configured to use input shape (None, 7). You cannot build it with input_shape [None, 7]

I am updating an old model which has a Lambda input:

model_4 = tf.keras.Sequential(
    [
        tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=1)),
        tf.keras.layers.Conv1D(
            filters=128, kernel_size=128, padding="causal", activation="relu"
        ),
        tf.keras.layers.Dense(units=HORIZON),
    ],
    name="model_4",
)

When saving it and reloading it in the new .keras format, I'm shown the error above. I have changed all the options within tf.keras.models.load_model(), but the error persists.

Thanks,

Best

[sherwoac]

@alanwilter this gist solves the *Conv2D keras 2->3 layer groups issue for me

[alanwilter]

I gave a look again into my legacy models and found out that actually they are ok with Keras3 (after #19438).

I've just found out that actually the issue now is elsewhere... and myself to blame now 🙂