model.save and load giving different result

[dipanjannag]

I am trying to save a simple LSTM model for text classification. The input of the model is padded vectorized sentences.

model = Sequential()
model.add(LSTM(40, input_shape=(16, 32)))
model.add(Dense(20))
model.add(Dense(8, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

For saving I'm using the following snippet:

for i in range(50):
    from sklearn.cross_validation import train_test_split

    data_train, data_test, labels_train, labels_test = train_test_split(feature_set, dummy_y, test_size=0.1, random_state=i)
    accuracy = 0.0
    try:
        with open('/app/accuracy', 'r') as file:
            for line in file:
                accuracy = float(line)
    except Exception:
        print("error")
    model.fit(data_train, labels_train, nb_epoch=50)
    loss, acc = model.evaluate(feature_set, dummy_y)
    if acc > accuracy:
        model.save_weights("model.h5", overwrite=True)
        model.save('my_model.h5', overwrite=True)
        print("Saved model to disk.\nAccuracy:")
        print(acc)
        with open('/app/accuracy', 'w') as file:
            file.write('%f' % acc)

But whenever I'm trying to load the same model

from keras.models import load_model
model = load_model('my_model.h5')

I'm getting random accuracy like an untrained model. same result even when trying to load weights separately.
If I set the weights

lstmweights=model.get_weights()
model2.set_weights(lstmweights)

like above. It is working if model and model2 are run under same session (same notebook session). If I serialize lstmweights and try to load it from different place, again I'm getting result like untrained model. It seems saving only the weights are not enough. So why model.save is not working. Any known point?

[Panache1]

I'm having a similar problem, but it has to do with setting stateful=True. If I do that, the prediction from the original model is different from the prediction of the saved and reloaded model.

`# DEPENDENCIES
import numpy as np

from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.layers.recurrent import LSTM

TRAINING AND VALIDATION FILES

xTrain = np.random.rand(200,10)
yTrain = np.random.rand(200,1)
xVal = np.random.rand(100,10)
yVal = np.random.rand(100,1)

ADD 3RD DIMENSION TO DATA

xTrain = xTrain.reshape(len(xTrain), 1, xTrain.shape[1])
xVal = xVal.reshape(len(xVal), 1, xVal.shape[1])

CREATE MODEL

model = Sequential()
model.add(LSTM(200, batch_input_shape=(10, 1, xTrain.shape[2])
#, stateful=True # With this line, the reloaded model generates different predictions than the original model
))
model.add(Dense(yTrain.shape[1]))
model.add(Activation("linear"))

model.compile(loss="mean_squared_error", optimizer="rmsprop")
model.fit(xTrain, yTrain,
batch_size=10, nb_epoch=2,
verbose=0,
shuffle='False',
validation_data=(xVal, yVal))

PREDICT RESULTS ON VALIDATION DATA

yFit = model.predict(xVal, batch_size=10, verbose=1)
print()
print(yFit)

SAVE MODEL

model.save('my_model.h5')
del model

RELOAD MODEL

from keras.models import load_model
model = load_model('my_model.h5')
yFit = model.predict(xVal, batch_size=10, verbose=1)
print()
print(yFit)

DO IT AGAIN

del model
model = load_model('my_model.h5')
yFit = model.predict(xVal, batch_size=10, verbose=1)
print()
print(yFit)`

[fighting41love]

Same problem. Is there sth wrong with the function "save model"?

[bibhashthakur]

I am having the exact same issue. Does anyone know exactly what the problem is?

[FTAsr]

Same issue using json format for saving a very simple model. I get different results on my test data before and after save/loading the model.

classifier = train(model, trainSet, devSet)

TEST BEFORE SAVING

test(model, classifier, testSet)

save model to json

model_json = classifier.to_json()
with open('../data/local/SICK-Classifier', "w") as json_file:
json_file.write(model_json)

load model fron json

json_file = open('../data/local/SICK-Classifier', 'r')
loaded_model_json = json_file.read()
json_file.close()
classifier = model_from_json(loaded_model_json)

TEST AFTER SAVING

test(model, classifier, testSet)

[HarshaVardhanP]

I crosschecked all these functions - they seem to be working properly.
model.save(), load_model(), model.save_weights() and model.load_weights()

[kswersky]

I ran into a similar issue. After saving my model, the weights were changed and my predictions became random.

For my own case, it came down to how I was mixing vanilla Tensorflow with Keras. It turns out that Keras implicitly runs tf.global_variables_initializer if you don't tell it that you will do so manually. This means that in trying to save my model, it was first re-initializing all of the weights.

The flag to prevent Keras from doing this is _MANUAL_VAR_INIT in the tensorflow backend. You can turn it on like this, before training your model:

from keras.backend import manual_variable_initialization manual_variable_initialization(True)

Hope this helps!

[HarshaVardhanP]

Hi @kswersky,

Thanks for your answer.

I am using Keras 2.0 with Tensorflow 1.0 setup. I am building model in Keras and using Tensorflow pipeline for training and testing. When you load the keras model, it might reinitialize the weights. I avoided tf.global_variables_initializer() and used load_weights('saved_model.h5'). Then model got the saved weights and I was able to reproduce correct results. I did not have to do the _manual_var_init step. (its a very good answer for just Keras)

May be I confused people who are just using Keras.

[pras135]

Use model.model.save() instead of model.save()

[lotempeledGong]

I'm stuck with the same problem. The only solution for now is move to python 2.7 ?

@pras135 , if I do as you suggest I cannot perform model.predict_classes(x), AttributeError: 'Model' object has no attribute 'predict_classes'

[kefth]

Same issue here. When I load a saved model my predictions are random.

[pras135]

@lotempeledGong

model_1 = model.model.save('abcd.h5') # save the model as abcd.h5
from keras.models import load_model
model_1 = load_model('abcd.h5') # load the saved model
y_score = model_1.predict_classes(data_to_predict) # supply data_to_predict

Hope it helps.

[lotempeledGong]

@pras135 What you suggested is in the same session, and it does work. Unfortunately I need this to work in separate sessions, and if you do the following:

(in first python session)
model_1 = model.model.save('abcd.h5') # save the model as abcd.h5
(close python session)
(open second python session)
model_1 = load_model('abcd.h5') # load the saved model
y_score = model_1.predict_classes(data_to_predict) # supply data_to_predict

I receive the following error: AttributeError: 'Model' object has no attribute 'predict_classes'

[dipanjannag]

@lotempeledGong That should not happen. Check if load_model means same as keras.models.load_model in your context. This should work just fine.

[lotempeledGong]

@deeiip thanks, but this still doesn't work for me. However this is not my main problem here. What I want eventually is to train a model, save it, close the python session and in a new python session load the trained model and obtain the same accuracy. Currently when I try to do this, the loaded model gives random predictions and it is as though it wasn't trained.
By the way, in case this is a versions issue- I'm running with Keras 2.0.4 with Tensorflow 1.1.0 backend on Python 3.5.

[BBarbosa]

@lotempeledGong I'm facing exactly the same issue you refer here. However, I'm using Tensorflow 1.1 and TFlearn 0.3 on Windows10 with Python 3.5.2.

[chenlihuang]

@deeiip have you solved this problem? how to?

[chenlihuang]

@kswersky l add the from keras.backend import manual_variable_initialization
manual_variable_initialization(True) but the error came:
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable
[[Node: Variable/_24 = _SendT=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_8_Variable", _device="/job:localhost/replica:0/task:0/gpu:0"]]
[[Node: Variable_1/_27 = _Recv_start_time=0, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_10_Variable_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]]

[chenlihuang]

@HarshaVardhanP how to avoided tf.global_variables_initializer() before load model?

[HarshaVardhanP]

@chenlihuang Now, I am using tf.global_variables_initializer(). It will init net variables. And use load_weights() to load the saved weights. This seems easier for me than using load_model().

[chenlihuang]

@HarshaVardhanP can you give some ideas for me on the keras level . l don't care the backend on TF.
only easy uase keras layers. that's mean how can l solve the problem in this beacuse I didn't touch the TF
.

[Ssica]

EDIT: i've noticed loading my model doesnt give different results so I guess I dont need the workaround.

[Atvar]

Unfortunately, I've run into the same issue that many others on here seem to have encountered -- I've trained what seems to be an extremely powerful text classifier (based on cross-validation, at least, with a healthy-sized dataset), but upon loading a saved model -- either using load_model or model.load_weights -- my model's performance is now completely worthless when tested in a new session. There's absolutely no way this is the same model that I trained. I've tried every suggestion offered here, but to no avail. Super disheartening to see what appeared to be such good results and have no ability to actually use the classifier in other sessions/settings. I hope this gets addressed soon. (P.S. I'm running with Keras 2.0.4 with Tensorflow 1.1.0 backend on Python 3.5.)

[ChandrahasJR]

@gokceneraslan @fchollet Many of us are facing this issue. Could you please take a look ?

Thanks!

[kevinjos]

I do not see any issue with model serialization using the save_model() and load_model() functions from the latest Tensorflow packaged Keras.

For example:

import tensorflow.contrib.keras as keras
m = train_keras_cnn_model() # Fill in the gaps with your model
model_fn = "test-keras-model-serialization.hdf5"
keras.models.save_model(m, model_fn)
m_load = keras.models.load_model(model_fn)
m_load_weights = m_load.get_weights()
m_weights = m.get_weights()
assert len(m_load_weights) == len(m_weights)
for i in range(len(m_weights)):
    assert np.array_equal(m_load_weights[i], m_weights[i])
print("Model weight serialization test passed"

[ChandrahasJR]

Hi @kevinjos

The issue is not with using the saved model in the same session. If i save a model from session 1 and load it in session 2 and use two exactly the same data to perform inference, the results are different.

To be more specific, the inference results from the session in which the model was built is much better compared to results from a different session using the same model.

How is tensorflow packaged keras different from vanilla keras itself ?

[kevinjos]

@Chandrahas1991 When I run a similar code as above with a fresh tf session I get the same results. Have you tried the suggestion offered by @kswersky to set a flag to prevent automatic variable initialization? Could the issue with serialization apply only to LSTM layers? Or more specifically stateful LSTM layers? Have you tried using only the Keras code packaged with TensorFlow?

import tensorflow.contrib.keras as keras
from tensorflow.contrib.keras import backend as K
m = train_keras_cnn_model() # Fill in the gaps with your model
model_fn = "test-keras-model-serialization.hdf5"
m_weights = m.get_weights()
keras.models.save_model(m, model_fn)
K.clear_session()
m_load = keras.models.load_model(model_fn)
m_load_weights = m_load.get_weights()
assert len(m_load_weights) == len(m_weights)
for i in range(len(m_weights)):
    assert np.array_equal(m_load_weights[i], m_weights[i])
print("Model weight serialization test passed"

[ChandrahasJR]

@kevinjos I got this error while working with CNN's and @kswersky's solution did not work for me.

I haven't tried with keras packed in tensorflow. I'll check it out. Thanks for the suggestion!

[gokceneraslan]

There are already tests in Keras to check if model saving/loading works. Unless you write a short and fully reproducible code (along with the data) and fully describe your working environment (keras/python version, backend), it's difficult to pinpoint the cause of all issues mentioned here.

I've just sent another PR to add more tests about problems described here. @deeiip can you check #7024 and see if it's similar to the code that you use to reproduce this? With my setup, (keras master branch, python 3.6, TF backend) I cannot reproduce any model save/load issues with either mlps or convlstms, even if I restart the session in between.

Some people mentioned reproducibility problems about stateful RNNs. As far as I know, RNN states are not saved via save_model(). Therefore, there must be differences when you compare predictions before and after saving the model, since states are reset. But keep in mind that this report is not about stateful RNNs.

[fhalamos]

Well I have made it work for me. The issue presented itself only when I had a vectorization layer inside my model. I finally had to export the vectorization layer separately, and being careful to export it as a ragged tensor (check this out). Then I reconstructed the model, loading weights for vectorization layer and inner model, and all good! Hope this helps

[zaneeto]

I have encountered the same issue as well, and none of the solutions suggested work for me. Has there been a fix yet?

[KeithYJohnson]

Did anyone inspect the weights themselves to see if they changed as they were loaded in memory to see if they changed between different calls to loading the saved model?

[popkristina]

I had the identical problem like 99% of you. But since it turns out that in most of the cases the cause of the problem is different, I thought to share how I fixed mine.
This might not help many, but make sure that you check your code carefully before. In my case at least, in every new session I was extracting my class labels with list(set(..)) and then enumerating them. Turns out, I didn't notice that the set function was getting my labels in different order each time, so after I would reload the model weights and try to make predictions, I was getting zero accuracy for each class, because their label numbers were mixed up.

[LanceNorskog]

@popkristina Yes, this is the problem I found a long time ago. Python deliberately makes sets and dictionaries use randomized orderings per creation, because it is so easy to write code that accidentally depends on the enumeration order of a particular set or dict.

Keras now has text 'preprocessing' layers to do this enumeration in a way that saves the enumeration order into the model.

[JonahYeoh]

`base_model = Arch(32, outdim=8, t=32, dropout=0.1)

x = np.random.uniform(0, 1, 2 * 32 * 160 * 160 * 3)
x = np.reshape(x, newshape=(2, 32, 160, 160, 3))

basey = base_model.predict(x)
base_model.save('./base_model')

loaded_model = tf.keras.models.load_model('./base_model')
loadedy = loaded_model.predict(x)`

I tested the above code with a trained model, I realize I don't get identical output but very close result
They are very close but not the same.

Base Model

PREDICTIONS
[0.2088217 0.45167127 0.05180011 0.03961079 0.106199 0.12914863
0.0040094 0.0087391 ]
[0.20879863 0.45169848 0.05179876 0.03960407 0.10620189 0.12915517
0.00400768 0.00873537]
[0.20883007 0.45435473 0.05046646 0.03836709 0.10564606 0.13050072
0.00370049 0.0081343 ]
[0.20918027 0.4554144 0.05004811 0.03805296 0.10549351 0.13019827
0.00361463 0.00799786]
[0.20889695 0.45809868 0.04976191 0.0378258 0.10494971 0.12905534
0.00352852 0.00788297]
[0.20802347 0.4553712 0.05105066 0.0387408 0.10540991 0.12932019
0.0037673 0.0083165 ]
[0.20773412 0.45495382 0.05132396 0.03893919 0.10549735 0.12930351
0.00382477 0.0084233 ]
[0.19982255 0.48187262 0.04821869 0.03645178 0.10228756 0.12053316
0.00328366 0.00752997]
[0.19985771 0.48211494 0.04807355 0.03638516 0.10223551 0.12060079
0.00325381 0.00747852]
[0.19940008 0.48217115 0.04807775 0.03682785 0.10239523 0.1202451
0.00329613 0.00758671]
END

Loaded Model

PREDICTIONS
[0.2088217 0.45167127 0.05180011 0.03961079 0.106199 0.12914863
0.0040094 0.0087391 ]
[0.20879863 0.45169848 0.05179876 0.03960407 0.1062019 0.12915517
0.00400768 0.00873537]
[0.20883009 0.45435485 0.05046646 0.0383671 0.10564605 0.13050073
0.00370049 0.0081343 ]
[0.2091803 0.4554144 0.0500481 0.03805296 0.10549352 0.13019827
0.00361463 0.00799786]
[0.20889693 0.4580988 0.04976192 0.03782581 0.10494972 0.12905534
0.00352852 0.00788297]
[0.20802346 0.4553712 0.05105066 0.0387408 0.10540989 0.12932019
0.0037673 0.00831649]
[0.20773406 0.45495382 0.05132396 0.03893919 0.10549735 0.12930353
0.00382477 0.0084233 ]
[0.19982259 0.48187256 0.0482187 0.03645178 0.10228758 0.12053318
0.00328366 0.00752997]
[0.1998577 0.482115 0.04807354 0.03638515 0.1022355 0.12060077
0.00325381 0.00747851]
[0.19940014 0.4821711 0.04807776 0.03682785 0.10239525 0.1202451
0.00329613 0.00758671]
END

[greeshmasmenon]

I still get the issue. Is there a solution for this?

[JonahYeoh]

I got it right with checkpoints, here

[BoulmaizMohamed]

i think the problem lay in data Transformation
you need to use the same scaler
like in my case i use
sc = StandardScaler()

[axel578]

I get the same issue, 5 years later

[ivan-rivera]

I'm also dealing with this issue. FYI, I've created a detailed description of what I'm dealing with in this StackOverflow

EDIT: FYI, I recently discovered that my problem was caused by a bug on my side, not Keras.

[shivalikasingh95]

Even I'm facing this issue when I'm trying to save and load a model with custom layers using TF SavedModel format.
However, the issue doesn't occur if I save and load model using HDF5 format.
But for my use case, I need it to work using TF SavedModel format but I'm observing big drop in accuracy after loading the model post saving using SavedModel format.

I've explained the issue in the detail here

Sample colab to reproduce issue

Tensorflow version - 2.9.1
Keras version - 2.9.0

[Jm-Paunlagui]

Even I'm facing this issue when I'm trying to save and load a model with custom layers using TF SavedModel format. However, the issue doesn't occur if I save and load model using HDF5 format. But for my use case, I need it to work using TF SavedModel format but I'm observing big drop in accuracy after loading the model post saving using SavedModel format.

I've explained the issue in the detail here

Sample colab to reproduce issue

Tensorflow version - 2.9.1 Keras version - 2.9.0

I'm facing the same issue too

[shouldsee]

Maybe we should add some sanity check for model.save, to see whether the saved model reproduce some expected results?

[hanneswilms]

I've had the same problem and changed two things in my jupyter notebook:

1. changed model.save('birds/mobilenet') to model.save('./birds/mobilenet')
2. removed physical_devices = tf.config.list_physical_devices("GPU") and tf.config.experimental.set_memory_growth(physical_devices[0], True) from the beginning of my notebook

also I tried the consequences of calling model._set_inputs and model.compute_output_shape. so I trained my Model, saved it and after saving my model with model.save I added these two commands and saved in between. So my code looked like that:

history = model.fit(normalized_train,validation_data = normalized_valid,epochs = 20,batch_size=batch_size,callbacks=[model_checkpoint_callback,earlyStopping])

model.save('./birds/mobilenet')
model.evaluate(normalized_test)
model._set_inputs(tf.random.uniform((150,150,3)))
model.save('./birds/mobilenet_set_inputs')
model.compute_output_shape(input_shape=(None, 150,150,3))
model.save('./birds/mobilenet_compute_output_shape')

Apparently all three models gave the same results (in another Jupyter Notebook / Kernel) as the training / validation / test predictions directly after training.
I didn't have the time yet to test, what exactly helped. But I guess one of those things may have done it (at least for me).

[samrajkodai]

@hanneswilms

hi ,i followed the same steps as you mentioned above ,it works fine in the same teriminal but when i restart the terminal or open the new jupyter notebook for testing it gives the random prediction only.

[bjfrbjx]

I forgotten /=255 。。。。

[gtmray]

Found out that the H5 file was the issue. We should save it in SavedModel format because of these limitations of the H5 file format:
i. External losses & metrics added via model.add_loss() & model.add_metric() are not saved (unlike SavedModel).
ii. The computation graph of custom objects such as custom layers is not included in the saved file. At loading time, Keras will need access to the Python classes/functions of these objects in order to reconstruct the model

Solution (Don't include .h5):
model.save("model_path_to_save")

[kxg202]

Still having this problem. I tried all the suggested solutions and they don't work. I built a model within Jupyter Notebook and saved it to my computer using model.save(). Then I run a python application which loads my model with load_model() and uses it for prediction. However, the predictions are entirely random. Something is still wrong with either saving or loading the model. Anyone find new solutions?

[jenabesaman]

i solved this problem in my case on keras ocr captcha predict model. (you can search it in google to see code)
its about data processing and your image and characters must be sorted
i add these lines to data processing and it fixed:

max_length = max([len(label) for label in raw_labels])
characters = sorted(list(characters))

and full code become like this:

images = sorted(list(map(str, list(data_dir.glob("*.png")))))
raw_labels = [img.split(os.path.sep)[-1].split(".png")[0] for img in images]
max_length = max([len(label) for label in raw_labels])
labels = [label.ljust(max_length) for label in raw_labels]
characters = set(char for label in labels for char in label)
characters = sorted(list(characters))

for predict on images which the name of them is different from the number in the image you have to set characters .which will be like this:

data_dir = Path("captcha2")
images = sorted(list(map(str, list(data_dir.glob("*.jpg")))))
raw_labels = [img.split(os.path.sep)[-1].split(".jpg")[0] for img in images]
max_length = max([len(label) for label in raw_labels])
labels = [label.ljust(max_length) for label in raw_labels]
characters={'4', 'n', 'w', '2', 'x', 'y', 'f', 'e', 'g', 'm', '8', '6', '9', 'b', '0', 'p', '5', ' ', '7', 'd', 'c', '1', '3'}
characters = sorted(list(characters))

now you can try on your captcha images data and save model for use it in another session!

[furqan4545]

If someone is still facing this issue. Here is the solution below:

The problem is not in the saved model but in the character list that you are using to map number back to string. Everytime you restart the notebook, it resets the character list and when you load your model, it can't accurately map the numbers back to string. To resolve this issue you need to save character list. Please follow the below code.

train_labels_cleaned = []
characters = set()
max_len = 0

for label in train_labels:
  label = label.split(" ")[-1].strip()
  for char in label:
    characters.add(char)

  max_len = max(max_len, len(label))
  train_labels_cleaned.append(label)

print("Maximum length: ", max_len)
print("Vocab size: ", len(characters))

# Check some label samples
train_labels_cleaned[:10]

ff = list(characters)

# save list as pickle file
import pickle
with open("/content/drive/MyDrive/Colab Notebooks/OCR_course/characters", "wb") as fp:   #Pickling
    pickle.dump(ff, fp)

# Load character list again
import pickle
with open("/content/drive/MyDrive/Colab Notebooks/OCR_course/characters", "rb") as fp:   # Unpickling
    b = pickle.load(fp)
    print(b)

AUTOTUNE = tf.data.AUTOTUNE

# Maping characaters to integers
char_to_num = StringLookup(vocabulary=b, mask_token=None)

#Maping integers back to original characters
num_to_chars = StringLookup(vocabulary=char_to_num.get_vocabulary(), mask_token=None, invert=True)

Now when you map back the numbers to string after prediction, it will retain the original order and will predict accurately.

If you still didn't understand the logic, you can watch my video in which I explained this project from scratch and resolved all the issues that you are facing.

https://youtu.be/ZiUEdS_5Byc

[longsc2603]

@furqan4545 Thanks for pointing that out, I thought my problem was due to the saving and loading process, but it turns out to be exactly what you mentioned. ^^

[gogsoon]

i think the problem lay in data Transformation you need to use the same scaler like in my case i use sc = StandardScaler()

this was also my case...
If you want to use the same trained model in other code you must also save the scalers used.

Saving:
import pickle
scalers = [scaler_1, scaler_2, scaler_3, scaler_4, scaler_5]
with open('scalers.pkl', 'wb') as file:
pickle.dump(scalers, file)
model.save("model.hdf5")

Loading:
import pickle
with open('scalers.pkl', 'rb') as file:
scalers = pickle.load(file)
model = load_model("model.hdf5")

[DanHumfeld]

I faced a very similar issue: I save a model, I run a program that loads the model and uses it to make a prediction, and the prediction varies from run to run. Run the same code with the same model and same inputs three times in a row, get three different results. Unlike the OP, my results were all reasonable like a well-trained model would have given, but as a deterministic (but large) algebra problem, they should have been the same every time.

I solved it! I don't know why this works.
Code that resulted in different predictions each time the code was run:

import tensorflow as tf
from tensorflow import keras
model = [tf.keras.models.load_model(trained_model) for trained_model in model_file]

Code that resulted in the same predictions each time the code was run:

import tensorflow.keras as k
model = [k.models.load_model(trained_model) for trained_model in model_file]

I wrote two programs at different times and I had done the import statements differently. One gave repeatable model[index].predict(input_data) results, one didn't. So now I have a solution. Not understanding, but a solution and hopefully this helps anyone else with the same issue.

BTW: my models were .keras format, saved with k.save_model(trained_model).

[ivolis]

I tried many things.
What worked for me is:

1. Seed every source of randomness:

# Helper libraries
import numpy as np
np.random.seed(10)  # for consistency of random numbers

# TensorFlow and tf.keras
import tensorflow as tf
tf.random.set_seed(10) # tensorflow uses MOSTLY numpy, just in case.

from tensorflow import keras
keras.utils.set_random_seed(10) # idem keras

from keras.backend import manual_variable_initialization
manual_variable_initialization(True) # https://github.com/keras-team/keras/issues/4875#issuecomment-296696536

2. Maybe irrelevant for many people who is smarter than me, but I was forgetting to rescale my images when loading my model in order to evaluate it (i.e dividing by 255.0)

[tharun-vemula]

I faced a similar issue and the only solution I found was to save the model in SaveModel format https://www.tensorflow.org/tutorials/keras/save_and_load#savedmodel_format. Weights are being reinitialized when the model is saved in .keras format.