How to Save Final Model Using Keras

what actually model.save() saves in Keras?

1. It saves weights
2. Yes
3. For saving weights for best epoch, use chunk of code i have given below
4. No

What actually keras model.save() is designed to save the weights after 100 epochs completion?. Yes it does, but have a look at following code for saving weights of only best epochs.

Use this chunk of code to:

1. Save weights of best epochs only
2. Update weights after every epoch only if given criteria is improved (val_loss is min)
3. Additionally, history after each epoch will be save in .csv file.

Code

import pandas as pd
from keras.callbacks import EarlyStopping, ModelCheckpoint

#Stop when val_loss is not decreasing
earlyStopping = EarlyStopping(monitor='val_loss', patience=10, verbose=0, mode='min')

#Save the model after every epoch.
checkpointer = ModelCheckpoint(filepath='Model_1_weights.h5', verbose=1, save_best_only=True)

#history variable will save training progress after each epoch 
history = model.fit(X_train, y_train, batch_size=20, epochs=40, validation_data=(X_valid, y_valid), shuffle=True, callbacks=[checkpointer, earlyStopping])
#Save progress of each epoch in .csv file 
hist_df = pd.DataFrame(history.history) 
hist_csv_file = 'History_Model_1.csv'
with open(hist_csv_file, mode='w') as f:
    hist_df.to_csv(f)

Link: https://keras.io/callbacks/#ModelCheckpoint

how to save best weights and best model using keras

Solution 1 (at the end of your training):

You can try using the below snippet, at the end of your training to save the weights and the model architecture separately.

from tensorflow.keras.models import model_from_json
model_json = model.to_json()
with open("model.json", "w") as json_file:
    json_file.write(model_json)
model.save_weights("model.h5")

Solution 2 (during the training):

We can observe that the model architecture does not change during the training, only the weights. Therefore, you can use this checkpoint to save only the best weights during the training, and at the beginning/end of the training save only the model_from_json.

checkpoint = ModelCheckpoint(filepath1, 
                             monitor='val_acc',
                             verbose=1,
                             save_best_only=True, 
                             save_weights_only=True, 
                             mode='max')

....training runs.....
......................
....training ends.....

from tensorflow.keras.models import model_from_json
model_json = model.to_json()
with open("model.json", "w") as json_file:
    json_file.write(model_json)

If there is nothing saved, ensure you have the correct filepath1.

Keras Save Model

Yes, you cannot save this model, because this part here is incorrect:

get_custom_objects().update({'swish': Activation(swish)})

Since you are using `"swish" as an activation function, it should not be a layer, so you just need to reference the function:

get_custom_objects().update({'swish': swish})

Then the model will train and can be saved successfully.

Keras: How to save models or weights?

First of all, it looks like you are using the tf.keras (from tensorflow) implementation rather than keras (from the keras-team/keras repo). In this case, as stated in the tf.keras guide :

When saving a model's weights, tf.keras defaults to the checkpoint
format. Pass save_format='h5' to use HDF5.

On the other hand, note that adding the callback ModelCheckpoint is, usually, roughly equivalent to call model.save(...) at the end of each epoch, so that's why you should expect three files to be saved (according to the checkpoint format).

The reason it's not doing so is because, by using the option save_weights_only=True, you are saving just the weights. Roughly equivalent to replace the call to model.save for model.save_weights at the end of each epoch. Hence, the only file that's being saved is the one with the weights.

From here, you can proceed in two different ways:

Storing just the weights

You need your model (the structure, let's say) to be loaded beforehand and then call model.load_weights instead of keras.models.load_model:

model = MyModel(...)  # Your model definition as used in training
model.load_weights(file_checkpoint)

Note that in this case, you won't have problems with custom definitions (my_cost_MSE) since you are just loading model weights.

Storing the whole model

Another way to proceed is to store the whole model and load it accordingly:

cp_callback = k.callbacks.ModelCheckpoint(
    checkpoint_dir,verbose=1,
    save_weights_only=False
)    
parallel_model.compile(
    optimizer=tf.keras.optimizers.Adam(lr=learning_rate),
    loss=my_cost_MSE,
    metrics=['accuracy']
)

model.fit(..., callbacks=[cp_callback])

Then you could load it by:

model = k.models.load_model(file_checkpoint, custom_objects={"my_cost_MSE": my_cost_MSE})

Note that in this latter case, you need to specify custom_objects since its definition is needed to deserialize the model.

Save Keras Model and Reuse It For Transfer Learning

The recommended way to save model, is saving with SavedModel format:

dir = "target_directory"
model_cnn.save(dir) # it will save a .pb file with assets and variables folders 

Then you can load it:

model_cnn = tf.keras.models.load_model(dir)

Now, you can add some layers and make another model. For example:

input = tf.keras.Input(shape=(128,128,3))
x = model_cnn(input)
x = tf.keras.layers.Dense(1, activation='sigmoid')(x)
new_model = tf.keras.models.Model(inputs=input, outputs=x)

---

Related Topics