Keras Sequential Model Input Layer

Keras Sequential model input layer

Well, it actually is an implicit input layer indeed, i.e. your model is an example of a "good old" neural net with three layers - input, hidden, and output. This is more explicitly visible in the Keras Functional API (check the example in the docs), in which your model would be written as:

inputs = Input(shape=(784,))                 # input layer
x = Dense(32, activation='relu')(inputs)     # hidden layer
outputs = Dense(10, activation='softmax')(x) # output layer

model = Model(inputs, outputs)

Actually, this implicit input layer is the reason why you have to include an input_shape argument only in the first (explicit) layer of the model in the Sequential API - in subsequent layers, the input shape is inferred from the output of the previous ones (see the comments in the source code of core.py).

You may also find the documentation on tf.contrib.keras.layers.Input enlightening.

Setting the shape of tensorflow sequential model input layer

The input shape should be equal to the length of the input X second dimension, while the output shape should be equal to the length of the output Y second dimension (assuming that both X and Y are 2-dimensional, i.e. that they don't have higher dimensions).

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from sklearn.datasets import make_classification
from sklearn.preprocessing import OneHotEncoder
tf.random.set_seed(0)

# generate some data
X, y = make_classification(n_classes=5, n_samples=5420, n_features=212, n_informative=212, n_redundant=0, random_state=42)
print(X.shape, y.shape)
# (5420, 212) (5420,)

# one-hot encode the target
Y = OneHotEncoder(sparse=False).fit_transform(y.reshape(-1, 1))
print(X.shape, Y.shape)
# (5420, 212) (5420, 5)

# extract the input and output shapes
input_shape = X.shape[1]   
output_shape = Y.shape[1]   
print(input_shape, output_shape)
# 212 5

# define the model
model = Sequential()
model.add(Dense(64, activation='relu', input_shape=(input_shape,)))
model.add(Dense(64, activation='relu'))
model.add(Dense(output_shape, activation='softmax'))

# compile the model
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])

# fit the model
history = model.fit(X, Y, epochs=3, batch_size=512)
# Epoch 1/3
# 11/11 [==============================] - 0s 1ms/step - loss: 4.8206 - accuracy: 0.2208
# Epoch 2/3
# 11/11 [==============================] - 0s 1ms/step - loss: 2.8060 - accuracy: 0.3229
# Epoch 3/3
# 11/11 [==============================] - 0s 1ms/step - loss: 2.0705 - accuracy: 0.3989

How to change the input shape of model in Keras

The problem is that you are trying to reduce (flatten) an output with multiple None dimensions, which will not work if you want to use the output as input to another layer. You can try using a GlobalAveragePooling2D or GlobalMaxPooling2D instead:

import tensorflow as tf

yolo3 = tf.keras.models.load_model("yolo3.h5")
yolo3.trainable = False
l3 = yolo3.get_layer('leaky_re_lu_71').output
l3_flat = tf.keras.layers.GlobalMaxPooling2D()(l3)
out3 = tf.keras.layers.Dense(100*(4+1+12))(l3_flat)
out3 = tf.keras.layers.Reshape((100, (4+1+12)), input_shape=(12,))(out3)
yolo3 = tf.keras.Model(inputs=yolo3.input, outputs=[out3])

How are the input layers in Keras defined?

The first dense layer is the first hidden layer. Keras automatically provides an input layer in Sequential objects, and the number of units is defined by input_shape or input_dim.

You can also explicitly state the input layer as follows:

def get_compiled_model():
    model = tf.keras.Sequential([
        tf.keras.layers.InputLayer((6,)),
        tf.keras.layers.Dense(50, activation='relu'),     
        tf.keras.layers.Dense(30, activation='relu'),     
        tf.keras.layers.Dense(30, activation='relu'),     
        tf.keras.layers.Dense(1, activation='linear'),   
    ])

keras: add constant number to all input values

You can replace the second input of your toy model with a call to tf.ones_like:

input1 = layers.Input(shape=())
added = layers.Add()([input1, tf.ones_like(input1)])
model = keras.models.Model(inputs=input1, outputs=added)

tf.ones_like creates a tensor full of ones of the shape of the tensor passed as an argument. As this op depends only on the shape of the input tensor, you can technically create your network without a specified input shape, and it will accept any shape as input:

>>> model(3)
<tf.Tensor: shape=(), dtype=float32, numpy=4.0>
>>> model(tf.ones((1,2,3)))
<tf.Tensor: shape=(1, 2, 3), dtype=float32, numpy=
array([[[2., 2., 2.],
        [2., 2., 2.]]], dtype=float32)>

Keras model with input multiply dense layer

The Dense layer has to receive some kind of input:

import tensorflow as tf

inputs = tf.keras.layers.Input(shape=256)
weightLayer = tf.keras.layers.Dense(256)
multipled = tf.keras.layers.Dot(axes=1)([inputs, weightLayer(inputs)])
model = tf.keras.Model(inputs, multipled)

Otherwise just define a weight matrix and multiply it with your input element-wise. For example, by using a custom layer:

import tensorflow as tf

class WeightedLayer(tf.keras.layers.Layer):
  def __init__(self, num_outputs):
    super(WeightedLayer, self).__init__()
    self.num_outputs = num_outputs
    self.dot_layer = tf.keras.layers.Dot(axes=1)

  def build(self, input_shape):
    self.kernel = self.add_weight("kernel",
                                  shape=[int(input_shape[-1]),
                                         self.num_outputs])

  def call(self, inputs):
    return self.dot_layer([inputs, self.kernel])

inputs = tf.keras.layers.Input(shape=256)
weighted_layer = WeightedLayer(256)
multipled = weighted_layer(inputs)
model = tf.keras.Model(inputs, multipled)

keras sequential tensor as argument

You cannot use tf.keras.applications.xception.preprocess_input() inside the sequential model. You have to define it outside the model and can pass the output of it to the sequential model by assigning values to the tensor argument in the input layer.

x=tf.random.uniform(shape=(1,128,128,3))
x= tf.keras.applications.xception.preprocess_input(x)
theModel=tf.keras.models.Sequential([    
   tf.keras.Input(tensor=x),
   base_model,
   tf.keras.layers.Flatten(),
   tf.keras.layers.Dense(128, activation='relu'),
   tf.keras.layers.Dense(1,activation='sigmoid')
])

For more details, Please refer to this gist.Thank You!

---

Related Topics