getting ValueError: Error when checking input: expected dense_33_input to have shape (2026,) but got array with shape (20,)

[sauravanand542]

I used this code to apply genetic algorithm on other data. I have changed the code accordingly
train.txt
main.txt
network.txt
optimizer.txt
I have made changes in the 'train' and 'main' files

instead of get_mnist()/get_cifar10(), I have created my own function get_bank()
which goes like this:
`"""
Utility used by the Network class to actually train.

Based on:
https://github.com/fchollet/keras/blob/master/examples/mnist_mlp.py

"""
from keras.datasets import mnist, cifar10
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.utils.np_utils import to_categorical
from keras.callbacks import EarlyStopping
import pandas as pd

Helper: Early stopping.

early_stopper = EarlyStopping(patience=5)

def get_bank():
"""Retrieve the CIFAR dataset and process the data."""
#set defaults
nb_classes = 2
batch_size = 100
input_shape = (2026,)

# Get the data.
data = pd.read_csv('clean.csv')
array = data.values
x_train = array[0:2026,:20]
x_test = array[2027:4053,:20]
y_train = array[2027:4053,21]
y_test = array[2027:4053,21]

#x_train = x_train.astype('float32')
#x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255


# convert class vectors to binary class matrices
y_train = to_categorical(y_train, nb_classes)
y_test = to_categorical(y_test, nb_classes)

return (nb_classes, batch_size, input_shape, x_train, x_test, y_train, y_test)

def compile_model(network, nb_classes, input_shape):
"""Compile a sequential model.

Args:
    network (dict): the parameters of the network

Returns:
    a compiled network.

"""
# Get our network parameters.
nb_layers = network['nb_layers']
nb_neurons = network['nb_neurons']
activation = network['activation']
optimizer = network['optimizer']

model = Sequential()

# Add each layer.
for i in range(nb_layers):

    # Need input shape for first layer.
    if i == 0:
        model.add(Dense(nb_neurons, activation=activation, input_shape=input_shape))
    else:
        model.add(Dense(nb_neurons, activation=activation))

    model.add(Dropout(0.2))  # hard-coded dropout

# Output layer.
model.add(Dense(nb_classes, activation='softmax'))

model.compile(loss='categorical_crossentropy', optimizer=optimizer,
              metrics=['accuracy'])

return model

def train_and_score(network, dataset):
"""Train the model, return test loss.

Args:
    network (dict): the parameters of the network
    dataset (str): Dataset to use for training/evaluating

"""

nb_classes, batch_size, input_shape, x_train, x_test, y_train, y_test = get_bank()


model1 = compile_model(network, nb_classes, input_shape)

model1.fit(  x_train, y_train, 
	batch_size=batch_size,
	epochs=10000, verbose=0, 
	validation_data=(x_test, y_test), 
	callbacks=[early_stopper] )

score = model1.evaluate(x_test, y_test, verbose=0)

return score[1]  # 1 is accuracy. 0 is loss.

`

`ValueError Traceback (most recent call last)
in ()
112
113 if name == 'main':
--> 114 main()

in main()
109 (generations, population))
110
--> 111 generate(generations, population, nn_param_choices, dataset)
112
113 if name == 'main':

in generate(generations, population, nn_param_choices, dataset)
61
62 # Train and get accuracy for networks.
---> 63 train_networks(networks, dataset)
64
65 # Get the average accuracy for this generation.

in train_networks(networks, dataset)
22 pbar = tqdm(total=len(networks))
23 for network in networks:
---> 24 network.train(dataset)
25 pbar.update(1)
26 pbar.close()

~\network.py in train(self, dataset)
46 """
47 if self.accuracy == 0.:
---> 48 self.accuracy = train_and_score(self.network, dataset)
49
50 def print_network(self):

~\train.py in train_and_score(network, dataset)
99 validation_data=(x_test, y_test),
100 callbacks=[early_stopper] )
--> 101
102 score = model1.evaluate(x_test, y_test, verbose=0)
103

~\Anaconda3\lib\site-packages\keras\engine\training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
1152 sample_weight=sample_weight,
1153 class_weight=class_weight,
-> 1154 batch_size=batch_size)
1155
1156 # Prepare validation data.

~\Anaconda3\lib\site-packages\keras\engine\training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
577 feed_input_shapes,
578 check_batch_axis=False, # Don't enforce the batch size.
--> 579 exception_prefix='input')
580
581 if y is not None:

~\Anaconda3\lib\site-packages\keras\engine\training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
143 ': expected ' + names[i] + ' to have shape ' +
144 str(shape) + ' but got array with shape ' +
--> 145 str(data_shape))
146 return data
147

ValueError: Error when checking input: expected dense_33_input to have shape (2026,) but got array with shape (20,)`

[xWuWux]

The error you're encountering is related to the input shape mismatch between the expected input shape of the neural network model and the shape of the input data. Here's a step-by-step breakdown of the issue and suggestions for improvement:

Issue:
The error message indicates that the neural network model expects an input shape of (2026,), but the actual input data has a shape of (20,).

Analysis:
The input shape is determined by the input_shape parameter when creating the first layer of the neural network. In your case, the first layer is defined in the compile_model function.

The get_bank function returns (2026,) as the input_shape, which seems to be the expected input shape for the neural network.

However, during training in the train_and_score function, it appears that the actual input data (x_train and x_test) has a shape of (20,).

Possible Solutions:
Check Data Dimensions:

Ensure that the input data loaded from the CSV file (clean.csv) has the correct dimensions. It seems that the data might be loaded incorrectly, resulting in a shape mismatch.
Review Data Loading:

Review the data loading process in the get_bank function. Confirm that the features (x_train and x_test) have the correct shape. You might need to reshape or preprocess the data appropriately.

x_train = array[0:2026, :20]
Check Feature Dimension:

Confirm that the number of features matches the expected input shape. If the neural network expects (2026,) as input, each training example should have 2026 features.
Debugging:

Print the shapes of x_train and x_test before training to understand their dimensions:
print("Shape of x_train:", x_train.shape) print("Shape of x_test:", x_test.shape)
Review Model Architecture:

Double-check the architecture of your neural network model in the compile_model function. Ensure that the first layer is correctly configured with the expected input shape.
model.add(Dense(nb_neurons, activation=activation, input_shape=input_shape))
Reshape Data if Necessary:

If the data has the correct information but is not shaped appropriately, you might need to reshape it. For example:
x_train = x_train.reshape((num_samples, 2026))
Check CSV Data:
Inspect the contents of your CSV file to ensure that the data is formatted correctly.

Additional Suggestions:

Validation Set:
Consider using a separate validation set during training to monitor model performance and prevent overfitting.

Normalization:
Ensure that data normalization is applied consistently during training and testing.

Callback Monitoring:
Monitor the model during training using Keras callbacks to save the best model or perform early stopping.
By addressing these points, you should be able to identify and resolve the input shape mismatch issue.