Hi Amol,

Fine tuning Lstm is always an intensive task, try something as below,

Once a sequence model is set try hyperparameter tuning!! example as below !! it is very intensive !! try with fewer data and one parameter after the another !!

batch_size = [10, 20, 40, 60, 80, 100]

epochs = [10, 50, 100]

optimizer = ['SGD', 'RMSprop', 'Adagrad', 'Adadelta', 'Adam', 'Adamax', 'Nadam']

learn_rate = [0.001, 0.01, 0.1, 0.2, 0.3]

momentum = [0.0, 0.2, 0.4, 0.6, 0.8, 0.9]

init_mode = ['uniform', 'lecun_uniform', 'normal', 'zero', 'glorot_normal', 'glorot_uniform', 'he_normal', 'he_uniform']

activation = ['softmax', 'softplus', 'softsign', 'relu', 'tanh', 'sigmoid', 'hard_sigmoid', 'linear']

weight_constraint = [1, 2, 3, 4, 5]

dropout_rate = [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]

neurons = [1, 5, 10, 15, 20, 25, 30]

param_grid = dict(neurons=neurons, dropout_rate=dropout_rate, weight_constraint=weight_constraint, activation=activation, batch_size=batch_size, epochs=epochs, optimizer=optimizer, learn_rate=learn_rate, momentum=momentum, init_mode=init_mode)

grid_search = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=-1, scoring='accuracy')

other things to do,

Best,

Athiq

Thank you Athiq. We will try incorporate all suggestion.

You cannot use GridSearchCV with LSTM the way that was shown in the first post. That will work if the model is simple MLP.

Emin Mamedov so any other suggestion to tune parameters of LSTM model please?

Rawia Sammout I am sure that quick google search will return many results but the popular options are talos, hyperas/hyperopt.

It is still possible to use GridSearch for LSTM but not in the way that was shown in the first post.