Good morning,

i’m trying to test my data with the following code. i tried to use dropout in my code but unfortunately my validation loss is lower than my training loss despite the mse for train and test seems the same 0.007 and 0.008 (without drop).

also the mse with dropout is more than without dropout as mentioned in the figures below.

https://ibb.co/ZKxB7fL

https://ibb.co/N3vpDCZ

even i tried to take 50% of my dataset as a validation/test set but the same thing

i tried to fund the reasons like mentioned here:

https://www.pyimagesearch.com/2019/10/14/why-is-my-validation-loss-lower-than-my-training-loss/

reason1: Regularization applied during training, but not during validation/testing (normally and by default the dropout is not used in validation/testing)

reason2: Training loss is measured during each epoch while validation loss is measured after each epoch

reason3: The validation set may be easier than the training set (or there may be leaks)

could you please tell me what shall i do?

should not use the dropout or what you suggest?

X_train=174200 samples X_test=85800 y_train=174200 y_test=85800

the code:

X = dataset[:,0:20].astype(float) y = dataset[:,20:22] scaler = StandardScaler() X = scaler.fit_transform(X) y = scaler.fit_transform(y) # split into train test sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33,random_state=1) print(X_train.shape, X_test.shape, y_train.shape, y_test.shape) # define the keras model model = Sequential() model.add(Dense(20, input_dim=20,kernel_initializer=’normal’)) model.add(LeakyReLU(alpha=0.1)) #model.add(Dropout(0.2)) model.add(Dense(7,kernel_initializer=’normal’)) model.add(LeakyReLU(alpha=0.1)) #model.add(Dropout(0.2)) model.add(Dense(2, activation=’linear’)) opt = SGD(lr=0.01, momentum=0.9) # compile the keras model model.compile(loss=’mean_squared_error’, optimizer=opt, metrics=[‘mse’]) # fit the keras model on the dataset history=model.fit(X, y, validation_data=(X_test, y_test),epochs=25,verbose=0) # evaluate the model _, train_mse = model.evaluate(X_train, y_train, verbose=0) _, test_mse = model.evaluate(X_test, y_test, verbose=0) print(‘Train: %.3f, Test: %.3f’ % (train_mse, test_mse)) #plot loss during training pyplot.title(‘Loss / Mean Squared Error’) pyplot.plot(history.history[‘loss’], label=’train’) pyplot.plot(history.history[‘val_loss’], label=’test’) pyplot.legend() pyplot.show()