This is actually an issue that how to recognize and approximate the parameters of an given system. I think, first, the enough data is necessary. Second, you can try neural networks with the adaptive learning rate.

Thanks.

Hi Andrew, I think the neural network is similar to car engine gears, and if the whole system is not working well, I think these tips could be helpful,

1 - check your data pipeline

-is your input data ok? I mean, did you check it?

It was situations that, after hours of tuning hyperparameters, I realized that my input data was garbage!

-Does your input data connected to the output data? check the connection of input to labels of your data, maybe by pairwise plotting

-shuffle your dataset! Ordered data will lead the learning process to biased results, and you should shuffle your training dataset. In some situations, shuffling will work like magic!

-reduce the imbalance of your data by adequately sampling them.

-is your data enough? If you can produce data, it is essential to have enough data.

2 - neural network issues -always start with an easy problem and then add complexity to it! For your case, I think you should start from one-dimensional to check if your neural network model is well connected and working well and, after that, add some complexity to it.

-activation functions for regression problems, you should not add something like the Relu activation function to the last layer (output layer), and you should put the Linear activation function

-Batch normalization in some cases, batch normalization is not a good option in regression problems!

-Try a different optimizer!

-Adjust your learning rate increase or decrease your model learning

-give your training time! maybe you should give your model some time to learn, so don't stop your model early and monitor the loss

I hope you find those tips helpful.

Regards

Morteza

Use this

1. # Construction of DNN Model

2. model = Sequential()

3. model.add(Dense(10, input_dim=total_cols, activation='relu'))

4. model.add(Dense(10, activation='relu'))

5. model.add(Dense(10, activation='relu'))

6. model.add(Dense(10, activation='relu'))

7. model.add(Dense(1))

8. model.compile(loss='mean_squared_error', optimizer='adam')

9. model.fit(X_train, Y_train, epochs=500,validation_split=0.2)