By hyper parameters If you mean the number of layers, layer width.. etc. you can do it by making the layer node number and layer depth as functions that could be looped and the idea is to get numbers for which the loss is minimum and accuracy is maximum. But you can infact use a fully connected nueral network for regression problems. I'm confused as why you need to use an LSTM.

If you need help with your model, you can contact me at:

[email protected]

If your data is time series, then you can use LSTM model. Otherwise, you can use fully connected neural network for regression problems. In case of, you need to select the best model it is better to apply hyper-parameters optimization for the proposed model. Finally, you can follow the following tutorials:

Choosing the right Hyperparameters for a simple LSTM using Keras

https://towardsdatascience.com/choosing-the-right-hyperparameters-for-a-simple-lstm-using-keras-f8e9ed76f046

How to Tune LSTM Hyperparameters with Keras for Time Series Forecasting

https://machinelearningmastery.com/tune-lstm-hyperparameters-keras-time-series-forecasting/

Kavaskar Sekar read this

https://machinelearningmastery.com/tune-lstm-hyperparameters-keras-time-series-forecasting/

Anoop A Nair , Thanks for your comments.

Actually , I have written LSTM code for Load forecasting problem by taking actual time series data. I have load values from the year 2003-2013 for each hour . I am giving five features as input (load, season code, week day or weekend code, Holiday or not, temperature). While training, My model gives RMSE around 1200 MAE around 900. testing process takes higher error. It is not decreasing even when i change parameters( stacked lstm 2,3,4) (hidden layers 64,32,128,256,512) all worked out. What kind of tuning required.