My model is not outputting the results I expected. I don't quite know my way around ANN. After learning how to use SupervisedDBNClassification from https://github.com/albertbup/deep-belief-network I decided to try SupervisedDBNRegression using the dataset located here https://raw.githubusercontent.com/Honeson/ANN-qos_qoe/main/QoS/qosqoeNew2. During pre-training stage, I get errors less than 0.05 but get losses up to 1468356 during the fine-tuning stage. My outputs are usually the same exact value. I tried changing the learning_rate, normalizing the date, e.t.c, according to the solution here Neural Network Always Produces Same/Similar Outputs for Any Input but to no avail, I keep getting from nan outputs, to all outputs being the same or in classes like in classification problem. A portion of the code to see what I have tried is here.
X = df[columns] Y = df['StartUpDelay'].values # Splitting data X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=1337) # Data scaling min_max_scaler = MinMaxScaler() X_train = min_max_scaler.fit_transform(X_train) # Training regressor = SupervisedDBNRegression(hidden_layers_structure=[100], learning_rate_rbm=0.05, learning_rate=0.1, n_epochs_rbm=20, n_iter_backprop=100, batch_size=16, activation_function='relu') regressor.fit(X_train, Y_train) # Test X_test = min_max_scaler.transform(X_test) Y_pred = regressor.predict(X_test) print(Y_pred) print('Done.\nR-squared: %f\nMSE: %f' % (r2_score(Y_test, Y_pred), mean_squared_error(Y_test, Y_pred)))
I expect to get Y_pred such as 1002, 1653, 2764, 1121, 2112, 1452, 1324, 1331, etc but I rather got values like nan, nan, nan, ..., nan or values like 1456, 1456, 1456, 1456, ..., 1456, or values like 1111, 1222, 1333, 1111, 1222, 1333, ..., 1111, 1222, 1333.
I will appreciate detailed help as I don't quite know ANN very well. Thanks in advance.
Edit 1: I changed the learning rate again from 0.01 to 0.00001 and I got different values for the Y Predicted but quite bigger than expected result. I also got absurd metrics as
R-squared: -37862.875097
MSE: 56730616135.090736
If R2 is negative but small, maybe your data is GIGO (garbage [coming] in, garbage [coming] out). Even one random input variable can spoil everything. If R2 is negative and big, you are receiving a spurious result in an early phase of the ANN training. A good advice in science is: Have a reasonable hypothesis. In your case, try first a simple multiple linear regression to identify meaningful input variables. Then use only those as input. Good luck.
George Mengov I changed the values of some parameters again and have been getting R-squared between 0.44 to 0.47. Now, It's not negative again but very close to 0, do u think it's still caused by randomness and that I still go ahead and try the MLL?
Let me remind you that R is between -1 and +1. The square root of your R2 is therefore around 0.6 to 0.7. That is certainly not "very close to 0" but is the opposite of it. In all likelihood, you have reached the limit of what your data permits. Hopefully you have a test sample and there your R will not be too much below that. Good luck.
George Mengov Thanks very much for your time in trying to help me out. I'll play with the dependent vairables and examine my result.
I recommend reading the following reference:
- Oshima, T. Chris and Dell-Ross, Theresa, "All Possible Regressions Using IBM SPSS: A Practitioner’s Guide to Automatic Linear Modeling" (2016). Georgia Educational Research Association Conference. 1. https://digitalcommons.georgiasouthern.edu/gera/2016/2016/1
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training