my regression is somehow stuck in certain number between 200, and i don't know why, i am already compared using 3 regression, i have 2300 data, and i split the train data and test data to (0.8 and 0.2) and here is the result:
RMSE : 29.459753 Parameter: Result Method: PLSRegression()
RMSE : 32.055038 Parameter: Result Method: SVR(kernel='sigmoid')
RMSE : 6.838304 Parameter: Result Method: RandomForestRegressor(n_estimators=400)
testing data that expected number : 174
result :
1. 207.0 Parameter: Hasil Method: PLSRegression()
2. 208.0 Parameter: Hasil Method: SVR(kernel='sigmoid')
3. 198.0 Parameter: Hasil Method: RandomForestRegressor(n_estimators=400)
and i have the plot for this, the first pic is PLS regression, second is SVR and last one is random forest. Random forest graph looks good but when in comes to predicting the external data the output is far, and i try to feature scaling and the result is worst
Take a look at the attached screenshot of a book that should help you with your problem. Best wishes David Booth
Koen Van de Moortel i'm working on cholesterol prediction with infrared as feature
So, x = intensity of some IR wavelength, and y = choleresterol% or something like that? It looks like the relationship is more or less logistic. Dit you test that?
It can be done easily with my software (25 day free trial):
www.lerenisplezant.be/fitting.htm
Koen Van de Moortel that is correct, and i did try logistic. What is the best logistic regression library for my case?
As I said, it's easy with my software "FittingKVdm". I can show me if you can share your data.
Koen Van de Moortel Can your software do the prediction with the external file? also here's my dataset, kindly please reply
Daffa Rabbani Can you also explain what the numbers are? What exactly is measured, in which units, what is the error margin?
It also seems to me that they are calulated from something else, since there are many numbers the same with many "significant" digits! So what are the original measurements?
Koen Van de Moortel so im using spectroscopy device called as7265x and the column of the feature represent the wavelength ( you can view the documentation here https://learn.sparkfun.com/tutorials/spectral-triad-as7265x-hookup-guide/all), and i'm also using conventional cholesterol test (you have to prick your finger to run the test). I choose random people for sample, first they using the conventional cholesterol test and after the result comes i use that number for the label, and after that they have to put his finger above the as7265x censor, i am looping the censor to take 100 or 20 shots of infrared so the number on the feature produced by the censor and i took the data from different 26 people
Daffa Rabbani So the colums are intensities/transmittances/reflectances??? at different wavelenghts? In what units? They are clearly calculated from something else; what? Log, ratio???? Or are these numbers the original data from the device? And then, where are the conventional test results to compare with?
Koen Van de Moortel that are the original numbers from the device (you can check this video for demonstration https://www.youtube.com/watch?v=qdHy9j06gng&t=449s&ab_channel=HowToElectronics ), column [result] is the number of the cholesterol with conventional device and all the numbers is from the person scanning their finger with as7265x. Here's the example person a run a cholesterol test with conventional device and got 200 of cholesterol amount, and after that he put his finger and scanned with as7265x and produced those numbers
This conventional result, what is it exactly? A concentration, a ratio, the log of a concentration,... Does it have an upper or lower boundary? What is the precision? These things are important to know for the modeling.
I would start by taking the average and standard deviation of the spectrometer readings, and then look which wavelengths are the most responsive to the cholesterol.
Daffa Rabbani Are you sure all your measurements were done in the same light circumstances (in a closed box)? I see some strange irregularities...
i also performed different circumstances, the data took in a room with enough light but i usually block the light to give different circumstances, because i want my device can be applied in different circumstances. Does that really affect the result much?
Oh yes, that will totally ruin the measurements! The instrument is calibrated to use it's own light source (the 3 different LEDs)! You have to put in in a box with a white inside (ideally covered with Barium sulphate).
oh i didn't think about this, okay i will add some lid in the top to cover enough the finger and the censor. I would like to give you the update about this research later Koen Van de Moortel , thanks for the suggestion
Daffa Rabbani And... how big are your samples? Looking at how the device is built, I think the sample surface should be at least the same size. I don't think you can just put one drop of blood in front of it. Check the manual.
Daffa Rabbani And I still don't understand where you got these long numbers from. The sensor produces normally short numbers (since the precision is only 12%); see:
https://how2electronics.com/interfacing-triad-spectroscopy-sensor-as7265x-with-arduino/
Koen Van de Moortel actually i didn't put the blood on the censor, i put the person finger above the censor (no blood in the finger)
Daffa Rabbani That will not work. You have to put a drop of blood in a thin flat recipient with water or so, as it is shown (very short) in the video.
I guess your professor will know these things.
Koen Van de Moortel my main goals is to build non-invasive device for cholesterol test, and why is that wont work? isn't the light penetrate the skin? i already observe the graph when the censor scan my finger, the graph is different to the different person
- Badges
- Science topic
- Similar topics
- Phytochemistry
- Extracts