I am currently envisioning a deep-learning model that classifies pulse waveforms (normal vs. pathologic).
At this time, the no. of pulse dataset is about 2,000,000, and the balance of normal and pathologic label is distributed about 3:1.
Stratified 10-fold cross-validation is currently too time-consuming and hard-burden for my computer system.
So, it's my question,
When creating a deep learning model with about 2 million data, is cross-validation necessarily required?
I would like to ask one question more.
I want to predict the patient outcome by the pulse-classification results from the deep-learning model to further evaluate the applicability of my deep learning model (2 million data were extracted from about 300 patients).
Does this approach make sense?
Patients with worse outcomes inevitably have many pathologic pulses, thus, I am concerned that the deep learning model would commit cheating.
I'm still very confused because I have very few experiences with machine learning studies in the medical domain.
Thank you for your help.
Sincerely,
Dear Young-Tak Kim
Cross-validation is remarkably one of the best tools to assess model generalization.
However, if the computational cost is limiting your application, you could employ hold-out validation. Especially since you have a lot of samples.
In one step from a recent paper, I employed a 70% 30% holdout validation:Article Ensemble learning by means of a multi-objective optimization...
It would also be interesting for you to separate the patients from training and testing sets. You could use data from 200 patients during training and 100 for testing. You should still try to maintain the class ratio though.
Regarding your last question, it is difficult for me to answer without further knowledge from the data.
Related to your query, I suggest you follow:: https://www.coursera.org/lecture/industrial-iot-project-planning-machine-learning/segment-10-cross-validation-9Pf98
https://medium.com/next-gen-machine-learning/types-of-cross-validation-in-machine-learning-8bd33bf3e12f
If you have a total of 300 patient observations, I would highly recommend cross validation regardless of how much data you have per patient. As Victor Henrique Alves Ribeiro suggested, holdout validation is a good choice.
Is your pulse data periodic, in other words are your features time-dependent? If so, try techniques such as Fourier transform to reduce the amount of features per patient (feed the frequencies as features to your deep NN model).
- Badges
- Science topic
- Similar topics
- Phytochemistry
- Extracts