Anyone knows how to add some noise data to my classification datasets? I am confused about add the noise data into training sets or testing sets or the whole datasets...what's more,I do not clear how can I make it
Thanks
Dear Tao Lee,
I would suggest you do the following:
Assuming you have a total of 100 data sample named "Dataset":
The random noise can be added as follows:
1. compute the random noise and assign it to a variable "Noise"
2. Add the noise to the dataset ( Dataset = Dataset + Noise)
3. Partition the Noisy Dataset into three parts:
a). 70% for training (Training data)
b). 15% for validation (Validation data)
c). 15% for testing (Testing data)
Alternatively, you can partition the Noisy Dataset into two part:
a). 75% for training (Training data)
b). 25% for testing (Testing data)
4. Then, you can then use a classifier ( Neural Network, SVM, LDA, ...) to classifier the Dataset
5. Finally, You can now examine the classification accuracy of the classifier.
I hope this information will be helpful for you
Good luck.
Add nose data into whole set and select 70% for training and 30% for validation to prevent over fitting.
Dear Tao Lee,
I would suggest you do the following:
Assuming you have a total of 100 data sample named "Dataset":
The random noise can be added as follows:
1. compute the random noise and assign it to a variable "Noise"
2. Add the noise to the dataset ( Dataset = Dataset + Noise)
3. Partition the Noisy Dataset into three parts:
a). 70% for training (Training data)
b). 15% for validation (Validation data)
c). 15% for testing (Testing data)
Alternatively, you can partition the Noisy Dataset into two part:
a). 75% for training (Training data)
b). 25% for testing (Testing data)
4. Then, you can then use a classifier ( Neural Network, SVM, LDA, ...) to classifier the Dataset
5. Finally, You can now examine the classification accuracy of the classifier.
I hope this information will be helpful for you
Good luck.
Oluwarotimi seems to have the entire learning process covered. I'll just expand a bit more on the adding the noise part.
The usual type of noise that is added to a classification dataset is Gaussian noise. Provided your dataset feature/attributes comprises of real numbers, it is actually a simple process:
Note that the scale factor w determines the degree of noise that could be added to your data. Have it too low, your dataset would become too noisy and your machine learning algorithm would not converge. If w is too high, then the noise itself would be negligible.
Hope this helps
Dear Tao Lee,
It would be more realistic to train your model on noise free dataset and then testing it on a noisy dataset.
Good luck
Ebenezer R.H.P. Isaac has said it all. The best idea for noise is the Gaussian noise. I have been using it all the time.
When a fewer training data is available, one can add a small amount of noise to create a larger data set. Each time a training sample is exposed to the model, random noise is added to the input variables making them different every time it is exposed to the model. In this way, adding noise to input samples is a simple form of data augmentation. The best way is to normalize the values and somehow add noise based on Gaussian distribution.
- Badges
- Science topic