Hi,
I'm writing a paper about Data Augmentation and I'm looking for some way of increasing the size of a dataset. I'm already aware of the techniques used for images (transformations, PCA, blurs, etc.) but I'm having trouble finding augmentation techniques for classic numeric datasets such as Iris for example.
I already explored some obvious methods which are the following :
1. Uniform Random Generation : This really naive method consists of creating a new instance based on the min and max of the existing ones, the value of each feature is generated randomly with a uniform probability. (The mins and maxs are calculated from the values of the concerned feature of the concerned class each time)
2. Normal Random Generation : Same as Uniform but the probability is now a gaussian curve. Which is of course less naive since generated value fit the initial data distribution.
3. Adding Noise : This method is a little bit different since it consists of cloning initial values, but each time adding some noise to it. This methods aims to strengthen the models and prevent overfitting.
I'm kinda new to ML and data augmentation and these 3 methods can look pretty obvious, i was wondering if anyone had an idea of different methods/algorithms to do such a task.
Thanks for your advices.
DP
NB: Text Augmentation is out of my project scope but if you have any info on that, please share !
Dear Damian Petroff,
Numeric.Datasets. The datasets package defines three different kinds of datasets: Tiny datasets (up to a few tens of rows) are embedded as part of the library source code, as lists of values. ... Larger data sets which need to be fetched over the network and are cached in a local temporary directory for subsequent use.
https://www.researchgate.net/post/How_to_decide_the_best_classifier_based_on_the_data-set_provided
Regards,
Shafagat
Shafagat Mahmudova The classifier is not the subject of my question. Before even taking a classifier into consideration, the purpose of my work is to find a way to increase the dataset's size (in terms of rows), by artificially generating significant data, in order to improve the learning of a given model (could be SVC, DecisionTree, NaivesBayes, CNN, you name it...).
You may try GANs (Generative Adversarial Networks - https://skymind.ai/wiki/generative-adversarial-network-gan).
use the SMOTE plugin in Weka, it provides the facility to increase and decrease as you required
Data augmentation means increasing the variable space in such a way that a model gets more options to find good predictions/classifications. For example, adding socio-economic variables to genomic data to improve predictions or to reduce spurious associations between genes and outcomes.
From a methodological point of view what you ask for is not data augmentation. You are simply creating new data which adhere exactly to what you already know.
Simply use the model parameters as data-generation (like in a parametric bootstrap procedure) and generate data to stick to your existing data.
The result will, in general, be the same as simply dividing your standard errors by some constant related to the number of observations you include additionally.
Artificially creating significance is questionable.
You can use Variational Autoencoder
This paper can help you more
Conference Paper Industrial Internet of Things Based Ransomware Detection usi...
- Badges
- Science topic