With the big data systems, one of the big questions for companies is how to keep these projects well-targeted and efficient. Many of the tools and resources built for big data are built to suction up vast amounts of information in a wide net. They're not always as attentive toward refining that data, and keeping it simple.
Sir,
After data collection then complete various stages of data mining process. See the following paper for applicable for stages.
http://www.iasri.res.in/sscnars/data_mining/1-Data%20Mining%20_Lecture%20Note.pdf
Meng (others also) has shown that adding more data (particularly administrative data or big data) via merging with high quality survey data can yield worse estimates that using the high quality sample survey data alone.
Meng. X. L. (2014), Chapter 45: A trio of inference problems that could win you a Nobel Prize in statistics (if you help fund it), in monograph related to 50th Anniversary of COPSS, http://www.stat.harvard.edu/Faculty_Content/meng/COPSS_50.pdf.
There are principled methods of cleaning up a data source. If you have 10% error in your data, how will you analyze it? How will you even know how much error that you have?
https://sites.google.com/site/dinaworkshop2015/invited-speakers
Quality is about making the right choices among many alternatives.
The best approach to eliminate noise or outliers in data set is a conscious cleaning of the dataset before the analysis.
By treat certain data sets as most critical to get the most business intelligence with the least effort.
By treat certain data sets as most critical to get the most business intelligence with the least effort.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Analysis
- Data Analysis
- Dataset
- Big Data