Hi guys,
I'm still newbie in big data analysis.
I'm currently looking to do incomplete data analysis for the big data in R rattle package.
I refer this book for my reference to do an analysis but it is not specifically focus on the big data.
http://mineriaddatos.wikispaces.com/file/view/Data+Mining+With+Rattle+and+R_+The+Art+of+Excavating+Data+for+Knowledge+Discovery+-+Graham+Williams.pdf
I wish to get your knowldege sharing or opinion to do analysis the big data:
1) Any recommended data set that pretty enough with the big data size?
2) What is best size of data set that we should considered as a big dataset?
3) Any the best recommended tutorial to practice for the incomplete big data analysis ?
Thank you in advance
the very first thing that i can suggest you is going through some coefficient analysis, like genetic algorithm. So you can merge your variables and reduce classes. Then you can go for KB forming. The most important thing in controlling large database is KB (knowledge base) system building up
Hi,
First, you will have to understand 3V of Big Data. Choice of tool for data exploration. You must have very keen knowledge in Machine learning Basic Technique to build up co-efficient system!
You are welcome to ask more if you will have specific query.
Regards
hello,
Muddsair Sharif & Mahdi Nakhavali Said right used co-efficient system for the analysis the big data set.
And also there are lots of tools available for big data analysis.
GridGain
Jaspersoft
Upsight
Karmasphere
Infochimps Cloud
check it out this paper u got more details about this tools.
http://ir.inflibnet.ac.in/bitstream/1944/1890/1/51.pdf
It is easy to jump into complex analyses without considering easy first looks at a data set. For example, a classical method for big-data matrices is principle component analysis (PCA) using an eigen value 1 cutoff, followed by with Varimax rotation -- many stat programs do this as a one-shot option. The intent is to get some idea of the number of component dimensions, their relative magnitudes and perhaps some understanding of their nature. Two of my early mentors [from different traditions], did this routinely with scans for extreme values and variables that had near unity, if not unity correlations -- as a check for inadvertent redundancies and gross errors in the input variables. It was often quite enlightening to see hidden in data gross values that were ignored when a previous analyst had jumped into a complex learning algorithm etc with impossible values and/or ..the off inadvertent redundancies clobbered the reaching of a stable result [after profound computer time costs]. Likewise, they often unexpectedly found that one or two components acounted for virtually all the relationships in the data (e.g,95% variation on a single component in one case and virtually no commonality (component variation on a primary variable of interest [both cases went unpublished due to the political embarrassment of months chasing respective data sets.]
I would suggest you scan the data -- perhaps using a classical PCA -- and screenings mentioned above -- as a first step.
Hi,
Below is the link related to the sample daily high-frequency data supplied by NYSE.
ftp://ftp.nyxdata.com/Historical%20Data%20Samples/Daily%20TAQ/
Best,
Yuxing
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Graphs