I do not know What is a Mathematical Definition of Big Data?
Can you help me?
This is an important topic.
A good start towards an answer to the big data question about the underlying mathematics is given in the 2017 paper from researchers at MIT:
Sparse Hierarchical Regression with Polynomials
https://arxiv.org/pdf/1709.10030.pdf
I think Big Data occurs when someone needs to select the most relevant experimental data, given an initial set of data, for example via the ABC curves method; or alternatively perform a statistical treatment of such data. In this context, the concept of Big Data is relative, it depends on the type of data set and researcher. For me, a set larger than 100,000 data is Big Data.
Dear Professor James F Peters,
I have downloaded your paper for future reading. But I donot know how to https://arxiv.org/ to upload paper. Can you help me
Best regards
Sun
It is a relative concept. However, it can be generally defined as a collection of data that we can not (or find difficulties to) deal manually. Numerically, a set of big data is not less than one hundred data
Pls refer this paper
https://www.4tu.nl/ami/en/BigData/di-bucchianico-mathematics-behind-big-data-2017.pdf
http://ksuweb.kennesaw.edu/~plaval/BigData/mathsurvey_slides.pdf
https://files.eric.ed.gov/fulltext/EJ1131006.pdf
I propose an non-academic industry-grade definition: Big data D(M,C1,C2,T) is an M-dimensional data that is or will be processed by a computer with capacity C1, interpreted by a specialist with competency C2, over a period of time T, such that C1 > q.C2, where q>1 is called the “sanity” factor. Qualitatively, if the data will require more than q people to analyze and get insights out of it, then it is already considered as big data and just let the computer practically handle it.
Thank you for your answers, my comments are as follow.
1. Dear Professor James F Peters, your recommended paper is important, but is not related to "Mathematical Definition of Big Data". 20 years ago I published a book on essentials of Discrete Mathematics. I hope to give a definition of big data like definition of group. I also think the definition of data warehouse, which has mathematical flavor. However, data mining has not Mathematical definition, machine learning has not Mathematical definition.
2. Dear Paulo Laerte Natti
" a set larger than 100,000 data is Big Data" is not a Mathematical definition. I have considered your idea in my
[1]. Sun Z, Wang PP (2017) A Mathematical Foundation of Big Data. Journal of New Mathematics and Natural Computation. 13(2): 83-99. DOI: 10.1142/S1793005717400014
3. Tareq Al-shami
"A collection of data that we cannot (or find difficulties to) deal manually. Numerically" has a little mathematical flavor, but it is not Mathematical definition.
For "A set of big data is not less than one hundred data" please also see Sun Z, Wang PP (2017), mentioned above.
4 Dear Santhosh Kumar Balan,
Thank you for your papers, I will download each of them for reading
5. Dear Emmanuel Gonzalez
Your non-academic industry-grade definition: Big data D(M,C1,C2,T) is an M-dimensional data that is or will be processed by a computer with capacity C1, interpreted by a specialist with competency C2, over a period of time T, such that C1 can be changed to
Big data D(C1,C2,T V) is data that is processed by an entity E1 with capacity C1, interpreted by an Entity 2 with competency C2, over a period of time T, such that data can be changed into big value V. where entity is a set of humans, computing machineries, robots, intelligent agents.
This is a market-oriented definition, or pragmatic definition.
For your definition, M-dimensional data should be replaced as data, have a look at topology, distance is nothing there, therefore, M-dimensional is too specific. "interpreted by a specialist", is replaced by entity, it is better.
a computer is replaced by entity. Similarly, in German, das Sein, is the most general concept in philosophy.
Best regards
Prof. Zhaohao Sun
2018-8-23
Dear Santhosh Kumar Balan,
Thank you for your provided three papers. It seems that they have drawn attention to mathematics behind big data. But they have not addressed what i have asked. Even so, I am happy because I have found a better explanation of Google Page Rank using matrix and graph theory from one of the three provided papers or presentation slides. It deserve me to think it and develop it.
Big data is a term used to refer to data sets that are too large or complex for traditional data-processing application software to adequately deal with. Data with many cases (rows) offer greater statistical power, while data with higher complexity (more attributes or columns) may lead to a higher false discovery rate
- Badges
- Science topic
- Similar topics
- Mathematics