Hi,
Authors have claimed the data set they used for the numerical example extracted from supplier database is big data.
However, such data-set cannot be considered big data as it doesn't have the 5v's. ( Article A Brief Introduction on Big Data 5Vs Characteristics and Had...
)An example of big data is the data transmitted by each sensor of each machine on the shop-floor at each 4 milliseconds!
Remember "Big Data" is often used as a marketing term, so it depends on who you ask and what kind of problem you are tackling, and what era the term was used.
I've not read the full paper, but it sounds like a great set of analysis for some complex problems where the data can't be easily partitioned. Importantly, these are problems where a 0.5% improvement can have big value to the customer.
To keep things in perspective, back in 2016 there were about 50 million companies in USA and less than 300 million companies of more than 20 employees globally. So not "big" by today's standards, but still "a lot of data". The challenge is the complexity of the data relationships, not the total volume.
Its only when sensor and GIS data comes into the picture that data volume and velocity really increases.
Historically, "big" has changed a lot. "Very Large Databases" (an ACM publication that I last read in the late 1990s) was an era where "very large" was often "more than one terabyte". "Big Data" was invented as marketing term in the 2000's when a server with 64GB of ram was considered "large" and storage of 2 TB took 20 rotating disks. Today, I have a one terabyte SSD and it is commonplace to have 10-100 Terabytes of rotating storage and a server with 1TB of ram. Even a few petabytes of sensor data is often straightforward since the problem is so easily partitioned.
Thanks Stan for the reply. However, you are not addressing a valid point as the paper I used as reference was published at the same year with the discussed paper.
Although the computation power of PCs can make a solution more/less feasible, this fact will not be changed that Big-Data should have those 5v's. NP-hard problem will always remain NP-hard, no matter you use a PC to solve them now or use a supercomputer 20 years later!
I would say that it should not have all the 5 Vs but few of the V's or rather a subset of V's could also do. For example: Velocity, Volume, and Veracity. The following manuscript describes all the aspects and the differences between each in a concise manner.
1. Chapter Applications of Hadoop Ecosystems Tools
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Analysis
- Data Analysis
- Dataset
- Big Data