Big Data is now-a-days a buzz word. Almost every one of us might have read, listened or even talked about it. Therefore, the question, ‘how big is really Big Data' seams a valid question. Please share your understanding, information, knowledge and experience about Big Data.
Dear Asmat, Big data is a term used for describing the exponential growth and availability of data.
Dear Asmat, Big data is a term used for describing the exponential growth and availability of data.
Big data is a blanket term for any collection of data sets so large and complex that it becomes difficult to process using on-hand data management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, transfer, analysis and visualization.
http://en.wikipedia.org/wiki/Big_data
Big data is a popular term used to describe the exponential growth and availability of data, both structured and unstructured. And big data may be as important to business – and society – as the Internet has become.
http://www.sas.com/en_us/insights/big-data/what-is-big-data.html
Dear @Asmat, "as far back as 2001, industry analyst Doug Laney (currently with Gartner) articulated the now mainstream definition of big data as the three Vs of big data: volume, velocity and variety". Have some more reading about, links follow!
http://www.sas.com/en_us/insights/big-data/what-is-big-data.html
Dear Gopinath,
I really like the term you introduced blanket.
Respected Ljubomir Jacić,
I appreciate your informative reply. Thanks for mentioning a useful link.
Dear Thoralf Klein ,
Thanks for joining. I see a kind of expert reply from you. It starts where the problem size grows faster than CPU speed, RAM size or disk space. Then how we process this kind of BIG data?
Dear @Asmat, there are many good answers in the following thread!
https://www.researchgate.net/post/Which_frameworks_are_available_for_Big_Data_processing?_sg=%2BvY%2F8M4qx4b8JuAy6uDa7BGtX1FAegRs10KrwkwTpn3NO0HpqcAGdNgkc4uoG6ct_QD83U9cxBq2Hr3mUmcOXmZzU%2FRgsEW62S6yi4exSHcRnlV1bPTSHhLoiExJ3Vakh_RwdRrAtZbRqJQlEDRSshEi48ufAiHG%2Fp70tip7jKXLC3o50%2B2Wg%2Fon%2FdVaS4I425
The term big data conntes data that is beyond the ability of common software tools to capture, and manage. Another way understanding the big data as opposed to small data is that big data is for the use of machines, and small data is for the use of people. .
Dear @Asmat, Big Data is the latest buzzword, and as such everybody is using it to their convenience. This has inevitably created some confusion. In this blog post we will try to clear it up. Big data is about two things: large sets of typically unstructured data and some relatively new techniques to deal with this kind of data. To get a good perspective we need to start by reviewing relational databases. In general, the traditional way of handling data is by using a relational database. When a database is used for on-line transaction processing we tend to see a separate setup for analytics. This is commonly known as a data warehouse, which provides processing relief from the main database. It also has some analytical or so-called business intelligence tool. Large relational databases tend to be expensive propositions, as the costs of the processing units and the disks are very high. See more at: http://www.opallios.com/what-really-is-big-data/#sthash.OsT2GcDb.dpuf
http://www.opallios.com/what-really-is-big-data/
Its both explicit and tacit, in which some can be captured while lots cant !
Even though Mahfuz' criteria is correct, and many of us support his view, in reality it is the management of data over petabytes. So it corresponds to the storage and processing of processes of peta, exa, zeta and yocto - bytes. Bellow tera bytes there is not really the necessity to speak of really "Big Data".
The subject is closely related to the capacity and storage - memory hence of spearhead computers.
I agree @ Mahfuz Judeh & Debi S. Saini. Now a days big data used in IBM.
Hello Asmat Shaib
Greetings and Good Wishes in Holy Month of RAMZAN
The X-Ray infiormation in two plane of X-Y co-ordinates which are recieved in Digital imaging and processing , using CT for imaging complicated body parts like brains is one the example for big data. This example is adoptable to MRI imaging too, there the source for energy is not X-Ray, but MR
Data reconstruction takes time due to voluminous information and requires high end computing...
Hope this example helps
Kindest Regards
Ramanan
How big is really, big data? Answer, not big enough! What is big today will not be so tomorrow.
@Issam Sahib, I love your views... more practical and true under the context of constant improvement on Technology, computing power and demand from customer for better quality...
I too agree with Carlos and Mahfuz.
Big Data Means new Data Processing and Decision Making Technologies and New Avenues of Income through 1) cost reductions, 2) time reductions, 3) new product development and optimized offerings, and 4) smarter business decision making.
The Following Links give a Bird's Eye View of Big Data and its Implications.
1. http://www.ibm.com/big-data/us/en/
2. https://www.researchgate.net/go.Deref.html?url=http%3A%2F%2Fwww.sas.com%2Fen_us%2Finsights%2Fbig-data%2Fwhat-is-big-data.html
Big data is being generated by everything around us at all times. Every digital process and social media exchange produces it. Systems, sensors and mobile devices transmit it. Big data is arriving from multiple sources at an alarming velocity, volume and variety. To extract meaningful value from big data, one needs optimal processing power, analytics capabilities and skills.
big data technology
New skills are needed to fully harness the power of big data. Leading organizations are developing new roles, focusing on key challenges and creating new business models to gain the most from big data.
Big data is changing the way people within organizations work together. It is creating a culture in which business and IT leaders must join forces to realize value from all data. Insights from big data can enable all employees to make better decisions—deepening customer engagement, optimizing operations, preventing threats and fraud, and capitalizing on new sources of revenue. But escalating demand for insights requires a fundamentally new approach to architecture, tools and practices.
To gain the competitive advantage that big data holds, one needs to infuse analytics everywhere, make speed a differentiator, and exploit value in all types of data. This requires an infrastructure that can manage and process exploding volumes of structured and unstructured data—in motion as well as at rest—and protect data privacy and security.
P.S.
According to Wikipedia, a very large database (VLDB), now termed as 'big data', is a database that contains an extremely high number of database rows called tuples, or occupies an extremely large physical file system storage space. In present practice, database that occupies more than 1 terabyte or contains several billion rows are termed as 'Big Data'. However, This maximum value may change over the time.
Dear all colleagues.
I agree mostly of your comments, however, big data depends on your research assumption and the instantaneous data can be collected with shortest time. For example investigating stress and presser tolerance on an object requires gathering minute data in small area with 3 dimensions and topological configuration. So what is big data in one research could be small in front of other research. The number and formulas with figures make the data bigger. The complexity of human modeling requires a lot of data and information processing with ample simulation of the model to reach the optimum solution.
"How big is really, Big Data?" is like asking someone to explain "How fast is too fast?" The answer can be relative.
From a point of view of a modeler/analyzer/forecaster/... it means you are faced with a situation/ problem called "curse of dimensionality" How to beat the curse of dimensionality is something that I am still interested to know (I asked it last year- please see the link below).
https://www.researchgate.net/post/How_to_beat_the_curse_of_dimensionality?_tpcectx=profile_questions
The definition of big data is fuzzy and relative (As Issam said above) and depends on volume, variety, velocity and veracity.
Here is Definition given by big companies:
IBM, Big Data…it's varied; it's growing; it's moving fast, and it's very much in need of smart management. IBM data scientists BM data scientists break big data into four dimensions: volume, variety, velocity and veracity
Oracle. Big data is the derivation of value from traditional relational database-driven business decision making, augmented with new sources of unstructured data.
Intel. Big data opportunities emerge in organizations generating a median of 300 terabytes of data a week. The most common forms of data analyzed in this way are business transactions stored in relational databases, followed by documents, e-mail, sensor data, blogs, and social media.
Microsoft. “Big data is the term increasingly used to describe the process of applying serious computing power—the latest in machine learning and artificial intelligence—to seriously massive and often highly complex sets of information.”
SAS, Big data is a popular term used to describe the exponential growth and availability of data, both structured and unstructured
As mentioned in my previews answer, database that occupies more than 1 terabyte or contains several billion rows are presently termed as 'Big Data' in industrial sector.
Dear Asmat Ali
Your question is time variable and my answer reflect this. Your question how big is it right now? Has also been answered! If you read my answer carefully you will see that the answer is right now it is not big enough.
Dear YOGESH CHANDRA TRIPATHI,
Thanks for contributing actively. Sorry, I missed that. I appreciate you, for highlighting and putting it again. It has added to my knowledge.
Dear Issam Sinjab,
Thanks for your generous contribution. Time is an independent variable. Generally, you are right to say not enough but is there are quantification?
The question is valid but the answers will deal with relativity. Not long time ago, a file having a size of 10 migabytes was considered by many as (big) & cannot be handled or stored by a floppy disc. Afterwards, data with 500 migabytes were considered (big) & the CD cannot take more data to store. Now, we are talking about several gigabytes of data stored in a small flash.
I think anyone of us has to look deep into the matter taking into account the limitations of human capacity. You can go on storing many gigabytes in your fields of interests & you can go on adding books to your own library but what is then?
Do you think that you are going to cover all what you collected within your life time? If it is really big data, the answer is: no way. Therefore, it is better to start "trimming" the big data , if you already have one, until you keep just (La Crème de la Crème).
In real, we can not say how big is the big data is,it is very big in its volume,verity and velocity.Bid data is unstructured and raw in nature,it needs high computational power to collect,classify, process and to store and it is possible only through Cloud computing.
This is the era of the big data and big data analytic.The Internet of the thing is contributing much towards it.currently the Government,private sectors and entire business entities around the world are depending on decision making power of big data analytic.
The Big Data can also be called as "actionable data".
@ Reeta Sony,
Thanks for joining. I agree with you, it is very big in its volume, verity and velocity. Is the growth uniform along three mentioned axis i.e. volume, verity and velocity?
A data which is “actionable’ is information or data?
A data which is not “actionable’ is really a data?
I agree with you.
For me, A data which is not “actionable’ is really a data, but it lucks time to pass to step to be actionable, it's need researches, investigations, thinking, etc,
Dear Asmai Ali
Thank you for your response. The whole idea of my answer is to be different and not to quantify, you have many interesting and excellent answers of this type. I was really trying to reflect the fact that technology is fast changing and constantly improving and so is computing power and the answer to your question is really relative. In this regard, Dr Nizar Matar's answer is an excellent illustration of what I meant in my short answer.
Big data is an evolving term that describes any voluminous amount of structured, semi-structured and unstructured data that has the potential to be mined for information. Big data is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.
I am quite skeptic about whether "Big Data is not bigger than human mind" as Asmat Ali said,if we observe the past and current trends of technology and its impact on all the aspect of our life, it rises question like; whether human being is working with the technology or working for the technology? or destructive end of the constructive technology making human being to work for the Technology?.
At present we are moving towards IPv6 and in the near future it is possible every molecule on the earth could be assigned an IP address according to the requirement and entire world will be surrounded by 0s,1s and bits and bites.The fundamental question is,whether the same human mind which is created the ocean of data, can regulate the same?
According to my thought, “Fire is good servant but a bad master”,it is good, if human being manages to control it, otherwise it can be turned to the bad ends.The same can be applicable to Big Data or any other technological trends.
@Reeta Sony,
Thanks for your nice comments on my post. Indeed, you have raised couple of quite interesting questions. Regarding your question, whether human being is working with the technology or working for the technology? I am of the opinion that both parts of the statement are true. Today's research need technological tools and of course new technology is developed on the bases on existing technology.
Creator always knows how to handle its creation so is the case with Big Data. whether the same human mind which is created the ocean of data, can regulate the same? Of course, human mind would certainly regulate it sooner or later.
You are right, it depends upon the hands which take control of commodities like fire as well as Big Data.
Time seems to become relativized once the data is stored. What may not look important now, might become important later
a) because of particular novel information
b) because of a generally increasing amount of correlations.
Dear Asmat,
in a belief system there could be a shift in the assessment of importance, too, Would you agree?
Thanks every one for sharing, reading and responding. It was really informative and knowledge adding to read your replies. Although, we could not reach at a conclusion that how big is really Big Data. I am sure the growth along its 3 axis i.e. velocity, variety and volume is tremendous which is posing new challenges for data managers to organize it efficiently.
Big Data is a Massive amount of data that is generated by Different kind of industries as well as our self only. Big data is a large set of the data that is generated through the FACEBOOK,U TUBE, and many social sites. That is in the form of video,image,text,graphics, sensors, and many other things. Big data has a different characteristics like a velocity, variety, volume, verity .Big data has devided in to 3 types Structured,Semi Structured, Unstructured data.
Read this two papers its explains all.
http://www.ijser.org/onlineResearchPaperViewer.aspx?Big-Data-challenges-and-Hadoop-as-one-of-the-solution.pdf
http://ir.inflibnet.ac.in/bitstream/1944/1890/1/51.pdf
It depends on many interrelated variables. No one definition for big data could be valid for all cases. However, the exact use must be justified carefully otherwise it will be only a kind of propaganda.
Regards
SM Najim
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Analysis
- Data Analysis
- Dataset
- Big Data