Say for example I need 100 TB of data to do some experiments on Big Data. How do I get if I find the source? What is the Media that is usually used here?
You can use tools like Apache Flume, Apache Chukwa or Scribe to pull in huge amounts of data into your systems. When we talk about BigData, it doesn't make much sense to simply start copying the data from one location to another via some removable media. This is where tools specified above come into picture. All of these tools are highly scalable and allow us to aggregate large amounts of data without much fuss. You just have to configure these tools once as per your requirements and they do the rest for you. For example, FB uses Scribe to collect millions of messages everyday.
Hadoop is used to process a large data.... u can compress a large file (big-data) using codec in Hadoop..... and then throuhg network u can change ur data location
Public cloud computers give limitless, elastic supply of computing power, networking, and storage. A scalable big-data file transfer solution through SaaS.
Businesses need to transfer data out of storage and into analysis without the threat of meeting storage capacity. Accelerating the transfer process allows companies to better manage their storage space, cutting storage expenses and freeing management.
With cloud computing taking foot as a business standard, moving data to the cloud for storage or analysis has become a part of every company’s workflow, particularly those that are becoming data driven.
Have more information about Big Data Here - https://data-flair.training/blogs/what-is-big-data/
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Analysis
- Data Analysis
- Dataset
- Big Data