There will be an estimated 26 billion units installed globally by 2020 with many more on the way in succeeding years as the price of processors drops. In the near future, it will be feasible to install a processor into just about everything.
Where is all the data provided by those processors going to be stored and what are the problems around them?
IoT deployments will generate large quantities of data that need to be processed and analyzed in real time. Processing large quantities of IoT data in real time will increase as a proportion of workloads of data centers, leaving providers facing new security, capacity and analytics challenges.
Are there any new ideas how to handle the problem of big data generated in the IoT??
Perhaps you can explore using compute & storage power at the IoT edge instead of sending real time big data generated at IoT back to data center or cloud for further analysis / processing. IoT edge compute includes e.g. Raspberry Pi, HPE Edgeline IoT systems etc. (see links below). Advantages of using such IoT edge processing power include:
https://www.theregister.co.uk/2015/05/09/book_review_iot_diy_at_home_projects_for_arduino_pi_beaglebone/
https://www.informationweek.com/software/enterprise-applications/10-raspberry-pi-projects-for-learning-iot/d/d-id/1320757
https://www.youtube.com/watch?v=FE-jGO6E1Vc
https://www.hpe.com/h30683/us/en/strategic-business-insights/c/it-operations/data-centre-management/bringing-your-data-center-to-the-iot-edge.html
https://www.hpe.com/us/en/servers/edgeline-iot-systems.html
There are bunch of cloud-based and centralized solutions for these requirements. However, expanding volume of IoT ecosystems will also push the adoption of decentralized web, data storage, security and messaging solutions. Although I don't have deep experience (and maybe this answer is broad), I can refer IPFS (interplanetary file system) for data transactions and storage, data processing&analysis with distributed ledgers, or directed graphs.
EDIT: The utilization of blockchain-based solutions are in discussions for a while. However, what I have seen so far, the majority of discussions are repeating themselves. I personally think blockchains are not feasible for IoT. Technically it can be utilized for data transfer and M2M transactions. However, there will be some major bottlenecks with blockchain such as increasing network size (scalability), high transaction fees (mining), transaction time (major delays). There is a 2-year old open source project called IOTA which is a distributed ledger based on tangle. They call it blockless-blockchain with its fee-free transactions, scalable and fast transaction arhitecture.
The use of fog computing besides cloud paradigms is important to handle this.
I agree with Nour answer,
The IoT uses the cloud computing to facilitate the storage and processing of massive amount of IoT data. However, for some latency sensitive IoT applications, network dependent nature of cloud is not suitable . So, fog computing is a preferable solution by bringing the cloud computing paradigms to the edge of network and close to the sensing data layer in IoT environment, and of course, the virtualization concept in both; cloud and fog computing makes the task easier.
Are there any new ideas how to handle the problem of big data generated in the IoT??
Perhaps you can explore using compute & storage power at the IoT edge instead of sending real time big data generated at IoT back to data center or cloud for further analysis / processing. IoT edge compute includes e.g. Raspberry Pi, HPE Edgeline IoT systems etc. (see links below). Advantages of using such IoT edge processing power include:
https://www.theregister.co.uk/2015/05/09/book_review_iot_diy_at_home_projects_for_arduino_pi_beaglebone/
https://www.informationweek.com/software/enterprise-applications/10-raspberry-pi-projects-for-learning-iot/d/d-id/1320757
https://www.youtube.com/watch?v=FE-jGO6E1Vc
https://www.hpe.com/h30683/us/en/strategic-business-insights/c/it-operations/data-centre-management/bringing-your-data-center-to-the-iot-edge.html
https://www.hpe.com/us/en/servers/edgeline-iot-systems.html
Hello Dr.
It's an interesting topic, thank you, for this question.
in my opinion, the processing of big data depends on the data type and degree of sensitivity and choose the location of storage depends on the ability of the organization (cost) or data unit center which owns data there are the possibilities of local and other online storage as well as the type of service available.There is a set of determinants which are must be taken into consideration when dealing with this subject.
Best regards..
Ameer Sameer
Home automation and Internet of Things are gaining popularity nowadays. In today’s world, Automatic systems are being preferred over the manual system. A Smart home automation system is based on making user life easier. The Internet of Things generates a huge amount of data. Additionally, it contains a huge number of sensors and their data which can control or monitor objects. This paper verbalizes about how big machine learning and data mining use for converting the normal home automation to the smart home automation system. Here the user of this system can control their home using their smartphone as well as computer or any devices which are connected to the internet. And also get suggestions [using prediction algorithm], reminder, and alerts additionally system interact with user
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Analysis
- Data Analysis
- Dataset
- Big Data