What are data collection methods and protocols for transferring sensor data from the IoT devices to a local server?

Anatol Badach

Dear Robbin Mchinzi,

the protocol CoAP is used here – instead of the protocol HTTP.

On layer 3, the 6LoWPAN protocol is used instead of the IP.

This is illustrated in the figures under the added addresses.

Best regards

Anatol Badach

https://www.researchgate.net/publication/312174308_CoAP_-_Constrained_Application_Protocol/figures?

https://www.researchgate.net/publication/313926805_IoT_CoAP_Messages_-_Structure_and_Types

https://www.researchgate.net/publication/313927079_IoT_HTTP-to-CoAP_Mapping

https://www.researchgate.net/publication/319205134_6LoWPAN_-_IPv6_over_Low_power_Wireless_Personal_Area_Network

Murtadha Shukur

There are a variety of data collection methods and protocols for transferring sensor data from IoT devices to a local server. The best method for a particular application will depend on a number of factors, including the type of sensor data being collected, the frequency of data collection, the distance between the IoT devices and the server, and the power constraints of the IoT devices.

Some common data collection methods include:

Polling: In polling, the server periodically sends a request to the IoT devices for sensor data. The IoT devices respond with the latest sensor data readings. Polling is a simple and reliable method, but it can be inefficient if the sensor data is not updated frequently.
Event-driven: In event-driven data collection, the IoT devices send sensor data to the server whenever a specified event occurs, such as when a sensor threshold is crossed or when a button is pressed. Event-driven data collection is more efficient than polling, but it requires more complex code to implement on the IoT devices.
Push-pull: In push-pull data collection, the IoT devices periodically send sensor data to the server, and the server can also request sensor data from the IoT devices on demand. Push-pull data collection is a hybrid approach that combines the benefits of polling and event-driven data collection.

Some common protocols for transferring sensor data from IoT devices to a local server include:

HTTP: HTTP is a general-purpose protocol that can be used to transfer sensor data in a variety of formats, such as JSON and XML. HTTP is a relatively simple protocol to implement, but it can be inefficient if the sensor data is updated frequently.
MQTT: MQTT is a lightweight messaging protocol that is designed for IoT applications. MQTT is very efficient for transferring sensor data, but it can be more complex to implement than HTTP.
CoAP: CoAP is another lightweight messaging protocol that is designed for IoT applications. CoAP is very similar to HTTP, but it is more efficient and optimized for low-power devices.

The best way to choose a data collection method and protocol is to consider the specific requirements of your application. If you are not sure which method and protocol to choose, there are a number of resources available online and in libraries to help you make a decision.

Here are some additional tips for collecting and transferring sensor data from IoT devices to a local server:

Use a data collection agent: A data collection agent is a software application that runs on the local server and is responsible for collecting and storing sensor data from the IoT devices. Using a data collection agent can simplify the process of collecting and transferring sensor data, and it can also provide additional features such as data filtering and aggregation.
Secure the data transfer: It is important to secure the data transfer between the IoT devices and the local server. This can be done by using encryption and authentication.
Monitor the data transfer: It is also important to monitor the data transfer to ensure that the sensor data is being collected and transferred successfully. This can be done by logging the data transfer activity and by setting up alerts to notify you if there are any problems.

Robbin Mchinzi

Murtadha Shukur Thank you so much I will take time to explore

Robbin Mchinzi

Anatol Badach Thank you so much for your response I will take time to explore.

Ahmad Abusukhon

Hi,

Please take a look at my papers:

■ A. Abusukhon, A. Al-Fuqaha, B. Hawashin, A Novel Technique for Detecting Underground Water Pipeline Leakage Using the Internet of Things. Journal of Universal Computer Science (JUCS). Vol. 29, No. 8.

■ A. Abusukhon, IOT Bracelets for Guiding Blind People in an Indoor Environment, in Journal of Communications Software and Systems, vol. 19, no. 2, pp. 114-125, April 2023, doi: 10.24138/jcomss-2022-0160.

■ A. Abusukhon (2021) Towards Achieving a Balance between the User Satisfaction and the Power Conservation in the Internet of Things, IEEE Internet of Things Journal, doi: 10.1109/JIOT.2021.3051764. impact factor 9.936. Published by IEEE. https://ieeexplore.ieee.org/document/9326414. [Science Citation Index].

■ Ahmad Abusukhon, Bilal Hawashin and Mohammad Lafi (2021) An Efficient Algorithm for Reducing the Power Consumption in Offices Using the Internet of Things, International Journal of Advances in Soft Computing and its Applications (IJASCA).

Hope this is useful

Regards,

Ahmad

Johirul Islam

Let me divide your question into 2 separate parts:

(1) data collection methods from the sensor: this part is basically defined by the sensor manufacturer. However, most of the manufacturers preferred Bluetooth/BLE (for short range) since most of the sensors (especially the medical IoT sensor) are operated through a small coin-size battery.

(2) for transferring sensor data: this part fully depends on you (or the application developer). You can choose any protocol you want (like HTTP and so on). But remember that the IoT resources e.g., Raspberry Pis are constraints in nature with limited computing resources (i.e., RAM, CPU, storage, etc.). As a result, in most cases, developers and researchers prefer to use CoAP (Constrained Application Protocol) or MQTT (Message Queue Telemetry Transport) protocols.

I recommend you to study a little more on HW interfacing protocols (I2C/I2C, SPI, BLE which will allow you to grab/collect data from a sensor) and SW/application interfacing protocols/application programming interface or APIs (and related to CoAP/MQTT which will allow you to design & develop your own application to transfer/receive data to/from another application).

Robbin Mchinzi

Johirul Islam and Ahmad Abusukhon Thank you so much for your helpful resources.

Mohit Tiwari

Collecting sensor data from IoT (Internet of Things) devices and transferring it to a local server involves several methods and protocols to ensure efficient and secure data transmission. Here are common data collection methods and protocols for IoT devices:

Data Collection Methods:

Polling and Pushing: IoT devices can send data to a local server through polling or pushing mechanisms. In polling, the server periodically requests data from devices. In pushing, devices send data to the server as soon as it becomes available. Pushing is often more efficient for real-time data.

HTTP/HTTPS: Using standard HTTP or its secure counterpart HTTPS is a common method for IoT data transfer. Devices can make HTTP/HTTPS requests to web services or APIs hosted on the server to send data. RESTful APIs are commonly used for this purpose.

MQTT (Message Queuing Telemetry Transport): MQTT is a lightweight and efficient publish-subscribe protocol designed for IoT. Devices publish data to specific topics, and subscribers (including the server) receive updates when data is published.

CoAP (Constrained Application Protocol): CoAP is another lightweight protocol designed for resource-constrained devices. It's suitable for IoT environments and supports UDP for efficient communication.

WebSocket: WebSockets provide full-duplex communication channels over a single TCP connection. IoT devices can establish WebSocket connections to the server, allowing bidirectional data transfer.

UDP (User Datagram Protocol): While less reliable than TCP, UDP is often used for real-time IoT applications where low latency is critical. Devices send data as datagrams to the server, which may require additional error handling.

Data Protocols:

JSON (JavaScript Object Notation): JSON is a lightweight and human-readable data interchange format. IoT devices can encode data in JSON format and send it to the server, which can then parse and process the data.

XML (eXtensible Markup Language): XML is another format for structuring data. IoT devices can use XML to encode data before transmission, and the server can parse the XML data.

Protocol Buffers (protobuf): Protocol Buffers is a binary serialization format designed for efficient and compact data transmission. It's often used in resource-constrained IoT environments.

MessagePack: MessagePack is another binary serialization format that provides a compact representation of data. It's suitable for IoT applications where bandwidth and efficiency are crucial.

Security Protocols:

TLS/SSL: Transport Layer Security (TLS) and its predecessor Secure Sockets Layer (SSL) provide encryption and authentication to secure data transmission. Using HTTPS or secure MQTT (MQTT over TLS) is common for IoT security.

OAuth: OAuth is used for secure authentication and authorization between IoT devices and servers. It ensures that only authorized devices can send data.

API Keys and Tokens: Devices can use API keys or tokens to authenticate with the server's APIs. Tokens can be short-lived and rotated for added security.

Digital Signatures and Certificates: IoT devices can sign data with digital signatures or use certificates to verify their identity to the server.

Firewalls and Intrusion Detection Systems: Implementing firewalls and intrusion detection systems on both the device and server sides can help detect and prevent unauthorized access and attacks.

The choice of data collection methods and protocols depends on factors such as the nature of the IoT devices, the required data transfer rate, security considerations, and the capabilities of the local server. A combination of methods and protocols may be used in a comprehensive IoT data collection and transmission strategy.

How to compare two groups with only two measurements?

What's the best way to measure growth rates in House sparrow chicks from day 2 to day 10?

How to increase a model accuracy ?

1. What is the impact of having different scales in a survey? and how can we solve this problem before and after data collection (Literature)?

What degree of missing data in a particular case/participant results in Expectation-Maximisation no longer working?

Is there a powerful system for the security of the systems distributed on IoT systems?

New Journal, "Integrative Psychological and Behavioral Science" -- do you not know, and have you not seen, this done before?

Which is a lucid standard comprehensive book to guide on psychological scale construction and its standardization?

How Relativistic effects is one reason for lanthanide contraction?

Can NFL theorem be valid in infinite search space in ML?