Ruchika, there are multiple communications layers. Seven, if you use the ISO/OSI model. And each one of those can have security protocols. Or at least, the physical layer, the data link layer, the network layer, and the application layer, have security protocols that may be applied.
But your question relates to IoT, so the layers that would impact a user of IoT would be layer 3 and above, in general. So, IPsec and TLS primarily, as Ayoub stated. VPNs, per se, I wouldn't call a security protocol, but they are usually made secure. You might want to start with this, for a VPN security discussion.
Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer (SSL), both frequently referred to as "SSL", are cryptographic protocols that provide communications security over a computer network. Encryption algorithms secure data at communication layer. You can use symmetric or asymmetric encryption as per setup need.
The physical layer uses adaptive security algorithm to secure the communication between two network devices. It can also secure the message through the use of encryption
In case of IoT, all these cryptographic protocols can only be used if the device supports cryptographic operations.Most of the IoT devices which mainly includes Wireless Sensor Devices have constrained resources and are not capable of crypto functions. So some of these protocols cannot be implemented from the IoT device side.
Another solution would be to use an aggreagator or controller which collects the data from IoT devices at the edge and from the controller to Cloud any secure communiication protocols can be used