It depends on which sensors you want to use. In many cases its the sensor developer who determines which software platform they use to program the device. I am saying this from experience in the field of medical sensors. The companies who develop the sensors want developers to use their sensors, it is in their best interests to make it as easy as possible for developers, so they often use open source software to prevent restrictions.

have a look at

https://en.wikipedia.org/wiki/OpenNI

https://github.com/OpenNI/OpenNI2

http://www.tuio.org/

and possibly

https://github.com/fraunhoferipk/tuiframework

for general approaches towards device and sensor integration in interactive systems.

Thanks you all for your answers,

I think the closest to what I'm looking for is Tangible User Interfaces suggested by Johann.

Many thanks for this, I have identified a couple of books on TUI that might fit my purpose.

This is Java language oriented

http://fivedots.coe.psu.ac.th/~ad/jg/