Would be interesting, that. If it is possible at all,

Take a peak at https://github.com/DPMI/trafficgenerators, udp case. The sender places a lot of information into the transmitted packets. Those are available at the server, either run-time or by reading the log the server generates. In the log file, you will find the send and receive time of the packets, subtract them and you get the 'delay'. But pay attention that you have control of the time synchronization between the different clocks in play (sender and receiver). 

Easier, just ping it and assume symmetrical links. Probably 'better' than assuming synchronized clocks. 

You could check this tool for measuring unidirectional packet delays:

http://software.internet2.edu/owamp/