The use of edge devices often increase the end-to-end latency as they are weaker than cloud counterparts. You can transform your inference model to be lightweight by introducing early exits (https://www.researchgate.net/publication/359758153_Combining_DNN_partitioning_and_early_exit). Bandwidth and cost reductions could be topics to explore (https://www.researchgate.net/publication/333395795_Multi-Objective_Reinforcement_Learning_for_Reconfiguring_Data_Stream_Analytics_on_Edge_Computing), but you should be realistic and also present how the use of edge devices increases your end-to-end latency.

Edge computing promises the opportunity to reduce latency and to improve availability, when the system sensitivity to "cloud round trip latency" or to "client-cloud communication challenges" is high.

To decide what to measure, you need to posit a system model that has these sensitivities, and that can be decoupled such that the edge can adequately host part of the system processing, in a more local and less contention sensitive way.

Any decouplable system, solving a problem that that requires near real-time response to processing or connectivity, despite internet outages or congestion, can be a candidate for such a model. Measure and plot latency/availability of transactions in that model's context. Time to response, time to insight, resilience to "down detector" events ... are all plausible measures/metrics strategies for the appropriate client-edge transactions in such a system.

Major cloud platforms have recognized that "edge" computing is a requirement in such situations, and have their own "edge" offerings to somewhat mitigate concerns for those kinds of system/problem sensitivities. Look at the showcase "solutions" on their edge platforms for compelling candidates:

https://www.metaswitch.com/azure-edge

https://www.lastweekinaws.com/blog/what-is-an-edge-location-in-aws-a-simple-explanation/

https://cloud.google.com/edge-tpu

Dear Bidyut Saha ;

I think this paper is very useful to you...

Article Modelling Cloud Service Latency and Availability using a Dee...

Regards;