Interesting question, and from what you write in the description, you should add to it the issue of "IoT at the Edge with mMTC" (massive machine type communication).

There may be an interesting scenario to consider: Quantization at the Edge prior to either aggregated distributed analysis or centralised analysis.

For instance a parameter value is between vmin and vmax, or above, or below. This is quantizable by a ternary model (-1,0,1) as states.

You could have a more refined n level quantizer, or if you collect vectors instead of just values, vector quantizers (see vectir quantization, Voronoi diagram, etc).

The algorithms on quantized data can be made light and fast (replace * by + and integer based representations, GF(n) for Galois fields also noted Z/nZ structures). This gives great algebra decompositions.

It fits on different hardware/VLSI.

You get one more degree of freedom through this quantized approach.

See my articles (on researchgate) on variable rate speech coding, and my patent on algebraic speech coding (method generalising to vector processing)

Interesting question, and from what you write in the description, you should add to it the issue of "IoT at the Edge with mMTC" (massive machine type communication).

There may be an interesting scenario to consider: Quantization at the Edge prior to either aggregated distributed analysis or centralised analysis.

For instance a parameter value is between vmin and vmax, or above, or below. This is quantizable by a ternary model (-1,0,1) as states.

You could have a more refined n level quantizer, or if you collect vectors instead of just values, vector quantizers (see vectir quantization, Voronoi diagram, etc).

The algorithms on quantized data can be made light and fast (replace * by + and integer based representations, GF(n) for Galois fields also noted Z/nZ structures). This gives great algebra decompositions.

It fits on different hardware/VLSI.

You get one more degree of freedom through this quantized approach.

See my articles (on researchgate) on variable rate speech coding, and my patent on algebraic speech coding (method generalising to vector processing)

Hi Renaud Di Francesco, thanks for your sharing. I read your paper< Real-Time Speech Segmentation Using Pitch and Convexity Jump Models:

Application to Variable Rate Speech Coding>, it's an outstanding approach to segment speech with a variable rate, without the training process. As you said, this question can be described as aggregated distributed analysis or centralized analysis for quantization at the Edge.

I think with the massive deployment of 5G (which enables the mMTC), a lot of problems can be solved, like the network latency, but I guess people will prefer a distributed solution where data are processed locally, so the preprocessed data uploaded to the network doesn't contain any sensitive information. For example, some hidden layers in deep learning can be performed locally, and leave the output layer to the cloud, or do the inferencing totally at the edge.

Hello Sizhen Bian , yes you are right: Edge processing is an interesting approach with distributed computing architectures.

(1) Every node has (network, compute, data) capability.

(2) mMTC strategy: very light weight computing requirement at Edge, together with unfrequent and low volume data upload from mMTC node to Cloud

(3) Because of (2), we perform "granularity controlled" data acquisition (data functionality of (network, data, compute) node functionality), followed by local compute.

Quantization folds the real data vector sensed into an index n (quantization class index) from a finite number N of quantization classes. Hence we have a transformation of vectors into integers. The algorithms then processes integers: multiplications become additions, etc. There is an order of magnitude gained in complexity.

Then you move the "use case" from requiring a GPU type of processor, to a very basic processor compatible with mMTC scenarios.

This is what I meant. Does it help you more concretely?

Hi, Renaud Di Francesco thanks a lot for this detailed information. It's quite a new knowledge for me. I will dig it more in depth later and will leave you messages when some confusion comes to me.

Appreciate:)

The convergence of Edge Computing and Deep Learning: A Comprehensive Survey

Xiaofei Wang, Senior Member, IEEE, Yiwen Han, Student Member, IEEE, Victor C.M. Leung, Fellow, IEEE, Dusit Niyato, Fellow, IEEE, Xueqiang Yan, Xu Chen, Member, IEEE

I agree with Renaud Di Francesco

hi, Isam Alkhalifawi thanks for sharing this survey paper. I took two days to go through the paper and may need more time(even months). To be honest, this paper is one of the best to comprehensively summarize the deployment issues of DL by edge computing, spanning from the field of networking, communication, to computation. Framework solutions on different platforms, like CoreML, SNPE, hardware platforms like FPGA, Mobile GPU, and lots more, are extensively discussed. To grasp the overall information of this survey, good background of network technique(NRF, SDN, etc), deep learning(GNN, DRL, etc), as well as the communication techniques are definitely needed. Unfortunately, I'm not at that level. I highly recommend this paper if anybody has an interest in the field of edge computing, it provides a close look at how to develop edge computing architecture to achieve the best performance of DL training and inference.

However, the hardware for edge computing mainly discussed in this paper is mobile ones(like SNPE from Qualcomm, APU from MediaTek) and FPGA based ones. which are actually not that far at the edge of the network, I mean they are not as weak as IoT devices, like Cortex-M microcontrollers.

For those very weak IoT devices, if we want to deploy deep learning model or classical machine learning models, (which is also a kind of edge computing, let's say, Tiny-Edge-Computing), would the model compressing solution or the asci solution be more acceptable, considering the cost, reliability, ease-to-use, etc?

I don't think you should assume that an IoT device necessarily has to hold the learning model - the edge model allows for local resources like full servers to be available to do work. We talked about this and other issues with IoT in a general, architectural sense:

Conference Paper An Architectural Vision for a Data-Centric IoT: Rethinking T...

In factories and utilities and in a whole host of buildings, you end up with lots of legacy IoT devices that you have no way of upgrading. Factories often use station controllers, which are PCs assigned locally, to control assembly, and they can be reassigned to do functions like machine learning driven fault detection:

https://www.intel.com/content/www/us/en/it-management/intel-it-best-practices/faster-more-accurate-defect-classification-using-machine-vision-paper.html

A real world instance of the architecture like the one described in the vision paper is Amazon's IoT Greengrass:

https://docs.aws.amazon.com/greengrass/latest/developerguide/what-is-gg.html

One of the use cases mentioned is running machine learning models trained in the cloud locally.

Hi, @Jeff Sedayao, I'm glad to have your opinion here. Thanks for your contribution. Yes, in most of IoT use cases, like factories, buildings, there is no need to move the inference to the edge, and a lot of real-life instances can be found, as you mentioned above. But there are still some use cases that need the inference locally at the edge, especially for some personal use cases. For example, some machine learning algorithms could detect gait freeze episodes in Parkinson’s disease with wrist-mounted inertial sensors in real time[1]. And also, some sports or exercise recognition tasks with only wearable sensors[2, 3]. A mature example is the speech input on wearables, Google's Voice Search support an end-to-end, all-neural, on-device speech recognizer to power speech input in Gboard, which uses an around 80M size RNN transducer model. So for those wearable applications, a real-time local inference is a demand. Thanks again for your opinion sharing about the convergence of edge computing and deep learning.

[1], Article The role of wrist-mounted inertial sensors in detecting gait...

Conference Paper Passive Capacitive based Approach for Full Body Gym Workout ...

[3], Conference Paper Using Wrist-Worn Activity Recognition for Basketball Game Analysis

Good question

Sizhen Bian, one point I was trying to make, perhaps not so well, is that in addition to some use cases being capable being handled by the cloud, there are already edge cases using commodity server type hardware. As an example, the defect detection through machine vision use case I mentioned has the computation on the edge, even if the data is stored in the cloud later on. I'd argue that the economics would push for more commodity processing rather than a specialized ASIC. As an example (disclaimer - I work for Intel), Intel has just put out processors built for edge applications:

https://www.intel.com/content/www/us/en/design/products-and-solutions/processors-and-chipsets/elkhart-lake/overview.html

Improvements to speed to inferencing and real time control have been added.

hi, Jeff Sedayao , I appreciate a lot for sharing your thought here, it's so important to here something from people like you working in Intel. I totally agree with what you wrote about the push form the side of economics. I think for most IoT cases that need local deep computing(the example you mentioned), some very new universal processors. like the one from Intel, is capable of the local deep data handling.

I'm working in a group mainly focused on the sensor-based human activity recognition in DFKI, especially the data from wearable sensors. Recently there are plenty of conference papers proposing novel deep learning approaches to recognize activities. But all inferencing are made on PC, instead of the wearable devices where the data was generated. That's the basic motivation for posting this question.

Whether a compressed deep model being fit into a week processor or a neural network-oriented chip will be more dominant, maybe again a neutral answer: use cases-dependent.

Ok, then let's talk about a special use case, how engineers will solve this problem: offline speech input on a smart band? I mean a full offline speech recognizer instead of just some wake-up words. Do you think they will need a front-end ASCI ship like NDP101[1] or keep compressing the current model[2]?

[1], https://www.syntiant.com/ndp101

[2], https://ai.googleblog.com/2019/03/an-all-neural-on-device-speech.html

Sizhen Bian: Great question! My guess (and it's only a guess) would be to compress the model, as software will generally evolve faster than hardware.