I recommend to read the following paper, "Action Recognition by Dense Trajectories"

https://hal.inria.fr/inria-00583818/document

There is also code available to describe trajectories.

I recommend to read the following paper, "Action Recognition by Dense Trajectories"

https://hal.inria.fr/inria-00583818/document

There is also code available to describe trajectories.

Thanks Sebastian, sounds good. 

I will check it out.

You can also turn them into time-series and apply "dynamic time warping".

https://en.wikipedia.org/wiki/Dynamic_time_warping

Thank you Tunc, 

As you know, DTW is to measure similarity between two sequences. Do you know if there is a way to describe a single signal by using DTW in orm of a FV (Feature Vector) . Since I have other elements in my FV which are not similar to the signal in concept, i.e "area" etc. Thus I might not be able to use DTW as a distance metric in my classifier.  In fact, I am not familiar with this metric.

I should know a bit more about your case. For example, people use DTW for kinect gesture detection. There are pre-recorded movement series and the newly recorded series are compared with those. Each time-series is a feature vector actually. So, you might add this DTW as one of your predictors along with other features. As I said, I can answer better if I know the case better.

Basically we are doing the same: we have recorded movements (or our train set) and also some samples for testing.

We extracted some features i.e. ratio between different components of face and put them together in our FV,  We are also interested to add some information related to motion, since we have trajectory of points. But I am afraid if it's possible to use DTW as a descriptor, and add it into my FV together with other features. As far as I studied, DTW compares two input signals. Is it right?

Yes, basically it compares two series which do not have to be at the same length. For example, if you are interested in different expressions, you can both use still images or face changes from neutral to that certain expression. DTW is used in gesture detection because some gestures cannot be comprehended without time and speed. For example, punching cannot be reduced to a single photo, you should have the movement from the beginning to the end. And, DTW can handle time variances like everybody does the movement in different speeds and times.

If you are working on expressions, still images might be enough. But, lets say if you are working on sign language or lip reading, you should have temporal information as well. It mostly depends on your problem.

That's right Tunc. We are working on facial expression for specific signs similar to sign language. So, as you already mentioned our objective is to combine the motion and still image information together in a form of FV.  So, I will keep working on this idea.

Thank you for the informative comments.

There is another hybrid approach to gesture/motion detection. AdaBoost trigger starts the motion and random forest is used for percentage detection: https://msdn.microsoft.com/en-us/library/dn785522.aspx

Dear reza

I used mapminmax function in matlab to map features (facial 3D landmarks and shape) in scale of 0-1. So as result my feature vector is like V(shape) = mapminmax ([x1,y1,z1,....,xn,yn,zn]) I got very good score with cross validation for facial expression.

For facial expression you can also test facial action units the accuracy for it depends on the number of emotions that your model need to recognize.

You can use CLM-framework to extract facial action units and 3D shape which is implemented by Cambridge university.

You can also use texture features like Local Binary Pattern and fuse it with other features that you are using. You can use any combination of texture, action units and 3d shape and fuse them by deep canonical correlation analysis or correlational neural networks (CorrNet).

Since the size of action units are only 20 you can simply append it to end of your feature vector.

 I hope you find my comment useful ;)

Thanks Muhammad,  you gave me fruitful clues. 

In fact, I am not very familiar with those techniques, so I need to study them at first in order to comprehend how to apply in my problem. However, I liked  Action Unit, but afraid of its applicability to describe a motion in sequence of frames, which is my first concern in this problem. 

 Best,