Dear

Look the link. May be useful.

http://faculty.ucmerced.edu/mhyang/papers/spie11a.pdf

http://www.nature.com/nmeth/journal/v11/n3/full/nmeth.2808.html

Regards, Shafagat

The question is if for your application the tracking information is really needed in the end? In multiple target tracking the tracker has to assign consistent labels to tracked objects. If it is important for you that the labels are correct than you should do an evaluation in the context of multi-target tracking.

However, I assume that you do the tracking just internally to get better detection results. Then you should evaluate the detection result. Very crucial to any evaluation is ground truth data, i.e. you need manually annotated bounding boxes of the pedestrians in the images. Then you can compare the output of your method to the ground truth bounding boxes. This is usually done by defining an acceptable amount of overlap between bounding boxes and measure miss rate and false positive rate of your detections. A good overview  and explanation is given in http://vision.ucsd.edu/~pdollar/files/papers/DollarPAMI12peds.pdf.

Hello again.

You ask some good questions! To answer it, you must make your requirements more precise. There are many performance metrics used for tracking systems and in a given application some are more important than others. I tend to stay away from combined metrics (like the wasserstein one the PHD-ers are pushing currently) since they don't really mean anything and combine lots of distinct ideas. As there are basically no analytical methods for tracking system performance, you have to use simulation and gather statistics. You can look at initiating track, losing track, track seduction, swaps, copies, etc. Or you can plot a ROC (TOC) curve in simple cases. Be careful to define exactly what you mean and what the scenario parameters are (PD, PFA, etc). The MTT field is actually in a sorry state IMO and there are  few researchers who seem to be interested in actually analysing what is going on rather than just pushing their own brand and doing 100 MC runs, without even bothering to simulate other approaches.

Here is a link to an attempt to model the SPRT which I did a while ago.

Please cite this if you find it useful, as not many people know about it ;-)

Some metrics are covered in the second paper (multi-target VDA). Come to think of  it, Wasserstein might be useful if you limit the types of errors that can occur.

Conference Paper Markov chain analysis of the sequential probability ratio te...

Article Multihypothesis Viterbi Data Association: Algorithm Developm...

hi go through the following link

http://vision.ucsd.edu/~pdollar/files/papers/DollarPAMI12peds.pdf.

Hi,

Graham pointed out the fact you must analyze what kind of evaluation is required.

In some MTT methods, if track labels or theoretical positions of pedestrians are known, you can use the OSPA-T distance. It can both combine the tracking accuracy and the cardinality estimation. Full details of this metric can be found in Ristic's paper :

Article A Metric for Performance Evaluation of Multi-Target Tracking...

There are many object tracking papers published in good journals. you can check how they evaluated  the performance of the system.

Some of the papers are:

http://faculty.ucmerced.edu/mhyang/papers/spie11a.pdf

http://link.springer.com/article/10.1007/s10043-015-0178-2

http://link.springer.com/article/10.1007/s11042-015-3211-3

http://link.springer.com/article/10.1007/s11042-014-2078-z

http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7001050&tag=1

Please follow the link instructions for evaluating your tracker results.

http://cvlab.hanyang.ac.kr/tracker_benchmark/