For two dimensional pareto fronts, the normalized area under the two pareto frontiers (with the axes set so less means better) is a very nice metric.

A note on 2d pareto AUC. Your two multi objective solutions may not produce the same max on both axes. In that case, you have to think through how that affects your problem. You may want to set problem appropriate max'es to make the two curves start and end at the same spot and then give each of the curves being compared an appropriate point on that max.

Once you move into 3 dimensions, AUC becomes a more difficult issue. You have to construct a convex hull and measure the area under it. That's a lot more programming than the 2d approach. It also requires assumptions about what to do in areas of Pareto space where there is little or no information, especially those near the "edges" of your pareto space--and you find those kind of low information areas not infrequently. (Also, defining a "low information area" is something you should think through). Once I start making assumptions and filling in data that doesn't exist, my confidence in the comparison metric goes down. And, what do you do if one learning algorithm produces solutions in an area of pareto space and the other one does not? Does the interpolation implicit in a convex hull for the later solution produce an appropriate comparison metric? There's probably a problem specific answer to that question. One way to fix this issue would be to produce a lot more solutions and hope pareto space fills up. But the more dimensions you use, the more sparse your pareto front will be.

I know the multi-objective optimization community works with a lot of 4d and up pareto spaces. I haven't seen work that measures the AUC in 4+ dimensions. I'm sure the math exists to do that but AUC for 4+ dimensions is not an intuitive concept.

Frank

Check this tutorial: ftp://ftp.tik.ee.ethz.ch/pub/people/zitzler/ZTLFG2002a.pdf

It's quite complete, and nothing really new has been proposed ever since, except for more efficient implementations.

Some useful links in addition:

- Hypervolume: http://iridia.ulb.ac.be/~manuel/hypervolume

- EAF: http://iridia.ulb.ac.be/~manuel/eaftools

You'll probably also need an epsilon indicator, but they're pretty easy to implement.

Sorry, I'll have to correct the link. That is the technical report version of a very good paper that was publish in IEEE TEC in 2003.

The link I intended to copy was this one: http://www.lania.mx/~ccoello/EMOO/knowles06a.pdf.gz

https://www.researchgate.net/publication/297245624_Particle_Swarm_Optimization_Algorithm_and_its_Codes_in_MATLAB

Research Particle Swarm Optimization: Algorithm and its Codes in MATLAB

https://www.researchgate.net/publication/296636431_Codes_in_MATLAB_for_Particle_Swarm_Optimization

Code Codes in MATLAB for Particle Swarm Optimization

There are many examples of quality measures in the literature. Some aim at measuring the distance of an approximation set to the Pareto-optimal front: Van Veldhuizen [21], e.g., calculated for each solution In the approximation set under consideration the Euclidean distance to the closest Pareto-optimal objective vector and then took the average over all of these distances. 

Other measures try to capture the diversity of an approximation set, e.g., the chi-square-like deviation measure used by Srinivas and Deb [18]. A Further example is the hypervolume measure which Considers the volume of the objective space dominated by an approximation set [26]. In these three cases, an approximation set is assigned a real number which is meant to reflect (certain aspects of) the quality of an approximation set. Alternatively, one can assign numbers to pairs of approximation sets.Zitzler and Thiele [26], e.g., introduced the coverage function which gives for a pair (A, B) of approximation sets the fraction of solutions in B that are weakly dominated by one or more solutions in A.

Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C. M., & Da Fonseca, V. G. (2003). Performance assessment of multiobjective optimizers: an analysis and review. Evolutionary Computation, IEEE Transactions on, 7(2), 117-132.

ftp://ftp.tik.ee.ethz.ch/pub/people/zitzler/ZTLFG2002a.pdf

There exist several paradigms to order non dominate points. From fuzzy to clasical niching methods are appropiates.

It seems that every year a new performance index is added

All the answer are great

Try the following link:

http://www.lania.mx/~ccoello/EMOO/knowles06a.pdf.gz

I had the same idea with Dr. Erik Cuevas , it seems to be many performance indexes added each year in much research. Thank you for sharing.

I found one based on

[1] Deb, Kalyanmoy. Multi-Objective Optimization Using Evolutionary Algorithms. Chichester, England: John Wiley & Sons, 2001.

Spread — The spread is a measure of the movement of the Pareto set. To calculate the spread, the gamultiobj algorithm first evaluates σ, the standard deviation of the crowding distance measure of points that are on the Pareto front with finite distance. Q is the number of these points, and d is the average distance measure among these points. The algorithm then evaluates μ, the sum over the k objective function indices of the norm of the difference between the current minimum-value Pareto point for that index and the minimum point for that index in the previous iteration. The spread is then

spread = (μ + σ)/(μ + Qd).

The spread is small when the extreme objective function values do not change much between iterations (that is, μ is small) and when the points on the Pareto front are spread evenly (that is, σ is small).

Link: https://www.mathworks.com/help/gads/gamultiobj.html

I was wondering about the smaller value of Spread for each Pareto front the better. So, when comparing the Pareto fronts in our solution, we can find a better result from this index. Could someone have experience with it? Thank you so much,

Regards, @

You can find the answer here:

This can provide some insight

Article Zimmermann, H.-J.: Fuzzy Programming and Linear Programming ...