Dear Pantelis Sotirelis · Panagiotis Nakopoulos · Theodora Valvi · Evangelos Grigoroudis · Elias Carayannis

I have read your article

Measuring Smart City Performance: A Multiple Criteria Decision Analysis Approach

My comments:

1- Very important and rarely addressed subject

2- Very good appreciation of the evolution of cities in the last 20 years. Very illustrative

3- In my opinion, the definitions of a smart city lack of a very important component. This is the ratio between inputs and outputs or circular economy, a fundamental concept nowadays, as well another important component: Resilience.

4- You should explain that ICT means Information and Communications Technologies

5- In the abstract you mention 52 indicators and 6 pillars. Sorry, but these expressions lead to confusion. I understand that there are 20 smart cities or alternatives to evaluate, subject to 52 criteria, that you call ‘indicators’ (please, consider that they are not, indicators are metrics that may change with time, for instance, the GDP, the number of people attending high school, the crime rate, etc.). Criteria are conditions that alternatives must meet.

‘Pillars’ is incorrect. You have 5 ‘Key areas’, and different ‘clusters’ within each one.

For instance, for the ‘Mobility’ key area, there are 3 clusters (Entrepreneurship, Income /Equality and Economic development). In Entrepreneurship you have 2 criteria (On line servicesand Ease of doing business, a so on. It is far for my intention to correct a colleague; however, this is not a matter of semantics, but in using the right MCDM wording that everybody is used to.

6- In page 3 you make a difference between smart cities and sustainability. In my opinion, the second is part of the first because it is related with a city future. If you consider a Venn diagram, normally used to express parts that integrate for form a whole, each part is a circle including components of smart cities as well as environment, sustainability, economics and society.

This space is then the intersection of all of them, and contains all solutions of the problem.

7- In page 3 you say “Nonetheless, the majority d a MCDM of the evaluation models construct composite indicators”

In my opinion this is incorrect, I am not aware of any MCDM method constructing composite indicators, except LP, that may be used for that, albeit its aim is the same as the other MCDM methods. That is, given a set of alternatives, subject to a set of criteria, determine the best option or project.

A concise definition of composite indicators can be found in:

Joze Rovan (2011) - Composite Indicators. In: Lovric, M. (eds) International Encyclopedia of Statistical Science. A composite indicator is formed when individual indicators are compiled into a single index, on the basis of an underlying model of the multidimensional concept that is being measured (OECD, Glossary of Statistical Terms).

8- You say that PROMETHEE is chosen to aggregate individual indicators, which is true, however, again, it is incorrect to evaluate individual criteria and then, add them up. All MCDM methods follows this procedure, except Linear Programming (LP), which considers all criteria and all alternatives simultaneously, and finds, it is exists, the dimensional space where alternatives interact, similar to a Venn diagram. Remember that a result is not always equal to the sum of the parts, but to their mutual intersection.

It is the only way to find a common space for solutions.

In other words, the composite indicator is not the aggregate of individual indicators, it is the result of their intersection.

9- In page 3 “PROMETHEE II is an outranking method that appears appropriate in the examined problem, since it can avoid the fully compensatory nature of composite indices and also avoid incomparabilities among alternatives (i.e., cities). Furthermore, the paper adopts a holistic approach for the assessment of “smartness” based on the six axes of Gifnger et al. (2007).

The only compensatory index I know is related to human health (Compensatory Reserve Index).

I don’t understand why the paper addresses this compensatory issue and without explaining it.

Since composite indices refer to indicators or metrics, as an example, suppose we have a composite index of say 5 indicators like Logging, Deforestation, Disposable Income, Education, and Water contamination, for year 2020. We get a unique figure that condenses the information for that year of these five metrics. There is no compensation here, since a variation in plus or minus of Education regarding 2019, is not related with say an increase in deforestation from 2019 to 2020. That is, there is no relationship between the variation of one indicator with respect to another one, or, in other words, there is no compensation between metrics.

Of course, deforestation is a consequence of logging that produces large benefits from the economic point of view, but this is another indicator. In addition, an increase of logging does not necessarily mean an increase of deforestation, since in accordance with existing laws, reforestation in mandatory.

10- Now, why should exist incomparability between cities if we are talking about the same issue? Of course, there could be different results but they are comparable. City A may have a crime rate of say 2.4% while city G could have 1.9%, and then, from this point of view G is in a better condition than A.

11- Page 5 - What is a dimension? You don’t explain its meaning. In MCDM mathematics, dimensions denote the mathematical solution spaces of a problem, and refers to the number of coordinate axes, and it is given by the number of alternatives. Thus, a 20 alternative problem has 20 dimensional spaces or 20 coordinate axes, while a 2 alternative problem has 2 dimensions or 2 coordinated axes. It appears that you are confused with the multiple criteria concept.

12- Page 6 - “while the entropy method was implemented for assigning weights to the 18 indicators of the model”

The DM does not assign weights using entropy. In here, the ‘weights’ are called ‘Quantity of information’ , they are intrinsic to each criterion and measure its evaluation capacity. The DM does not have any participation in them.

13- Your paper produces a very valuable literature references about smart cities.

14- Page 15, Table 1 and 2. Where did you get the percentual values from? In my opinion, the ‘Pillars’ should not have the same weight, and if you believe they do, I suggest you explain your reasons

15- Page 19, Table 4. You mention ‘Vancouver’. There are 2 cities with that name, one in the US Aand the other in Canada. Which of them is in the list, both are close in geographical area, on the Pacific Ocean and very similar?

16- Page 22. The paper refers to sensitivity analysis, a crucial subject. It suggests using the standard deviation to determine the strength of the solution. However, frankly, I don’t see how.

This strength is related to the best alternative holding the first position, when some criteria are increased or decreased. Where is the relation of your comment on sensitivity analysis regarding the mentioned condition, which is universally accepted as measure of strength? I don’t think that sigma has anything to do with this, and you don’t explain either.

I hope that these comments may help

Nolberto Munier