it may depend on what you want to measure - actual participation in the meetings, or the quality/ outcomes of the meetings. seems you are looking at the latter.

i dont think you would be able to evaluate the meetings in a direct way. one would need to implement suggestions, etc first to see how good they were, and there are other issues related to implementation. it may seem a meeting had good outcomes, only to later find very little can be implemented or is practical.

hence, i think you can really only evaluate the level of participant engagement - the level to which participants gave themselves - and the level of thought - the quality and extent of reasoning.

you would likely measure both using qualitative methods that interrogate the meetings and recordings thereof

This is a difficult task if your objective is to evaluate quantitatively the success of community meetings.  I think one approach is to narrow your focus to the effectiveness of communication at those meetings.  That objective should be easily quantified.  However, if your objective is to evalute the effects of those meetings on plan making and implementation, that will be a much more difficult job.  You need to consider many broader socioeconomic factors, especially the institutional background of urban planning.  Peilei Fan published a paper on the effects of planning exhibition halls in China.  Your research sounds similar to hers although her study analyzed the effects on plans, plan making and implementation.

A good question that should be more thoroughly investigated. My suggestions for planning discourse support systems -- aimed mainly at online projects (not community meetings) include include potential measures for argument plausibility, plan proposal plausibility and quality, and the merit of discourse contributions in general. They use basic 'contribution rewards' that serve as initial participation incentives as the vehicle for later assessment of their plausibility and merit, with the possibility of focusing on different aspects such as number of ideas generated, depth of evaluation, improvement of participants' judgments as a result of added information. These assessments are based on evaluation by the participants in each project, so the comparison of the outcome of different approaches (with different groups of participants) face some methodological challenges; it would require a 'third' group of evaluators who will of course have different perspectives, values and stakes in the project. But the approach would provide a meaningful basis for studies. This is -- for me -- several steps into the future; my concern is to get some projects to use the approach, which will first call for some programming of the support platform to integrate the various functions. I have a number of articles on the subject I can make available to interested parties.

Hello Dongying

In my experience with projects of social intervention through the participation of the beneficiaries, the success of the meeting is based on the integration of the teams where the leaderships are distributed so that they promote participation. The methodologies or dynamics of groups can be effective for some and in others a failure, therefore, the meetings must be dynamic, with material available, with people trained to lead a group.

Brian explanis that "you can really only evaluate the level of participant engagement - the level to which participants gave themselves - and the level of thought - the quality and extent of reasoning", I agree with the .

Good luck

Leticia Peña-Barrera

Hello Dongying,

You are asking a very good question. There is a need for more systematic and semi-experimental comparisons of different community engagement methods in planning.

Maybe you can adapt the methodologies from the following sources:

Al-Kodmany 1999 Using visualization techniques for enhancing public participation in planning and design: process, implementation, and evaluation [this article compares the effectiveness of different methods using visualisation ]

Behrer & Breux 2012 The diversity of public participation tools: Complementing or competing with one another?

Pocewicz et al 2012 An Evaluation of Internet Versus Paper‐based Methods for Public Participation Geographic Information Systems (PPGIS)

Jankowski et al 2017 Evaluating the scalability of public participation in urban land use planning: A comparison of Geoweb methods with face-to-face meetings

Brown etal 2014 An empirical evaluation of workshop versus survey PPGIS methods

Brown etal 2012 Evaluation of an online (opt-in) panel for public participation geographic information systems surveys

Brown etal 2014 Which 'public'? Sampling effects in public participation GIS (PPGIS) and volunteered geographic information (VGI) systems for public lands management

Many researchers have compared how different technologies tend to attract different people, which can influence both the type and quality of output data that will be produced through different community engagement methods, and the planning decisions that would be based on them.

I highly recommend in this respect to have a look at my book "Public Participation as a Tool for Incorporating Local Knowledge into Spatial Planning" published by Springer (2017). I compare between the capabilities of various participatory methods - e.g., Charrettes, Citizen Forums, Focus Groups - to uncover citizens' needs and interests and incorporate them into plans. In addition, because I have asked similar questions in ResearchGate you might find the answers to my questions useful.