I am not an expert in this but I do find this question interesting. From the user point of view , it is a lot easier to just click "like" than to vote on stars. Researchgate itself uses upvote vs downvote binary system for the supposedly intelligent and sophisticated users (researchers/scientists) . Emotional scales would probably be too complicated for the users .  Good luck with your research!

Thank you for your kind comment, it makes sense and help thinking :) 

It's not only a question of scale but also data collection méthod. I just studied influence of stars on demand and price in a carpooling platforme. Surprising the effect is quietly not  significant. An explanation is the skewness of distribution that result from a bargaining behavior : because passengers are alors evaluate by drivers, they give 4 ou 5 stars that don't discriminate enough to signal any quality. Number of rating seems more predictive ( as a number of like) ...

Thank you Christophe, it is very interesting. The expected reciprocity is problematic when we want "independant" people, as required for crowd wisdom effect to happen. I just finished a report about voting systems. It seems that basically all the current systems (and methods) are not adequate. So for me basically, all websites are wrong. It might be OK for 'popularity' but to predict quality it is just a poor method. To me, websites shall not show the evaluation before people vote for it... and your point about expected reciprocity implies that websites shall NOT show who (or more precisely 'how much')  voted for you... Independence VS Community... Some new evaluation methods are in development, and other indicators can be used (like the number / valence of comments). A lot of research is still needed !

Sinan Aral (with Taylor among others) has published a number of relevant studies, e.g..: Muchnik, L., S. Aral and S. J. Taylor (2013). "Social Influence Bias: A Randomized Experiment." Science 341: 647-651.

Put briefly, it surely is no coincidence that Google and Facebook only have "positive" like evaluations, and no negative - since these turn out to be useless, while positive likes persist over time in some contexts.

My own paper, https://www.researchgate.net/publication/261285310_How_to_Solve_Problems_with_Crowds_A_Computer-Based_Simulation_Model?ev=prf_pub, also investigates this, and similar to your thoughts, simulation models show that one should not expose the ideas of others too soon.

Article How to Solve Problems with Crowds: A Computer-Based Simulation Model

Thank you Carsten for your answer, I will look at those papers carefully.

Spoken in another way, you are dealing with the issue of social effects on online systems and want to somehow moderate these effects.

You could go by traditional crowd wisdom studies aka. Surowiecki (crowd wisdom requires diverse groups, who are independent and have local knowledge) and Jaron Lanier (crowd wisdom idea will only work when crowd does not define the question).

Golub, B., & Jackson, M. O. (2010). Naive learning in social networks and the wisdom of crowds. American Economic Journal: Microeconomics, 112-149. has a nice model using DeGroot learning that theorizes how wisdom can occur when independence cannot be controlled (ie. in almost every crowdsourcing case). It would be interesting to conduct an experiment to test interfaces using this model as an evaluative tool.

Thank you very much for those elements, it is so cool to have that kind of feedback. And also, sorry for answering late.

The DeGroot model and formalization is very interesting. I find it very enlighting for conditions where you have a "community" in a broad sense, ie. when some people have a better reputation than others. I am wondering if it could not be easier si simply hide the identity of the author until the end of an evaluation process (like for ex. in prediction markets). Anyway, thanks for the sharing.

I don't know the work of Lanier : why does he think that when crowd defines the question it will not be wise enough to find a good answer ? 

About voting systems, for those interested, it seems that the Bags of Lemon approach from Klein & Garcia (2014, 2015) is a more powerful and a fastest evaluation technique. It is interesting since it do not display previous votes.

Klein, M., & Garcia, A. C. B. (2014). The Bag of Stars: High-Speed Idea Filtering for Open Innovation.  Collective Intelligence conference 2014

Hi Mehdi,

Lanier is a futurist with a great amount of experience in computer systems etc. The main thrust of his argument is that when we rely on digital markets for decision-making we forget what is actually producing the wisdom in the first place: people thinking hard about what the answer might be. Broadly, you can't take the social out of social.

I think anonymizing the users would be a _possible_ solution to removing this bias, but I think skepticism would be in order. Much better to test it. Lots of variables to consider: interface, feedback, selection, problem structure, level of controversy, promotion etc.

The question of whether collective intelligence is social or not is really important. Too many scientists consider it just 'aggregating' (mathematically) individual opinions. In the other hand, social influence is strong and not a must at all... I think it shall depend on the nature of the activity taken in charge (routine tasks vs evaluation vs creating vs deciding)...  An interesting topic ! Thank you very much for those elements.

Best