Wisdom of the crowd

Jury theorems from social choice theory provide formal arguments for wisdom of the crowd given a variety of more or less plausible assumptions.

Aristotle is credited as the first person to write about the "wisdom of the crowd" in his work Politics.

At a 1906 country fair in Plymouth, 800 people participated in a contest to estimate the weight of a slaughtered and dressed ox.

[7] This has contributed to the insight in cognitive science that a crowd's individual judgments can be modeled as a probability distribution of responses with the median centered near the true value of the quantity to be estimated.

[8] In recent years, the "wisdom of the crowd" phenomenon has been leveraged in business strategy, advertising spaces, and also political research.

[12] Although classic wisdom-of-the-crowds findings center on point estimates of single continuous quantities, the phenomenon also scales up to higher-dimensional problems that do not lend themselves to aggregation methods such as taking the mean.

A few examples of higher-dimensional problems that exhibit wisdom-of-the-crowds effects include: In further exploring the ways to improve the results, a new technique called the "surprisingly popular" was developed by scientists at MIT's Sloan Neuroeconomics Lab in collaboration with Princeton University.

[20] In the digital age, the potential for collective intelligence has expanded with the advent of information technologies and social media platforms such as Google, Facebook, Twitter, and others.

These platforms enable the aggregation of opinions and knowledge on a massive scale, creating what some have defined as "intelligent communities.

To mitigate these issues, researchers have suggested using a multi-media approach to aggregate intelligence from various platforms or employing factor analysis to filter out biases and noise.

[24] The insight that crowd responses to an estimation task can be modeled as a sample from a probability distribution invites comparisons with individual cognition.

The answers on the ends of the spectrum will cancel each other, allowing the wisdom of the crowd phenomena to take its place.

In general, these results suggest that individual cognition may indeed be subject to an internal probability distribution characterized by stochastic noise, rather than consistently producing the best answer based on all the knowledge a person has.

[27] Rauhut and Lorenz (2011) expanded on this research by again asking participants to make estimates of continuous quantities related to real world knowledge.

Ultimately, they argue that the results of Vul and Pashler (2008) overestimate the wisdom of the "crowd within" – as their results show that asking oneself more than three times actually reduces accuracy to levels below that reported by Vul and Pashler (who only asked participants to make two estimates).

[30] Herzog and Hertwig (2009) attempted to improve on the "wisdom of many in one mind" (i.e., the "crowd within") by asking participants to use dialectical bootstrapping.

[31] Hirt and Markman (1995) found that participants need not be limited to a consider-the-opposite strategy in order to improve judgments.

[37] Miller and Stevyers reduced the independence of individual responses in a wisdom-of-the-crowds experiment by allowing limited communication between participants.

[40] Wisdom-of-the-crowd algorithms thrive when individual responses exhibit proximity and a symmetrical distribution around the correct, albeit unknown, answer.

Conversely, these algorithms may falter when the subset of correct answers is limited, failing to counteract random biases.

This challenge is particularly pronounced in online settings where individuals, often with varying levels of expertise, respond anonymously.

Specifically, the algorithm identifies experts by presuming that their responses will be relatively "closer" to each other when addressing questions within their field of expertise.

This approach enhances the algorithm's ability to discern expertise levels in scenarios where only a small subset of participants possess proficiency in a given domain, mitigating the impact of potential biases that may arise during anonymous online interactions.

Social influence can cause the average of the crowd answers to be inaccurate, while the geometric mean and the median are more robust.