Group decision making in honey bee swarms (2006)

This Article From Issue

May-June 2006

Volume 94, Number 3

The problem of social choice has challenged social philosophers and political scientists for centuries. The fundamental decision-making dilemma for groups is how to turn individual preferences for different outcomes into a single choice for the group as a whole.

This problem has been studied mainly with respect to human groups, which have developed a variety of voting procedures to single out one option from a list of possible choices: majority rule, plurality wins, various weighted-voting systems and others. Social choice in animal groups is less well studied, although examples are abundant: A baboon troop must decide where to go following a rest period; an ant colony decides whether or not to attack a neighboring colony.

A striking example of decision making by an animal group is the choice of a nesting site by a swarm of up to 10,000 honey bees. This process involves several hundred bees from the swarm working together to find a dozen or more candidate nesting cavities in trees and then selecting the best one of these options for their new home. We've been investigating this process for the past decade using a variety of observational, experimental and mathematical-modeling studies. This work has revealed a set of behavioral mechanisms in a swarm that consistently yields excellent collective decisions. It has become clear that this group intelligence is a product of disagreement and contest, not consensus or compromise, among different groups of bees representing different alternatives in the decision-making task. We have found that evolution has supplied an intriguing answer to the question of how to make a group function as an effective decision-making unit.

For centuries beekeepers have known that in late spring or early summer a strong colony of honey bees will divide itself by swarming, a process in which the queen and approximately half the worker bees leave their hive to establish a new colony; meanwhile a daughter queen and the balance of the workers remain behind to perpetuate the old colony. Beekeepers also have known that after a swarm leaves its parental hive, the bees will coalesce into a beardlike cluster on a nearby tree branch, conduct a search for a home and, if left alone, eventually launch into flight and move off together to their new abode, usually a far-off hollow tree. People have long captured the bivouacked swarms that they have found and installed them in manmade hives, cutting short the bees' nest-site search. Thus it is not surprising that this decision-making process long remained a deep mystery.

This situation began to change in the 1950s when Martin Lindauer, a German zoologist, published his seminal paper on house hunting by honey bees. Lindauer was then a postdoctoral student at the University of Munich, studying with the famous Karl von Frisch, who had shortly before decoded the waggle dance of honey bees. This communication behavior allows successful foragers to inform hive mates of the locations of rich food sources through a specific series of movements. A dancing bee runs forward and performs the waggle run, vibrating her abdomen laterally, then circles back to her starting point, producing one dance circuit. A single bout of dancing contains many of these circuits. Von Frisch found that the length of a bee's waggle run translates into the distance to the food source, and the angle of the dance represents direction.

Lindauer was a keen observer. On one occasion when he was using his skills on a swarm of bees that had settled outside the university's Zoological Institute, he noticed that bees on the surface of the swarm were performing waggle dances. Moreover, he observed that these dancers on the swarm, unlike those in a hive, did not bear loads of nectar or pollen. Evidently, these were not foragers advertising profitable food sources. Might they be scouts reporting potential nest sites? This was a previously unknown use of waggle dancing.

Lindauer answered this question by patiently observing all the dancers on several swarms, a marathon task that demanded many days of steady bee watching and frantic note taking. Whenever he saw a new dancing bee, Lindauer noted the location coded in her dance and gave her a paint dot to avoid repeatedly recording her dance information.

This painstaking work yielded several remarkable discoveries. One was that during the decision-making process, only a few hundred of the thousands of bees in a swarm were active—flying to and from the swarm, presumably finding and inspecting potential nest sites, then performing and following dances. Most bees remained quiescent, probably to conserve the swarm's energy supply, until a decision had been made and it was time to fly to the chosen site. A second curious find was that, at first, the bees' dances indicated various sites around the swarm, but hour by hour the number of sites advertised by the dances declined until just one site remained, which was excitedly reported by dozens of dancing bees. Lindauer also found that shortly after the bees' dances had become focused on one site, the entire cluster of bees would suddenly take off and fly toward this site. Sometimes he managed to sprint beneath the swarm throughout its cross-country flight and so learned its precise destination—always a cavity in a tree or building and always at the spot indicated in the final dances. There was no doubt then that the dancing bees were reporting nest sites. Indeed, it seemed these bees were holding a kind of plebiscite on the swarm's future home, although exactly how they conducted their deliberations was still unknown.

In the mid 1990s we decided to look more deeply at this intriguing example of animal democracy. In the years since Lindauer's work, several investigators had studied the real-estate preferences of honey bees and had found that a first-rate home for a honey bee colony has a cavity volume greater than 20 liters and an entrance hole that is smaller than 30 square centimeters, perched several meters off the ground, facing south and located at the floor of the cavity. But no one had figured out exactly how the scout bees in a swarm implement these housing preferences during their collective choice of a new home.

Our first step in renewing the analysis was to repeat Lindauer's observations of the scout bees' dances, but using modern video equipment to get a more complete picture than had been possible in the 1950s. We worked with small swarms of about 4,000 bees and labeled each bee for individual identification, so we could attribute each dance to a particular individual and thus ascertain her contribution to a swarm's decision making.

From our recordings of every dance performed by each scout bee, we found a pattern of dancing by nest-site scouts that closely resembles what Lindauer reported based on his records of only each scout's first dance. For example, in a swarm we observed on July 20 to 22, 1997, the entire decision-making process required about 16 hours of dance activity spread over three days. During the first half of the process, the scouts reported all 11 of the potential nest sites that they would consider, and no one site dominated the dancing. During the second half, however, one of the sites gradually began to be advertised much more than the others and ultimately became the chosen site. Indeed, during the last few hours of the decision making, the site that had emerged as the frontrunner became the object of all the dances.

Given the striking way that the dances on a swarm come to represent one site and then the swarm moves to this site, it was tempting to conclude that a swarm's decision-making process is essentially one of consensus building, rather like the arrival of the "Sense of the Meeting" among Quakers. By this hypothesis, a scout bee "votes" in favor of a site by dancing for it, somehow the scouts act and interact so that gradually their votes come into agreement in favor of a superior site, and somehow the voting pattern of the scouts is steadily monitored so that they know when they've reached an agreement and can start acting on their decision.

There were, however, two factors that cast doubt on this appealing hypothesis. First, neither Lindauer nor we had seen any sign of scout bees polling their fellow dancers, something that surely they must do to know when they've reached an agreement. Second, both Lindauer and we had occasionally seen a swarm launch into flight without a dance consensus, that is, when there were two strong coalitions of dancers advertising two distinct sites. Were these rare cases of takeoffs with dissent simply bizarre anomalies that we could ignore, or were they valuable clues that we should heed?

We chose to heed them, because we had long wondered whether the essence of a swarm's decision making might be sensing a quorum (sufficient number of scouts) at one of the nest sites rather than sensing a consensus (agreement of dancing scouts) at the swarm cluster. By this quorum-sensing hypothesis, a scout bee "votes" for a site by spending time at it, somehow the scouts act and interact so that their numbers rise faster at superior sites, and somehow the bees at each site monitor their numbers there so that they know whether they've reached the threshold number (quorum) and can proceed to initiating the swarm's move to this site. This hypothesis can explain the cases of liftoff with dissent as instances where a quorum was reached at one site before the competition between dancers from different sites had eliminated the dancing for all but one site.

We tested these two hypotheses with experiments performed on Appledore Island, site of the Shoals Marine Laboratory of Cornell University. This island, off the coast of Maine, is nearly treeless and so is lacking in natural nesting cavities for honey bees. Each swarm that we ferried to this island was thus compelled to pay attention to the special nest boxes that we provided. In our first experiment, we presented several swarms, one at a time, with two identical nest boxes, each one a superb nest site. The swarm was positioned at the island's center, whereas both nest boxes were placed near the rocky shore, each one 250 meters from the swarm but in different directions. We found that when swarms were forced to choose between two first-rate nest sites, they would routinely take off when scout bees were still dancing strongly for both sites. Consensus among dancers was certainly not necessary for these swarms to start flying to one of the sites, hence we could reject the consensus-sensing hypothesis. At the same time, we gained support for the quorum-sensing hypothesis, because we noticed that swarms consistently started preparing for flight once 15 or more bees were seen together at one of the nest boxes. It should be noted, however, that because the bees spend the majority of their time at the swarm, seeing at least 15 bees at a nest site at any one time means that approximately 150 bees overall are visiting the site.

In our second experiment on Appledore Island, we explicitly tested the quorum-sensing hypothesis by checking a falsifiable prediction of it: Delaying the formation of a quorum at a swarm's chosen nest site, while leaving the rest of the decision-making process undisturbed, will delay the swarm's flight to the site. To delay quorum formation, we placed five desirable nest boxes close together at one location on the island. This caused the scouts visiting the site to be dispersed among five identical nest cavities rather than concentrated at one. We then saw how long it took a swarm, once it had discovered the site of the nest boxes, to make its decision and take off to fly to the site. We also performed with each swarm another control trial  with just one nest box. The two trials for each swarm were performed using two different sites on the island, so each trial began in the same way, with one scout bee discovering an attractive nest cavity in a new site. In all four swarms that we tested, there was indeed a marked delay to takeoff in the five-nest-box treatment (442 minutes on average) relative to the one-nest-box treatment (196 minutes on average). Thus this experiment yielded strong support for the quorum-sensing hypothesis.

Exactly how scout bees sense a quorum remains an enigma. They may use visual, olfactory or even tactile information to assess the number of fellow scouts at a site, but this remains a subject for future study.

Once the quorum threshold is reached at one of the sites, the bees start a behavior that is well understood. The scouts at this site will return to the swarm cluster and begin to produce a special, high-pitched acoustical signal that stimulates the nonscouts in the swarm cluster to begin warming their flight muscles, by shivering, to the 33 to 35 degrees Celsius needed for flight. In producing this signal, which we call worker piping, a scout scrambles through the swarm cluster, pausing every second or so to press her thorax against another bee and activate her wing muscles. Although most of the vibrational energy probably transfers directly into the contacted bee, this action produces an audible vibration that is reminiscent of the revving of a race-car engine. The piping signal lasts about 0.8 seconds and has a fundamental frequency of about 200 hertz. Because the stimulus for worker piping is a quorum of scouts at the chosen site, not a consensus among the scouts for this site, the process of swarm warming generally begins before the scouts have reached a consensus. But because the warm-up usually takes an hour or more, there is usually time for the scouts to achieve a consensus for the chosen site before the entire swarm launches into flight.

By eavesdropping on the decision making of swarms through observation of the dances of their scout bees, Lindauer and our group have shown clearly that a swarm chooses one nest site from an array of five or more alternatives. The next question that naturally arises is whether a swarm chooses the best site, and if so, how? To assess the accuracy of nest-site choice by swarms, we presented swarms on Appledore Island with a five-alternative choice in which four of the alternatives were mediocre nest sites and one was a superb nest site. The four so-so nest boxes were attractive in all ways except that each box provided only 15 liters of living space. The excellent nest box was identical to the other four except that it provided 40 liters of room, a volume that better meets a colony's space needs for its various activities (rearing brood, storing food et cetera).

Nearly all of the test swarms chose the excellent nest box. Specifically, we observed that although the excellent site was never the first one to be found, once a scout bee had discovered the prime site, the number of bees visiting this site rose more rapidly than at the other sites and reached the quorum threshold first. Moreover, as the number of bees increased at the excellent site, it decreased at each of the mediocre sites, indicating that rising interest in the top-quality site depressed interest in the others. This inhibition of buildup at the poorer sites by buildup at the best site is important, because it helps to ensure that the quorum threshold is crossed first at the best site and to generate the pattern of consensus among dancers that almost always appears shortly before a swarm flies to its new home.

What are the behavioral mechanisms at the level of individual scout bees that underlie these dynamics? One is the scout bees' careful tuning of dance strength, in terms of the number of waggle dance circuits they perform for a site, as a function of site quality. We studied this phenomenon by presenting a swarm on Appledore Island with two nest boxes simultaneously, one excellent and one mediocre, and analyzing the waggle dances for the two boxes as they were performed side-by-side on the swarm. We found that the first time a scout returns to the swarm from a first-rate site, she is apt to perform a waggle dance containing 100 or more dance circuits. Scouts also report mediocre but acceptable nest sites, presumably in case nothing better is located. But the first time a scout returns from a so-so site, we found that she is likely to perform a waggle dance containing only a dozen or so dance circuits. The greater the strength of dancing for a particular site, the larger the stream of newcomers to it, hence the buildup of scouts will be most rapid at the best site.

The difference in recruitment-signal strength between scouts from excellent and mediocre sites is amplified by another curious feature of their behavior. If a scout bee commits herself to a site, she will make multiple visits to the site (probably both to show support for "her site" and to stay informed about the buildup there of fellow scouts), and after each visit she will advertise her site with a waggle dance. She will, however, decrease the strength of her dance advertisement by about 15 dance circuits each time she returns to the swarm and performs a dance. The result is that the overall difference in the recruitment-signal strength between two sites is a nearly exponential function of the difference in quality between the sites. If two bees advertising excellent and mediocre sites perform 90 and 30 dance circuits, respectively, on their first return to the swarm, then the total difference in their recruitment signal will not be merely threefold, but sevenfold (90 + 75 + 60 + 45 + 30 + 15 + 0 = 315 circuits total versus 30 + 15 + 0 = 45 circuits total). Moreover, there is strong positive feedback in this recruitment process, such that the greater the number of bees committed to a site, the greater the number of recruiters, which in turn gives rise to a still greater number of bees committed to the site. Consequently, small differences in nest-site quality and waggle-dance strength between two sites can snowball into large differences in the number of scouts affiliated with these sites.

The differences in strength of waggle dancing and the positive feedback inherent to this recruitment process explain the variance in the number of scouts committed to candidate sites, with the best site gaining scout bees the fastest. But what causes the collapse in the number of supporters at inferior sites as it balloons at a superior one? The fundamental basis for the drop in the number of scout bees affiliated with inferior sites is the reality that all scouts, even ones that are committed to excellent sites, will eventually abandon their sites. Usually, a bee ceases making visits to a site shortly after she has ceased performing dances for the site, hence bees abandon poor sites more rapidly than they do excellent ones.

Once a scout abandons a site, she "resets" and can be recruited to another site, or even re-recruited to the same site. However, when a bee finishes dancing for a site, about 80 percent of the time she will cease dancing entirely. Scout bees therefore depend on the recruitment of other scouts who were unable to find any candidate sites on their searches and so remain uncommitted to any site. But when a bee is recruited to visit a site, if she feels it is poor, she may not immediately commit to the site by dancing for it upon her return. An uncommitted scout may therefore visit several sites before finding one she feels is worthwhile.

As long as the rate of recruitment to a site exceeds the rate of abandonment, the number of scouts affiliated with this site will grow. Eventually, however, the rate of recruitment for the highest quality site will snowball, at which time the rate of recruitment for each inferior site will melt away: the pool of uncommitted scout bees is finite, and most are being recruited to the best site. When the recruitment rate falls below the abandonment rate at each inferior site, the number of scouts committed to these sites starts to shrink. In short, as the group committed to the best site grows large, it automatically excludes from the competition the groups affiliated with the inferior sites.

Mary R. Myerscough, a mathematical biologist at the University of Sydney, has created mathematical models of the population dynamics of scout bees performing dances for different nest sites. She has elegantly demonstrated that, given enough time, the dancing scouts in a swarm will almost always become focused on the one best site that has been found. This certainly matches what Lindauer and we witnessed in the scout bees' debates: Almost always, a consensus among the dancers arises before the swarm takes off to fly to its home.

Although unanimity among the dancers shortly before takeoff is a conspicuous feature of the dance records of swarms, we now understand that reaching a quorum, not building a consensus, is the essence of the bee's group decision-making process. Nevertheless, we should not view the dancer consensus as an unimportant, incidental by-product of the bee's decision-making process. On the contrary, consensus is necessary for a swarm to perform a successful flight to its new home. Occasionally we have seen a swarm take off with the scouts dancing strongly for multiple home sites, and each time the airborne swarm has been unable to fly away. The mechanisms of swarm flight guidance remain poorly understood, but it is clear from such observations that the steering process depends on a sufficient number of scouts providing coherent directional information to the rest of the flying swarm bees. When a split decision happens, the swarm seems to need to resettle and continue deliberations until one site predominates.

A fundamental problem faced by any decision maker is finding a suitable compromise between swift decisions and good decisions. If an animal, or a group, must make a quick decision, it is susceptible to making a poor one because it either cannot sample its options sufficiently broadly, cannot evaluate them sufficiently deeply or both. Assuming that a honey bee swarm experiences such a trade-off between speed and accuracy in choosing a nest site, we wondered whether the behavioral parameters of the bees' process of group decision making have been tuned by natural selection so that a swarm incurs low time and energy costs while minimizing its chances of choosing a poor site. To see whether this is the case, we built a stochastic, discrete-time mathematical model of the decision-making process of swarms and then used our model to create "pseudomutant" swarms, ones with different values for various behavioral parameters. This enabled us to see how increases or decreases in particular parameters affect the speed and accuracy of a swarm's choice of a home.

An obvious candidate parameter for alteration was quorum size, since quorum sensing lies at the heart of a swarm's decision making. When we varied the quorum size in the model, while holding everything else at normal levels, the model made it clear that a low quorum yields relatively rapid but often inaccurate decisions and that a high quorum produces slower but more accurate decisions. It was especially noteworthy that the model's prediction of the quorum size that achieves a good balance between speed and accuracy, some 15 to 20 bees, essentially matches the empirical finding that scout bees initiate the process of swarm warming, in preparation for takeoff, when the number of bees at one of the sites has reached 10 to 20.

We also examined one of the curious features of scout bee behavior that presumably contributes to a swarm's decision making, namely the way that a scout reduces the strength of her dancing for a prospective nest site over repeated visits to the site. It is striking that each time a scout visits a potential nest site and then returns to the swarm cluster to advertise the site, she produces a dance with fewer waggle dance circuits than before and so advocates for her site less and less strongly. Varying the rate of dance-circuit reduction in our model revealed just how critical this factor is to the decision-making process. If the number of  dance circuits is reduced at a rate faster than what is observed in nature, then the time needed to reach a decision steadily increases, because a rapid decay in the number of circuits makes it difficult for a swarm to reach a quorum at any one site. Conversely, if the number of circuits is reduced at a rate slower than what is observed in nature, then an even greater problem arises: Swarm decision making fails altogether as split decisions (that is, quorums reached quickly at multiple sites) become common. Again, it is noteworthy that our model's prediction of the rate of dance-circuit reduction that provides a good balance between speed and accuracy—15 to 20 dance circuits per nest-site visit—essentially matches the empirical finding that, on average, scout bees shorten their dances by 15 dance circuits per visit to a nest site.

Given these findings about quorum size, rate of dance-circuit reduction and other parameters, we conclude that the behavior of the scouts in honey bee swarms has indeed been tuned by natural selection to create a group decision-making process with a favorable balance between the competing demands of speed and accuracy.

Henry David Thoreau lamented in one of his journal entries from 1838 about the difficulty that human groups have in achieving a collective intelligence: "The mass never comes up to the standard of its best member, but on the contrary degrades itself to a level with the lowest." Likewise, Friedrich Nietzsche wrote in Beyond Good and Evil: "Madness is the exception in individuals but the rule in groups." Although it is true that groups can make bad decisions, it is also the case that groups can make good decisions. What are the circumstances under which groups will be highly intelligent and able to act collectively to make good choices? We suggest that bees' nest-site selection behavior can provide guidance on this topic, for it is clear they are successful at making collective judgments.

The first relevant factor is that the scout bees are organized in a way that promotes diversity of knowledge within the group. Specifically, they are not led or dominated by a small number of bees; instead, the decision-making process is broadly diffused among all the scout bees in a swarm. Consequently, a swarm's decision-making process is based on the actions of hundreds of individuals, each one an autonomous agent capable of providing unique information for solving the house-hunting problem. As an example, note how the bees accomplish the first stage of their decision-making task—uncovering the possible alternatives from which to choose. Searching independently, widely and simultaneously, the hundreds of scout bees from a swarm bring back to the group diverse information—knowledge of superb, mediocre and even lousy sites—which can be shared with the other scouts by means of waggle dances. All discoveries of potential nest sites are freely reported; no scout is stifled. Thus a swarm takes full advantage of its inherently collective nature to assemble rather quickly—often in just a few hours—a profusion of alternatives from which to choose. The larger this set, the more likely it includes a first-rate site. Thus, we see that one key feature of a swarm's decision making is its decentralized organization, which helps ensure that it has a broad set of options.

A second feature of the bees' behavior that promotes their collective intelligence is that the scouts show no tendency toward conformity or slavish imitation of others as they contribute to the decision-making process. We have explained that the heart of this process is a competition among the various coalitions of scouts affiliated with different sites, each one vying to attract uncommitted scouts to her site. The members of each coalition recruit additional members by performing waggle dances that vary in strength in relation to site quality, so that the higher the site quality, the stronger the waggle dance and the greater the stream of newcomers. What is critically important here is that when an uncommitted scout is recruited to a site, she does not blindly support the bee whose dance she followed. Instead, she examines the advertised site herself, and only if she too judges it to be a worthy site does she perform a dance for it and thereby recruit still more bees to the site. Through this independence of opinions, the scouts avoid propagating errors in the assessments of sites. Only at a truly good site will dancers attract more dancers, hence will there be a strong addition to the number of scout bees at the site. The net result is that scout bees avoid mass manias over poor options.

The third key to the swarm's success is how the quorum-sensing process aggregates the diverse and independent opinions of the scouts in a way that balances the competing needs of decision-making accuracy and speed. The quorum level is high enough that many bees must independently assess a site's quality before it is chosen. Quick selection of a home based on only one or a few bees' favorable assessments is not possible. The quorum-sensing process filters out extreme or inaccurate opinions and provides a balanced, group-level assessment of the chosen site. This assessment process takes time but ensures that there is enough of an interval for true diversity of opinion to arise and for discovered sites to be independently evaluated before one of them is chosen. Thus, the quorum-sensing method of aggregating the bees' information allows diversity and independence of opinion to thrive, but only long enough to ensure that a decision error is improbable.

These considerations illustrate how the study of group decision making by honey bees might help human groups achieve collective intelligence and thus avoid collective folly. Good group decisions, the bees show us, can be fostered by endowing a group with three key habits: structuring each deliberation as an open competition of ideas, promoting diversity of knowledge and independence of opinions among a group's members and aggregating the opinions in a way that meets time constraints yet wisely exploits the breadth of knowledge within the group.

• Beekman, M., R. L. Fathke and T. D. Seeley. 2006. How does an informed minority of scouts guide a honey bee swarm as it flies to its new home? Animal Behaviour 71:161-171.
• Black, D. 1986. The Theory of Committees and Elections. Dordrecht: Kluwer.
• Camazine, S., P. K. Visscher, J. Finley and R. S. Vetter. 1999. House-hunting by honey bee swarms: collective decisions and individual behaviors. Insectes Sociaux 46:348-360.
• • Conradt, L., and T. J. Roper. 2005. Consensus decision making in animals. Trends in Ecology and Evolution 20:449-456.
  • Franks, N. R., S. C. Pratt, E. B. Mallon, N. F. Britton and D. J. T. Sumpter. 2002. Information flow, opinion polling and collective intelligence in house-hunting social insects. Philosophical Transactions of the Royal Society of London B 337:1567-1583.
  • Lindauer, M. 1955. Schwarmbienen auf Wohnungssuche. Zeitschrift für vergleichende Physiologie 37:263-324.
  • Myerscough, M. R. 2003. Dancing for a decision: A matrix model for nest-site choice by honey bees. Proceedings of the Royal Society of London B 270:577-582.
  • Passino, K. M., and T. D. Seeley. 2006. Modeling and analysis of nest-site selection by honey bee swarms: The speed and accuracy trade-off. Behavioral Ecology and Sociobiology 59:427-442.
  • Seeley, T. D. 2003. Consensus building during nest-site selection in honey bee swarms: The expiration of dissent. Behavioral Ecology and Sociobiology 53:417-424.
  • Seeley, T. D., and S. C. Buhrman. 1999. Group decision making in swarms of honey bees. Behavioral Ecology and Sociobiology 45:19-31.
  • Seeley, T. D., and S. C. Buhrman. 2001. Nest-site selection in honey bees: How well do swarms implement the "best-of-N" decision rule? Behavioral Ecology and Sociobiology 49:416-427.
  • Seeley, T. D., and J. Tautz. 2001. Worker piping in honey bee swarms and its role in preparing for liftoff. Journal of Comparative Physiology A 187:667-676.
  • Seeley, T. D., and P. K. Visscher. 2003. Choosing a home: How the scouts in a honey bee swarm perceive the completion of their group decision making. Behavioral Ecology and Sociobiology 54:511-520.
  • Seeley, T. D., and P. K. Visscher. 2004. Quorum sensing during nest-site selection by honey bee swarms. Behavioral Ecology and Sociobiology 56:594-601.
  • Surowiecki, J. 2004. The Wisdom of Crowds. New York: Doubleday.

from Hacker News https://ift.tt/E8S0qtv