Using the Crowd as an Innovation Partner
by Kevin J. Boudreau and Karim R. Lakhani
Courtesy HBR
Photography: Michele Sereni
Artwork: Jacob Hashimoto, The Other Sun, 2012, acrylic, paper, thread, bamboo, Ronchini Gallery, London
To answer the most vexing innovation and research questions, crowds are becoming the partner of choice. Apple has turned to large numbers of users and developers distributed around the world to propel its growth by creating apps and podcasts that enhance its products. Biologists at the University of Washington used crowds of external contributors to map the structure of an AIDS-related virus that had stumped academic and industry experts for more than 15 years. Despite a growing list of success stories, only a few companies use crowds effectively—or much at all.
Managers remain understandably cautious. Pushing problems out to a vast group of strangers seems risky and even unnatural, particularly to organizations built on internal innovation. How, for example, can a company protect its intellectual property? Isn’t integrating a crowdsourced solution into corporate operations an administrative nightmare? What about the costs? And how can you be sure you’ll get an appropriate solution?
These concerns are all reasonable, but excluding crowdsourcing from the corporate innovation tool kit means losing an opportunity. The main reason companies resist crowds is that managers don’t clearly understand what kinds of problems a crowd really can handle better and how to manage the process. Over the past decade we’ve studied dozens of company interactions with crowds on innovation projects, in areas as diverse as genomics, engineering, operations research, predictive analytics, enterprise software development, video games, mobile apps, and marketing. On the basis of that work, the supporting body of economic theory, and rigorous empirical testing, we’ve identified when crowds tend to outperform the internal organization and, equally important, when they don’t. In this article we offer guidance on choosing the best form of crowdsourcing for a given situation. We also review how technology is helping managers address these concerns. Crowds are moving into the mainstream; even if you don’t take advantage of them, your competitors surely will.
Beyond “Make or Buy”
Let’s start by noting the fundamental differences between crowd-powered problem solving and traditional organizational models. Companies are relatively well-coordinated environments for amassing and marshaling specialized knowledge to address problems and innovation opportunities. In contrast, a well-functioning crowd is loose and decentralized. It exposes a problem to widely diverse individuals with varied skills, experience, and perspectives. And it can operate at a scale that exceeds even that of the biggest and most complex global corporation, bringing in many more individuals to focus on a given challenge.
In certain situations, that means we can solve problems more efficiently. For example, we worked with the Harvard Clinical and Translational Science Center (known as Harvard Catalyst) to design a contest to solve a tough computational biology problem that had immediate research and commercial implications. To provide a platform for the contest, we enlisted TopCoder, a company that administers computer programming competitions. The two-week contest attracted viable solutions from 122 solvers—a staggering number. Many of the solutions surpassed the quality of those developed over the years by the school’s own scientists and by experts at the National Institutes of Health.
In addition to benefits of scale and diversity, crowds offer incentives that companies find difficult to match. Companies operate on traditional incentives—namely, salary and bonuses—and employees are assigned clearly delineated roles and specific responsibilities, which discourages them from seeking challenges outside their purview. But crowds, research shows, are energized by intrinsic motivations—such as the desire to learn—that are more likely to come into play when people decide for themselves what problems to attack. (Can you imagine any company paying a salary to an employee who’s just floating around looking for a problem to solve?) The opportunity to burnish one’s reputation among a large community of peers is another strong motivator (as is money, to be sure). Also, crowds are often more cost-effective per output or per worker than traditional company solutions.
So although internal, crowdlike approaches to creativity and idea generation, such as “jams,” “idea marketplaces,” and “personal entrepreneurial projects,” may increase the scope for exploration and flexibility inside companies, they are qualitatively different from and fall short of the full capability of external crowds. At the same time, it should be said that the benefits of the crowd do nothing on their own to offset the management worries mentioned above. We will describe the safeguards and other mechanisms that address those worries.
Crowdsourcing as a way to deal with innovation problems has existed in one form or another for centuries. Communities of innovators have helped kick-start entire industries, including aviation and personal computing. The difference today lies in technology. Over the past decade tools for development, design, and collaboration have been radically transformed; they’re getting more powerful and easier to use all the time, even as their prices plummet. At least as important, online crowdsourcing platforms have become much more sophisticated, making it ever simpler to manage, support, and mediate among distributed workers. Companies can reinvigorate (with incentive systems, for example) and redeploy crowds across a continual stream of problems. In essence, the crowd has become a fixed institution available on demand.
Having determined that you face a challenge your company cannot or should not solve on its own, you must figure out how to actually work with the crowd. At first glance, the landscape of possibilities may seem bewildering. But at a high level, crowdsourcing generally takes one of four distinct forms—contest, collaborative community, complementor, or labor market—each best suited to a specific kind of challenge. Let’s examine each one.
Crowd Contests
The most straightforward way to engage a crowd is to create a contest. The sponsor (the company) identifies a specific problem, offers a cash prize, and broadcasts an invitation to submit solutions. Contests have cracked some of the toughest scientific and technological challenges in history, including the search for a way to determine longitude at sea. The Longitude Prize was established by an act of Britain’s Parliament in 1714 after a host of brilliant scientists, including Giovanni Domenico Cassini, Christiaan Huygens, Edmond Halley, and Isaac Newton, had tried and failed to come up with an answer. The winning solution, one of more than 100 submissions, was a highly accurate chronometer that enabled the exact triangulation of location. It came from John Harrison, a carpenter and clockmaker from the English countryside, who was eventually awarded about £15,000.
Contests work well when it’s not obvious what combination of skills or even which technical approach will lead to the best solution for a problem. Running a contest is akin to running a series of independent experiments in which, ideally, we can see some variation in outcomes. Therefore, of the four forms of crowdsourcing, contests are most useful for problems that would benefit from experimentation and multiple solutions. Today online platforms such as TopCoder, Kaggle, and InnoCentive provide crowd-contest services. They source and retain members, enable payment, and protect, clear, and transfer intellectual property worldwide.
Although a company might in the end use only one of the solutions it receives, the assessment of many submissions can provide insight into where the “technical frontier” lies, especially if the solutions cluster at some extreme. (In contrast, internal R&D may generate far less information—and a lingering question about whether an even better solution might still be found.)
We have learned that contests are most effective when the problem is complex or novel or has no established best-practice approaches. This is especially true when you don’t know in advance what a good solution will look like. Just last fall the pharmaceutical company Merck worked with Kaggle, a predictive analytics crowdsourcing site, to streamline its drug discovery process. The standard practice for identifying chemicals that might be effective in targeting particular diseases involves testing hundreds of thousands of compounds, and there is no cost-effective way to test all of them against all potential disease mechanisms. So Merck set up an eight-week, $40,000 contest in which it released data on chemical compounds it had previously tested and challenged participants to identify which held the most promise for future testing. The contest attracted 238 teams that submitted well over 2,500 proposals. The winning solution came from computer scientists (not professionals in the life sciences) employing machine-learning approaches previously unknown to Merck. The results were spectacular enough to merit a front-page story in the New York Times, and the company is now implementing the solutions.
Contests are also useful for solving design problems, in which creativity and subjectivity influence the evaluation of solutions. Tongal, a crowd-powered advertising agency, routinely solicits submissions for campaigns for consumer products firms. In the summer of 2012 Colgate-Palmolive worked with the Tongal community on a two-month, $17,000 challenge to develop ads for Speed Stick’s “Handle It” campaign, and selected one of the resulting submissions for its $4 million Super Bowl buy. In the Kellogg School of Management’s ninth annual Super Bowl advertising review, the Tongal ad ranked 12 out of 36—outperforming ads by Calvin Klein, Volkswagen, Coke, Toyota, and Pepsi.
Tongal is just one of a number of contest platforms available to companies facing design challenges. HYVE has worked extensively with firms as diverse as Intel and Procter & Gamble to deploy crowds in the thousands to invent uses for both new and existing products and technologies. Other firms in this space include Quirky (new product and service concepts) and crowdSPRING, DesignCrowd, and 99designs (logos and graphic design).
There are, of course, management challenges in running a crowdsourcing contest. First is identifying a problem important enough to warrant dedicated experimentation. The problem must then be “extracted” from the organization—translated or generalized in order to be immediately understandable to large numbers of outside solvers. It must also be “abstracted” to avoid revealing company-specific details. That may involve breaking it down into multiple subproblems and contests. And finally, the contest must be structured to yield solutions the organization can feasibly implement.
A contest should be promoted in such a way—with prizes and opportunities to increase stature among one’s peers—that it appeals to sufficiently skilled participants and receives adequate attention from the crowd. The sponsor must devise and commit to a scoring system at the outset. In addition, explicit contractual terms and technical specifications (involving platform design) must be created to ensure the proper treatment of intellectual property.
Crowd Collaborative Communities
In June of 1998 IBM shocked the global software industry by announcing that it intended to abandon its internal development efforts on web server infrastructure and instead join forces with Apache, a nascent online community of webmasters and technologists. The Apache community was aggregating diverse inputs from its global membership to rapidly deliver a full-featured—and free—product that far outperformed any commercial offering. Two years later IBM announced a three-year, $1 billion initiative to support the Linux open-source operating system and put more than 700 engineers to work with hundreds of open-source communities to jointly create a range of software products.
In teaming up with a collaborative community, IBM recognized a twofold advantage: The Apache community was made up of customers who knew the software’s deficits and who had the skills to fix them. With so many collaborators at work, each individual was free to attack his or her particular problem with the software and not worry about the rest of the components. As individuals solved their problems, their solutions were integrated into the steadily improving software. IBM reasoned that the crowd was beating it at the software game, so it would do better to join forces and reap profits through complementary assets such as hardware and services.
Like contests, collaborative communities have a long and rich history. They were critical to the development of Bessemer steel, blast furnaces, Cornish pumping engines, and large-scale silk production. But whereas contests separate contributions and maximize diverse experiments, communities are organized to marshal the outputs of multiple contributors and aggregate them into a coherent and value-creating whole—much as traditional companies do. And like companies, communities must first assess what should be included in the final aggregation and then accomplish that through a combination of technology and process.
The strength of the community is its diversity, but it lacks cohesiveness. Companies create cohesion with structures and systems (such as incentives) that align values. They hire employees for “fit” and colocate them so that they can interact directly, become socialized, and share a culture. Moreover, employees gain specific experience and knowledge in the narrow fields where the company focuses. A crowd, in contrast, may draw participants from around the globe—from varying companies, domains, and industries—who have their own interests and motivations. That makes crowds harder to control.
Consider Wikipedia. In less than a decade the internet-based encyclopedia has disrupted the reference world and demonstrated the value of large-scale, highly diverse collaboration within a new organizational model. Wikipedia uses an automated process to coordinate and aggregate the crowd’s edits and keep track of all changes. The size of the Wikipedia crowd, with multiple people typically examining any given article, ensures a thorough monitoring of content quality.
Wikipedia shows that collaborative communities are most effective when they tackle projects whose orchestration is relatively simple. Crowd collaboration relies on extensive task modularization, standardized routines, and technology to facilitate coordination. Norms, knowledge sharing, teams, and leadership emerge to deal with what little decision making and coordination are required, but these structures are much looser than the ones found in most companies.
Organizations can assemble their own communities, but doing so may be difficult and time-consuming, especially when resources must be dedicated to curating the platforms. Most corporate crowd initiatives involve only modest amounts of coordination—for example, FAQ pages to which customers can contribute. Some companies, particularly technology and electronics firms, take this a step further by building systems that allow customers to support one another as well as seek answers from the company itself. Verizon, the U.S. telecommunications company, relies on its community of users to help address one another’s technical questions. Facebook relied on its vast user community to translate its website and services into multiple languages. Lego, the Danish toy company, now works with its community of fans to come up with new designs and products. And IDEO, the design and innovation firm, has developed the OpenIDEO platform to create a global community of design professionals interested in solving tough social problems in areas as diverse as human rights, urbanization, maternal health, and water sanitation.
But collaborative communities work best when participants can accumulate and recombine ideas, sharing information freely. So protecting intellectual property is next to impossible. Companies should maintain a strict division between proprietary assets and community assets and attempt to derive profits from complementary businesses. Google makes its Android operating system for mobile devices free and open; its profits come from the monetization of mobile search, the algorithms for which are proprietary.
Crowd Complementors
The third type of crowd-powered innovation enables a market for goods or services to be built on your core product or technology, effectively transforming that product into a platform that generates complementary innovations. Consider iTunes, organized around Apple’s core mobile products—the iPod, the iPhone, and the iPad. Through iTunes, vast pools of geographically distributed developers create a staggering array of complementary innovations such as software apps and user-generated podcasts.
Unlike contests or communities, complementors provide solutions to many different problems rather than just one. The opportunity lies in the sheer volume of solutions. Platforms like iTunes allow the core business to collect licensing or transaction revenues from complementors, who sell their products to customers of the core product (such as iPhone owners). The variety of complementary goods does more than generate revenue. It can expand demand for the product itself, by making it more useful. Increased demand, in turn, can prompt an increase in the supply of complementary innovations, and pretty soon you have a nice set of network effects.
To be sure, crowds aren’t always the best way to create complementary products. They make sense only when a great number and variety of complements is important. Otherwise, a few partners or even an internal organization will better serve the goal. Clearly, we don’t need thousands upon thousands of tennis ball developers.
When deployed in the right context, crowd complementors can yield powerful competitive advantages. They account for Apple’s ability to muscle in on both the audio market, edging out such well-regarded manufacturers as Bang & Olufsen and Bose, and the smartphone market, edging out Research In Motion (BlackBerry), Nokia, and Sony. Ford Motor Company is following suit, with plans to convert its vehicle electronics, entertainment, and hardware systems into an open platform that will enable outside developers to innovate. Ford’s rationale is that their innovations will create more demand and value for its products and allow it to compete with Google and Facebook, which are themselves competing in automotive-related services such as mapping, traffic notification, geopositioning, and social information.
The first challenge to using the crowd as a complementor is providing access to the functions and information in the core product. That is accomplished through technological interfaces or hooks that enable external developers to create complementary innovations in a frictionless way. This may be relatively easy when the core product is simple, such as a data feed from a website. More challenging are cases in which complementors have to dial into core product functions and build on them. Third-party developers, for instance, must use application programming interfaces (APIs) to access a software vendor’s capabilities in order to develop complementary applications. For example, Canada’s Yellow Pages Group worked with Mashery, a third-party provider, to create an API that makes its geolocalized business listings and associated content available to app developers.
There are also advantages to assembling complementor crowds that are specific to a company’s own platform. Think of the enormous ecosystems around Microsoft, Facebook, and Apple, each of which operates on a model that stimulates adoption on both the complementor and customer sides to kick-start positive interactions and initiate growth. (How to get this started is a classic chicken-and-egg problem that has received much research attention in the past 20 years and goes beyond the scope of this article.) The strategies of those companies require considerable industry experience and support and depend on the particulars of the situation. They involve the design of the core product, setting prices for different sides of the platform, setting expectations, and creating a wider set of inducements, among other issues.
If you are exposing your technology and assets to outsiders, you must make sure they’re protected. Unlike contests, which can carefully control the exposure of assets to elicit a single, narrow solution, complementor platforms must give outsiders more-flexible access to develop a wide range of solutions. Some form of developer’s agreement or contract is now accepted industry practice.
Crowd Labor Markets
Whereas contests offer crowds rewards for coming up with solutions to specific problems, labor markets match buyers and sellers of services and employ conventional contracting for services rendered. These are not platforms that a company would want to build itself but, rather, third-party intermediaries such as Elance, oDesk, Guru, Clickworker, ShortTask, Samasource, Freelancer, and CloudCrowd. Rather than matching workers to jobs within companies for long-term employment (as more traditional labor market intermediaries do), these highly flexible platforms serve as spot markets, matching skills to tasks. They often perform on-demand matching to give immediate support at an unprecedented scale. For example, the California-based oDesk boasts 2.5 million workers and more than 495,000 registered clients; it monitors its own performance not in terms of placements but in terms of hours worked.
Critical to the success of these flexible spot markets is the growing sophistication of their technology infrastructure and platform design, which allow transactions to be effectively governed. The supporting platforms provide reputation and skills evaluations, bidding systems, procedures for recourse, monitoring technologies, and escrow services that keep payments in a third-party account to minimize conflict between buyers and sellers. This arrangement means that labor contracts can function outside the context of long-term employment relationships, radically reducing start-up and transaction costs. In this sense, the model, along with collaborative communities, comes closest to overlapping with (if not necessarily substituting for) a traditional company organization.
Spot labor markets work when you know what kind of solution you are looking for and what an appropriate solver looks like. Because spot labor markets must identify qualified workers before the fact and collect meaningful performance data, they organize projects and participants in familiar categories such as web development, software development, design and multimedia, sales and marketing, and customer service. Such standardization makes it easier to evaluate workers’ skills and productivity, make good matches, and set expectations on all sides. The platforms themselves go even further to help ensure high-quality matches by measuring the skills and capabilities of workers and needs of employers, collecting abundant data on performance and feedback, and then allowing these data to be used in future matches. The data can be translated into algorithm-based analytic support of pinpoint matching for future jobs, going beyond the usual search functions of traditional employment sites like Monster.com. (Try doing that with employees buried inside organizations, who are largely unobserved and unmeasured.)
Labor markets’ low transaction costs allow for extremely “bite-size” outsourcing. For example, services may cost literally pennies per task—as on the crowdsourcing marketplace Amazon Mechanical Turk. Particularly suited to labor markets are repetitive tasks that require human intelligence but for which it would be difficult and expensive to hire full-time employees—for example, simple data entry, annotating photographs, or cleaning data sets. It is now even economical to perform large-scale “human computation”—essentially outsourcing tasks that a person can perform better than a computer can, such as identifying people in photographs. Instead of relying on machine-based algorithms, Twitter uses the spot labor market on Amazon Mechanical Turk to respond to user search queries for trending topics. National Geographic recently deployed a crowd of about 28,000 to sift through satellite imagery of Mongolia in search of Genghis Khan’s tomb.
Spot labor markets should be less a radical departure from than an extension of current hiring and outsourcing practices. Like outsourcing, they give companies flexibility and access to a greater variety and depth of skills. They do a better job of matching talent to tasks than was ever before possible.
The management challenges in exploiting spot labor markets are minor compared with those in other forms of crowdsourcing. The biggest concern may be identifying which tasks to farm out and who within your organization should manage them. A few personal assistants within a company, for example, might match the productivity of many more if they were invited to manage a crowd pool of support.
Ideally, our mode of organization will always produce high motivation, achieve seamless coordination, and harness world-class knowledge. But any approach to organizing comes with challenges. This is true of both companies and crowds; each has strengths and weaknesses and comes with trade-offs. In the end, though, crowds expand the capabilities of companies; they should be viewed as another tool for organizational problem solving.
The technology that has turbocharged crowdsourcing’s potency and application is still relatively new, so it is too early to fully understand the depth and reach of crowds across the economy. Nonetheless, recent experience and mounting research suggest that we may just now be seeing the outlines of a genuine expansion in capabilities—one with important implications for solving the most enigmatic problems, which might go unsolved if kept inside companies, and for taking care of a number of more-categorizable tasks. The management challenges, while real, are hardly crippling. But they demand that we put as much energy and intelligence into designing systems for organizing work outside company walls as we do for work within them.
