Why crowdsourcing never became the next big thing, and what tech leaders can learn from it

Image: iStock/gmast3r

No problem is too big for a crowd of intelligent people, whether it’s designing a new automobile, identifying a security flaw, or mapping the human genome. Crowdsourcing, the process of sourcing labor from a pool of Internet users, was supposed to provide an unlimited pool of specialist resources, at low or zero cost, all without the pesky problems of hiring full-time employees. Somewhere along the way to labor arbitrage utopia, however, crowdsourcing failed to catch on except in narrow applications. What went wrong?

Labor arbitrage is hard

Labor arbitrage, the act of moving labor to lower-cost locations, was a cornerstone of IT cost reductions through much of the last decade. It started with the move to outsourcing and increased use of consultants and “staff augmentation” providers, and crowdsourcing was billed as the next logical extension. Technology enabled work to be quickly sent outside the four walls of most companies, and theoretically crowdsourcing would provide access to a near-infinite pool of talent.

One of the major barriers to labor arbitrage is that it’s far more difficult in practice than in theory. Ultimately you’re still dealing with human beings who, like your own employees, get sick, have bad days, and work and think at varying levels of quality and speed. On top of these challenges, add one or more additional layers of organizations between you and the actual worker, conflicting and questionable loyalties, and a complex administrative system to track everything from hours billed, to logins, and temporary access rights that these workers require.

The main selling point of crowdsourcing also revealed its largest weakness. Crowdsourcing was supposed to represent “micro outsourcing” where single projects could be sent to multiple parties for a quick solution. However, the administrative and support structures of outsourcing don’t lend themselves to short-duration projects with an ever-changing pool of resources. If I want to crowdsource design of a vehicle component, how do I share my changing specifications with a nameless, faceless crowd? How do I allow them access to my design standards and specifications? How do I allow them access to my intellectual property without releasing it to the wider world, and finally the ultimate bugaboo of labor arbitrage: what do I do when I have hundreds of responses at wildly varying levels of quality? In the worst case, more time is spent doing the administration and quality control than doing the work internally, for an inferior result.

Where the crowd does work

One of the shining successes of crowdsourcing has been in the IT security arena. Companies have offered bounties for security specialists to identify flaws in their applications, as long as the exploit is confidentially reported to the company first for resolution. This is one of the few examples where crowdsourcing has lived up to its promise: a higher quality result is delivered for less cost than doing the work internally.

SEE: How to develop a bug bounty program

What’s interesting about this example is that several unique factors are at play that allow crowdsourcing to be successful:

1. No unique company IP is required. In fact, knowing internal details about the company would invalidate security testing to some extent, so less information is actually more beneficial in this example.

2. Anyone can participate, with zero incremental cost to the company. Participating in the crowd requires no setup or even acknowledgement from the provider.

3. Payment occurs after the work is accomplished, and there are near-zero administrative costs to launch the program beyond publishing its existence. Facebook doesn’t need to know or care whether five or 500,000 people are attempting to find an exploit; they just need to write a single check to a single person once they come forward with a valid exploit.

4. There’s a strong non-cash incentive for the worker. If someone identifies a major exploit, they get their name widely published and get significant credibility for finding a weakness in an otherwise “secure” system. Cash is only part of the reward, and the credibility component essentially costs the company nothing.

5. It’s cheaper to crowdsource. Even a six-figure bounty is significantly cheaper than hiring a cadre of security experts without a guaranteed outcome.

If you can identify a unique project or process that meets some or all of the above criteria, crowdsourcing may be appropriate. However, these situations are few and far between outside the relatively unique situation of something like finding security exploits. When you do consider crowdsourcing, take care to determine what the administrative requirements of the program will be. If you need to provide information or identify each member of the “crowd,” you’ll likely drown in the administrative processes. If the reward you’re offering the crowd is minimal and has no cachet beyond a small check, you may build the ultimate crowdsourcing program, but not generate enough interest to actually assemble a crowd. Like most of the “next big thing” trends in IT, crowdsourcing has legitimate application and benefits to a small set of scenarios. However, it’s far from the solution to every labor challenge that was once promised.

Also see:
Crowdsourcing analytics: How to find the wisdom in the crowd
Swarm AI predicts the 2016 Kentucky Derby
Is tech turning contract work into the future of employment?
Tech CEOs: Automation will increase, but human workforce to grow 6% in next 3 years