reCAPCHA: Crowdsourcing Intelligence

reCAPCHA has digitized books and mapped the world with crowdsourcing… Now it uses humans’ comparative advantage to crowdsource the creation of a truly intelligent computer.

CAPCHA (or Completely Automated Turing test to tell Computers and Humans Apart) is a test used in computing to distinguish between computers and humans by presenting the user with a problem that is much more difficult for computers to solve than for humans. Examples of these types of problems include pattern recognition, object segmentation, and contextual understanding.

This technology was originally deployed on websites to prevent spammers from using automated “bots” to create fraudulent accounts on websites.  Initial efforts used background images and colors, irregular spacing, and image distortion to confuse computers.

Realizing that humans were constantly solving difficult problems for computers, a group of researchers created reCAPCHA to put the crowd to work.

Digitizing Books

Initiatives to digitize books generally began with Optical Character Recognition (OCR). However books have different text fonts, sizes, spacing, and irregularities, making perfect recognition of all words difficult.  By taking words that could not be deciphered by computer and giving them to humans to solve, reCAPCHA greatly accelerated the digitization of books, all while reducing fraud on websites.

Value creation was clear for websites who could filter out malicious users and for digitization initiatives (reCAPCHA began by digitizing the NYT Archive). Adoption was fast because the reCAPCHA plugin could be provided to websites for free, end users were a captive audience, and the cost of deciphering words was much lower than paying individuals. Value Capture was initially more difficult, relying on contracting with companies to digitize archived content.

Acquisition by Google

In 2009, Google bought reCAPCHA and has since expanded its use beyond books. It now uses reCAPCHA to assist in recognition of street signs for StreetView to improve Google Maps and of images to improve its image search tool.  These use cases directly improve Google’s services allowing it to capture value through more targeted advertising.

More importantly and somewhat ironically, using crowds to outsmart computers is now allowing Google to make computers smarter.  Google can learn about how humans use context and segmentation to understand images.  They are crowdsourcing data on how humans think and can then design algorithms that mimic that process or, if done systematically, automatically integrate that mode of thinking into its architecture using machine learning.  In doing so, Google is crowdsourcing the creation of an intelligent computer.

Governance and Challenges

The system has however faced significant challenges.  First, the words need to be sufficiently difficult for bots to recognize but easy enough for humans to recognize – striking this balance as spammers continually attempt new approaches to OCR has proven difficult and reCAPCHA has periodically shown vulnerability. Second, reCAPCHA can be a roadblock for those with visual disabilities. And finally, some end users want to be compensated for the service they are providing to reCAPCHA.  To address these issues reCAPCHA has improved each stage of the process: using better algorithms to make sure the words are sufficiently difficult; creating other traps for bots on websites; creating audible and math driven forms to assist those who cannot do the visual task; and making the user experience simpler to lower to amount of time spent on the task.

The real beauty of reCAPCHA’s use of crowds is that the crowd has no choice but to participate.  Websites are the customers and as long as the tool is free and effective and blocking fraud, they will continue to use the service.  No crowd governance required!

Previous:

Reddit: A bastion of free speech

Next:

The Netflix Prize: Crowdsourcing to Improve DVD Recommendations

Student comments on reCAPCHA: Crowdsourcing Intelligence

  1. This is great – I didn’t realize the content being fed through the reCAPCHA’s was actually being used in any way! But it makes a lot of sense and I understand the value proposition way more now.

    On the other side of the table, I remember hearing about some operations a while back that used low cost overseas labor to defeat reCAPCHA’s for bots and other nefarious entities. If the cost is low enough, it can still make sense in order to be able to run bots through these systems. I wonder what the state of the art looks like today?

  2. I think that its been a brilliant idea so far – an extremely simple application which has been leveraged to solve real world challenges!

  3. Great post! I love the fact the they have created so much value for society from a mundane task.

  4. I believe Google’s acquisition of reCAPTHA was truly one of their smartest, as it provided a substantial preexisting user base that they could harness to improve their already best-in-class map and book products. With the newest update to the product (“I am not a robot”), while this clearly benefits users I’m interested to see how Google will use this user data to further enhance their existing suite of products.

Leave a comment