It pains us to think of this, but every day thousands of sexually explicit images of children proliferate in the worst corners of the internet. Under US law, Internet service providers such as Google and Facebook are required to report all cases of child exploitation by providing “cyber tips” to the National Center for Missing & Exploited Children (NCMEC).

The challenge is that NCMEC receives around 60,000 cyber tips each day, and with a team of just 30 analysts, this is far too much data for human eyes to sort through. That’s where software company Palantir steps in.

Before working with Palantir, the typical approach to solving crimes involving child exploitation material was to have analysts pour over thousands of images and videos – many of them duplicates from closed cases – and then send tips to law enforcement. Now Palantir, a software company offering solutions that enable clients to analyze large sets of data, has devised technology to help NCMEC locate children at risk.

The software saves NCMEC’s analysts a tremendous amount of time by directing staff toward reports which may contain brand new content, versus reports related to images and videos that have been previously seen. Palantir’s software can identify connectivity between reports and it allows the dissection of these reports to be rapidly scaled to the millions and billions. It easily completes tasks that a person couldn’t complete in their entire lifetime. Palantir also helped devise new methods to spot insights in the data that could assist law enforcement in recovering children from harm and investigating offenders.

As a result of the partnership between NCMEC and Palantir, the time it takes to rescue a child and get a child predator into custody has significantly decreased. What used to take days and weeks now takes, in some cases, just hours from the time of a report to the time of an arrest.

For those of us at the Optimist Daily, this story is a prime example of how big data can actually help, not hurt, humanity.