Algorithmia

How to Censor Faces with Video Processing Algorithms

Simon Pegg and Nick Frost faces blurred

Simon Pegg and Nick Frost from Wikimedia Commons

Earlier this week we introduced Censorface, an algorithm that finds the faces in images and then either blurs or puts a colored box over the faces to censor them. We thought it would be fun to pair it up with some of our video processing algorithms to show how you can use different algorithms together to censor a video clip when you don’t want to run the whole video.

Maybe you have some embarrassing videos that you want to share, but don’t want anyone to know it’s you! Or maybe you have a potentially viral video that you want to post on YouTube, but you need to protect the innocent. No matter what your use case is, let’s dive into creating non-nude video clips with censored faces!

Step 1: Install the Algorithmia Client

This tutorial is in Python. But, it could be built using any of the supported clients, like Scala, Ruby, Java, Node and others. Here’s the Python client guide for more information on using the Algorithmia API.

Install the Algorithmia client from PyPi:

pip install algorithmia

You’ll also need a free Algorithmia account, which includes 5,000 free credits a month – more than enough to get started.

Sign up here, and then grab your API key.

Step 2: Call Scene Detection

Like most algorithms on the Algorithmia platform, Scene Detection takes input in the form of a JSON object. There are many parameters for Scene Detection so be sure to check out the algorithm description page for more than is used here.

import Algorithmia

client = Algorithmia.client("YOUR_API_KEY")

def scene_detection():
    """Find subclips of videos."""
    input = {
        "video": "https://www.youtube.com/watch?v=CLrkUWWiQ70",
        "threshold": 40,
        "output_collection": "testSceneDetection"
    }
    algo = client.algo('media/SceneDetection/0.1.2').set_options(timeout=1000)
    scene_timestamps = algo.pipe(input).result
    return scene_timestamps["output_collection_videos"]

Our function above called scene_detection has an input object with the video URL as the first key-value pair.  Your video can be a YouTube video (make sure it’s either yours or a Creative Commons video) or a Data API URL pointing to your content in Dropbox, Amazon S3, or Algorithmia’s Hosted Data.

Next there is the threshold field that cuts the scenes according to a pixel density that you can set. This field requires an 8-bit intensity value that each pixel value (R, G, and B) must be less than or equal to in order to trigger a fade in/out. The default is 30, but we’ve set it to 40.

Finally notice the output_collection field where you’ll pass in the name of your data collection. Note that this is different than most algorithms that ask for your entire path.

Check out the algorithm description page for Scene Detection to see all the other parameters available such as minimum scene length and others.

Now that we have our input we can call our algorithm. Make sure you always include the version number (check the algorithm description page for the latest version).

Next we pipe our data into the algorithm and get the result back. Notice that we are only returning the video clip data called output_collection_videos and not the timestamps.

Here is a sample of the output:

['data://.my/testSceneDetection/output_0.mp4', 'data://.my/testSceneDetection/output_1.mp4', 'data://.my/testSceneDetection/output_2.mp4', 'data://.my/testSceneDetection/output_3.mp4']

Step 3: Find Clips Without Nudity

Here comes the meat of our recipe. We’ll want to first call Nudity Detection on all the video clips, then we’ll sift through the results that are in JSON files, looking for clips that don’t contain nudity.

If you haven’t used our Data API to work with files saved in your Dropbox, S3, or Algorithmia Data Collections, check out the docs here.

def nudity_detection():
    """Check for nudity returning json files in data collection."""
    data = scene_detection()
    algo = client.algo("media/VideoMetadataExtraction/0.4.2")
    for video_file in data:
        output_file_name = video_file.split('/')[-1].split('.')[0]
        input = {
            "input_file": video_file,
            "output_file": "data://.algo/temp/{0}.json".format(output_file_name),
            "algorithm": "algo://sfw/nuditydetectioni2v"
        }
        algo.pipe(input).result
    print(data)
    return data


def safe_videos():
    """For video clips check if they are nude and return non-nude clips."""
    image_dir = client.dir("data://.algo/media/VideoMetadataExtraction/temp/")
    clips = nudity_detection()
    if image_dir.exists():
        safe_clips = []
        for file in image_dir.list():
            file_contents = client.file(file.path).getJson()
            nude_list = [f["data"]["nude"]
                         for f in file_contents["frame_data"]]
            # Check to see if there is nudity in clip
            if True not in nude_list:
                clean_file = file.path.split('/')[-1].split('.')[0]
                for cf in clips:
                    if clean_file in cf:
                        # Return only the safe video clip paths
                        safe_clips.append(cf)
        return safe_clips

Ok, that’s a lot of code, but bear with me. The first function nudity_detection simply calls the Nudity Detection algorithm using the Video Metadata Extraction algorithm (check out this blog post for details about it) which outputs JSON files in the data collection of your choice. Since we want to check later if the path for a file contains nudity and match that with the original video clip, we’ve made sure to make the output file names similar to the original clip’s name.

The second function iterates through all those JSON files in the temporary directory of the Video Metadata Extraction algorithm. If the file doesn’t have any nudity, then that file name is checked against the file name of the original sub clip from the Scene Detection algorithm and we create a list of clean video clips.

Here is sample output of our family-friendly subclips:

['data://.my/testSceneDetection/output_0.mp4', 'data://.my/testSceneDetection/output_1.mp4']

Step 4: Create Censored Subclips

Now that we have our list of non-nude subclips from our original YouTube video, we can create censored video clips.

def censor_video():
    """For any non-nude clips censor the faces."""
    algo = client.algo(
        "media/VideoTransform/0.5.5")
    for video_file in safe_videos():
        input = {
            "input_file": video_file,
            "output_file": "data://quality/testCensorface/testCensor.mp4",
            "algorithm": "cv/CensorFace/0.1.3",
            "advanced_input": {
                "images": "$BATCH_INPUT",
                "output_loc": "$BATCH_OUTPUT",
                "fill_color": "blur"
            }
        }
        response = algo.pipe(input).result
        print(response)

censor_video()

Now is the fun part. Using the Video Transform algorithm we can easily create censored clips from our non-nude videos. Notice that we use the “blur” effect for the censoring boxes and that we chose to do batch input to speed the image processing along. Note that the fill_color is an optional parameter for the Censorface algorithm and you place those in advanced_input of the Video Transform input.

Here is one of our non-nude subclips that we created. This video is from a blind dating tv show that had a slightly embarrassing moment:

And that’s it! You have a video pipeline to create censored family-friendly video clips. Be sure to check out these other video processing posts and let us know what you think @Algorithmia.

References:

For ease of use check out the full code here or on GitHub:

import Algorithmia

client = Algorithmia.client("YOUR_API_KEY")

def scene_detection():
    """Find subclips of videos."""
    input = {
        "video": "https://www.youtube.com/watch?v=CLrkUWWiQ70",
        "threshold": 40,
        "output_collection": "testSceneDetection"
    }
    algo = client.algo('media/SceneDetection/0.1.2').set_options(timeout=1000)
    scene_timestamps = algo.pipe(input).result
    print(scene_timestamps["output_collection_videos"])
    return scene_timestamps["output_collection_videos"]


def nudity_detection():
    """Check for nudity returning json files in data collection."""
    data = scene_detection()
    algo = client.algo("media/VideoMetadataExtraction/0.4.2")
    for video_file in data:
        output_file_name = video_file.split('/')[-1].split('.')[0]
        input = {
            "input_file": video_file,
            "output_file": "data://.algo/temp/{0}.json".format(output_file_name),
            "algorithm": "algo://sfw/nuditydetectioni2v"
        }
        algo.pipe(input).result
    print(data)
    return data


def safe_videos():
    """For video clips check if they are nude and return non-nude clips."""
    image_dir = client.dir("data://.algo/media/VideoMetadataExtraction/temp/")
    clips = nudity_detection()
    if image_dir.exists():
        safe_clips = []
        for file in image_dir.list():
            file_contents = client.file(file.path).getJson()
            nude_list = [f["data"]["nude"]
                         for f in file_contents["frame_data"]]
            # Check to see if there is nudity in clip
            if True not in nude_list:
                clean_file = file.path.split('/')[-1].split('.')[0]
                for cf in clips:
                    if clean_file in cf:
                        # Return only the safe video clip paths
                        safe_clips.append(cf)
        return safe_clips


def censor_video():
    """For any non-nude clips censor the faces."""
    algo = client.algo(
        "media/VideoTransform/0.5.5")
    for video_file in safe_videos():
        print(video_file)
        input = {
            "input_file": video_file,
            "output_file": "data://quality/testCensorface/testCensor.mp4",
            "algorithm": "cv/CensorFace/0.1.3",
            "advanced_input": {
                "images": "$BATCH_INPUT",
                "output_loc": "$BATCH_OUTPUT",
                "fill_color": "blur"
            }
        }
        response = algo.pipe(input).result
        print(response)

censor_video()