All posts in Algorithm Spotlight

Censoring Faces Automatically

Privacy issues are a big concern when recording public videos. Professional photographers and public institutions (such as police departments) run into this problem when publishing or releasing public images or footage. Generally speaking, professionals are required to obscure faces when there is a reasonable expectation of privacy and the individual(s) being filmed have not signed a release form. Similarly, many governmental institutions censor faces when releasing video footage, in order to maintain the privacy of those in the video.

Censor face tries to solve these problems by automating the process of censoring faces, efficiently and at scale. Read More…

Automatic Scene Detection

Hundreds of thousands of videos are uploaded each day to Youtube, Facebook, Instagram, Snapchat and other sites. One of the many issues that these services face is the extraction of useful metadata. At Algorithmia, we’ve been automating a lot of this process, enabling rapid auto-tagging and feature detection of videos. But there’s still more work to be done…

For example, if you wanted to put an ad in the middle of a video, as an advertiser you’d probably prefer to show the ad in between scene cuts, where it would be less intrusive. Or perhaps you’re curating a stock footage gallery but are working with multi-scene footage which needs to be split up. In both of these cases, you don’t want to manually scrub through many thousands of videos each day to determine the best insertion or cut-points.

Scene Detection is an algorithm which automates this task at scale, and is now available for you to use via Algorithmia.  Let’s dig into the details of how we detect scene changes in videos… Read More…

Deep Dive into Parallelized Video Processing

Where it all began

At Algorithmia, one of our driving goals is to enable all developers to stand on the shoulders of the algorithmic giants. Like Lego our users can construct amazing devices and tools by utilizing our algorithmic building blocks like FaceDetection or Smart Image Downloader.

As a platform, Algorithmia is unique in that we’re able to scale to meet any volume of concurrent algorithm requests, meaning that even though your algorithm might be making 10,000 API requests to a particular image processing algorithm, it won’t influence the experience of other users quality of service.

One of the earliest projects I worked on at Algorithmia was to construct a video processing pipeline which would leverage our existing image processing algorithms as building blocks. The project was designed to improve the reach of our image processing algorithms by automatically enabling them to become video processing algorithms.

After the first couple of weeks the first Interface and process flow was starting to come together and by using ffmpeg we were able to easily split videos into frames and concatenate them back into any video format we wanted. However, it quickly became apparent how fragile this initial process flow was, and how difficult it was to use for an end user. Read More…

Introduction to Video Metadata Extraction

Last week we talked about how Video Transform was able to change the way users handled video transformation tasks. What’s even better than being able to Transform Videos at will? Getting actual, structured information out of them! This week we introduce you Video Transform’s sister, Video Metadata Extraction.

What’s the difference between Metadata Extraction and Transform?


Video Metadata Extraction is a Rust algorithm which functions very similarly to Video Transform, however instead of utilizing algorithms that transform images, it uses algorithms that classify or extract information from images, and returns the information in a structured, timestamped json array file.

This key difference unlocks a whole universe of potential, allowing us to extract any kind of information from any video, given we have the right image processing algorithm. Read More…