Algorithmia Blog

Checking out Content Aware Resize

media/ContentAwareResize is an algorithm that discovers the most important features of an image and attempts to preserve them when resizing, letting you resize images without cropping out what’s important.

How it works

media/ContentAwareResize works by utilizing two existing algorithms on our platform, dlib/FaceDetection and deeplearning/SalNet:

FaceDetection is fantastic at detecting faces by using dlib’s builtin CNN model, and Salnet is a neural network that detects saliency in an image and generates a heat map. By utilizing both the centroid of saliency and the centroid of detected faces, we’re able to generate a content aware centroid. By using this centroid we’re able to preserve the focus of an image when resizing.

This algorithm is a direct improvement in quality over SmartThumbnail, as the underlying face detection algorithm is significantly improved in both performance and accuracy. If you’ve used SmartThumbnail before you should consider trying out media/ContentAwareResize.

It should be mentioned that if your image contains salient features in multiple parts of an image, such as a multi-subject photo, media/ContentAwareResize may end up removing one or more of the subjects. In that situation, it might be worthwhile to take a look at the anowell/ContentAwareResize algorithm. The anowell/ContentAwareResize algorithm uses the seam carving technique to resize an image instead of cropping around a centroid, but it also distorts the image by removing unimportant content.

Why you need it

Have you ever encountered a website that just automatically resized images to center fit? Maybe the website doesn’t even have resizing functionality and forces their users to alter images themselves? In either event, this causes friction among your users and leads to churn. This algorithm removes that hassle from your users and your development team so you can focus on more important aspects of your application

How do I use Content Aware Resize?

Getting started with media/ContentAwareResize and using it in your existing pipeline is easy, just follow these steps and you can recreate our demo above:

import Algorithmia
input = {
  "height":300,
  "width":100,
  "ratio":0.45,
  "output":"data://.algo/media/contentAwareResize/temp/leaning_girl.png",
  "image":"https://static.pexels.com/photos/160463/vietnam-vietnamese-girl-young-160463.jpeg"
}

result = client.algo('media/ContentAwareResize/0.1.1).pipe(input).result
output_file = result['output']

And that’s all you need to get media/ContentAwareResize working with your existing pipeline. Want more image processing? Check out Nudity Detection, Censor Face, and Saliency