Algorithmia Blog

Introduction to Image Saliency Detection

Whenever we look at a photo or watch a video, we always notice certain things over others. This could be a person’s face, or a sports car, or even a symbol that is located in the corner of a video.

SalNet is an algorithm that automates the process of finding the salient regions of an image using deep learning.

What is Saliency Detection

Saliency is what stands out to you and how you are able to quickly focus on the most relevant parts of what you see.

In neuroscience, saliency is described as an attention mechanism in organisms to narrow down to the important parts of what they see.

In UX design, saliency is a feedback loop for understanding what parts of a design are useful, and which are not. They use the information they gather from usability and eye tracking studies this to design better interfaces.

Advertisers are well aware that many people don’t have long attention spans, hence they try to catch the eye of a user with a single glance. Saliency detection methods are used to better design ads and posters.

Saliency detection, essentially, can be used in any area in which you’re trying to automate the process of understanding what stands out in an image.

Why You Need Saliency Detection

We use saliency detection to make our algorithms smarter. One example of this would be the Smart Thumbnail Algorithm. This microservice uses the Saliency Detector algorithm to get information about the important parts of an image. Smart Thumbnail then uses this information to perfectly crop and resize the image every time.

Using Saliency Detection will make your app/service smarter by detecting the relevant (salient) parts in your images automatically. You can use this information to improve your service, and make your app smarter!

How to Use Saliency Detection

First step is to create a free account on Algorithmia.

After creating your account, go to your profile page and navigate to the Credentials tab. There you will find your API key. Copy that to a safe location, so we can use it later.

Now, let’s find a test image. I’ve picked a photo of a guy from Unsplash, but you can pick another photo if you’d like.

Last and not least, install the Python Algorithmia client using the command pip install algorithmia

Now by running the sample code (in Python) below, we’ll be able to get our stylized image in a few seconds!

Sample Input

import Algorithmia

client = Algorithmia.client("YOUR_API_KEY_GOES_HERE")

# Create testing directory if it doesn’t exist

if not client.dir("data://.my/SalNetTest").exists():
    client.dir("data://.my/SalNetTest").create()

input = {
    "image": "data://deeplearning/example_data/guy.jpg",
    "location": "data://.my/SalNetTest/guy_saliency.png",
    "saliencyLocation": "data://.my/SalNetTest/saliencyMatrix.json"
}

result = client.algo("deeplearning/SalNet/0.2.0").pipe(input).result

print result

Sample Input Image:

Sample Output:

{
    "output": "data://.my/SalNetTest/guy_saliency.png",
    "saliencyMatrix": "data://.my/SalNetTest/saliencyMatrix.json"
}

Sample Output Image:

You can now find the stylized image in your newly created collection in the data API portal.

For more information, go to the SalNet algorithm page here. You can find all relevant documentation about the algorithm there.

Conclusion

Saliency detection is a way to spend less time and energy determining what’s most relevant in an image.

SalNet is an algorithm that utilizes the power of deep learning to extract information about important, relevant parts of any given image by finding salient regions and points.

The heavy lifting is handled behind a serverless microservices API, so you can focus on developing your app, and not worry about servers or infrastructure.

Let us know if you’ve used SalNet in your app/service via @algorithmia on Twitter!