Algorithmia Blog

How to use an Unsharp Mask to Improve the Accuracy of Saliency Detection

In our introduction to saliency detection post, we showed how to harness the power of the human brain using a saliency algorithm to detect the most distinct and noticeable objects in an image.

This algorithm can be used in a variety of fields, such as user experience testing, neuroscience research, and even product placement in advertisements or commercials.

SalNet is a saliency detection microservice hosted on Algorithmia, which allows you to easily integrate saliency detection into your product via a serverless API.

Most images do very well using SalNet, however some older, fuzzier images might need a little help from another algorithm called Sharpen Image. This algorithm uses an unsharp mask to sharpen the edges in the image, which increases the ability of the saliency detection algorithm to detect the most relevant shapes in an image.

In this recipe we’ll go through how to use the Sharpen Image algorithm to clean the image before running it through Sal Net, the saliency detection algorithm. This way you can pass in less than perfect images through the saliency algorithm and get great results.

Step 1: Install the Algorithmia Client

This tutorial is in Python. But, it could be built using any of the supported clients, like Scala, Ruby, Java, Node and others. Here’s the Python client guide for more information on using the Algorithmia API.

Install the Algorithmia client from PyPi:

pip install algorithmia

You’ll also need a free Algorithmia account, which includes 5,000 free credits a month – more than enough to get started with saliency detection.

Sign up here, and then grab your API key.

Step 2: Sharpen Your Images

Now that we have our account credentials, we can call the first algorithm that will sharpen our image in order to prepare the photo for the saliency algorithm. In this example we are showing how to pass in an image that is located in a data collection on Algorithmia. You could also use S3 or Dropbox.

import Algorithmia

client = Algorithmia.client("your_api_key")

def sharpen_image(path_dict):
    # Pass in your image path hosted on Algorithmia platform
    input = {
        "image": path_dict["sharpen_input_path"],
        "location": path_dict["sharpen_output_path"]
    }
    algo = client.algo("opencv/SharpenImage/0.1.10")
    try:
        result = algo.pipe(input).result
        print(result)
        # Return the image path for the sharpened image
        return result["output"]
    except Exception as e:
        print(e)

In the above code notice there is a parameter called path_dict that when we call our function, we’ll pass a dictionary in as our argument. This dictionary has two keys, the “image” key and the “location” key. The “image” key’s value will hold our file path to our original photo and “location” will be the path that our sharpened image will go into.

The “image” path should be the location of your original image in your data collection. To get this path, go to the Hosted Data page (requires sign in). In the collection your original photo is in, you’ll find its path under the filename: data://:username/:collection/:filename

You’ll want to set the “location” path to somewhere other than the temporary folder (data://.algo/temp/), because our next algorithm will need access to that image.

For details about permissions and special directories such as the .algo directory, see the Hosted Data Guide.

Even though we’ll wait to run it until the whole script is finished, here is a sneak peak of what we’ve done so far:

Original
After Sharpen Image Algorithm

Step 3: Apply Saliency to Find Notable Objects

Now that we have our sharpened image, let’s put them through the saliency algorithm:

def salnet(path_dict):
    new_image_file = sharpen_image(path_dict)
    # Make sure image exists in temporary folder
    if client.file(new_image_file).exists() is True:
        # Get the file name of sharpened image
        file_name = new_image_file.split("/")[-1]
        input = {
            "image": new_image_file,
            "location": "data://.algo/temp/salnet-{0}".format(file_name)
        }
        algo = client.algo('deeplearning/SalNet/0.2.0')
        result = algo.pipe(input).result
        print(result)
        return result
    else:
        print("Please make sure your file path is correct.")

salnet({"sharpen_input_path": "data://your_username/your_data_collection_name/gilligan-guys.jpg",
"sharpen_output_path": "data://your_username/your_data_collection_name/gilligan-guys-sharpened.jpg"})

The above code calls the Sal Net algorithm, passing in the dictionary that holds our input and output for our first algorithm Sharpen Image. The Sal Net algorithm’s output will be held in the algorithm’s temporary folder (.algo/deeplearning/SalNet) in your Hosted Data.

After running the algorithm, you’ll print out the paths for the Sharpen Image file, the Sal Net image path and the matrix of the saliency results in a JSON file:

{
    'output': 'data://quality/testing/gilligan-guys-sharpened.jpg'
}
{
    'saliencyMatrix': 'data://.algo/deeplearning/SalNet/temp/saliencyMatrix.json',
    'output': 'data://.algo/temp/salnet-gilligan-guys-sharpened.jpg'
}

Now you can go to your Hosted Data page and click on your saved images in the temporary folder .algo/deeplearning/SalNet to view your results.

To give you an idea of how the Sharpen Image algorithm increased the accuracy of Sal Net, here is the original photo ran through the saliency algorithm:

Without Sharpening
With Sharpening
As you can see, after running the image through the Sharpen Image algorithm there are three distinct objects that are more clearly defined than when the original, unsharpened image was used.

And that’s it! Now you can confidently use saliency detection even on images that aren’t high resolution or are blurry.

Tools Used:

For easy reference, find this code in our Recipes repo, or just copy the whole code snippet:

import Algorithmia

client = Algorithmia.client("your_api_key")

def sharpen_image(path_dict):
# Pass in your image path hosted on Algorithmia platform
input = {
"image": path_dict["sharpen_input_path"],
"location": path_dict["sharpen_output_path"]
}
algo = client.algo("opencv/SharpenImage/0.1.10")
try:
result = algo.pipe(input).result
print(result)
# Return the image path for the sharpened image
return result["output"]
except Exception as e:
print(e)

def salnet(path_dict):
new_image_file = sharpen_image(path_dict)
# Make sure image exists in temporary folder
if client.file(new_image_file).exists() is True:
# Get the file name of sharpened image
file_name = new_image_file.split("/")[-1]
input = {
"image": new_image_file,
"location": "data://.algo/temp/salnet-{0}".format(file_name)
}
algo = client.algo('deeplearning/SalNet/0.2.0')
result = algo.pipe(input).result
print(result)
return result
else:
print("Please make sure your file path is correct.")

salnet({"sharpen_input_path": "data://your_username/your_data_collection_name/gilligan-guys.jpg",
"sharpen_output_path": "data://your_username/your_data_collection_name/gilligan-guys-sharpened.jpg"})