Algorithmia Blog - Deploying AI at scale

Create a Dropbox Image Processing Pipeline in Python

Before and after example of Dropbox image processing

Need to create a Dropbox image processing pipeline to batch edit images? Now that Algorithmia supports Dropbox integration, here’s a quick way to automatically create thumbnails with custom dimensions.

In this demo, we’ll use SmartThumbnail, a microservice that uses face detection to perfectly crop every photo to the same size without awkwardly cropped heads or faces.

While manually cropping photos isn’t bad when it’s just a few. Cropping hundreds or thousands of images in real-time would be extremely expensive, time consuming, and tedious.

So, instead of doing this by hand so that every face in every photo is perfectly preserved, we can run all the photos through SmartThumbnail. The output is both intuitive and expected, each and every time.

We’re going to connect to your Dropbox, process the images in a folder, and then save a new thumbnail images back to the folder. We’ll be using Python for this tutorial, but this could easily be done in JavaScript/Node, Rust, Scala, Java, or Ruby.

Don’t use Dropbox? Want to use Amazon S3 instead? No problem. Here’s our guide to creating an Amazon S3 image processing pipeline.

Ready? Let’s go.

Step 1: Create a Free Account and Install Client

You’ll need a free Algorithmia account for this tutorial. Use the promo code “dropbox” to get an additional 50,000 credits when you signup.

Next, make sure you have the latest Algorithmia Python client on your machine. Let’s do that really quick:

pip install algorithmia

and to check that installation was successful…

pip show algorithmia

The Algorithmia Python Client should be version 1.0.5.


Step 2: OAuth Dropbox

Now that you have an Algorithmia account with the latest client installed, let’s OAuth your Dropbox so that Algorithmia’s microservices can read and write to it.

Once logged in, navigate to the Algorithmia Data Portal, where you manage all your data, collections, and connected services, like Dropbox and Amazon S3.

Connect a new Data Source

  1. Select Add New Data Source 
  2. Connect to Dropbox. A new Dropbox window will open.
  3. Click Allow to grant Algorithmia access to your Dropbox files and folders
  4. Click Manage Dropbox
  5. For this tutorial, Algorithmia will need both READ and WRITE access to your Dropbox. Check Write Access and then Save

Manage Dropbox Settings

Now, when we want to READ or WRITE data to Dropbox from an Algorithmia microservice, we refer to it as dropbox://*. Let’s get to the fun part, and write the code to process our Dropbox images.


Step 3: Dropbox Image Processing

We’re going to write a simple Python script to initialize the Algorithmia client, set the API key, loop through all the files in a specified Dropbox folder, process each image, and then save a new thumbnail image back to Dropbox.

There are three things you’ll need here:

  1. Your Algorithmia API key, which can be found under Credentials on your Algorithmia Profile page.
  2. The Dropbox folder path you want to process. In our example below, we are going to process the /Camera Uploads folder.
  3. And, the image size of your new thumbnail. In this example, we’re generating 300×300 thumbnails.
##############################
#Author: Diego Oppenheimer ###
#						   ###
# Algorithmia, Inc         ###
##############################


import Algorithmia


#Set your Algorithmia API Key
apiKey = 'YOUR API KEY GOES HERE'

#Initialize Algorithmia Python client
client = Algorithmia.client(apiKey)

#Pick Algorithm to use
algo = client.algo('opencv/SmartThumbnail/1.0.4')

#Set folder URI path
uri = "dropbox://Camera Uploads"

#Iterate over the Dropbox folder containing images
for f in client.dir(uri).list():
	
	#Check file type is an image
	if f.getName().lower().endswith(('.png','.jpg','.jpeg','.bmp','.gif')):
		#Image progress write
		print "Reading " + f.getName()

		#Define input for Algorithm + Parameters 
		input = [uri + '/' + f.getName(), uri + '/thumbnail_' + f.getName(), 300, 300, "FALSE"]
		
		#Call Algorithm
		output = algo.pipe(input)
		
		print "Thumbnailing: thumbnail_" + f.getName()
       
	else:
		print "File:" + f.getName() +  "is not a type that is supported."

print "Done processing..."

Above, we’re calling Algorithmia, and asking for a list of files in the directory /Camera Uploads. We then iterate through all the files, checking to see if they’re a PNG, JPG, etc. If we find an image file, we’ll then pass it to the SmartThumbnail microservice, which processes the image.

To ensure images are perfectly cropped, SmartThumbnail uses face detection to ensure heads and faces are in the frame. It then crops the image to the desired dimension (in our case it’s a 300×300 thumbnail), and then writes it back in the same format (i.e. PNG, JPG, etc.) to the Dropbox folder with the “thumbnail_” suffix. Get the Gist here.

Ready to process your images? Simply copy the above, change your settings, and save the file as processImages.py. Run it from the command line by typing:

python processImages.py

The Dropbox image processing pipeline in action

Pretty cool, right? There’s more than 2,000 microservices in the Algorithmia library you could use to process Dropbox files. For instance, you could batch convert files from one type to another, convert audio to text (speech recognition), automatically tag and update the metadata on images, detect and sort images of people smiling, and more.

You could easily create an Amazon Lambda function to watch your Dropbox for new images, and then run this script to automatically process images as they’re uploaded.

We’d love to hear what you think @Algorithmia.

Product manager at Algorithmia helping to give developers super powers.

More Posts - Website

Follow Me:
TwitterFacebookLinkedIn