Algorithmia

Turn Your Deep Learning Model into a Serverless Microservice

Hosting Deep Learning Models

Introducing Cloud Hosted Deep Learning Models

At Algorithmia, we believe in democratizing access to state-of-the-art algorithmic intelligence. That’s why we’re introducing a solution for hosting and distributing trained deep learning models on Algorithmia using GPUs in the cloud.

Today, researchers and developers can train their neural nets locally, and deploy them to Algorithmia’s scalable, cloud infrastructure, where they become smart API endpoints for other developers to use.

We’re excited to announce initial native support for the Caffe, Theano, and TensorFlow frameworks, and have added 16 open source deep learning models that run as microservices to start. Support for Torch and MxNet are coming soon.

Grab out developer guides for CaffeTensorFlow, and Theano here.

Plus, we’ve created two demos to showcase how easy it is to build applications off of hosted models:

We want Algorithmia to be the place for researchers and developers creating deep learning models to showcase their work, distribute it, and make it available for other developers to use.

We believe algorithms and models should be implemented once, reusable by developers anywhere, in any language.

You should train your model using the tools you’re comfortable with. And, when you’re ready, deploy it to our infrastructure, where your model will join a collection of more than 2,200 algorithmic microservices other developers can use to obtain real-time predictions, and build production-ready, machine intelligent apps.


Why Is Deep Learning So Important?

Digital Data vs Cost of Storage

Thanks to an abundance of digital data, and powerful GPUs, we are now capable of teaching computers to read, see, and hear.

Just this year, a handful of high-profile experiments came into the spotlight, including Microsoft Tay, Google DeepMind AlphaGo, and Facebook M.

These experiments all relied on a technique known as deep learning, which attempts to mimic the layers of neurons in the brain’s neocortex. This idea – to create an artificial neural network by simulating how the neocortex works – has been around since the 1980s.

During the training process, the algorithm learns to discover useful patterns in the digital representation of data, like sounds and images. In a very real sense, we’re teaching machines to teach themselves.

As a result, it’s become clear that deep learning is the next frontier in machine learning and artificial intelligence.

Yet, despite plentiful data, and abundant computing power, deep learning is still very hard.

The bottleneck is the lack of developers trained to use these deep learning techniques. Machine learning is already a highly specialized domain, and those with the knowledge to train deep learning models are even more select.

For instance, Google can’t recruit enough developers with deep machine learning experience. Their solution? Teach their developers to use ML instead. When Facebook’s engineers were struggling to take advantage of machine learning, they created an internal tool for visualizing ML workflows.

But, where does that leave the other 99% of developers that don’t work at one of these top tech company?

Trends in Machine Learning and Deep LearningVery few people in the world know how to use these tools.

“Machine learning is a complicated field,” S. Somasegar says, venture partner at Madrona Venture Group and the former head of Microsoft’s Developer Division. “If you look up the Wikipedia page on deep learning, you’ll see 18 subcategories underneath Deep Neural Network Architectures with names such as Convolutional Neural Networks, Spike-and-Slab RBMs, and LTSM-related differentiable memory structures.”

“These are not topics that a typical software developer will immediately understand.”

Yet, the number of companies that want to process unstructured data, like images or text, is rapidly increasing. The trend will continue, primarily because deep learning techniques are delivering impressive results. Which is why it’s important for the people capable of training neural nets are also able to share their work with as many people as possible.

Every industry needs machine intelligence

“Algorithmia provides an effective way for developers to seamlessly use deep learning modules,” Somasegar says.

GPUs on-demand and running in the cloud, eliminate the manual work required for teams and organizations to experiment with cutting-edge deep learning algorithms and models, which allows them to get started for a fraction of the cost.

“Deep learning has proven remarkably powerful, but it is far from plug-n-play,” Oren Etzioni says, CEO of the Allen Institute for Artificial Intelligence. “That’s where Algorithmia’s technology comes in – to accelerate and streamline the use of deep learning.”

We have a unique perspective on using deep learning frameworks, having been forced to deal with the idiosyncrasies of deploying TensorFlow, Caffe, and Theano into highly scalable, cloud-based production environments.

While GPUs were originally used to accelerate graphics and video games, more recently they’ve found new life powering AI and deep learning tasks, like natural language understanding, and image recognition.

“We’ve had to build a lot of the technology and configure all of the components required to get GPUs to work with these deep learning frameworks in the cloud,” Kenny Daniel says, Algorithmia founder and CTO. “The GPU was never designed to be shared in a cloud service like this.”

Hosting deep learning models in the cloud can be especially challenging due to complex hardware and software dependencies. While using GPUs in the cloud is still nascent, they’re essential for making deep learning performant.

CPU vs GPU performance

“For anybody trying to go down the road of deploying their deep learning model into a production environment, they’re going to run into problems pretty quickly,” Daniel says. “Using GPUs inside of containers is a challenge. There are driver issues, system dependencies, and configuration challenges. It’s a new space that’s not well-explored, yet. There’s not a lot of people out there trying to run multiple GPU jobs inside a Docker container.”

“We’re dealing with the coordination needed between the cloud providers, the hardware, and the dependencies to intelligently schedule work and share GPUs, so that users don’t have to.”

On Algorithmia, algorithms run as serverless microservices. And, our solution for hosting and distributing trained deep learning models using GPUs is one of the many building blocks developers can use to create, share, and remix algorithmic intelligence at scale.

By making algorithms composable, interoperable, and extensible, algorithms can be written in any supported language, reused, and made available to application developers where the code is always “on,” and available via a simple REST API.

We’re committed to making algorithms accessible and discoverable by everyone, and look forward to seeing what you build with Algorithmia.

Thanks,
– Diego M. Oppenheimer, Algorithmia founder and CEO

Let us know what you think

If you run into any issues with hosting your deep learning model, or want us to support a different framework, please reach out. Are you excited about turning your deep learning model into a microservice? Let us know @Algorithmia.

Useful Links: Developer CenterAPI Docs | Data Portal Guides

Facts and Figures

Application developers can access the API via our clients for Java, Scala, Python, JavaScript, Node.js, Ruby, Rust, CLI, and cURL. Our AWS Lambda blueprint is perfect for those working on event-driven and IoT projects.

Algorithmia is the largest algorithm marketplace in the world, with more than 19,000 developers leveraging more than 2,200 algorithms. Algorithms and models on Algorithmia include research from MIT, University of Washington, Carnegie Mellon University, University of California Berkeley, Caltech, University of Texas at Austin, University of Tokyo, University of Toronto, among others.

16 Open Source Deep Learning Models Running as Microservices

Open Source Deep Learning Models

If you’ve ever worked with deep learning frameworks you know they require a good amount of time, knowledge and, most of all, commitment just to get them up and running on your machine. And, that’s before you’ve even built your model or trained your data.

This assumes you have a GPU-enabled server that you don’t mind provisioning and maintaining. Algorithmia is excited to introduce three deep learning frameworks that are now supported on our platform: TensorFlow, Theano and Caffe!

To showcase these commonly used frameworks, we’ve added 16 open source deep learning models to the platform for you to try out, or use in your applications. Here’s a roundup of the deep learning algorithms available on Algorithmia:

Places 365 Classifier
An image classifier using a pre-trained CNN and is based on Places: An Image Database for Deep Scene Understanding B. Zhou, et al., 2016. This algorithm is trained using the deep learning framework Caffe, and it classifies images as a particular location, such as a courtyard, drugstore, hotel room or even a landscape tag such as a glacier or hot spring. This algorithm is particularly useful in travel, real estate or map applications. Try out the place recognition demo we created here.

Deep Face Recognition 
A classifier trained to recognize celebrities using Caffe based on the work of Parkhi, O. M., et al., 2015.This classifier takes an image of a person’s face, and returns the likely celebrity names, along with confidence values. Of course, this algorithm can be used to detect celebrities in a newsfeed or other source, but it might also be fun to use it to see what celebrity you look most like. Try building an application around that.

Real Estate Classifier 
A Caffe algorithm we trained in-house. It was built specifically for real estate applications, and includes image classification for features such as pools, a walk-in-closets, and house types, like a beach house or colonial house.

Colorful Image Colorization 
An algorithm originally developed by Richard Zhang, Phillip Isola, Alexei A. Efros, which takes black and white pictures, and returns the image in color. The algorithm uses the deep learning caffe framework to classify objects/regions within the image and colors them accordingly. So go ahead and take those old pictures from your photo albums and see what they look like in color! Try the colorize photo demo we created here.

Illustration Tagger 
An image tagging algorithm using Caffe based on Illustration2Vec: A Semantic Vector Representation of Illustrations by Saito, Masaki and Matsui, Yusuke. This algorithm takes an image and calculates the similarity between the image and available tags. It returns four categories of predicted tags when available: NSFW rating, character, copyright, and general (tags your image belongs to such as “male,” or “black hair”). This algorithm can help with grouping alike images into categories or detect images that aren’t appropriate for some settings.

InceptionNet 
A direct implementation of Google’s InceptionNet using Tensorflow. It takes an image (such as a car), and returns the top 5 classes the model predicts are relevant to the image. This could include “race car,” “sports car,” or ‘”convertible” classes for a car, while an image of a dog could return the breed of dog such as “husky.” With this algorithm you can bring object recognition to your app easily via the Algorithmia API.

Parsey McParseface
A popular language parser that Google recently open-sourced using Tensorflow. The Parsey McParseface neural model is an incredibly accurate sentence parser and Parts-of-Speech tagger that can be used for computational linguistic problems such as sentiment analysis and comparative opinions. Use it to build intelligent chatbots.

ArtsyNetworks 
A fun algorithm from ArtsyNetworks GitHub using the deep learning framework Theano. The algorithm takes an image and stylizes it according to various artists or art forms. Users can find an artists style they like such as Alphonse Mucha’s Art Nouveau, and then submit an image of an example of that style (such as Dance by Mucha) along with their own image including metadata and the algorithm returns a stylized image.

Here’s a few more of our deep learning models:

The above examples are already available on Algorithmia. We’re working to add more, and since we’re an open platform, we encourage everyone from academics to developers to data scientists to add their work and make it available to others.

Ready to host your deep learning models on Algorithmia? Get started with our model guides.

Machine Learning Ninjas, Autonomous Cars, Computer Vision, Delivery Drones, and More

Machine Learning, Computer Vision, and moreIssue 15
This week we learn about the
machine learning ninjas at Google, check in with a company retrofitting old cars with new tech, discover Apple’s computer vision capabilities, and anxiously wait for delivery drones taking flight. Plus, what we’re reading and a few things for you to try at home. 

Not a subscriber? Join the Emergent // Future newsletter here.


Google’s Machine Learning Ninjas 👀

You Might Have Heard: Google is remaking itself as a machine learning company by teaching its employees to use the latest learning algorithms.

The Machine Learning Ninja Program — no joke – embeds select Googlers with their machine learning team for six months where they work on ML projects, like Smart Reply in Gmail, and RankBrain.

In an earnings call late last year, CEO Sundar Pichai laid out Google’s new vision: “Machine learning is a core, transformative way by which we’re rethinking how we’re doing everything. We are thoughtfully applying it across all our products, be it search, ads, YouTube, or Play. And we’re in early days, but you will see us — in a systematic way — apply machine learning in all these areas.”

If Google’s going to build machine learning into all its products, it needs engineers who have mastery of those techniques.

But Did You Know: Google’s RankBrain machine learning system is now used to process every search query, and is now considered the third most important ranking factor?

PLUS: Benedict Evans weighs in on the state of AI, Apple and Google


A Billion Cars Going Autonomous 🚗 🚕 🚙

While mums the word on Apple’s car plans, Pearl Automation has big plans to transform millions of today’s cars into the self-driving vehicles of tomorrow.

Armed with 50 former Apple employees, the team has plans for a slew of devices that can upgrade your car by augmenting it with the latest driving capabilities.

Their first product? A rear-view camera.

While their entry into the auto accessory aftermarket is sort of an anticlimactic entry to the market, the Verge writes. “The differentiator is the attention to quality and detail that Apple is known for. This craftsmanship shines through in Pearl’s offering, as well.”

PLUS: Behind the race in Silicon Valley and Detroit to build a self-driving car


Apple’s Advanced Computer Vision 📸

You might remember that Apple added facial and object recognition to the Photos app for iPhone and iPad to bundle photos according to events and places.

Well, a developer has already uncovered that the new Photos app can distinguish 7 facial expressions, generate 33 Moments categories, and detect 4,432 searchable objects.

So, what’s so great about this?

Unlike Google Photos, Apple has made sure that Photos only uses machine learning to analyze the photos that are stored locally on your iPhone, eliminating the need to upload your photos to Apple’s servers for processing.


Drones Taking Flight ✈️

The FAA unveiled new rules last week to allow for commercial operation of low-altitude drones weighing less than 55 pounds.

As long as you can pass the skill test, you can operate a drone legally.

However, drones cannot be flown at night, and must always remain within eyeshot of the pilot, making it unlikely that autonomous drone delivery will be flying anytime soon.

The FAA says the rules could “could generate more than $82 billion for the U.S. economy and create more than 100,000 new jobs over the next 10 years.”


What We’re Reading 📚

  • Society in the Loop Artificial Intelligence. At the heart of human-in-the-loop computation is the idea of building models not just from data, but also from the human perspective of the data. (Joi Ito)
  • Meet The New Wave Of Wearables. Stretchable Electronics. Scientists have figured out how to make electronics as pliable as a temporary tattoo—meaning the next big tech platform may be your skin. (Fast Company)
  • The Return of the Machinery Question. After many false starts, artificial intelligence has taken off. Will it cause mass unemployment or even destroy mankind? History can provide some helpful clues. (The Economist)
  • Internet Archive Is a Beautiful Storage Bin for Our Online Memories. The Wayback Machine is an incredible resource for historians, researchers, journalists looking to see when sites get stealthily edited, people who like gawking at corny web design … really anyone who enjoys puttering around online. And it’s getting a search engine in 2017, which will make digging around even easier. (The Ringer)
  • Artificial Intelligence’s White Guy Problem / Artificial Intelligence Has a ‘Sea of Dudes’ Problem. Two articles this week by The New York Times and Bloomberg look at how bias creeps into datasets used to train machine learning algorithms. AI software depends on data sets, and data sets have to be created by computer scientists. What happens when most of those researchers are men?
    (New York Times) (Bloomberg)

Try This At Home 🛠


Emergent Future is a weekly, hand-curated dispatch exploring technology through the lens of artificial intelligence, data science, and the shape of things to come. Subscribe here.

 

Algorithmia Now Supports Amazon S3 and Dropbox Integrations

Algorithmia Now Integrates with Amazon S3 and Dropbox We know how hard and frustrating it is to have data stored in one location when you need it in another. Wouldn’t it be better to not have to deal with uploading your data to a new service in order to work with it?

That’s why we’re excited to announce the Algorithmia Data Portal. This dedicated I/O hub – a starting point for reading and writing your data from any data source – makes it easy to connect with Amazon S3 and Dropbox to access your data where it’s at. Now, application developers can read their data from an external source, process it using the algorithmic microservices from Algorithmia, and then write the output where it’s needed. No DevOps required.

“Algorithms allow you to gain insights from your data, but without data, what do you need algorithms for?” Diego Oppenheimer says, Algorithmia CEO and co-founder. “That’s why we’re enabling Algorithmia users to access their data where it’s stored, removing the friction of using state-of-the-art algorithms to interpret, and extract insights from data.”

With the Algorithmia Data Portal, we’re addressing one of the core issues of data portability and interoperability. Algorithmia can now retrieve your data on-demand, removing the need for developers to ship or on-board their data in advance. We believe that ingesting, processing, and writing data should be as simple as an API call. 

To demonstrate, we’ve created an image processing pipeline in 10-lines of Python code. Simply by connecting to an existing data source, like Amazon S3 or Dropbox, we can easily batch process an entire folder of images. In this demo, we make an API call to our data source to list the files, then make another API call to Algorithmia’s SmartThumbnail microservice. The service processes each image, and then writes a new file to a folder in the same data source.

Here’s the image processing pipeline tutorial for Amazon S3, and for Dropbox.

Learn how to connect, configure, and read/write from your data in a few easy steps with our data portal guides below. If you have any questions or suggestions, please get in touch by email or @Algorithmia.

Don’t use Amazon S3 or Dropbox? No problem. Algorithmia also offers a free hosted data service for storing large files, preserving state, creating collections, and more.


Data Portal Guides

For the algorithm developer, Algorithmia hosted data is perfect for storing trained machine learning models and instantly turning them into a live, scalable API. For more, check out our guides for hosting your NLTK and scikit-learn models on Algorithmia.


Documentation

For Application Developers

For Algorithm Developers


More About Algorithmia

We’ve created an open marketplace for algorithms and algorithm development, making state-of-the-art algorithms accessible and discoverable by everyone. On Algorithmia, algorithms run as microservices, creating the building blocks of algorithmic intelligence developers can use to create, share, and remix at scale.

By making algorithms composable, interoperable, and portable, algorithms can be written in any supported language, and then made available to application developers where the code is always “on,” and available via a simple REST API.

Application developers can access the API via our clients for Java, Scala, Python, JavaScript, Node.js, Ruby, Rust, CLI, and cURL. Our AWS Lambda blueprint is perfect for those working on event-driven and IoT-type projects.

Algorithmia is the largest marketplace for algorithms in the world, with more than 18,000 developers leveraging 2,000+ algorithms.

Create an Amazon S3 Image Processing Pipeline in Python

Amazon S3 Image Processing with Algorithmia

Need to create a simple Amazon S3 image processing pipeline to batch edit images? Now that Algorithmia supports Amazon S3 integration, here’s a quick way to automatically create thumbnails with custom dimensions.

In this demo, we’ll use SmartThumbnail, a microservice that uses face detection to perfectly crop every photo to the same size without awkwardly cropped heads or faces.

While manually cropping just a handful of photos isn’t bad. Cropping hundreds or thousands of images in real-time would be extremely expensive, time consuming, and tedious.

So, instead of doing this by hand so that every face in every photo is perfectly preserved, we can run all the photos through SmartThumbnail. The output is both intuitive and expected, each and every time.

We’re going to connect to an Amazon S3 bucket, process the images in a folder, and then save a new thumbnail images back to the folder. We’ll be using Python for this tutorial, but this could easily be done in JavaScript/Node, Rust, Scala, Java, or Ruby.

Don’t use Amazon S3? Want to use Dropbox instead? No problem. Here’s our guide to creating a Dropbox image processing pipeline.

Ready? Let’s go.

Step 1: Create a Free Account and Install Client

You’ll need a free Algorithmia account for this tutorial. Use the promo code “s3” to get an additional 50,000 credits when you signup.

Next, make sure you have the latest Algorithmia Python client on your machine. Let’s do that really quick:

pip install algorithmia

and to check that installation was successful…

pip show algorithmia

The Algorithmia Python Client should be version 1.0.5.


Step 2: Add Amazon S3 Credentials

Now that you have an Algorithmia account with the latest client installed, let’s connect your Amazon S3 account so that Algorithmia’s microservices can read and write to it.

Once logged in, navigate to the Algorithmia Data Portal, where you manage all your data, collections, and connected services, like Dropbox and Amazon S3.

Connect a new Data Source

  1. Select Add New Data Source 
  2. Connect to Amazon S3.
  3. Add your AWS Access Key ID and your Secret Access Key
  4. Check the Write Access box, and click Connect to Amazon S3

Connect to Amazon S3 with Algorithmia

 

Note: The best practice with Amazon Web Services is to create an AWS IAM identity, and then grant AmazonS3FullAccess permissions so that you can read/write to the bucket.  

Now, when we want to read/write data to Amazon S3 from an Algorithmia microservice, we refer to it as s3://*. Let’s get to the fun part, and write the code to process our images.


Step 3: Amazon S3 Image Processing

We’re going to write a simple Python script to initialize the Algorithmia client, set the API key, loop through all the files in a specified Amazon S3 bucket, process each image, and then save a new thumbnail image back to the bucket.

There are three things you’ll need here:

  1. Your Algorithmia API key, which can be found under Credentials on your Algorithmia Profile page
  2. The Amazon S3 bucket path you want to process. In our example below, we going to process the myimageassets bucket.
  3. And, the image size of your new thumbnail. In this example, we’re generating 300×300 thumbnails.
##############################
#Author: Diego Oppenheimer ###
#						   ###
# Algorithmia, Inc         ###
##############################


import Algorithmia


#Set your Algorithmia API Key
apiKey = 'YOUR API KEY GOES HERE'

#Initialize Algorithmia Python client
client = Algorithmia.client(apiKey)

#Pick Algorithm to use
algo = client.algo('opencv/SmartThumbnail/1.0.4')

#Set folder URI path
uri = "s3://myimageassets"

#Iterate over folder containing images in S3 
for f in client.dir(uri).list():
	
	#Check file type is an image
	if f.getName().lower().endswith(('.png','.jpg','.jpeg','.bmp','.gif')):
		#Image progress write
		print "Reading " + f.getName()

		#Define input for Algorithm + Parameters 
		input = [uri + '/' + f.getName(), uri + '/thumbnail_' + f.getName(), 300, 300, "FALSE"]
		
		#Call Algorithm
		output = algo.pipe(input)
		
		print "Thumbnailing: thumbnail_" + f.getName()
        
	else:
		print "File:" + f.getName() +  "is not a type that is supported."

print "Done processing..."

Above, we’re calling Algorithmia, and asking for a list of files in the bucket /myimageassets. We then iterate through all the files, checking to see if they’re a PNG, JPG, etc. If we find an image file, we’ll then pass it to the SmartThumbnail microservice, which processes the image.

To ensure images are perfectly cropped, SmartThumbnail uses face detection to ensure heads and faces are in the frame. It then crops the image to the desired dimension (in our case it’s a 300×300 thumbnail), and then writes it back in the same format (i.e. PNG, JPG, etc.) to the Amazon S3 bucket with the “thumbnail_” suffix. Get the Gist here.

Ready to process your images? Simply copy the above, change your settings, and save the file as processImages.py. Run it from the command line by typing:

python processImages.py

Processed S3 Images

Pretty cool, right? There’s more than 2,000 microservices in the Algorithmia library you could use to process Amazon S3 files. For instance, you could batch convert files from one type to another, convert audio to text (speech recognition), automatically tag and update the metadata on images, detect and sort images of people smiling, and more.

You could easily create an Amazon Lambda function to watch for new images, and then run this script to automatically process images as they’re uploaded.

We’d love to hear what you think @Algorithmia.

Create a Dropbox Image Processing Pipeline in Python

Before and after example of Dropbox image processing

Need to create a Dropbox image processing pipeline to batch edit images? Now that Algorithmia supports Dropbox integration, here’s a quick way to automatically create thumbnails with custom dimensions.

In this demo, we’ll use SmartThumbnail, a microservice that uses face detection to perfectly crop every photo to the same size without awkwardly cropped heads or faces.

While manually cropping photos isn’t bad when it’s just a few. Cropping hundreds or thousands of images in real-time would be extremely expensive, time consuming, and tedious.

So, instead of doing this by hand so that every face in every photo is perfectly preserved, we can run all the photos through SmartThumbnail. The output is both intuitive and expected, each and every time.

We’re going to connect to your Dropbox, process the images in a folder, and then save a new thumbnail images back to the folder. We’ll be using Python for this tutorial, but this could easily be done in JavaScript/Node, Rust, Scala, Java, or Ruby.

Don’t use Dropbox? Want to use Amazon S3 instead? No problem. Here’s our guide to creating an Amazon S3 image processing pipeline.

Ready? Let’s go.

Step 1: Create a Free Account and Install Client

You’ll need a free Algorithmia account for this tutorial. Use the promo code “dropbox” to get an additional 50,000 credits when you signup.

Next, make sure you have the latest Algorithmia Python client on your machine. Let’s do that really quick:

pip install algorithmia

and to check that installation was successful…

pip show algorithmia

The Algorithmia Python Client should be version 1.0.5.


Step 2: OAuth Dropbox

Now that you have an Algorithmia account with the latest client installed, let’s OAuth your Dropbox so that Algorithmia’s microservices can read and write to it.

Once logged in, navigate to the Algorithmia Data Portal, where you manage all your data, collections, and connected services, like Dropbox and Amazon S3.

Connect a new Data Source

  1. Select Add New Data Source 
  2. Connect to Dropbox. A new Dropbox window will open.
  3. Click Allow to grant Algorithmia access to your Dropbox files and folders
  4. Click Manage Dropbox
  5. For this tutorial, Algorithmia will need both READ and WRITE access to your Dropbox. Check Write Access and then Save

Manage Dropbox Settings

Now, when we want to READ or WRITE data to Dropbox from an Algorithmia microservice, we refer to it as dropbox://*. Let’s get to the fun part, and write the code to process our Dropbox images.


Step 3: Dropbox Image Processing

We’re going to write a simple Python script to initialize the Algorithmia client, set the API key, loop through all the files in a specified Dropbox folder, process each image, and then save a new thumbnail image back to Dropbox.

There are three things you’ll need here:

  1. Your Algorithmia API key, which can be found under Credentials on your Algorithmia Profile page.
  2. The Dropbox folder path you want to process. In our example below, we are going to process the /Camera Uploads folder.
  3. And, the image size of your new thumbnail. In this example, we’re generating 300×300 thumbnails.
##############################
#Author: Diego Oppenheimer ###
#						   ###
# Algorithmia, Inc         ###
##############################


import Algorithmia


#Set your Algorithmia API Key
apiKey = 'YOUR API KEY GOES HERE'

#Initialize Algorithmia Python client
client = Algorithmia.client(apiKey)

#Pick Algorithm to use
algo = client.algo('opencv/SmartThumbnail/1.0.4')

#Set folder URI path
uri = "dropbox://Camera Uploads"

#Iterate over the Dropbox folder containing images
for f in client.dir(uri).list():
	
	#Check file type is an image
	if f.getName().lower().endswith(('.png','.jpg','.jpeg','.bmp','.gif')):
		#Image progress write
		print "Reading " + f.getName()

		#Define input for Algorithm + Parameters 
		input = [uri + '/' + f.getName(), uri + '/thumbnail_' + f.getName(), 300, 300, "FALSE"]
		
		#Call Algorithm
		output = algo.pipe(input)
		
		print "Thumbnailing: thumbnail_" + f.getName()
       
	else:
		print "File:" + f.getName() +  "is not a type that is supported."

print "Done processing..."

Above, we’re calling Algorithmia, and asking for a list of files in the directory /Camera Uploads. We then iterate through all the files, checking to see if they’re a PNG, JPG, etc. If we find an image file, we’ll then pass it to the SmartThumbnail microservice, which processes the image.

To ensure images are perfectly cropped, SmartThumbnail uses face detection to ensure heads and faces are in the frame. It then crops the image to the desired dimension (in our case it’s a 300×300 thumbnail), and then writes it back in the same format (i.e. PNG, JPG, etc.) to the Dropbox folder with the “thumbnail_” suffix. Get the Gist here.

Ready to process your images? Simply copy the above, change your settings, and save the file as processImages.py. Run it from the command line by typing:

python processImages.py

The Dropbox image processing pipeline in action

Pretty cool, right? There’s more than 2,000 microservices in the Algorithmia library you could use to process Dropbox files. For instance, you could batch convert files from one type to another, convert audio to text (speech recognition), automatically tag and update the metadata on images, detect and sort images of people smiling, and more.

You could easily create an Amazon Lambda function to watch your Dropbox for new images, and then run this script to automatically process images as they’re uploaded.

We’d love to hear what you think @Algorithmia.

Twitter’s Machine Learning Plans, Is Deep Learning Magic?, Net Neutrality Upheld, and More

Twitter, Machine Learning, and Net NeutralityIssue 14
This week we check in on
Twitter’s machine learning plans, ask if deep learning is magic or just math, celebrate the latest net neutrality ruling, and share some videos to watch. Plus, what we’re reading and a few things for you to try at home. 

Not a subscriber? Join the Emergent // Future newsletter here.


Twitter Dives Into Machine Learning 📡

You Might Have Heard: Twitter announced Monday that it acquired the AI company Magic Pony Technology.

The London-based company uses machine learning and neural networks to identify features in video, enhance imagery, and create graphics for virtual and augmented reality.

In other words, Twitter’s big bet is that the algorithms will improve video streaming for Vine and Periscope by automatically filling in patchy video feeds, and increasing the resolution of pixelated video and images.

The new tech might come in handy this Fall when Twitter begins providing free, live streaming video of the NFL’s Thursday Night Football games to more than 800 Million users worldwide on mobile phones, tablets, PCs and connected TVs.

Magic Pony is the third machine learning acquisition by Twitter, joining Madbits, and Whetlab. Sources say the deal was worth $150M and includes a team of 11 PhDs with expertise in computer vision, machine learning, high-performance computing, and computational neuroscience.

PLUS: Google opened a dedicated machine learning research center in Zurich to focus on machine intelligence, natural language processing, and machine perception.


Deep Learning Isn’t Magic 🔮

Deep learning isn’t a dangerous magic genie, and it’s far from magic, Oren Etzioni says, CEO of the Allen Institute for Artificial Intelligence and a computer scientist at the University of Washington.

Google, Facebook, Microsoft and others continue to push AI into everyday online services while pundits describe deep learning as an imitation of the human brain. It’s really just simple math, Oren says, executed on an enormous scale.

And, amazingly, the artificial intelligence we were promised is finally coming as deep learning algorithms can now recognize images, voice, text, and more.

There is almost nothing we can think of that cannot be made new, different, or interesting by infusing it with some extra IQ,” Kevin Kelly writes.

It’s not all peaches and cream. Oxford philosopher and author Nick Bostrom warns that artificial intelligence is a greater threat to humanity than climate change.

So, what’s next for AI? The best minds in AI weigh in on what life will look like in the age of the machines.

Confused About AI? This two-part series on the AI revolution and the road to superintelligence is an excellent primer. (Part 1) (Part 2)

PLUS: A guide to staying human in the machine age


Net Neutrality: Can You Hear Me Now? 💬

A federal appeals court has ruled that high-speed internet can be defined as a utility, putting it on par with other essential services like power and the phone.

The ruling clears the way for rigorous policing of broadband providers like AT&T, Comcast, and Verizon, limiting their ability to shape the user experience by blocking content, prioritizing paid content, or creating fast and slow lanes on the internet.

“This is an enormous win for consumers,” said Gene Kimmelman, president of the public interest group Public Knowledge. “It ensures the right to an open internet with no gatekeepers.”

With the win, the internet will remain a platform for innovation, free expression, and economic growth, Tom Wheeler, chairman of the F.C.C., said in a statement.

But, hold the celebration, the fight for net neutrality isn’t over.

AT&T is already vowing to fight the decision, and expects this to ultimately be decided by the Supreme Court. 🙈

 


Videos to Watch 📺

 


What We’re Reading 📚

  • Serverless Architectures. Unlike traditional architectures, serverless is run in stateless compute containers that are event-triggered, ephemeral, and fully managed by a 3rd party. Think of this as “Functions as a service / FaaS.” AWS Lambda is one of the most popular implementations of FaaS. (Martin Fowler)
  • What’s Next for Artificial Intelligence. The best minds in the business on what life will look like in the age of the machines. (WSJ)
  • A Dozen Things I’ve Learned from Elon Musk About Business and Investing. Elon Musk is a classic missionary founder who is more interested in changing the world and creating enduring businesses than just the financial rewards that may flow to him from the product or service. (25iq)
  • The Forrest Gump of the Internet. Ev Williams became a billionaire by helping to create the free and open web. Now, he’s betting against it. (The Atlantic)
  • A Simple Explanation of How Image Recognition Works.Are you tired of reading endless news stories about deep learning and not really knowing what that means? Let’s change that! This time, learn how to write programs that recognize objects in images using deep learning. (Medium)

Try This At Home 🛠


Emergent Future is a weekly, hand-curated dispatch exploring technology through the lens of artificial intelligence, data science, and the shape of things to come. Subscribe here.

Improving Nudity Detection and NSFW Image Recognition

Detecting Nudity In ImagesLast June we introduced isitnude.com, a demo for detecting nudity in images based on our Nudity Detection algorithm. However, we wouldn’t be engineers if we didn’t think we could improve the NSFW image recognition accuracy.

The challenge, of course, is how do you do that without a labeled dataset of tens of thousands of nude images? To source and manually label thousands of nude images would have taken months, and likely years of post-traumatic therapy sessions. Without this kind of labeled dataset, we had no way of utilizing computer vision techniques such as CNNs (Convolutional Neural Networks) to construct an algorithm that could detect nudity in images.

The Starting Point for NSFW Image Recognition

The original algorithm used nude.py at its core to identify and locate skin in an image. It looked for clues in an image, like the size and percentage of skin in the image, to classify it as either nude or non-nude.

We built upon this by combining nude.py with OpenCV’s nose detection, and face detection algorithms. By doing this, we could draw bounding polygons for the face and nose in an image, and then get the skin tone range for each polygon. We could then compare the skin tone from inside the box to the skin found outside the box.

OpenCV Face Detection AlgorithmAs a result, if we found a large area of skin in the image that matched the skin tone of the person’s face/nose, we could return that there was nudity in the image with high confidence.

Additionally, this method allowed us to limit false positives, and check for nudity when there were multiple people in an image. The downside to this method was that we could only detect nudity when a human face was present.

Detecting nudity using Algorithmia

All told, our Nudity Detection algorithm was ~60% accurate using the above method.

Improving Nudity Detection In Images

The first step in improving our ability to detect nudity in images was to find a pre-trained model that we could work with. After some digging, one of our algorithm developers found the Illustration2Vec algorithm. It’s trained on anime to extract feature vectors from illustrations. The creators of the Illustration2Vec scraped 1.2M+ images of anime and the associated metadata, creating four categories of tags: general, copyright, character, and ratings.

This was a good starting point. We then created the Illustration Tagger microservice on Algorithmia, our implementation of the Illustration2Vec algorithm.

With the microservice running, we could simply call the API endpoint, and pass it an image. Below we pass an image of Justin Trudeau, Canada’s prime minister, to Illustration Tagger like so:

{
  "image": "https://upload.wikimedia.org/wikipedia/commons/9/9a/Trudeaujpg.jpg"
}

Testing Nudity Detection Illustration Tagger

And, the microservice will return the following JSON:

{
  "rating": [
    {"safe": 0.9863441586494446},
    {"questionable": 0.011578541249036789},
    {"explicit": 0.0006071273819543421}
  ],
  "character": [],
  "copyright": [{"real life": 0.4488498270511627}],
  "general": [
    {"1boy": 0.9083328247070312},
    {"solo": 0.8707828521728516},
    {"male": 0.6287103891372681},
    {"photo": 0.45845481753349304},
    {"black hair": 0.2932664752006531}
  ]
}

Side-note: Illustration2Vec was developed to make a keyword-based search engine to help novice drawers find reference images to base new work on. The pre-trained caffe model is available through Algorithmia, and available under the MIT License.

In all, Illustration2Vec can classify 512 tags, across three different rating levels (“safe,” “questionable,” and “explicit.”).

The important part here is that the ratings were used to … wait for it … label nudity and sexual implications in images. Why? Because anime tends to have lots of nudity and sexual themes.

Because of that, we were able to piggyback off their pre-trained model.

We used a subset of the 512 tags to create a new microservice called Nudity Detection I2V, which acts as a wrapper for Illustration Tagger, but only returns nude/not-nude with corresponding confidence values.

The same Trudeau image when passed to Nudity Detection I2V returns the following:

{
  "url": "https://upload.wikimedia.org/wikipedia/commons/9/9a/Trudeaujpg.jpg", 
  "confidence": 0.9829585656606923, 
  "nude": false 
}

For comparison, here’s the output of the famous Lenna photo from Nudity Detection, Nudity Detection I2V, and Illustration Tagger.

Image Recognition and NSFW

Nudity Detection

{
  "nude": "true", 
  "confidence": 1
}

Nudity Detection I2V

{
  "url": "http://www.log85.com/tmp/doc/img/perfectas/lenna/lenna1.jpg", 
  "confidence": 1, 
  "nude": true
}

Illustration Tagger

{ 
  "character": [], 
  "copyright": [{"original": 0.24073825776577}], 
  "general": [ 
    {"1girl": 0.7676181197166442},
    {"nude": 0.7233675718307494},
    {"photo": 0.5793498158454897},
    {"solo": 0.5685935020446777},
    {"breasts": 0.38033962249755865},
    {"long hair": 0.24592463672161105}
  ], 
  "rating": [ 
    {"questionable": 0.7255685329437256},
    {"safe": 0.23983716964721682},
    {"explicit": 0.032572492957115166}
  ] 
}

For every image analyzed using Nudity Detection I2V, we check to see if any of these nine tags are returned: “explicit”, “questionable”, “safe”, “nude”, “pussy”, “breasts”, “penis”, “nipples”, “puffy nipples”, or “no humans.”

To capture nudity related information from the above tags, we labelled a small ~100 image dataset and pushed our data to Excel. Once there, we ran Excel’s Solver plugin to neatly fit weights to each tag, maximizing our new detector’s accuracy across our sample set.

One interesting observation from the linear fit operation is that some tags have less correlation with nudity (like breasts), than others (like nipples). Logically this makes sense, since exposed breasts aren’t necessarily nude, but if you can see nipples, it is.

By using this method, we were able to increase the accuracy rate from ~60% to ~85%.

Conclusion

Constructing a nudity detection algorithm from scratch is tough. Building a large unfiltered nudity dataset for CNN training is expensive and potentially damaging to one’s mental health. Our solution was to find a pre-trained model that “almost” fit our needs, and then fine tune the output to become an excellent nudity detector.

Positive Accuracy

72.73%

83.64%

Negative Accuracy

48.39%

87.10%

Overall Accuracy

60.56%

85.37%

 

As you can see from the above results, we’ve definitely succeeded in improving our accuracy. Don’t just take our word for it, try out Nudity Detection I2V.

In a future post, we’ll demonstrate how to use an ensemble to combine the two methods for even greater accuracy.

Emergent // Future Report: Flying Cars, Understanding AI and Machine Learning, WWDC News, and More

Flying Cars, Understanding AI and Machine Learning, WWDC News, and MoreIssue 13
This week we check in on
Larry Page’s flying-car project, get a primer in AI, ML, and Deep Learning, take a look at all the Apple WWDC news, and check out the Vive VR headset. Plus, what we’re reading and a few things for you to try at home.

Not a subscriber? Join the Emergent // Future newsletter here.


We Were Promised… Flying Cars? 🚗

You Might Have Heard: Larry Page has a secret flying-car factory.

Not only that, but he’s invested over $100M into the flying-car startup Zee.Aero, and is also an investor in the competing firm, Kitty Hawk.

Flying cars have always been a part of our collective imagination, for better or worse, but for flying cars to become a reality they need to be able to fly autonomously.

“With ultralight composites and better battery tech we may actually be drawing near a basic functional design,” TechCrunch writes. “In a few decades it may seem absurd that we drove our own cars for a century, with the dead from traffic accidents totaling in the millions.”


A Primer on AI, and Machine Learning 🤖

This video presentation by a16z Deal and Research head Frank Chen, walks through the basics of AI, deep learning, and machine learning, explaining how we got here, why now, and what the next breakthrough is.

Every company is now a data company, capable of using machine learning in the cloud to deploy intelligent apps at scale, thanks to three machine learning trends: data flywheels, the algorithm economy, and cloud-hosted intelligence.

Despite playing catchup to Google and Microsoft, Facebook wants to dominate in AI and machine learning. They’ve tripled their investment in processing power for research, and hired up more than 150 people.

Meanwhile, a patent for “Face Detection Using Machine Learning” was just granted. 😳

PLUS: A few considerations when setting up deep learning hardware.


News from WWDC 📱🖥⌚️

The big news from Apple WWDC this year is that they’re opening up Siri to app developers in an effort to keep pace with Amazon, Google, Facebook, and Microsoft, all of which are betting that voice commands and chatbots will be one of the next big computing paradigms.

HomeKit, the Apple Internet of Things platform, got a big updateand a dedicated app to control all the devices in your home, andfeatures access to connected cameras from the lock screen, geofencing, automation, Siri commands, and Apple Watch support.

One More Thing: Apple added facial and object recognition to the iPhone and iPad.

The computer vision tech runs natively on the device, and doesn’t require you to upload all the images to the cloud. With the Photos app, you can now recognize faces and objects across your photo library to help you to find a specific photo with ease.

In addition, the app has a new feature called Memories, which bundles photos according to events and places.

PLUS: The 13 biggest announcements from Apple WWDC 2016


Vive VR Headset Now Shipping 🎮

The VR headset from HTC started shipping last week in 24 countries, with delivery in two to three days. Head to a Microsoft, GameStop, and Micro Center store for a demo before plopping down $799.

But, should you buy an HTC Vive right now?

Still unsure? Here’s an HTC Vive vs Oculus Rift comparison.

tl;dr
The Rift has better sound and ergonomics; The Vive has better controllers and smoother tracking of head and hand motions.

“Both of these devices are good enough to deliver a great VR experience to millions of people. This is the beginning of something big.”

PLUS: VR rollercoasters are coming, and they look amazing.

ALSO: Augmented reality startup Blippar unveils its “visual browser,” an app for recognizing real-world objects using machine learning.


What We’re Reading 📚

  • What are the odds we are living in a computer simulation?Citing the speed with which video games are improving, Elon Musk suggested that the development of simulations “indistinguishable from reality” was inevitable. The likelihood that we are living in “base reality,” Musk concluded, was just “one in billions.” (The New Yorker)
  • Ray Kurzweil’s four big insights for predicting the future.“How we eat, work, play, communicate, and travel are deeply affected by the development of new technology. But what is the underlying engine that drives technological progress? Does technological change progress at a steady rate? Can we predict what’s coming in 5 or 10 years?” (Singularity Hub)
  • Mastering Programming. “The theme here is scaling your brain. The journeyman learns to solve bigger problems by solving more problems at once. The master learns to solve even bigger problems than that by solving fewer problems at once.” (Kent Beck)
  • How big data and poker-playing bots are blurring the line between man and machine. “Science, mathematics, and gambling have long been intertwined, and thanks to advances in big data and machine learning, our sense of what’s predictable is growing, crowding out the spaces formerly ruled by chance.” (Kernal Mag)
  • A New Theory Explains How Consciousness Evolved. “If the wind rustles the grass and you misinterpret it as a lion, no harm done. But if you fail to detect an actual lion, you’re taken out of the gene pool.” (The Atlantic)
  • Jessica Livingston’s Pretty Complete List on How Not to Fail. “Nothing else you do will matter if you’re not making something people want. You can be the best spokesperson, the best fundraiser, the best programmer, but if you aren’t building a product that satisfies a real need, you’ll never succeed.” (The Macro)

Try This At Home 🛠


Emergent Future is a weekly, hand-curated dispatch exploring technology through the lens of artificial intelligence, data science, and the shape of things to come. Subscribe here.

Powered by Algorithmia

Machine Learning Trends and the Future of Artificial Intelligence 2016

The Future of AI and Machine Learning Trends 2016

Every company is now a data company, capable of using machine learning in the cloud to deploy intelligent apps at scale, thanks to three machine learning trends: data flywheels, the algorithm economy, and cloud-hosted intelligence.

That was the takeaway from the inaugural Machine Learning / Artificial Intelligence Summit, hosted by Madrona Venture Group* last month in Seattle, where more than 100 experts, researchers, and journalists converged to discuss the future of artificial intelligence, trends in machine learning, and how to build smarter applications.

With hosted machine learning models, companies can now quickly analyze large, complex data, and deliver faster, more accurate insights without the high cost of deploying and maintaining machine learning systems.

“Every successful new application built today will be an intelligent application,” Soma Somasegar said, venture partner at Madrona Venture Group. “Intelligent building blocks and learning services will be the brains behind apps.

Below is an overview of the three machine learning trends leading to a new paradigm where every app has the potential to be a smart app.


Data Flywheels

Digital data and cloud storage follow Moore’s law: the world’s data doubles every two years, while the cost of storing that data declines at roughly the same rate. This abundance of data enables more features, and better machine learning models to be created.

“In the world of intelligent applications, data will be king, and the services that can generate the highest-quality data will have an unfair advantage from their data flywheel — more data leading to better models, leading to a better user experience, leading to more users, leading to more data,” Somasegar says.

Digital data growth and cost of storageFor instance, Tesla has collected 780 million miles of driving data, and they’re adding another million every 10 hours.

This data is feed into Autopilot, their assisted driving program that uses ultrasonic sensors, radar, and cameras to steer, change lanes, and avoid collisions with little human interaction. Ultimately, this data will be the basis for their autonomous, self-driving car they plan to release in 2018.

Compared to Google’s self-driving program, which has amassed just over 1.5 million miles of driving data. Tesla’s data flywheel is in full effect.


The Algorithm Economy

All the data in the world isn’t very useful if you can’t leverage it. Algorithms are how you efficiently scale the manual management of business processes.

“Everything at scale in this world is going to be managed by algorithms and data,” says Joseph Sirosh, CVP of Data Group and Machine Learning at Microsoft. In the near-future, “every business is an algorithmic business.”

This creates an algorithm economy, where algorithm marketplaces function as the global meeting place for researchers, engineers, and organizations to create, share, and remix algorithmic intelligence at scale. As composable building blocks, algorithms can be stacked together to manipulate data, and extract key insights.

Intelligent App Stack, Artificial Intelligence and Machine LearningIn the algorithm economy, state-of-the-art research is turned into functional, running code, and made available for others to use. The intelligent app stack illustrates the abstraction layers, which form the building blocks needed to create intelligent apps.

“Algorithm marketplaces are similar to the mobile app stores that created the ‘app economy,'” Alexander Linden, research director at Gartner said. “The essence of the app economy is to allow all kinds of individuals to distribute and sell software globally without the need to pitch their idea to investors or set up their own sales, marketing and distribution channels.”


Cloud-Hosted Intelligence

For a company to discover insights about their business, using algorithmic machine intelligence to iteratively learn from their data is the only scalable way. It’s historically been an expensive upfront investment with no guarantee of a significant return.

“Analytics and data science today are like tailoring 40-years ago,” Sirosh said. “It takes a long time and a tremendous amount of effort.”

For instance, an organization needs to first collect custom data, hire a team of data scientists, continually develop the models, and optimize them to keep pace with the rapidly changing and growing volumes of data — that’s just to get started.

The Machine Learning Trend at GoogleWith more data becoming available, and the cost to store it dropping, machine learning is starting to move to the cloud, where a scalable web service is an API call away. Data scientists will no longer need to manage infrastructure or implement custom code. The systems will scale for them, generating new models on the fly, and delivering faster, more accurate results.

“When the effort to build and deploy machine learning models becomes a lot less — when you can ‘mass manufacture’ it — then the data to do that becomes widely available in the cloud,” Sirosh said.

Emerging machine intelligence platforms hosting pre-trained machine learning models-as-a-service will make it easy for companies to get started with ML, allowing them to rapidly take their applications from prototype to production.

“As companies adopt the microservices development paradigm, the ability to plug and play different machine learning models and services to deliver specific functionality becomes more and more interesting,” Somasegar said.

When open source machine learning and deep learning frameworks running in the cloud, like Scikit-Learn, NLTK, Numpy, Caffe, TensorFlow, Theano, or Torch, companies will be able to easily leverage pre-trained, hosted models to tag images, recommend products, and do general natural language processing tasks.


Recap of Machine Learning Trends

“Our world view is that every company today is a data company, and every application is an intelligent application,” Somasegar said. “How can companies get insights from huge amounts of data and learn from that? That’s something that has to be brought up with every organization in the world.”

As the data flywheels begin to turn, the cost to acquire, store, and compute that data will continue to drop.

This creates an algorithm economy, where the building blocks of machine intelligence live in the cloud. These pre-trained, hosted machine learning models make it possible for every app to tap into algorithmic intelligence at scale.

The confluence of data flywheels, the algorithm economy, and cloud-hosted intelligence means:

  • Every company can now be a data company
  • Every company can now access algorithmic intelligence
  • Every app can now be an intelligent app

“We have come a long way,” Matt McIlwain says, Madrona Venture Group managing director. “But we still have a long way to go.”


*Disclosure: Madrona Venture Group is an investor in Algorithmia