Algorithmia

Accessing Algorithmia through Python

Today’s blog post is brought to you by one of our community members John Hammink

Imagine – you are a novice programmer and want to harness the power of algorithms.   Or maybe you are an academic, or an expert in another area that has nothing to do with programming and want to incorporate algorithms into your own work.  Now you can!   

To leverage the power of Algorithmia’s service, you must first understand a thing or two about REST APIs as well as how to do something with the output.  (Incidentally, what you learn about REST APIs here can be applied across the board to many web services!).

In this article, we’ll use python to create a small program to use a few REST APIs.

Why python?  Python’s easy, like using a notebook.  It’s very demoable code for any skill level.  So here’s what we’ll do:

  1. First, we’ll start out using cURL at the command line and walking through various usage scenarios.
  2. Next, we’ll look at a basic Python example for using a REST API.
  3. After that, we’ll get more advanced:  there are different ways to using the REST API within Python.  And why you might prefer different alternatives depending on the algorithm and the performance you want to achieve.
  4. Next, we’ll look at some different APIs.
  5. Our goal is to build an extremely simple console application to demonstrate the power of a couple of Algorithmia’s algorithms.
  6. Finally, we’ll end with a question: which problems would you, our users, via comments, want to solve?  

Starting off with cURL

First things first!  Head over to Algorithmia.com and sign-up if you have not already so you can get a free authentication token. You can find your API Key under your account summary:

image

Let’s look at the profanity detection algorithm.  You can read the summary docs of the algorithm here:  https://algorithmia.com/algorithms/nlp/ProfanityDetection

Let’s use cURL at the command line to access the API and pass a request.  

curl -X POST -d ’[“He is acting like a damn jackass, and as far as I am concerned he can frack off.”,[“frack”,“cussed”],false]’ -H ‘Content-Type: application/json’ -H 'Authorization: <insert_auth_token>’ https://api.algorithmia.com/api/nlp/ProfanityDetection

Before you move ahead, let’s explain what each parameter does.

curl -X POST -d ’[“He is acting like a damn jackass, and as far as I am concerned he can frack off.”,[“frack”,“cussed”],false]’ -H  

-X is how you use curl to send a custom request with data, specified with -d.   The data to be passed to the profanity detection algorithm comes in three parts:

  1. The sentence or phrase to be analyzed for profanity.  This can also be a variable, or a string scraped from somewhere else in the internet.
  2. Additional words or phrases to be added to the profanity corpus when analyzing a.
  3. a boolean value – when set to true, the additional words or phrases are ignored.

-H Allows us to send an extra header to include in the request when sending HTTP to a server. Here, we’re calling it twice – once to add the content type spec, the second time to add an authorization header.  Note that in our documentation and blog, this auth key will automatically be populated with your auth key….and finally, the url for the algorithm in the Algorithmia API that we want to use.

…enter python!

Python just might be the easiest language with which to make HTTP requests.  

Taking a step back, let’s look at how we might run a cURL-like request using python.  

First, let’s look at how one might use python with  cURL using a subprocess call.  Try this in python interactive mode:

Python 2.7.6 (default, Sep  9 2014, 15:04:36)[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwinType “help”, “copyright”, “credits” or “license” for more information.>>> data = “”“["I would like to eat some flaming cheesecake.”,[“cheesecake”,“cussed”],false]“”“>>> import subprocess>>> subprocess.call(['curl’, ’-X’, 'POST’, ’-d’, data, ’-H’, 'Content-Type: application/json’, ’-H’, 'Authorization:51619aab16b44ad0baef1d2c5110fb21’,  'http{"duration”:0.0009742940000000001,“result”:{“cheesecake”:1}}0>>>

This works, and passes the data, but there is always going to be a performance hit when calling cURL or any subprocess, for that matter, from within Python.  Generally, native function calls simply work best!   Python handles the whole data-passing thing much more cleanly.   Let’s look at a simple ‘dataless’ implementation (an urllib2 request where we don’t pass any data to the algorithm) might look like in python using urllib2 library.

import urllib2, jsonrequest = urllib2.Request('https://api.algorithmia.com/api/nlp/ProfanityDetection’)request

What is going on here?   We are simply creating a request and storing it to a variable.   Running it returns this:

<urllib2.Request instance at 0x7fcb52d5e3f8>

In this example, the imported json library and input variables are just placeholders for what we’re about to do – pass and return actual data to an urllib2 request.   Let’s try that now:

import urllib2url = 'https://api.algorithmia.com/api/nlp/ProfanityDetection’data = “”“["I would like to eat some damn cheesecake.”,[“cheesecake”,“cussed”],false]“”“req = urllib2.Request(url, data, {'Content-Type’: 'application/json’})req.add_header('Authorization’, ’<insert_auth_token>’)f = urllib2.urlopen(req)for x in f:    print(x)

This actually prints out the contents the data, returned by Algorithmia’s API:

{"duration”:0.089443493,“result”:{“damn”:1,“cheesecake”:1}}

To close the object, just add:

f.close()

So now, we have the basics.  You can use the above example for calling more or less any REST API in Python.  

In my study of algorithms over REST APIs, I’ve learned that once we’ve mastered the basics of calling the API from whatever language or framework we’re using, it’s time to play around with the parameters we’re passing to the algorithm.

Consider this:

data = “”“["I would like to eat some damn cheesecake.”,[“cheesecake”,“cussed”],false]“”“

There are actually several things going on here.   There is the main data we’re parsing:

"I would like to eat some damn cheesecake.”

Then there are the additional words we add to our cuss-corpus, in this case:

[“cheesecake”,“cussed”]

Last, but not least,  there is a boolean value being passed at the end of the string.   This value tells us whether or not to ignore known profanities (and only go with cuss-words you provide!).   The input format is documented on the algorithm summary page on Algorithmia.  I tried actually passing several parameters to the data string; you can have fun with this too:

data = “”“["I wouldd totally go bananas with his recommendations, if he weren not such a pompous ass”, [“cuss”, “banana”] , false]“”“data = ”“”[“Ernst is a conniving little runt.”, [“eat”, “little”] , false]“”“data = ”“”[“Ernst is a conniving little runt.”, [] , false]“”“

You can have a load of fun with the data set on this one.  Did you try these?  Did you invent some others? What did you get?

Other Algorithms
Now, let’s try calling some of the other APIs.

Using the template we provided earlier, let’s try calling a few different algorithms.  We’ll do this simply by switching out our url and data parameters.

Note that we can get some clues by studying the sample input above the API console for each.

Let’s look first at Do words rhyme ? (algorithm summary available at: https://algorithmia.com/algorithms/WebPredict/DoWordsRhyme)

import urllib2url = 'http://api.algorithmia.com/api/WebPredict/DoWordsRhyme’data = ”“”[“fish”, “wish”]“”“req = urllib2.Request(url, data, {'Content-Type’: 'application/json’})req.add_header('Authorization’, ’<insert_auth_token>’)f = urllib2.urlopen(req)for x in f:      print(x)

This returns:

{"duration”:0.23639225300000002,“result”:true}

Hint:  you can isolate just the result value you need by using python’s dictionary manipulations.   So for example, if you are getting your response this way:

import jsonresponse = urllib2.urlopen(req, json.dumps(data)

You can pass it to a python dictionary, and then just take the result:

f = response.read()rhyminDict = json.loads(f)rhyminResult = rhyminDict['result’]

You can mix it up with the input data and try again.    And you can benchmark different methods duration times by calling it directly with cURL or with cURL called as a subprocess in python.   See the performance difference?

Now let’s try some other algorithms:

Word Frequency Counter

algorithm summary available at https://algorithmia.com/algorithms/diego/WordFrequencyCounter

url = 'http://api.algorithmia.com/api/diego/WordFrequencyCounter’data = ’“Does this thing need a need like this need needs a thing?  Cause the last thing this thing needs is a need.”’

Geographic Distance

algorithm summary available at https://algorithmia.com/algorithms/diego/GeographicDistance

url = 'http://api.algorithmia.com/api/diego/GeographicDistance’data = “”“{"lat1”: 60.1,“long1”:24.5,“lat2”:37.2,“long2”:122.4}“”“

Are you getting the hang of it now?   

Putting it all together

Now let’s put together a simple console application in Python.

NOTE:  This app is not intended to take the place of legitimate psychological or arousal research.  It’s just code to illustrate what you can do with these algorithms.

In this app, we’ll use two algorithms, profanity detection and sentiment analysis to get a general read on the user’s state of mind, from the input they give.   

Listing:  howsyourday.py  Full source code available at: https://github.com/algorithmiaio/samples/blob/master/HowIsYourDay.py

############################################################## Simple python app that uses 2 algorithms in algorithmia API                     ## 	- Profanity Detection                                                                                    ## 	- Sentiment Analysis                                                                                     ##                                                                                                                       ## Author: John Hammink <john@johnhamm.ink>                                           #############################################################
import urllib2, json
def get_cuss_index(sentiment):    url = 'https://api.algorithmia.com/api/nlp/ProfanityDetection’    filterwords = ["cuss”, “blow”, “spooge”, “waste”]    ignore = True    data = “”“[”%s", [“%s”], %s]“”“ % (sentiment, filterwords, ignore)    req = urllib2.Request(url, data, {'Content-Type’: 'application/json’})    req.add_header('Authorization’, 'cb7a25d50ddd4deeaed9e4959cfa1ffe’)    response = urllib2.urlopen(req, json.dumps(data))    f = response.read()    curseDict = json.loads(f)    curseResult = curseDict['result’]    cuss_index = sum(curseResult.values())    return cuss_index
def analyze_sentiment(sentiment):    url2 = 'http://api.algorithmia.com/api/nlp/SentimentAnalysis’    data2 = str(sentiment)    req2 = urllib2.Request(url2, data2, {'Content-Type’: 'application/json’})    req2.add_header('Authorization’, 'cb7a25d50ddd4deeaed9e4959cfa1ffe’)    response = urllib2.urlopen(req2, json.dumps(data2))    g = response.read()    analysis = json.loads(g)    analysisResult = analysis['result’]    return analysisResult
def gimme_your_verdict(cuss_index, analysisResult):    highArousal = ’"My, we are feisty today! Take five or go for a skydive!”’    mediumArousal = ’“Seems like you are feeling pretty meh”’    lowArousal = ’“Hey dude, are you awake?”’    if cuss_index >= 2 or analysisResult >= 3:   	 	print highArousal    elif cuss_index >= 1 or analysisResult >= 2:   	 	print mediumArousal    else:   	 	print lowArousal    print “Come back tomorrow!”        if __name__ == ’__main__’:    print “Como estas amigo?”    sentiment = str(raw_input(“How was your day? Swearing is allowed , even encouraged: ”))    cuss_index = get_cuss_index(sentiment)    analysisResult = analyze_sentiment(sentiment)    gimme_your_verdict(cuss_index, analysisResult)

We’ve put together three functions.   get_cuss_index and analyze_sentiment both query and parse the returns from their respective algorithms;  both functions load their respective json responses into a python dictionary, then parse the dictionary for only the result value; get_cuss_index actually sums the values in the dictionary, since there may be many.

gimme_your_verdict returns a snarky comment based on a conditional derived from the values returned by the first two functions.    

Having a bad day today, oh feisty one?  Well, there’s always tomorrow!

-John

A question for you, the reader….

Which other problems would you like to solve? Do you have a favorite algorithm you’d like to see demonstrated?  In a different programming language or framework?  Maybe you’d like to propose one – an algorithm or a solution with an existing one – that doesn’t exist yet?   

Let us know in the comments below! Follow and tweet us at @algorithmia We’ll get right on it – and you’ll see more on the topic, either in our blog or in our knowledge base.   Thanks for reading!

Product manager at Algorithmia helping to give developers super powers.

More Posts - Website

Follow Me:
TwitterFacebookLinkedIn