All posts in Blog Posts

Getting Started with the Algorithmia Command Line Interface (CLI)

These instructions are out-of-date. Please See https://algorithmia.com/developers/clients/cli/ instead.

image

Algorithmia has released a command line interface (CLI)! This new, cross-platform tool allows you to use the command line to call more than 1,600 algorithms from the Algorithmia API.

The official docs can be found in the README on GitHub, but we’ll do a quick walkthrough to get you up and running.

Installation:

Setting up the Algorithmia CLI is easy.

OSX

Download and extract the OSX tarball with this cURL command:

$ curl -L https://github.com/algorithmiaio/algorithmia-cli/releases/download/v1.0.0-beta.2/algorithmia_osx.tar.gz | tar -xz

We recommend putting it within your PATH with the following:

$ sudo mv algo /usr/local/bin/

Windows (64-bit)

First, install OpenSSL for Windows.

Next, download and extract the Windows zip file. We recommend putting it within your PATH.

Looking for instructions for Debian/Ubuntu or Arch installs? Find them on the official README.

Configure the CLI:

Before we can start making calls to the APIs, first we need configure our authentication. Run the command:

$ algo auth

You’ll be prompted to input your API key for a default profile. If you’d like to create more than one profile, simply run algo auth [<profile>] for the same interactive prompt. For more information, see the auth command help with algo auth --help.

Check that your profile is configured by running algo ls.

Usage:

Now that we’re set up, let’s try executing some algorithms! We’re going to start with something simple, the Factor algorithm that takes in integer and returns its prime factors. You can find the details about this algorithm on the description page.

$ algo run kenny/factor -d 19635
[3,5,7,11,17]

Here we see the basic syntax: algo run, followed by the algorithm’s username and algorithm name (kenny/factor), the data options, and finally the input. The response we got back was just the result of the algorithm, but we can also add the option --response-body to see the full JSON response:

$ algo run kenny/factor -d 19635 --response-body
{"result":[3,5,7,11,17],"metadata":{"content_type":"json","duration":0.001427314}}

Run algo run --help to see more command options, including the data input and output options.

The Algorithmia CLI is also great for interacting with the Algorithmia API. You can use the CLI to create and manage your data directories. Let’s say you want to create a new data directory and upload some files-it’s easy with the CLI.

First, create a new data directory:

$ algo mkdir .my/favoritepics

Created directory data://.my/favoritepics

Copy a file from your local directory to the new data directory:

$ algo cp puppies.png data://.my/favoritepics

Uploaded data://.my/favoritepics/puppies.png

When you’ve finished uploading, check that everything you expect is in your data directory:

$ algo ls .my/favoritepics

puppies.png kittens.jpg baby_turtles.jpg

Example:

Algorithmia CLI use of opencv/SmartThumbnail

Help:

Run algo --help for additional usage information and to learn about other command options. Be sure to check out the official README for even more detailed instructions on using the Algorithmia CLI and new release updates!

We’ve been invited to present at AWS re:Invent!

image

Algorithmia is excited to announce that we’ve been selected to present at AWS re:Invent 2015 in Las Vegas on Tuesday, October 6th!

Our founders Diego Oppenheimer and Kenny Daniel will be presenting a solution for building and deploying serverless applications using Algorithmia and AWS Lambda during the breakout session Building Tomorrow’s Applications: Serverless Solutions in the Cloud.

“The combination of AWS’s limitless infrastructure, and Algorithmia’s algorithmic intelligence, allows developers from around the globe to build tomorrow’s smart applications today,” Oppenheimer said.

Sign-up for Algorithmia today, and receive 10,000 free credits to get you started building brilliant apps. 

If you’re in Las Vegas, please join us on Tuesday, October 6th at 2:45 PM in Lido 3001B. This event is free and open to APN members. Check out the complete AWS re:Invent schedule here.

We’ll be sharing a detailed blog post from the presentation, including a video demo, code samples, and a how-to guide for creating your own serverless apps using Algorithmia and Lambda.

Watch the AWS re:Invent 2014 Day 1 Keynote with Andy Jassy:

And, Day 2 Keynote with Werner Vogels:

About Algorithmia:

Build brilliant apps with Algorithmia, the largest marketplace for algorithms in the world. We help application developers solve complex problems with ease and efficiency by making algorithmic intelligence approachable and accessible in less than 5-lines of code.

Algorithmia unlocks the building blocks of human understanding, helping you make every app a smart app. Use Algorithmia to recognize patterns in your data, extract visual knowledge, understand audio, classify unstructured data, and derive meaning from language. Focus on what matters most, and let Algorithmia take care of the rest

Features:

  • Infrastructureless Deployment
  • REST API
  • Production Ready

Algorithmia wins the Startup Showcase at Strata + Hadoop World

Wow! This morning we were selected as this year’s Startup Showcase winner at Strata + Hadoop World NY 2015. The panel of judges chose Algorithmia as the winner based on our team, technology, and innovations.

image

It’s an honor to take part in this competition, where 12 of the top big data startups demonstrated their technologies to a room packed full of investors, entrepreneurs, and researchers. 

Use the code STRATA15 to get $100 in free credits when you sign-up for Algorithmia.

Three other companies stood apart from the rest, including second-place sense.io, third-place timbr.io, and the audience favorite, Blue Talon.

About Algorithmia:

Build brilliant apps with Algorithmia, the largest marketplace for algorithms in the world. We help application developers solve complex problems with ease and efficiency by making algorithmic intelligence approachable and accessible in less than 5-lines of code.

Algorithmia unlocks the building blocks of human understanding, helping you make every app a smart app. Use Algorithmia to recognize patterns in your data, extract visual knowledge, understand audio, classify unstructured data, and derive meaning from language. Focus on what matters most, and let Algorithmia take care of the rest

Features:

  • Infrastructureless Deployment 
  • REST API
  • Production Ready

Getting Started With Natural Language Processing (NLP)

Natural Language Processing Summary:

Natural Language Process, or NLP for short, is a field of study focused on the interactions between human language and computers. It sits at the intersection of computer science, artificial intelligence, and computational linguistics (Wikipedia).

“Nat­ur­al Lan­guage Pro­cessing is a field that cov­ers com­puter un­der­stand­ing and ma­nip­u­la­tion of hu­man lan­guage, and it’s ripe with pos­sib­il­it­ies for news­gath­er­ing,” Anthony Pesce said in Natural Language Processing in the kitchen. “You usu­ally hear about it in the con­text of ana­lyz­ing large pools of legis­la­tion or other doc­u­ment sets, at­tempt­ing to dis­cov­er pat­terns or root out cor­rup­tion.”

NLP is a way for computers to analyze, understand, and derive meaning from human language in a smart and useful way. By utilizing NLP, developers can organize and structure knowledge to perform tasks such as automatic summarization, translation, named entity recognition, relationship extraction, sentiment analysis, speech recognition, and topic segmentation.

“Apart from common word processor operations that treat text like a mere sequence of symbols, NLP considers the hierarchical structure of language: several words make a phrase, several phrases make a sentence and, ultimately, sentences convey ideas,” John Rehling, an NLP expert at Meltwater Group, said in How Natural Language Processing Helps Uncover Social Media Sentiment. “By analyzing language for its meaning, NLP systems have long filled useful roles, such as correcting grammar, converting speech to text and automatically translating between languages.”

What Can I Use Natural Language Processing For?

  • Summarize blocks of text using Summarizer to extract the most important and central ideas while ignoring irrelevant information. 
  • Automatically generate keyword tags from content using AutoTag, which leverages LDA, a technique that discovers topics contained within a body of text.
  • Identify the type of entity extracted, such as it being a person, place, or organization using Named Entity Recognition.
  • Use Sentiment Analysis to identify the sentiment of a string of text, from very negative to neutral to very positive.
  • Reduce words to their root, or stem, using PorterStemmer, or break up text into tokens using Tokenizer.

These are just some of the natural language processing algorithms web developers can use.

What Are Some Real World Examples of Natural Language Processing?

Social media analysis is a great example of NLP use. Brands track conversations online to understand what customers are saying, and glean insight into user behavior.

“One of the most compelling ways NLP offers valuable intelligence is by tracking sentiment — the tone of a written message (tweet, Facebook update, etc.) — and tag that text as positive, negative or neutral,” Rehling said.

Build your own social media monitoring tool

  1. Start by using the algorithm Retrieve Tweets With Keyword to capture all mentions of your brand name on Twitter. In our case, we search for mentions of Algorithmia.
  2. Then, pipe the results into the Sentiment Analysis algorithm, which will assign a sentiment rating from 0-4 for each string (Tweet).

Similarly, Facebook uses NLP to track trending topics and popular hashtags.

“Hashtags and topics are two different ways of grouping and participating in conversations,” Chris Struhar, a software engineer on News Feed, said in How Facebook Built Trending Topics With Natural Language Processing. “So don’t think Facebook won’t recognize a string as a topic without a hashtag in front of it. Rather, it’s all about NLP: natural language processing. Ain’t nothing natural about a hashtag, so Facebook instead parses strings and figures out which strings are referring to nodes — objects in the network. We look at the text, and we try to understand what that was about.”

It’s not just social media that can use NLP to it’s benefit. Publishers are hoping to use NLP to improve the quality of their online communities by leveraging technology to “auto-filter the offensive comments on news sites to save moderators from what can be an ‘exhausting process’,” Francis Tseng said in Prototype winner using ‘natural language processing’ to solve journalism’s commenting problem.

Other practical uses of NLP include monitoring for malicious digital attacks, such as phishing, or detecting when somebody is lying.

Use NLP to build your own RSS reader

You can build a machine learning RSS reader in less than 30-minutes using the follow algorithms:

  1. ScrapeRSS to grab the title and content from an RSS feed.
  2. Html2Text to keep the important text, but strip all the HTML from the document.
  3. AutoTag uses Latent Dirichlet Allocation to identify relevant keywords from the text.
  4. Sentiment Analysis is then used to identify if the article is positive, negative, or neutral.
  5. Summarizer is finally used to identify the key sentences.

Recommended NLP Books for Beginners

  • Speech and Language Processing: “The first of its kind to thoroughly cover language technology – at all levels and with all modern technologies – this book takes an empirical approach to the subject, based on applying statistical and other machine-learning algorithms to large corporations.”
  • An Introduction to Information Retrieval: “Class-tested and coherent, this groundbreaking new textbook teaches web-era information retrieval, including web search and the related areas of text classification and text clustering from basic concepts.”
  • Foundations of Statistical Natural Language Processing: “This foundational text is the first comprehensive introduction to statistical natural language processing (NLP) to appear. The book contains all the theory and algorithms needed for building NLP tools. It provides broad but rigorous coverage of mathematical and linguistic foundations, as well as detailed discussion of statistical methods, allowing students and researchers to construct their own implementations. The book covers collocation finding, word sense disambiguation, probabilistic parsing, information retrieval, and other applications.”
  • Handbook of Natural Language Processing: “The Second Edition presents practical tools and techniques for implementing natural language processing in computer systems. Along with removing outdated material, this edition updates every chapter and expands the content to include emerging areas, such as sentiment analysis.”
  • Statistical Language Learning (Language, Speech, and Communication): “Eugene Charniak breaks new ground in artificial intelligenceresearch by presenting statistical language processing from an artificial intelligence point of view in a text for researchers and scientists with a traditional computer science background.”
  • Natural Language Understanding: “This long-awaited revision offers a comprehensive introduction to natural language understanding with developments and research in the field today. Building on the effective framework of the first edition, the new edition gives the same balanced coverage of syntax, semantics, and discourse, and offers a uniform framework based on feature-based context-free grammars and chart parsers used for syntactic and semantic processing.”

NLP Tutorials

  • Natural Language Processing Tutorial: “We will go from tokenization to feature extraction to creating a model using a machine learning algorithm. You can get the source of the post from github.”
  • Basic Natural Language Processing: “In this tutorial competition, we dig a little “deeper” into sentiment analysis. People express their emotions in language that is often obscured by sarcasm, ambiguity, and plays on words, all of which could be very misleading for both humans and computers.“
  • An NLP tutorial with Roger Ebert: “Natural Language Processing is the process of extracting information from text and speech. In this post, we walk through different approaches for automatically extracting information from text—keyword-based, statistical, machine learning—to explain why many organizations are now moving towards the more sophisticated machine-learning approaches to managing text data.”

Additional Research

If you’re interested in learning more, this free introductory course from Stanford University will help you will learn the fundamentals of natural language processing, and how you can use it to solve practical problems.

Once you’ve gotten the fundamentals down, apply what you’ve learned using Python and NLTK, the most popular framework for Python NLP.

  • Natural language processing (Wikipedia): “Natural language processing (NLP) is a field of computer science, artificial intelligence, and computational linguistics concerned with the interactions between computers and human (natural) languages. In 1950, Alan Turing published an article titled ‘Computing Machinery and Intelligence’ which proposed what is now called the Turing test as a criterion of intelligence. Starting in the late 1980s, however, there was a revolution in NLP with the introduction of machine learning algorithms for language processing.”
  • Outline of natural language processing (Wikipedia): “The following outline is provided as an overview of and topical guide to natural language processing: Natural language processing – computer activity in which computers are entailed to analyze, understand, alter, or generate natural language.”
  • Apache OpenNLP: “The Apache OpenNLP library is a machine learning based toolkit for the processing of natural language text.”
  • Natural Language Toolkit: “NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning, wrappers for industrial-strength NLP libraries, and an active discussion forum. Natural Language Processing with Python provides a practical introduction to programming for language processing.”

Algorithmia selected as Strata + Hadoop World 2015 Startup Showcase finalist

image

We’re thrilled to be one of the twelve leading big data startups selected to present at Strata + Hadoop World 2015 at 6:30pm on Tuesday, September 29th at the Javits Center in New York City. Diego and Kenny will be showing off our intelligent algorithms to a packed room of developers, entrepreneurs, and researchers as part of Startup Showcase. Winners will be announced during Wednesday’s keynote – fingers crossed!

Check out this video of winners from Strata + Hadoop in San Jose to get a taste of the competition:

Are you in New York? Startup Showcase is free to attend, and open to the public as part of NYC DataWeek. Register here for your free ticket, and be sure to come say hi to Diego and Kenny, who will be handing out Algorithmia credits. Or reach out to us @Algorithmia.