Algorithmia Blog - Deploying AI at scale

Getting Started with Algorithmia in Spark

Fashion model

Apache Spark is one of the most useful tools for large scale data processing. It allows for data ingestion, aggregation, analysis and more on massive amounts of data and has been widely adopted by data engineers and other professionals.

With Spark Streaming and Spark SQL you can perform ETL operations in real-time on data coming from a variety of sources such as Kafka or Flume. And now if you want to do some basic machine learning, you can do that with SparkML, which is a library where they bring core statistical models like KMeans or decision tree models to users in a high level API.

But what if you want to analyze thousands of Tweets in real time, yet you don’t have a trained dataset to discover the sentiment of those tweets. Or maybe you want to classify documents on the fly or remove profanity from text or nudity from images?

Algorithmia’s over 4,000 pre-trained models and functions cover all of the above use cases and perfectly compliment Spark’s core functionality. These pre-trained models can easily integrate into Spark via a REST API endpoint. And just like Spark, Algorithmia has Python, R, Java, and Scala clients so you can stay in the language you’re familiar with while building robust machine and deep learning pipelines that scale with your data.

Today, we’ll show you how easy it is to integrate a deep fashion pre-trained model into Spark by simply calling the Algorithmia API from your Spark cluster in a few lines of Scala code. Read More…

Advanced grammar and Natural Language Processing with Syntaxnet

Parsey McParseface

Lets play a game: can you tell the difference between these two sentences?

“Most of the time, travellers worry about their luggage.”

“Most of the time travellers worry about their luggage.”

Whoa, remove the comma and all of a sudden we’re having an entirely different conversation!

The little nuances of language can be hard enough for a human to understand, let alone a computer! How could we possibly teach a computer to understand the difference?

Read More…

DevOps for AI – The AI Layer

When Google’s Gradient Ventures invested in us, they did so with an understanding that it is incredibly hard to deploy AI/ML infrastructure — and that every dev team is going to need to solve this problem.
Our solution, the AI Layer, is the best-in-class architecture.As our co-founder, Kenny Daniel says: Tensorflow is open-source, but scaling it is not. 
DevOps for AI presents massive challenges to all sizes of organizations. If your team is only a couple of developers, you don’t want to distract them from their primary mission by requiring them to put in a ton of effort supporting infrastructure.For large organizations, AI/ML models require a completely customized DevOps stack. Let us show you how…

  Read More…

Advanced Algorithm Design

We host more than 4000 algorithms for over 50k developers. Here is a list of best practices we’ve identified for designing advanced algorithms. We hope this can help you and your team. Read More…

Chatbot Workshop at Seattle’s Building Intelligent Applications Meetup

Recently Jon Peck, who is a Software Engineer and Developer Evangelist at Algorithmia, wrote a fun post on how to get an emotionally aware chatbot up and running in about 15 minutes.

In this hands-on micro workshop, Jon will show you how to create a chatbot using Dexter, a company that makes building chatbots easy and accessible. Then he’ll show you how to make the chatbot emotionally aware using Algorithmia. Our open marketplace that hosts over 4,000 algorithms and microservices that are all available via a scalable API endpoint.

Jon will also go through some use cases covering why you would need a chatbot, especially one enabled with machine learning and provide some examples of other machine learning algorithms that work well in chatbots, but aren’t covered in the demo.

Please join us for a fun evening of food and drinks provided by Algorithmia and learn how to build an emotionally intelligent chatbot!

For more information or to RSVP check out the Seattle Building Intelligent Applications Meetup.