All posts by Stephanie Kim

Getting Started with Algorithmia in Spark

Fashion model

Apache Spark is one of the most useful tools for large scale data processing. It allows for data ingestion, aggregation, analysis and more on massive amounts of data and has been widely adopted by data engineers and other professionals.

With Spark Streaming and Spark SQL you can perform ETL operations in real-time on data coming from a variety of sources such as Kafka or Flume. And now if you want to do some basic machine learning, you can do that with SparkML, which is a library where they bring core statistical models like KMeans or decision tree models to users in a high level API.

But what if you want to analyze thousands of Tweets in real time, yet you don’t have a trained dataset to discover the sentiment of those tweets. Or maybe you want to classify documents on the fly or remove profanity from text or nudity from images?

Algorithmia’s over 4,000 pre-trained models and functions cover all of the above use cases and perfectly compliment Spark’s core functionality. These pre-trained models can easily integrate into Spark via a REST API endpoint. And just like Spark, Algorithmia has Python, R, Java, and Scala clients so you can stay in the language you’re familiar with while building robust machine and deep learning pipelines that scale with your data.

Today, we’ll show you how easy it is to integrate a deep fashion pre-trained model into Spark by simply calling the Algorithmia API from your Spark cluster in a few lines of Scala code. Read More…

Chatbot Workshop at Seattle’s Building Intelligent Applications Meetup

Recently Jon Peck, who is a Software Engineer and Developer Evangelist at Algorithmia, wrote a fun post on how to get an emotionally aware chatbot up and running in about 15 minutes.

In this hands-on micro workshop, Jon will show you how to create a chatbot using Dexter, a company that makes building chatbots easy and accessible. Then he’ll show you how to make the chatbot emotionally aware using Algorithmia. Our open marketplace that hosts over 4,000 algorithms and microservices that are all available via a scalable API endpoint.

Jon will also go through some use cases covering why you would need a chatbot, especially one enabled with machine learning and provide some examples of other machine learning algorithms that work well in chatbots, but aren’t covered in the demo.

Please join us for a fun evening of food and drinks provided by Algorithmia and learn how to build an emotionally intelligent chatbot!

For more information or to RSVP check out the Seattle Building Intelligent Applications Meetup.

Bring Deep Learning to iOS and Android


We’ve all read about machine learning in the headlines, but many iOS and Android developers haven’t made the leap to integrating machine learning intelligence into their applications. This is partly due to the time commitment needed to learn enough statistics to understand the math behind the models, and to determine which models are appropriate for your use case.  Once a developer has this knowledge under their belt, they now have to move their trained model to production which requires a whole other set of skills, especially when it’s a deep learning algorithm that requires a GPU environment.

Between learning the algorithms and productionizing them for mobile devices, integrating ML into an application can seem like a daunting task. But there are big benefits to adding machine learning: you can take your mobile app from a basic CRUD architecture, to much more advanced uses:

Fortunately, there is an easier way. You don’t have to be an expert in machine learning to take advantage of its benefits. And if you are an expert, you can host your models for free in our scalable, serverless AI cloud. Read More…

Making Algorithms Discoverable and Composable

Just like a music producer creates a beat, then combines it with instrumentals and a baseline to form something catchy that lyrics can be applied to… developers need a way to compose algorithms together in a clean and elegant way.

Whether you’re creating a sentiment analysis pipeline for your social data or doing image processing on thousands of photos, you’ll need an easy way to combine the various tools available so you aren’t writing spaghetti code.

It isn’t always easy to combine the libraries you need. Sometimes a library or machine learning model is written in a different language than the one you’re using. Other times there might simply be a performance difference between languages which (is why we chose Rust to create a Video Metadata Extraction pipeline). And even though GitHub offers thousands of libraries, frameworks, and models to choose from, it’s sometimes difficult to find the one you need to solve your problem.

To solve these problems — and allow you to write elegant code while using machine learning models — Algorithmia provides an easy way to find, combine, and reuse models regardless of language. Each one gets a RES API endpoint, so you can mix & match them with each other and with external code. Read More…