Algorithmia Blog - Deploying AI at scale

Serverless Computing

Serverless architecture is making cloud deployment even easier by removing the need to design your own server-side systems. Integrated properly, this paradigm can get your applications out the door faster and free up company resources to build more.

In a nutshell, serverless, also called Functions as a Service (FaaS), is a further abstraction on what cloud computing platforms like AWS already do—making it easier than ever to get your applications up and running at scale. Serverless takes the power of a hosted cloud to a software level – it abstracts away the entire concept of the server. Instead, you just write functions. The provider takes care of how and where to run those functions, ensuring that you focus on code and not the hardware and systems that operationalize that code.

Serverless as an Extension of Public Cloud

To understand what serverless is and why it matters, it helps to understand how the shift to the public cloud impacted businesses. If you wanted to start a company that had anything to do with the internet before Amazon Web Services, you needed to setup your own infrastructure. That meant buying your own servers, networking them, monitoring and optimizing them, and a myriad of other complex and time consuming tasks. This would be a huge up-front investment and take a long time. Imagine having to spend your entire seed round on servers!

Enter public clouds—instead of building your own infrastructure, you can just pay to use someone else’s over the internet. Instead of paying for an entire server set, you just pay for the compute you use. If you build a sustainable business, AWS ends up being more expensive over time (usually an 18 month catch up) – but the fact that you didn’t have to invest all of that up front was transformational. A popular mental model became “turn your capital expenditures into operational expenditures,” or CapEx to OpEx.

At the core though, the architecture for applications remained largely the same. Deployment still required server side logic and code, and you had to write it. But you got to design that instead of physically building it. If we’re going to paint a broad stroke, public cloud computing was a hardware and networking advancement; not a software one. Serverless is an attempt at advancing the software that runs these things too.

What is Serverless / FaaS?

Mike Roberts says it well: “fundamentally FaaS is about running back end code without managing your own server systems or your own server applications.” Now instead of just running the hardware for your deployments, cloud platforms will run the server side logic. As far as developers are concerned, there are no more servers – there are only functions.

Well, that’s not entirely true. Your applications will still need to deal with the realities of running on hardware – you’ll have downtime, errors, bugs, and cost management. There’s a startup called IOPipe that will help monitor and manage your FaaS for you. Algorithmia also offers it’s Serverless AI Layer—which automates away the need to deploy or manage infrastructure (it’s especially well suited for deploying AI/ML models).

Fowler lists a few more important features of serverless that separate it from traditional server side architecture:

Coding Frameworks

Since serverless functions run by themselves, they can trigger other functions written in different languages or leveraging different frameworks. As long as you define the required dependencies in your deployment package, you’re good.

Upload

The process of actually getting your code to run in the FaaS framework is super easy – you literally can copy and paste it into the service provider’s console. Once you adjust the other parameters (e.g. upload dependencies), you can just click deploy.

Scaling

This is probably the most important feature of serverless – the provider takes care of the complexity around scaling your functions. Theoretically, you could go from 10 API calls today to 15 Million tomorrow and be totally confident that your infrastructure would hold. There’s still some complexity to navigate around handling multiple copies of the same function running in parallel, but that’s written into the functions themselves. There’s nothing else to write it into.

To accomplish the same with current cloud computing, you’d use something like Amazon’s Elastic Load Balancing. You need to set up the logic, explicitly tell it when to spin up and shut down servers, and take care of the event handling. But by then, you’ve already built something like Lambda.

To summarize here, FaaS is basically a pre-built, easy to use combination of some of the services that most cloud providers already offer. The end product is an auto-scaled, easy-upload, and framework agnostic service where your functions run the show. The difference is that you don’t need to set most of it up, which can make a big difference for certain business use cases.

Business Use Cases of Serverless

It’s easy to understand why serverless can make it easier for you to deploy your applications without having to worry about coordinating multiple parts of what your cloud providers offers. But in today’s business world you pay for simplicity, and serverless’ cost increase needs to be worth it. In addition to the cost factor, FaaS platforms like Lambda also have compute limitations that make it a bad fit for larger scale projects. For example, disk space is temporary and limited to 512MB, and functions will timeout if taking more than 5 minutes to execute.

What all if this means is that serverless needs to be a good fit – it’s not realistic to port over and entire application and say “we’re running this here now.” That’s why most of the well documented business use cases for serverless rarely represent a core product function. For example, here are some of the examples on Amazon’s Lambda page for use cases:

None of these businesses are shifting their entire applications over to a serverless architecture, but many are starting to integrate it into smaller parts of their stack or workflow.

As far as technical use cases, serverless can be used for almost anything. A few popular examples include:

  • Operating serverless websites with functions as the serverless interface between the client browser and a database instance
  • Interfacing with 3rd Party services with a function handling the request
  • Scheduling events with cron, but only paying for execution time

Another excellent use case for serverless is for deploying Machine Learning, which is where Algorithmia’s scalable platform fits in. We’ll include more information on that below, and expect the suite of use cases to expand as these services proliferate and improve.

Serverless Works for More Than Just Computing

Another interesting and promising application of serverless is data storage. When you’re dealing with large amount of data, it gets technically challenging to maintain the right sized database and manage ETL processes and a data warehouse. Traditional database cloud services like Amazon’s RDS can also get prohibitively expensive – especially for early stage companies – when you scale a SQL based infrastructure to tens of millions of rows.

A few new managed databases services are relying on the serverless paradigm and promise the same benefits of FaaS in computing: we’ll take care of everything, and you worry about your business. Predictably, there are three major players in the serverless data space: Google, Amazon, and Microsoft. IBM’s Cloudant and Fauna are new entrants in the early stages.

Google’s offering is called BigQuery, and it’s a “serverless, highly scalable, low cost enterprise data warehouse designed to make all your data analysts productive.” Like with FaaS, you just upload your data and Google takes care of the rest. The pricing model of serverless data solutions charges both for storage and for querying, so you’ll be paying a premium for what you’re getting.

Amazon launched their Athena service in late 2016, and it’s a similar offering to Google. In their words, it’s “an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.” Unlike with BigQuery, you need to set up your data in a separate service (S3) first.

The major difference between these two options is how they’re architected. Google’s BigQuery is entirely managed, from data storage to query. It can be safely labeled as a serverless data warehouse. Athena, on the other hand, is a bit more abstracted – instead of pushing your data to the warehouse, you can just query directly from your sources in S3. The end product ends up accomplishing the same task, but if you’ve already been storing your data on S3 then Athena might be easier to onboard.

Bigger picture, both of these options are presented as data warehouses, and as such are only solutions for analysis. You’ll still need to worry about how to store your core data and deal with that set of complexities. Look out for serverless to make its way into cold storage soon as the technology improves.

Another Emerging Area: Machine Learning

A promising and nascent area where serverless can be a game-changer is deployment of Machine Learning and AI. Algorithmia’s platform deploys models as serverless microservices that are infinitely scalable and in a way that’s easy for developers to utilize. Machine Learning has some unique deployment challenges that make it a good candidate for serverless:

  • Data Scientists use a large variety of programming languages (Python, Java, SAS, R) and very little of their code design is optimized for deployment
  • Machine Learning models effectively need to run on GPUs in 2018, and the infrastructure for designing and coordinating scaled GPU compute is still an open question

Serverless platforms like Algorithmia make it easy to overcome these production issues by relying on a managed platform that takes care of scaling for you. Think of it as the Heroku (PaaS) for serverless AI.

Serverless, Containers, Microservices, and PaaS

There are a number of very important architectural shifts going on in the computing and storage infrastructure world right now, and it’s easy to get lost in the terminology. Here’s how some of these interlocking terms relate to each other

Idea Description Connection
Serverless / FaaS Run functions instead of servers – your cloud provider manages how they run. N/A
Containers Ship your code with dependencies attached in a self contained package. Run multiple of these on one VM. You can deploy your application with a container-based architecture, or let a provider do so with FaaS. Algorithmia uses a container architecture but offers a serverless solution.
Microservices A development philosophy where applications are built through self-contained components. A microservices focused architecture can be deployed on a serverless platform, a VM-based platform, and a container-based platform. This is a way to design applications and split them up.
Platform as a Service (PaaS) A more packaged version of traditional cloud computing where the vendor also manages things like the OS and databases (like Heroku). Serverless can be viewed as a type of PaaS, but PaaS usually requires you to worry about scaling. Serverless platforms spin up and shut down servers for every request, while PaaS providers usually don’t.

As the ecosystem develops, the relationships between these interlocking pieces will evolve. For more information about serverless and how to integrate it into your application stack, check out the resources below.

Serverless Vendors (Barebones infrastructure)

Vendors in serverless can be split into two broad categories: barebones infrastructure and managed platforms. Alluded to in the chart above, some would call the managed platforms PaaS and not serverless vendors. The difference is just how much of the process they take care of for you, and managed platforms do a lot more.

AWS Lambda – “AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume – there is no charge when your code is not running.

Google Cloud Functions – “Cloud computing has made possible fully serverless models of computing where logic can be spun up on-demand in response to events originating from anywhere. Construct applications from bite-sized business logic billed to the nearest 100 milliseconds, only while your code is running. Serve users from zero to planet-scale, all without managing any infrastructure.

IBM Cloud Functions – “Based on Apache OpenWhisk, IBM Cloud Functions is a polyglot functions-as-a-service (FaaS) programming platform for developing lightweight code that scalably executes on demand.

Microsoft Functions – “Take advantage of serverless compute with Functions. Easily build the apps you need using simple, serverless functions that scale to meet demand. Use the programming language of your choice, and don’t worry about servers or infrastructure.

Heroku for Serverless

Algorithmia – “The vast majority of Data Science and ML investments are being underutilized, and often produce little beyond business dashboards and PowerPoint slides. Algorithmia’s AI Layer powers the ML DevOps of the world’s largest organizations, as well as offering the same scalable solutions in the cloud to companies of all sizes.

Algorithmia takes the power of serverless to the next level by automating much of the devops process. You can learn more about the AI Layer here.

Explainers and Papers

Serverless Architectures (Martin Fowler) – “Serverless is a hot topic in the software architecture world. We’re already seeing books, open source frameworks, plenty of vendor products, and even a conference dedicated to the subject. But what is Serverless and why is (or isn’t) it worth considering? Through this evolving publication I hope to enlighten you a little on these questions.

Serverless Computing: Economic and Architectural Impact – “This paper presents two case industrial studies of early adopters [of serverless], showing how migrating an application to the Lambda deployment architecture reduced hosting costs – by between 66% and 95% – and discusses how further adoption of this trend might influence common software architecture design practices.”

The Comprehensive Guide to Serverless Architecture (Simform) – “What is ‘serverless’ and why you should be or shouldn’t be considering it? What are its benefits and drawbacks? What is FaaS? Who are all using serverless architecture? All these questions coming to your mind? Calm down! In this blog, we’ll discuss them all!

Serverless Computing: Current Trends and Open Problems – “Serverless computing has emerged as a new compelling paradigm for the deployment of applications and services. It represents an evolution of cloud programming models, abstractions, and platforms, and is a testament to the maturity and wide adoption of cloud technologies. In this chapter, we survey existing serverless platforms from industry, academia, and open source projects, identify key characteristics and use cases, and describe technical challenges and open problems.

Serverless Computation with OpenLambda – “We present OpenLambda, a new, open-source platform for building next-generation web services and applications in the burgeoning model of serverless computation. We describe the key aspects of serverless computation, and present numerous research challenges that must be addressed in the design and implementation of such systems. We also include a brief study of current web applications, so as to better motivate some aspects of serverless application construction.

Tutorials and Walkthroughs

Serverless Stack – “Serverless Stack is an open source guide for building and deploying full-stack apps using Serverless and React on AWS. Create a note taking app from scratch using the Serverless Framework and Create React App. Follow our step-by-step tutorials with screenshots and code samples.

A Serverless REST API in Minutes with the Serverless Framework – “The Serverless Framework (previously known as JAWS) debuted several months ago and has been rapidly maturing to help engineers build scalable applications without the hassle of maintaining servers. Let’s get a REST API up and running quickly with the Serverless Framework by using a starter/boilerplate project called the Serverless Starter.”

Serverless Framework Tutorial for Beginner using AWS Lambda – “Using AWS Lambda service, we can deploy any number of serverless applications and pay for only what is really used which is time to process the request. Let’s see first what Serverless applications are.

A Crash Course on Serverless With Node.js – “Regardless of your developer background, it’s inevitable you’ve heard the term Serverless in the past year. The word has been buzzing around in my ears for longer than I dare say. For too long have I been putting off looking into it in more detail. Well here goes nothing.”

AWS Tutorial: Intro to Using Lambda With the Serverless Framework – “Lately, I’ve been turning to AWS Lambda for building server-side logic – whether for client work, product development, or even personal projects. Lambda is a managed service, so there’s no need to patch or monitor servers. It’s pay-as-you-go, so you only get charged for usage, rather than uptime. And it’s elastic, so it scales up to handle enterprise level traffic, or shrinks to zero for those pet projects that never take off.

Publishing Your First Algorithm on Algorithmia

Courses and Videos

AWS Lambda and the Serverless Framework (Udemy) – “This course is designed for Developers, DevOps and Solution Architects who want a deep understanding about how to use Serverless Framework to deploy AWS Lambda functions. This course helps to learn how to develop, deploy and manage AWS Lambda functions easily with Serverless Framework with real world examples.

Serverless Concepts (Linux Academy) – “Serverless Concepts will teach you the basics of this hot new technology. You’ll learn about what constitutes a serverless application. You’ll watch how-to videos that show off services like AWS Lambda, Google Firebase, and more. You’ll be able to review note cards to remember key elements from the sessions.

Getting Started with Serverless Computing on AWS (Cloud Academy) – “In this group of lectures, we explore what serverless computing is and how “functions as a service” differs from traditional computing models.”

Serverless: the Future of Software Architecture – YouTube.