Algorithmia Blog - Deploying AI at scale

Why a multi-cloud infrastructure is an important part of application and Machine Learning deployment

Source: Forgeahead

Multi-cloud is quickly becoming the de facto strategy for large companies looking to diversify their IT efforts. At Algorithmia, we deploy across multiple clouds and recommend it for Machine Learning pipelines and portfolios. This post will outline the pros and cons of a multi-cloud architecture, as well as its applicability to Machine Learning workloads.

The first thing to understand about this emerging strategy is that it’s very popular. Almost 80% of enterprises that utilize public clouds use two or more of them, and 44% of those enterprises use 3 or more. Overall, 61% of all enterprises surveyed here are using two or more cloud providers.


Benefits of Multi-Cloud

When you’re deploying an application or a Machine Learning model, you’re always going to need to deal with conflicting priorities. Speed, accuracy, uptime, and cost are all parts of the equation. And while in the past technology companies had no choice but to develop their own data centers, in 2018 there’s a broad selection of public cloud platforms available. You can take advantage of multiple ones to balance these competing values.

1) Prevent downtime

Despite the overall reliability of public cloud providers like Amazon Web Services, they still have a high likelihood of experiencing significant downtime at some point during the year.

Your customers depend on your application running 24/7, so diversifying that downtime risk across multiple clouds can ensure that your software will always be running somewhere.

2) Avoid vendor lock-in

One of the major lessons businesses learned from deploying internet infrastructure over the past few decades is how nasty it can be to depend on one vendor. IT managers are concerned with the potential pitfalls of committing to one cloud provider: the more services you use, the harder it is to transfer; and a small price hike or feature change could doom your applications. Deploying applications across multiple clouds can help avoid becoming too reliant on one platform.

3) Price competition

Different public cloud providers offer distinct pricing models and tiers. Owing to the fact that every application is unique, that enables potential pricing fits that can lower your overall cost. For example: if your application has heavy database use and heavy compute use, you might want to host most of your database instances on a cloud where data storage is cheaper, and most of your compute use on a cloud where compute is cheaper.

4) Feature variety

As with any competing set of product offerings, public clouds seek to differentiate themselves through special functionality. For example: Amazon Web Services boasts interoperability and modular architecture, while Google’s Cloud Platform has the fastest chips for Machine Learning around (sort of). Diversifying across multiple cloud providers can give you access to all of these product benefits instead of just one set.

5) Location diversity

Public cloud providers locate their physical data centers all over the world. For applications where low or no latency is a key feature, you can optimize for lightning fast speed by using a cloud provider with data centers close to your clients. Engineers at High Frequency Trading funds have been doing this for years already, and have proven that it works.

6) Data sovereignty / compliance

Compliance has been making it into the more public headlines these days as new GDPR regulations fill our inboxes with Terms of Service changes. There are vastly different regulations surrounding data across the world: Europe is generally more strict with user data than the U.S. is, among other complexities. Given that public cloud providers all have different ways of dealing with these regulations, some may be a better fit in different geographies.

Source: TechTarget


Challenges with Implementing Multi-Cloud

Multi-cloud isn’t all rosy: there are some downsides, and a plethora of complexity to deal with. When deciding on your personal architecture, balance these with the benefits and your goals.

1) Usage based pricing

Public cloud typically becomes more expensive than operating on-premise over time, but volume discounts can soften that blow. By diversifying your application deployment across multiple clouds, you’ll lose some of those pricing benefits if you don’t have enough on each individual service. For seriously large apps though, this likely won’t be a problem.

2) Cross-cloud expertise needed

Public clouds are all built very differently, and there’s a learning curve for deploying on each one. Your IT teams are going to need to understand the nuances of each provider and how they fit in with your application and goals.

3) Information flow

When you’re deployed across multiple clouds, they’ll need to be interacting with each other and sharing information. Your team is going to need to coordinate permissions, connect parts of your application that need to talk, and ensure that scaling doesn’t lead to messed up communication between parts.


Multi-Cloud and Machine Learning

Machine Learning is a type of software, but it introduces new constraints and goals into how products are built. A lot of the promise in multi-cloud for Machine Learning isn’t across vendors, but across cloud types. Here are some use cases where combining public and private clouds, as well as multiple cloud vendors, can help with Machine Learning deployment:

  • Sensitive customer data: train with on-prem data and deploy on public cloud
  • Legacy infrastructure: train on public cloud with specialized hardware (GPUs), deploy on systems of record
  • Public datasets: train on public cloud with public data, and deploy on-prem (algorithmic trading)

At Algorithmia, we understand why multi-cloud is important and how it can be the right decision for many businesses. We support deployments across multiple clouds and on-prem for our enterprise offering.

For more information on the Algorithmia platform and deploying Machine Learning at scale, check out our whitepaper here or set up a demo.

Further Reading

What is multicloud? The next step in cloud computing (InfoWorld) – “We live in a world where we like to name things. In the case of cloud computing, the names revolve around patterns of use: public cloud, private cloud, and hybrid cloud. Now there’s a new term, multicloud, for an emerging pattern of use for cloud computing. Multicloud means using more than a single public cloud. That usage pattern arose when enterprises tried to avoid dependence on a single public cloud provider, when they chose specific services from each public cloud to get the best of each, or when they wanted both benefits.

What is multicloud? (RedHat) – “Multicloud is a cloud approach made up of more than 1 cloud service, from more than 1 cloud vendor—public or private.

Organizations surprised at technical challenges of deploying multi-cloud, VMware finds (CloudTech) – “Multi-cloud continues to be a major trend as we move into 2018 – and according to a new research report from VMware, organizations need to overcome the inherent technical and skill challenges of multi-cloud deployments for success. Over half (57%) of the more than 1,300 global IT decision makers in large enterprises polled said that technical challenges, as well as the demand for new skills and staff, were unexpected, critical learnings from managing multiple clouds.”

3 Scenarios for Machine Learning on Multicloud (Inside Machine Learning) – “As machine learning (ML) continues to pervade enterprise environments, we need to understand how to make ML practical on multicloud — including those architectures that span the firewall. Let’s look at three possible scenarios.