All posts in Blog Posts

Data Scientists and Deploying Models From any Framework

any framework

Asking a data scientist to work with only one framework is like asking a carpenter to work with only a hammer. It’s essential that professionals have access to all the right tools for the job.

It’s time to rethink best practices for leveraging and building ML infrastructure and set the precedent that data scientists should be able to use whichever tools they need at any time.

Now, certainly some ML frameworks are better suited to solve specific problems or perform specific tasks, and as projects become more complex, being able to work across multiple frameworks and with various tools will be paramount.  

For now, machine learning is still in its pioneering days, and though tech behemoths have created novel approaches to ML (Google’s TensorFlow, Amazon’s SageMaker, Uber’s Michelangelo), most ML infrastructure is still immature or inflexible at best, which severely limits data scientists and DevOps. This should change.

Flexible frameworks and ML investment

Most companies don’t have dozens of systems engineers who can devote several years to building and maintaining their own custom ML infrastructure or learning to work within new frameworks, and sometimes, open-source models are only available in a specific framework. This could restrict some ML teams from using them if the models don’t work with their pre-existing infrastructure. Companies can, and should, have the freedom to work concurrently across all frameworks. The benefits of doing so are multifold:

Increase interoperability of ML teams

Often machine learning is conducted in different parts of an organization by data scientists seeking to automate their processes. These silos are not collaborating with other teams doing similar work. Being able to blend teams together while still retaining the merits of their individual work is key. It will de-duplicate efforts as ML work becomes more transparent within an organization.

Allow for vendor flexibility and pipelining

You don’t want to end up locked-in to only one framework or only one cloud provider. The best framework for a specific task today may be overtaken next month or next year by a better product, and businesses should be able to scale and adapt as they grow. Pipelining different frameworks together creates the environment for using the best tools.

any framework

Reduce the time from model training to deployment

Data scientists write models in the framework they know best and hand them over to DevOps, who rewrite them to work within their infrastructure. Not only does this usually decrease the quality of a model, it creates a huge iteration delay.

Enable collaboration and prevent wasted resources

If a data scientist is accustomed to PyTorch but her colleague has only used TensorFlow, a platform that supports both people’s work saves times and money. Forcing work to be done with tools that aren’t optimal for a given project is like showing up with knives to a gunfight.

Leverage off-the-shelf products

There’s no need to constantly reinvent the wheel; if an existing open-source service or dataset can do the job, then that becomes the right tool.

Position your team for future innovations

Because the ML story is far from complete, being flexible now will enable a company to pivot more easily come whatever tech developments arise.

How to deploy models in any framework

Attaining framework flexibility, however, is no small feat. The main steps to enable deploying ML models from multiple frameworks are as follows:

Dependency management

It’s fairly simple to run a variety of frameworks on a laptop, but trying to productionize them requires a way to manage all dependencies for running each model, in addition to interfacing with other tools to manage compute.

Containerization and orchestration

Putting a model in a container is straightforward. When companies have only a handful of models, they often task junior engineers with manually containerizing, putting the models into production, and managing scale-up. This process unravels as usage increases, model numbers grow, and as multiple versions of models run in parallel to serve various applications.

Many companies are using Kubernetes to orchestrate containers—there are a variety of open-source projects on components that will do some of the drudge work of machine learning for container orchestration. Teams who have attempted to do this in-house have found that it requires constant maintenance and becomes a Frankenstein of modular components and spaghetti code that falls over when trying to scale. Worse still, after models are in production, you discover that Kubernetes doesn’t deal well with many machine learning use cases.

API creation and management

Handling many frameworks requires a disciplined API design and seamless management practice. When data scientists begin to work faster, a growing portfolio of models with an ever-increasing number of versions needs to be continuously managed, and that can be difficult.

Languages and DevOps

Machine learning has vastly different requirements than traditional compute, including the freedom to support models written in many languages. A problem arises, however, when data scientists working in R or Python sync with DevOps teams who then need to rewrite or wrap the models to work in the language of their current infrastructure.

Down the road

Eventually, every company that wants to extend its capabilities with ML is going to have to choose between enabling a multi-framework solution like the AI Layer or undertaking a massive, ongoing investment in building and maintaining an in-house DevOps platform for machine learning.

We Run the World’s Machine Learning. Literally.

Satellite imagery

Algorithmia’s AI Layer Powers the UN Methods Service

Algorithmia is renewing its commitment to global humanitarian efforts by making powerful ML tools available to everyone.

Economic and population data are key elements of planning and decision-making in first-world countries, but access to sophisticated analytic and compute power is limited or non-existent in developing countries.

Meeting the Problem Head On

Working in conjunction with the United Nations Global Platform for Official Statistics, Algorithmia built a repository of algorithms that are readily available to any data scientist of any member state at any time. There are models for predicting economic, environmental, and social trends to enable smarter decision-making for strategies like agricultural planning, flooding probabilities, and curbing deforestation.

The United Nations Global Platform for Official Statistics sought to build the algorithm repository as part of the Sustainable Development Goals (SDGs). The SDGs aim to meet global challenges in healthcare, poverty, environmental degradation, and inequality by 2030. The algorithm repository will serve member states to “establish strategies to reuse and adapt algorithms across topics and to build implementations for large volumes of data.” UN Big Data

Building an Algorithm Marketplace for the Developing World

The UN wanted a way to share models with underdeveloped countries to curate economic, environmental, and social data to save lives and improve health and environmental conditions. Using the UN algorithm repository, for example, a developing country could model farmland satellite imagery to predict draughts, urbanization trends, or migration patterns.

Such statistical information can be used in myriad ways by both humanitarian organizations and policy-making, governmental bodies to make smarter resource-allocation decisions, better understand urban planning needs from population data, and even predict migration crop cycles using geospatial imagery.

The UN’s partnership with Algorithmia demonstrates our dedication to leveraging AI and machine learning to seek solutions to global problems. We are so looking forward to empowering the developing world, one algorithm at a time.

Read the Case Study

Navigating the Machine Learning Roadmap

Roadmap whitepaper cover

Machine learning (ML) will drastically alter how many industries operate in the future. Natural language processing will enable seamless and instantaneous language translation, forecasting algorithms will help predict environmental trends, and computer vision will revolutionize the driverless car industry.

Nearly all companies that have initiated ML programs have encountered challenges or roadblocks in their development. Despite efforts to move toward building robust ML programs, most companies are still at nascent stages of building sophisticated infrastructure to productionize ML models.

Roadmap Overview

After surveying hundreds of companies, Algorithmia has developed a roadmap that outlines the main stages of building a robust ML program as well as tips for avoiding common ML pitfalls. We hope this roadmap can be a guide that companies can use to position themselves for ML maturity. Keep in mind, the route to building a sophisticated ML program will vary by company and team and require flexibility.

Using the Roadmap

Every company or team is situated at a different maturity level in each stage. After locating your current position on the roadmap, we suggest the following:

  • Chart your path to maturity
  • Orient and align stakeholders
  • Navigate common pitfalls

The roadmap comprises four stages: Data, Training, Deployment, and Management. The stages build on one another but could also occur concurrently in some instances.

Data: Developing and maintaining secure, clean data
Training: Using structured datasets to train models
Deployment: Feeding applications, pipelining models, or generating reports.
*Models begin to generate value at this stage.*
Management: Continuously tuning models to ensure optimal performance

Pinpointing Your Location on Algorithmia’s Roadmap

ML Roadmap

At each stage, the roadmap charts three variables to gauge ML maturity: people, tools, and operations. These variables develop further at every stage as an ML program becomes more sophisticated.

For more information about building a sophisticated machine learning program and to use the roadmap, read our whitepaper, The Roadmap to Machine Learning Maturity.

Download Roadmap

Data Providers Added for Algorithmia Users

Algorithmia data providers list

Azure Blob and Google Cloud Storage

In an effort to constantly improve products for our customers, this month we introduced two additional data providers into Algorithmia’s data abstraction service: Azure Blob Storage and Google Cloud Storage. This update allows algorithm developers to read and write data without worrying about the underlying data source. Additionally, developers who consume algorithms never need to worry about passing sensitive credentials to an algorithm since Algorithmia securely brokers the connection for them.

How Easy is it?

By creating an Algorithmia account, you automatically have access to our Hosted Data Source where you can store your data or algorithm output. If you have a Dropbox, Azure Blob Storage, Google Cloud Storage, or an Amazon S3 account, you can configure a new data source to permit Algorithmia to read and write files on your behalf. All data sources have a protocol and a label that you will use to reference your data.

We create these labels because you may want to add multiple connections to the same data provider account and they will each need a unique label for later reference in your algorithm. You might want to have multiple connections to the same source so you can set different access permissions to each connection, such as read from one file and write to a different folder.

We’re Flexible

These providers are available now in addition to Amazon S3, Dropbox, and the Algorithmia Hosted Data service. These options will provide our users with even more flexibility when incorporating Algorithmia’s services into their infrastructures.

Learn more about how Algorithmia enables data connection on our site.

We’d love to know which other data providers developers are interested in, and we’ll keep shipping new providers in future releases. Get in touch if you have suggestions!

Finding an Algorithm Marketplace by the Marketplace

Pike Place Market

Sometimes the best advertising is a small, nondescript company name etched onto an equally nondescript door in a back alley, only accessible by foot traffic. Lucky for us, Paul Borza of TalentSort—a recruiting search engine that mines open-source code and ranks software engineers based on their skills—was curious about Algorithmia when he happened to walk by our office near Pike Place Market one day.

“It’s funny how I stumbled on Algorithmia. I was waiting for a friend of mine in front of
The Pink Door, but my friend was late so I started walking around. Next door I noticed a cool logo and the name ‘Algorithmia.’ Working in tech, I thought it must be a startup so I looked up the name and learned that Algorithmia was building an AI marketplace. It was such a coincidence!”

Paul Needed an Algorithm Marketplace

“Two weeks before I had tried monetizing my APIs on AWS but had given up because it was too cumbersome. So rather than waste my time with bad development experiences, I was willing to wait for someone else to develop a proper AI marketplace; then I stumbled upon Algorithmia.”

Paul Found Algorithmia

“I went home that day and in a few hours I managed to publish two of my machine learning models on Algorithmia. It was such a breeze! Publishing something similar on AWS would have taken at least a week.”

We asked Paul what made his experience using Algorithmia’s marketplace so easy:

“Before I started publishing algorithms, I wanted to see if Algorthmia fit our company’s needs. The “Run an Example” feature was helpful in assessing the quality of an algorithm on the website; no code required. I loved the experience as a potential customer.”

“To create an API, I started the process on the Algorithmia website. Each API has its own git repository with some initial boilerplate code. I cloned that repository and added my code to the empty function that was part of the boilerplate code, and that was it! The algorithm was up and running on the Algorithmia platform. Then I added a description, a default JSON example, and documentation via Markdown.”

“The beauty of Algorithmia is that as a developer, you only care about the code. And that’s what I wanted to focus on: the code, not the customer sign-up or billing process. And Algorithmia allowed me to do that.”

Paul is Smart; Be like Paul

Paul’s algorithms are the building blocks of TalentSort; they enable customers to improve their recruiting efficiency. The models are trained on 1.5 million names from more than 30 countries and have an accuracy rate of more than 95 percent at determining country of origin and gender. Also, the algorithms don’t call into any other external service, so there’s no data leakage. Try them out in the Algorithmia marketplace today:

Gender By Name

Origin By Name

Paul’s relentless curiosity led him to Algorithmia’s marketplace where his tools became part of more than 7,000 unique algorithms available for use now.