How to do machine learning without an army of data scientists

3 years ago 291

Commentary: Machine learning is still harder than it needs to be. The open-source tool ModelDB and the ML model management platform Verta can help.

Jennifer Flynn had a problem. Shortly after joining LeadCrunch as a senior data scientist, she wanted to push out one small update of the company's software, which uses machine learning to find sales leads for its business customers. The problem? The data science team consisted of just five engineers, including her. That simple update took days and required help from the company's product development team, too.

"It wasn't tenable," Flynn said, now LeadCrunch's principal data scientist. "We wanted to do major overhauls of our models, but just putting one small update out there was a major pain point for us."

The artificial intelligence/machine learning software development and deployment lifecycle is still very nascent. The challenge of moving models into production is exacerbated by a demand for speed and a shortage of qualified ML engineers. But there's hope that things may soon get better.

There's a need for MLOps

We're still early enough in ML that it lacks the mature tooling and workflow processes of traditional software development. There, concepts like agile development and continuous integration and continuous deployment let entrenched companies and scrappy startups push new features to market quickly.

While AI is embedding itself into the products and processes of virtually every industry, deploying models into production remains a headache. Data scientists struggle to keep track of which version of a machine-learning model works best—a problem that grows when multiple models are involved—and even when a model is deployed, companies often have nothing in place to monitor its performance.

Something's got to give ... and it has.

SEE: Implementing DevOps: A guide for IT pros (free PDF) (TechRepublic)

A new crop of platforms and tools are sprouting, defined loosely as MLOps, something I've discussed before. MLOps stands for machine learning operations and is itself a derivative of DevOps, or software development and IT operations. Open-source versioning tool ModelDB, created by Manasi Vartak, a Ph.D. in computer science from MIT, is helping many organizations get started in ML.

Many machine learning solutions are actually assemblies of models. They run several models to get one prediction. Then, after all that, data scientists need to monitor model performance, retrain when needed and redeploy.

There are two key aspects to monitoring. The first is tracking CPU utilization and other metrics similar to traditional software. But the second aspect is different. It looks at what's happening to the data, how the distribution is changing as new data is acquired and how this distribution will affect the model's ability to make predictions.

Investing in an MLOps future

Intel Capital is among a group of savvy strategic investors who are placing big bets on ML/AI. The world's largest chipmaker, under new CEO Pat Gelsinger, sees its future in the spread of ML and AI. The more ML and AI workloads that run, the more chips Intel can sell or white label for others. Ecosystems around AI/ML accelerate chip demand.

Among the MLOps companies Intel Capital has invested in is Verta, founded by Vartak. Verta is an ML model management platform that tracks versions of models and data, can run multiple experiments simultaneously to find the best performing data-model combination and monitors those models and the data once they are deployed.

LeadCrunch, ranked number two in the advertising and marketing category of the Inc. 500 list of fastest-growing private companies, tried using an open-source tool but felt it wasn't robust enough. "We couldn't search through them, and we couldn't collate them, and we couldn't compare them easily," Flynn noted. Verta, however, seemed promising. "It's really a productivity tool for us," Flynn said. "This was something we could drop in underneath our workflow and do the stuff we were trying to do much faster and more reliably without having to build it ourselves."

SEE: Digital transformation: A CXO's guide (free PDF) (TechRepublic)

Vartak created Verta to commercialize ModelDB, which helps data scientists make sense of their ML models. As data scientists develop machine-learning models, they go through many iterations and often test multiple iterations simultaneously. They need a way to track those models, how they have changed and how the data used to train them has changed. ModelDB solves the problem by registering each version of the model and each version of the data, saving the metadata for reproducibility and for troubleshooting later on.

Then, to go into production, Verta packages the data efficiently in a container with all the dependencies. Data scientists are not experts in registering models or data; they are also not experts in building containers or putting things inside containers and making sure that they run on various platforms. Then, after a model is deployed, data scientists are not experts at scaling it up and down–Verta takes care of that.

Before Verta, it took three senior team members from the data science and development teams to get a model update deployed, Flynn said. That bottleneck allowed for only one or two major model upgrades per year. After using Verta, the company pushes major upgrades monthly–and that job is handled by the data science team's most junior member.

"We went from needing about 20 years of development experience to handle the necessary bushwhacking to someone with less than a year of development experience handling deployment alone," Flynn said. "We now have time to make even better models to better serve our customers." It's the sort of thing that Verta and similar open-source tools hope to conquer to make data science more accessible.

Disclosure: I work for AWS, but the views expressed herein are mine.