Automating Model Training and Model Deployment

4 min readNov 16, 2022

Automated pipelines are something that a lot of individuals and companies are using these days. These pipelines based on certain rules and instructions can do things without or a very minimal Human Interference.

These automated pipelines can be anything. For Example, Training a model and deploying that trained model so that it can be used for various business applications. The pipelines can have time-based triggers or manual triggers or they can get triggered when a new dataset version is released, etc.

In Today’s blog, we are going to create an automated pipeline that when triggered will train an ML Model and then deploy it as a Live Demo as a gradio App.

Let’s begin

Directory Structure

ROOT
- deploy.py # For Job Deployment
- service.py # For Model Training and Service Deployment
- model.py # Source code for model
- app.py # Gradio Source Code (gets created in run time)
- requirements.txt # requirements

Creating our Pipeline

Here, we will a series of functions that will help us to implement this pipeline

1. Deploy.py

This file is responsible for tracking the experiment and deploying the triggerable job.

2. Model.py

Here, we are defining the source code for our model. The model that we are using is Random Forest Regressor from My own AutoML named LuciferML. Here we are training the model on salary_data which is a minimal dataset that I’ve chosen for this blog, you can use any model and dataset as per your choice and make sure to return the trained model, features, and labels when this function is called.

3. Service.py

This file is responsible for training and deploying the model and creating gradio source code (app.py)

Our Pipeline finishes here, therefore we will move to the deployment stage

Deployment

Now as we have created the pipeline we will deploy the Triggerable job on TrueFoundry.

We can deploy it in two ways one is CLI and the other is TrueFoundry WebApp.

Deployment through CLI

Use this command in your terminal and sit back it will automatically deploy the job for you which will Trigger every day at 9:00 am, and train as well as deploy the model as a Service on TrueFoundry.

python deploy.py

Deployment through WebApp

Logging into TrueFoundry

Heading to Deployment Section

1) Creating a new deployment

2) Select the Job option and Workspace name

3) Fill out properties and submit

Make sure the code is already in the repository. Here, we will schedule the job to run at 9:00 AM. Do not forget to enter the values for your Environmental variables.

4) Deploying

5) Successful Deployment

6) Trigger the Job

You can wait till 9:00 am for the job to be triggered but I can’t so I will manually trigger the job and it will train the model and create a new service. After the job is done it will make a new Service named “trained-rfr-from-job” and deploy the trained model.