Sklearn How to Save a Model Created From a Pipeline and GridSearchCV Using Joblib or Pickle?

As a data scientist or software engineer, one of the most important tasks is to build models that can accurately predict the outcome of a given problem. However, building a model is just the first step. The next step is to save the model so that it can be used in the future. In this blog post, we will learn how to save a model created from a pipeline and GridSearchCV using Joblib or Pickle.

Table of Contents

1. Introduction to Scikit-Learn
2. What is a Pipeline?
3. What is GridSearchCV?
4. Saving a Model
5. Saving a Pipeline and GridSearchCV Model
6. Common Errors and Solutions
7. Conclusion

Introduction to Scikit-Learn

Scikit-Learn is an open-source machine learning library for Python. It is built on top of NumPy, SciPy, and matplotlib, and provides a simple and efficient tool for data mining and data analysis. Scikit-Learn is widely used in the industry and academia for building machine learning models.

What is a Pipeline?

A pipeline is a sequence of data processing components that are chained together. Each component in the pipeline takes the output of the previous component as input and performs some operation on it. Pipelines are commonly used in Scikit-Learn to automate the machine learning workflow.

What is GridSearchCV?

GridSearchCV is a technique used to find the best hyperparameters for a machine learning model. It is a brute-force approach that searches through a specified subset of the hyperparameter space to find the optimal hyperparameters.

Saving a Model

Once you have built a machine learning model, the next step is to save it so that it can be used in the future. Scikit-Learn provides two methods for saving a model: Joblib and Pickle.

Joblib

Joblib is a set of tools to provide lightweight pipelining in Python. It is particularly useful for big data and memory-intensive tasks. Joblib provides two functions for saving and loading models: dump and load.

To save a model using Joblib, you need to import the dump function from the joblib library and call the dump function with the model and the file name.

from sklearn.externals import joblib

joblib.dump(model, 'filename.pkl')

To load the saved model, you need to import the load function from the joblib library and call the load function with the file name.

from sklearn.externals import joblib

model = joblib.load('filename.pkl')

Pickle

Pickle is a Python module used for serializing and de-serializing Python objects. It can be used to save and load machine learning models.

To save a model using Pickle, you need to import the pickle module and call the dump function with the model and the file name.

import pickle

with open('filename.pkl', 'wb') as f:
    pickle.dump(model, f)

To load the saved model, you need to import the pickle module and call the load function with the file name.

import pickle

with open('filename.pkl', 'rb') as f:
    model = pickle.load(f)

Saving a Pipeline and GridSearchCV Model

Saving a pipeline and GridSearchCV model is slightly different from saving a regular model. You need to save the entire pipeline object, including the GridSearchCV object and the model object. To do this, you can use either Joblib or Pickle.

Let’s start by building a machine learning model using Scikit-learn’s pipeline and GridSearchCV. This combination allows us to efficiently explore a hyperparameter search space and encapsulate preprocessing steps.

# Import necessary libraries
from sklearn.datasets import load_iris
from sklearn.model_selection import GridSearchCV, train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

# Load dataset
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2, random_state=42)

# Create a pipeline with preprocessing and classifier
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('classifier', RandomForestClassifier())
])

# Define hyperparameter grid for GridSearchCV
param_grid = {
    'classifier__n_estimators': [50, 100, 200],
    'classifier__max_depth': [None, 10, 20, 30]
}

# Create GridSearchCV object
grid_search = GridSearchCV(pipeline, param_grid, cv=5)
grid_search.fit(X_train, y_train)

Saving a Pipeline and GridSearchCV Model using Joblib

To save a pipeline and GridSearchCV model using Joblib, you need to import the dump function from the joblib library and call the dump function with the pipeline object and the file name.

# Import Joblib
import joblib

# Save the model to a file
joblib.dump(grid_search, 'model.joblib')

To load the saved pipeline and GridSearchCV model, you need to import the load function from the joblib library and call the load function with the file name.

# Load the model using Joblib
loaded_model_joblib = joblib.load('model.joblib')

Saving a Pipeline and GridSearchCV Model using Pickle

To save a pipeline and GridSearchCV model using Pickle, you need to import the pickle module and call the dump function with the pipeline object and the file name.

# Import Pickle
import pickle

# Save the model to a file
with open('model.pkl', 'wb') as file:
    pickle.dump(grid_search, file)

To load the saved pipeline and GridSearchCV model, you need to import the pickle module and call the load function with the file name.

# Load the model using Pickle
with open('model.pkl', 'rb') as file:
    loaded_model_pickle = pickle.load(file)

Prediction Step

Now, let’s perform predictions using the loaded models:

# Make predictions using the loaded models
predictions_joblib = loaded_model_joblib.predict(X_test)
predictions_pickle = loaded_model_pickle.predict(X_test)

print("Predictions Joblibs: ", predictions_joblib)
print("-------------")
print("Predictions Pickle: ", predictions_pickle)

Output:

Predictions Joblibs:  [1 0 2 1 1 0 1 2 1 1 2 0 0 0 0 1 2 1 1 2 0 2 0 2 2 2 2 2 0 0]
-------------
Predictions Pickle:  [1 0 2 1 1 0 1 2 1 1 2 0 0 0 0 1 2 1 1 2 0 2 0 2 2 2 2 2 0 0]

Common Errors and Solutions

Error 1: AttributeError: Can't get attribute 'function' on <module '__main__' (built-in)>

This error occurs when trying to load a model with custom functions.

Solution: Define custom functions in a separate module.

# Save custom functions in a module named custom_functions.py
# Then, in the main script, import and use them
from custom_functions import custom_function

Error 2: ModuleNotFoundError: No module named 'module_name'

Occurs when trying to load a model with missing dependencies.

Solution: Ensure all required modules are installed.

pip install module_name

Error 3: ValueError: Buffer dtype mismatch, expected 'INT_TYPE' but got 'INT_TYPE_ANOTHER'

This error may arise due to inconsistent NumPy versions.

Solution: Use the same NumPy version when saving and loading the model.

pip install numpy==<version>

Conclusion

In this blog post, we learned how to save a model created from a pipeline and GridSearchCV using Joblib or Pickle. Saving a model is an important task in machine learning, and it is essential to know how to do it. By saving a model, you can reuse it in the future, which can save a lot of time and effort.

SHARE:

About Saturn Cloud

Saturn Cloud is your all-in-one solution for data science & ML development, deployment, and data pipelines in the cloud. Spin up a notebook with 4TB of RAM, add a GPU, connect to a distributed cluster of workers, and more. Request a demo today to learn more.