How to Convert a Tensorflow Frozen Graph to SavedModel

As a data scientist or software engineer, you may have come across the need to convert a Tensorflow frozen graph to a SavedModel. This can be necessary when deploying machine learning models in production environments, where SavedModel is the recommended format for serving models.

In this article, we will explain what frozen graphs and SavedModel are and provide a step-by-step guide on how to convert a Tensorflow frozen graph to SavedModel.

Table of Contents

1. What are Frozen Graphs and SavedModel?
2. Converting a Tensorflow Frozen Graph to SavedModel
3. Common Errors and How to Handle Them
4. Pros and Cons
5. Conclusion

What are Frozen Graphs and SavedModel?

Frozen Graphs

In Tensorflow, a frozen graph is a serialized version of a Tensorflow graph that contains both the graph definition and the trained variables. The graph definition describes the structure of the model, while the trained variables contain the weights and biases learned during the training process.

Freezing the graph is an optimization technique that simplifies the graph by removing unnecessary nodes and operations, and freezing the variables by converting them into constants. This reduces the size of the graph and speeds up inference, making it more suitable for deployment on production systems.

SavedModel

SavedModel is a format for saving and loading machine learning models in Tensorflow. It provides a standardized way to package and serve models, making it easier to share and deploy models across different platforms and environments.

A SavedModel contains the graph definition, the weights and biases, and metadata about the model, such as the input and output signatures. It also includes information about how to restore the model from a checkpoint, making it easier to train and fine-tune the model in the future.

Converting a Tensorflow Frozen Graph to SavedModel

Converting a Tensorflow frozen graph to SavedModel involves two main steps:

1. Loading the frozen graph
2. Saving the graph as a SavedModel

Step 1: Loading the Frozen Graph

To load the frozen graph, we first need to create a GraphDef object from the frozen graph file. The GraphDef object contains the serialized graph definition, which we can use to create a Graph object.

import tensorflow as tf

# Path to frozen graph file
frozen_graph_path = "/path/to/frozen_graph.pb"

# Load frozen graph
with tf.io.gfile.GFile(frozen_graph_path, "rb") as f:
    graph_def = tf.compat.v1.GraphDef()
    graph_def.ParseFromString(f.read())

# Create graph from graph_def
with tf.Graph().as_default() as graph:
    tf.import_graph_def(graph_def, name="")

In the above code, we use the tf.io.gfile.GFile function to read the frozen graph file and create a GraphDef object from it. We then create a new Graph object and import the GraphDef into it using the tf.import_graph_def function.

Step 2: Saving the Graph as a SavedModel

Once we have loaded the frozen graph into a Graph object, we can save it as a SavedModel using the tf.saved_model.builder.SavedModelBuilder class.

# Path to SavedModel directory
saved_model_path = "/path/to/saved_model"

# Create SavedModel builder
builder = tf.compat.v1.saved_model.builder.SavedModelBuilder(saved_model_path)

# Define input and output signatures
inputs = {
    "input_tensor": tf.compat.v1.saved_model.utils.build_tensor_info(graph.get_tensor_by_name("input_tensor:0"))
}

outputs = {
    "output_tensor": tf.compat.v1.saved_model.utils.build_tensor_info(graph.get_tensor_by_name("output_tensor:0"))
}

# Define signature definition
signature_def = tf.compat.v1.saved_model.signature_def_utils.build_signature_def(
    inputs=inputs,
    outputs=outputs,
    method_name=tf.compat.v1.saved_model.signature_constants.PREDICT_METHOD_NAME
)

# Add graph to SavedModel
builder.add_meta_graph_and_variables(
    sess=tf.compat.v1.Session(),
    tags=[tf.compat.v1.saved_model.tag_constants.SERVING],
    signature_def_map={
        tf.compat.v1.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature_def
    }
)

# Save SavedModel
builder.save()

In the above code, we create a SavedModelBuilder object and define the input and output signatures of the model. We then create a signature definition using the tf.saved_model.signature_def_utils.build_signature_def function.

Finally, we add the graph to the SavedModel using the add_meta_graph_and_variables method of the SavedModelBuilder class, passing in the signature definition and the session used to create the Graph object. We then save the SavedModel using the save method.

Common Errors and How to Handle Them:

GraphDef Parse Error:

If you encounter issues parsing the GraphDef, ensure that you are using the correct TensorFlow version. TensorFlow 2.x users may need to utilize the tf.compat.v1 module for compatibility.

with tf.compat.v1.Session(graph=graph) as sess:

Tensor Name Mismatch:

Double-check and verify that the input and output tensor names specified during the SavedModel conversion match those defined in your original frozen graph.

input_tensor_name = 'input_tensor_name:0'
output_tensor_name = 'output_tensor_name:0'

Node Not Found Error:

The specified input or output node is not found in the graph. To solve this, it’s crucial to correctly identify and reference the input and output nodes when converting the graph. Follow these steps to handle this error:

• Use TensorBoard: TensorBoard is a powerful tool for visualizing TensorFlow graphs. You can use it to inspect the nodes and their names in your frozen graph. Run the following command to launch TensorBoard:

tensorboard --logdir=path/to/log_directory

Open your web browser and navigate to http://localhost:6006 to explore the graph structure.

• Inspect the Graph Using tf.compat.v1.graph_util.extract_sub_graph: The tf.compat.v1.graph_util.extract_sub_graph function allows you to extract a subgraph containing specified nodes. Use it to inspect the graph and verify the presence and correctness of the input and output nodes:

subgraph_def = tf.compat.v1.graph_util.extract_sub_graph(graph_def, ["input_tensor", "output_tensor"])

Unsupported Ops Error:

This error means that the graph contains operations that are not supported in TensorFlow Serving. This is because TensorFlow Serving has specific requirements for the types of operations it supports. To address this issue, follow these steps:

• Identify Unsupported Ops: Use TensorFlow’s tf.compat.v1.graph_util.list_operations to list all operations in the graph. Identify any operations that are not compatible with TensorFlow Serving.

ops = tf.compat.v1.graph_util.list_operations(graph_def)

• Remove or Replace Unsupported Ops: Remove or replace the unsupported operations with equivalent operations compatible with TensorFlow Serving. You may need to modify the original model or use tools like tf.compat.v1.graph_transforms to apply transformations.

transformed_graph_def = tf.compat.v1.graph_transforms.TransformGraph(graph_def, ["input_tensor"], ["output_tensor"], ["remove_unsupported_ops"])

Session Initialization Error:

If variables are not initialized in the session, this error will appear. To handle this issue, ensure that all necessary variables are properly initialized before converting the graph. Follow these steps:

• Add Initialization Ops: If your model contains variables, add initialization operations to your session before running the conversion process.

with tf.compat.v1.Session(graph=graph) as sess:
    # Initialize variables
    sess.run(tf.compat.v1.global_variables_initializer())

SavedModel Already Exists Error:

This error occurs when you attempt to save a model to a directory that already contains a SavedModel. To avoid conflicts, choose a new directory for saving your model or remove the existing SavedModel. Follow these steps:

• Choose a New Directory: Specify a new directory path when creating the SavedModel to avoid overwriting existing models.

saved_model_dir = "path/to/new_saved_model"

• Remove Existing SavedModel: If you intend to use the same directory, manually delete the existing SavedModel or use code to handle this:

# Remove existing SavedModel directory
import shutil
shutil.rmtree("path/to/existing_saved_model")

Pros and Cons of using SavedModel:

Pros:

• Versioning and Flexibility: SavedModel format supports versioning, allowing you to easily manage and deploy multiple versions of your model simultaneously.

• Ease of Deployment: SavedModel is the recommended format for deploying models with TensorFlow Serving, making it simpler to integrate your models into production environments.

• Consistency Across Platforms: SavedModel ensures consistent behavior across different platforms and TensorFlow versions, reducing potential compatibility issues.

Cons:

• Size Increase: SavedModel files tend to be larger than frozen graphs due to additional metadata and versioning information.

• Complexity: While the conversion process is straightforward, SavedModel introduces more complexity compared to a single frozen graph file.

Conclusion

In this article, we have explained what frozen graphs and SavedModel are and provided a step-by-step guide on how to convert a Tensorflow frozen graph to SavedModel. Converting a frozen graph to SavedModel is an important step in deploying machine learning models in production environments, and we hope that this guide has been helpful in showing you how to do it.

SHARE:

About Saturn Cloud

Saturn Cloud is your all-in-one solution for data science & ML development, deployment, and data pipelines in the cloud. Spin up a notebook with 4TB of RAM, add a GPU, connect to a distributed cluster of workers, and more. Request a demo today to learn more.