How to Invoke SageMaker Endpoint using Boto3 client from AWS Lambda

As a data scientist or software engineer, you may encounter a situation where you need to invoke a SageMaker endpoint from an AWS Lambda function. In this article, we will discuss how to invoke a SageMaker endpoint using the Boto3 client from an AWS Lambda function, specifically for a TensorFlow model.

Table of Contents

1. What is SageMaker?
2. What is AWS Lambda?
3. Step-by-step invoking SageMaker Endpoint using Boto3 client from AWS Lambda
4. Conclusion

What is SageMaker?

Amazon SageMaker is a fully-managed platform for building, training, and deploying machine learning models at scale. With SageMaker, you can easily build and train machine learning models using popular deep learning frameworks such as TensorFlow, PyTorch, and MXNet. SageMaker also provides pre-built algorithms and frameworks for common machine learning tasks such as regression, classification, and clustering.

What is AWS Lambda?

AWS Lambda is a serverless computing platform provided by Amazon Web Services. With Lambda, you can run your code without provisioning or managing servers. Lambda functions can be triggered in response to events such as changes to data in an S3 bucket, or an HTTP request to an API Gateway endpoint.

AWS Lambda serves various purposes in modern computing. Below are some typical scenarios where AWS Lambda proves invaluable:

• Automation and Orchestrations: Automate workflows and execute tasks based on specific triggers.

• Back-end for Mobile and Web Applications: Construct back-end services triggered by HTTP requests, facilitated by Amazon API Gateway or other AWS services.

• Real-time Stream Processing: Process and analyze real-time streaming data from services like Amazon Kinesis or Apache Kafka.

• Chatbots and Virtual Assistants: Implement chatbots or virtual assistants that respond intelligently to user inputs.

• Event-driven Applications: Automatically run code in response to diverse events, such as changes to data in an Amazon S3 bucket or updates to a DynamoDB table.

• Machine Learning Inference: Run machine learning models for real-time inference without the need for dedicated infrastructure.

• Real-time File Processing: Process files as they are uploaded to S3, handling tasks like image or video processing, data validation, or transformation.

• IoT Backend Processing: Process data from Internet of Things (IoT) devices and execute code based on sensor data.

One of the standout advantages of AWS Lambda lies in its pricing model, where you only pay for the compute time consumed, alleviating concerns about underlying infrastructure. It seamlessly scales applications in response to incoming traffic, ensuring that your code runs in a highly available environment.

Step-by-step invoking SageMaker Endpoint using Boto3 client from AWS Lambda

To invoke a SageMaker endpoint from an AWS Lambda function using the Boto3 client, follow these steps:

Step 1: Create a Lambda Function

You can either go to https://console.aws.amazon.com/lambda/home or you can access AWS Lambda by looking for it on the search bar:

To create a Lambda Function, click on “Create Function”

Select “Author from Scratch”

Complete the fundamental details for the Lambda function:

1. Function Name:

   • Enter the desired name for the function.

2. Runtime:

   • Choose the preferred runtime; in our case, Python 3.8 was selected, but you can opt for a different version.

3. Architecture:

   • Leave it as the default setting.

4. Execution Role:

   • An execution role in AWS Identity and Access Management (IAM) grants necessary permissions to the Lambda function for accessing AWS services and resources. AWS Lambda assumes this role to execute functions on your behalf. The execution role should be equipped with appropriate permissions policies, enabling the Lambda function to access services like Amazon S3, Amazon DynamoDB, Amazon SNS, etc.
   • Upon creating a Lambda function, specify the execution role, and AWS Lambda utilizes the credentials of this role for executing actions on your behalf.
   • Two options are available:
     • Use an existing role or create a new one from AWS policy templates.
     • Create a new role with basic Lambda permissions.

Step 2: Install the Boto3 Library

To use the Boto3 client in your Lambda function, you must first install the Boto3 library. You can do this by including the following line of code at the beginning of your Python script:

import boto3

Step 3: Invoke the SageMaker Endpoint

To invoke the SageMaker endpoint from your Lambda function, use the Boto3 client’s invoke_endpoint method. This method takes the following parameters:

• EndpointName: The name of the SageMaker endpoint to invoke.
• ContentType: The content type of the input data.
• Body: The input data to send to the endpoint.

Here is an example Python script that invokes a SageMaker endpoint using the Boto3 client:

• The function receives the test event content through the event argument.
• Utilizing runtime.invoke_endpoint, the endpoint is invoked, and the input data is transmitted to the deployed model.
• Lastly, it is imperative to process the response, preparing it for the function’s output.

import boto3
import json

def lambda_handler(event, context):

    # Create a Boto3 client for SageMaker
    client = boto3.client('sagemaker-runtime')

    # Set the name of the SageMaker endpoint to invoke
    endpoint_name = 'my-endpoint'

    # Set the content type of the input data
    content_type = 'application/json'

    # Set the input data
    input_data = {
        'data': ['input_value_1', 'input_value_2']
    }

    # Convert the input data to JSON format
    payload = json.dumps(input_data)

    # Invoke the SageMaker endpoint
    response = client.invoke_endpoint(
        EndpointName=endpoint_name,
        ContentType=content_type,
        Body=payload
    )

    # Get the response from the SageMaker endpoint
    result = response['Body'].read().decode('utf-8')

    # Return the result
    return {
        'statusCode': 200,
        'body': result
    }

In this example, the Lambda function takes an input event and context as parameters, which are not used in the function. The function creates a Boto3 client for SageMaker, sets the name of the endpoint to invoke, sets the content type of the input data, and sets the input data. The function then converts the input data to JSON format and invokes the SageMaker endpoint using the Boto3 client’s invoke_endpoint method. The result from the endpoint is returned as the function’s response.

Save the file and deploy changes. After some seconds it will be ready to test.

Click on Test when it finishes. A new tab will open with the Execution result

Conclusion

In this article, we discussed how to invoke a SageMaker endpoint using the Boto3 client from an AWS Lambda function. We identified the most common cause of the “Received server error (503)” error message and provided a step-by-step solution for configuring the Lambda function’s VPC, installing the Boto3 library, and invoking the SageMaker endpoint. By following these steps, you can easily invoke a SageMaker endpoint from an AWS Lambda function and integrate your machine learning models into your serverless architecture.

SHARE:

About Saturn Cloud

Saturn Cloud is your all-in-one solution for data science & ML development, deployment, and data pipelines in the cloud. Spin up a notebook with 4TB of RAM, add a GPU, connect to a distributed cluster of workers, and more. Join today and get 150 hours of free compute per month.