How to Put an Object to Amazon S3 Using a Presigned URL

How to Put an Object to Amazon S3 Using a Presigned URL

In the realm of cloud computing, the ability to securely and efficiently manage data is crucial. Amazon’s Simple Storage Service (S3) is a popular choice among data scientists and software engineers because of its scalability, data availability, and security features. One of these features is the ability to use presigned URLs. This article will explain how to put an object to Amazon S3 using a presigned URL.

What is a Presigned URL?

A presigned URL gives you access to an object in your S3 bucket for a temporary period. This URL can be used to download or upload an object without needing AWS security credentials or permissions. This is especially vital when you want to grant temporary access to your S3 resources to third parties.

Generating a Presigned URL

To generate a presigned URL, you’ll need the AWS SDK. AWS provides SDKs for multiple languages, including Python, JavaScript, and Java. In this example, we’ll use the Python SDK, Boto3.

First, you need to install Boto3 if you haven’t done it already:

pip install boto3

Then, you can generate a presigned URL with the following snippet:

import boto3

def create_presigned_url(bucket_name, object_name, expiration=3600):
    """
    Generate a presigned S3 PUT URL 

    :param bucket_name: string
    :param object_name: string
    :param expiration: Time in seconds for the presigned URL to remain valid
    :return: Presigned URL as string. If error, returns None.
    """
    # Generate a presigned S3 PUT URL
    s3_client = boto3.client('s3')
    try:
        response = s3_client.generate_presigned_url('put_object',
                                                    Params={'Bucket': bucket_name,
                                                            'Key': object_name},
                                                    ExpiresIn=expiration)
    except ClientError as e:
        logging.error(e)
        return None

    # The response contains the presigned URL
    return response

This function generates a presigned URL that allows uploading (PUT) an object to your S3 bucket. The URL will expire in an hour (3600 seconds).

Uploading an Object using the Presigned URL

Once you have the presigned URL, you can upload an object to S3. Here’s an example using the requests library in Python:

import requests

def upload_object(file_path, presigned_url):
    """
    Upload an object to S3 using a presigned URL

    :param file_path: string
    :param presigned_url: string
    :return: True if file was uploaded, else False
    """
    # Load the file to be uploaded
    with open(file_path, 'rb') as file:
        files = {'file': (file_path, file)}

    # PUT the file
    response = requests.put(presigned_url, files=files)

    # If successful, the response should have a status code of 200
    if response.status_code == 200:
        return True
    return False

In this function, we use requests.put to upload a file to the presigned URL. If the upload is successful, the function returns True.

Conclusion

Presigned URLs offer a powerful way to control access to your S3 objects while maintaining security and privacy. They are a key tool in a data scientist’s or software engineer’s toolkit when working with AWS. With this guide, you should now be able to generate and use a presigned URL to upload objects to your S3 bucket. Happy coding!

Remember to replace bucket_name, object_name, and file_path with your actual bucket name, object name, and file path.

About Saturn Cloud

Saturn Cloud is your all-in-one solution for data science & ML development, deployment, and data pipelines in the cloud. Spin up a notebook with 4TB of RAM, add a GPU, connect to a distributed cluster of workers, and more. Join today and get 150 hours of free compute per month.