Amazon S3 Multipart Upload with Plupload and Rails 3

Amazon S3 Multipart Upload with Plupload and Rails 3

Amazon S3 provides a highly reliable, scalable, and low-latency data storage infrastructure. However, uploading large files can be a challenge. Multipart upload allows you to upload a single object as a set of parts, enabling improved throughput, quick recovery from network issues, and pausing/resuming file uploads. Today, I’ll guide you on how to implement Amazon S3 multipart upload with Plupload and Rails 3.

What is Plupload?

Plupload is a JavaScript API for dealing with file uploads; it supports features like multiple file selection, file type filtering, request chunking, client-side image scaling, and more. It has cross-browser support and can use HTML5, Silverlight, Flash, and other technologies.

Why Rails 3?

Rails 3 is a popular web application framework written in Ruby. It includes everything needed to create database-backed web applications according to the Model-View-Controller (MVC) pattern.

Step-by-step Guide

Firstly, ensure you have an Amazon S3 account and Rails 3 installed. You will also need to add the ‘aws-sdk’ gem to your Gemfile:

gem 'aws-sdk'

After running bundle install, create a new initializer config/initializers/[aws](https://saturncloud.io/glossary/aws).rb:

AWS.config(
  :access_key_id => 'Your AWS Access Key',
  :secret_access_key => 'Your AWS Secret Key'
)

Step 1: Set up Plupload

Download Plupload and include the Plupload core, the queue widget, and any runtimes in your JavaScript manifest:

//= require plupload.full
//= require jquery.plupload.queue

Step 2: Prepare Your View

In the view where you want to upload files, add a div for the Plupload widget:

<div id="uploader">
    <p>You browser doesn't have Flash, Silverlight, or HTML5 support.</p>
</div>

Then, initialize Plupload with your S3 bucket information:

$(function() {
    $("#uploader").pluploadQueue({
        runtimes: 'html5,flash,silverlight',
        url: 'https://your-bucket-name.s3.amazonaws.com/',
        multipart: true,
        multipart_params: {
            'key': '${filename}', // use filename as a key
            'Filename': '${filename}', // adding this to keep consistency across the runtimes
            'acl': 'private',
            'Content-Type': 'binary/octet-stream',
            'AWSAccessKeyId': 'Your AWS Access Key',        
            'policy': 'Your Policy Document in Base64 Format',
            'signature': 'Your Calculated Signature'
        },
        max_file_size: '10mb',
        flash_swf_url: '/plupload/js/plupload.flash.swf',
        silverlight_xap_url: '/plupload/js/plupload.silverlight.xap'
    });
});

Step 3: Generate the Policy Document and Signature in Rails

You need to generate a policy document and a signature for the S3 upload. In your Rails controller, add the following method:

def s3_access_token
  render json: {
    policy:    s3_upload_policy_document,
    signature: s3_upload_signature,
    key:       "uploads/#{SecureRandom.uuid}/${filename}"
  }
end

Create the s3_upload_policy_document and s3_upload_signature methods:

def s3_upload_policy_document
  Base64.encode64({
    "expiration" => 1.hour.from_now.utc.xmlschema,
    "conditions" => [ 
      { "bucket" =>  "your-bucket-name" },
      [ "starts-with", "$key", "" ],
      { "acl" => "private" },
      { "success_action_status" => "201" },
      [ "starts-with", "$Filename", "" ],
      [ "content-length-range", 0, 10 * 1024 * 1024 ]
    ]
  }.to_json).gsub(/\n/,'')
end

def s3_upload_signature
  Base64.encode64(
    OpenSSL::HMAC.digest(
      OpenSSL::Digest.new('sha1'),
      'Your AWS Secret Key', s3_upload_policy_document
    )
  ).gsub("\n","")
end

Step 4: Modify Your Plupload Setup

Modify your Plupload setup to fetch the policy, signature, and key from your Rails app:

$.ajax({
  url: '/s3_access_token', 
  type: 'GET',
  dataType: 'json'
}).done(function(data) {
  var uploader = $('#uploader').pluploadQueue();
  uploader.settings.multipart_params = {
    'key': data.key,
    'Filename': data.key,
    'acl': 'private',
    'Content-Type': 'binary/octet-stream',
    'AWSAccessKeyId': 'Your AWS Access Key',
    'policy': data.policy,
    'signature': data.signature
  };
});

And that’s it! You should now be able to upload files directly to your S3 bucket from your Rails 3 application using Plupload.

Conclusion

Amazon S3 multipart uploads with Plupload and Rails 3 allow for efficient, manageable file uploads. This setup provides a robust, scalable solution for handling large files that can be applied to a wide range of web applications. Happy coding!

Keywords: Amazon S3, multipart upload, Plupload, Rails 3, file upload, web applications, Ruby, JavaScript, how to, guide, data storage, AWS.

About Saturn Cloud

Saturn Cloud is your all-in-one solution for data science & ML development, deployment, and data pipelines in the cloud. Spin up a notebook with 4TB of RAM, add a GPU, connect to a distributed cluster of workers, and more. Join today and get 150 hours of free compute per month.