Custom Tagging on EBS Volumes Provisioned Dynamically by Kubernetes: A Guide

As data scientists, we often find ourselves in the midst of managing complex data workflows. Kubernetes, the open-source platform for automating deployment, scaling, and management of containerized applications, has been a game-changer in this regard. One of its powerful features is the dynamic provisioning of storage volumes, such as AWS Elastic Block Store (EBS) volumes. In this blog post, we will delve into the process of applying custom tags to EBS volumes provisioned dynamically by Kubernetes.

By Saturn Cloud | Tuesday, August 08, 2023 | Miscellaneous

Custom Tagging on EBS Volumes Provisioned Dynamically by Kubernetes: A Guide

Why Custom Tagging?

Before we dive into the how, let’s discuss the why. Custom tagging is a powerful tool for managing resources in a cloud environment. It allows you to categorize resources based on purpose, owner, environment, or any other criteria that make sense for your organization. This can greatly simplify cost tracking, resource management, and automation tasks.

Prerequisites

To follow along, you’ll need:

A Kubernetes cluster running on AWS
AWS CLI installed and configured
Kubernetes command-line tool kubectl installed
Familiarity with Kubernetes Persistent Volumes (PV) and Persistent Volume Claims (PVC)

Step 1: Enable Dynamic Provisioning

Kubernetes supports dynamic provisioning through the use of StorageClasses. A StorageClass provides a way for administrators to describe the “classes” of storage they offer.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-sc
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
  fsType: ext4

This YAML file defines a StorageClass named ebs-sc that provisions gp2 type EBS volumes with the ext4 file system.

Step 2: Create a Persistent Volume Claim

Next, we create a Persistent Volume Claim (PVC) that uses our StorageClass. Kubernetes will dynamically provision an EBS volume for this PVC.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ebs-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: ebs-sc
  resources:
    requests:
      storage: 20Gi

This YAML file creates a PVC named ebs-pvc that requests a 20Gi EBS volume.

Step 3: Apply Custom Tags

To apply custom tags to the dynamically provisioned EBS volume, we need to modify the StorageClass. Kubernetes v1.19 introduced the volumeBindingMode: WaitForFirstConsumer parameter, which delays the binding and provisioning of a PersistentVolume until a Pod using the PVC is created. This allows us to use the EBS CSI driver to apply custom tags.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-sc
provisioner: ebs.csi.aws.com
parameters:
  type: gp2
  fsType: ext4
  csi.storage.k8s.io/fstype: ext4
  encrypted: "false"
volumeBindingMode: WaitForFirstConsumer
allowedTopologies:
- matchLabelExpressions:
  - key: topology.ebs.csi.aws.com/zone
    values:
    - us-west-2a

In the metadata section of your PVC, add the volume.beta.kubernetes.io/additional-resource-tags annotation with your custom tags.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ebs-pvc
  annotations:
    volume.beta.kubernetes.io/additional-resource-tags: "Environment=Dev,Project=Kubernetes"
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: ebs-sc
  resources:
    requests:
      storage: 20Gi

Conclusion

Custom tagging on EBS volumes provisioned dynamically by Kubernetes is a powerful tool for managing your AWS resources. It allows for better organization, tracking, and automation. With the steps outlined in this guide, you can start leveraging this feature in your own Kubernetes deployments.

Remember, the world of Kubernetes is vast and ever-evolving. Stay tuned for more deep dives into its powerful features and how they can be leveraged in the world of data science.

Keywords: Kubernetes, AWS, EBS, Dynamic Provisioning, Custom Tagging, Data Science, StorageClass, Persistent Volume Claim, PVC, Persistent Volume, PV, EBS CSI driver

About Saturn Cloud

Saturn Cloud is your all-in-one solution for data science & ML development, deployment, and data pipelines in the cloud. Spin up a notebook with 4TB of RAM, add a GPU, connect to a distributed cluster of workers, and more. Join today and get 150 hours of free compute per month.

Try Saturn Cloud Now