Kubernetes Custom Resource Definitions (CRDs) Use Cases

Kubernetes is an open - source container orchestration platform that has become the de facto standard for managing containerized applications in production environments. One of the most powerful features of Kubernetes is its extensibility, and Custom Resource Definitions (CRDs) play a crucial role in this regard. CRDs allow users to define and manage their own custom resources within a Kubernetes cluster, effectively extending the Kubernetes API. This blog post will explore various use cases of Kubernetes CRDs, covering core concepts, typical usage examples, common practices, and best practices.

Table of Contents

  1. Core Concepts of Kubernetes CRDs
  2. Typical Usage Examples
  3. Common Practices
  4. Best Practices
  5. Conclusion
  6. References

Core Concepts of Kubernetes CRDs

What are CRDs?

A Custom Resource Definition is a way to teach Kubernetes about a new kind of resource. In Kubernetes, a resource is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind. For example, Pod, Service, and Deployment are built - in resources. With CRDs, you can create new types of resources that are specific to your application or organization.

How do CRDs work?

When you create a CRD, Kubernetes adds a new API endpoint to the Kubernetes API server. You can then create, read, update, and delete instances of the custom resource just like you would with built - in resources. These custom resources can have their own schema, which defines the structure and validation rules for the objects.

Custom Controllers

To make the custom resources useful, you often need a custom controller. A custom controller is a piece of software that watches the Kubernetes API for changes to your custom resources and takes appropriate actions based on those changes. For example, if you have a custom resource representing a database instance, the custom controller could be responsible for creating, scaling, or deleting the actual database pods.

Typical Usage Examples

Database Provisioning

Imagine you are running a microservices application that requires multiple database instances. You can create a custom resource definition for a DatabaseInstance. The CRD can define the properties such as the database type (e.g., MySQL, PostgreSQL), the storage size, and the number of replicas.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: databaseinstances.example.com
spec:
  group: example.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                databaseType:
                  type: string
                storageSize:
                  type: string
                replicas:
                  type: integer
  scope: Namespaced
  names:
    plural: databaseinstances
    singular: databaseinstance
    kind: DatabaseInstance
    shortNames:
      - dbi

A custom controller can then watch for new DatabaseInstance resources and create the necessary Deployment and PersistentVolumeClaim objects to provision the database.

Machine Learning Model Deployment

In a machine - learning environment, you can use CRDs to manage the deployment of models. Create a custom resource named MLModelDeployment. The CRD can specify details such as the model name, the input data source, and the inference service type.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: mlmodeldeployments.example.com
spec:
  group: example.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                modelName:
                  type: string
                inputDataSource:
                  type: string
                inferenceServiceType:
                  type: string
  scope: Namespaced
  names:
    plural: mlmodeldeployments
    singular: mlmodeldeployment
    kind: MLModelDeployment
    shortNames:
      - mld

A custom controller can handle the deployment of the model as a Kubernetes Deployment and expose it as a Service for inference.

Common Practices

Schema Design

When creating a CRD, pay close attention to the schema design. Use the openAPIV3Schema to define the structure and validation rules for your custom resources. This helps ensure that users create valid custom resource objects. For example, you can set required fields, define data types, and specify allowed values for certain fields.

Versioning

Kubernetes supports versioning of CRDs. When you make changes to the schema of your custom resource, it’s important to manage the versions properly. You can have multiple versions of a CRD served simultaneously, with one version marked as the storage version. This allows for a smooth transition when upgrading the schema.

Error Handling

In your custom controller, implement proper error handling. When a custom resource is created or updated, there could be various issues such as resource conflicts, permission errors, or external service failures. Log the errors clearly and, if possible, provide meaningful status messages in the custom resource’s status field.

Best Practices

Keep It Simple

When defining a CRD, start with a simple schema. Avoid over - complicating the resource definition with too many fields or complex relationships. You can always add more features and fields as your use case evolves.

Follow Kubernetes Conventions

Adhere to Kubernetes naming conventions and best practices. Use plural and singular names for your custom resources that are easy to understand. Also, follow the same patterns as built - in resources for metadata, status, and spec fields.

Testing

Thoroughly test your CRDs and custom controllers. Write unit tests for the custom controller logic and integration tests to ensure that the custom resources interact correctly with the Kubernetes API and other components in the cluster.

Conclusion

Kubernetes CRDs are a powerful tool for extending the Kubernetes API and enabling custom resource management. They can be used in a wide range of scenarios, from database provisioning to machine - learning model deployment. By understanding the core concepts, using typical usage examples, following common practices, and adhering to best practices, intermediate - to - advanced software engineers can effectively leverage CRDs to build more flexible and customized Kubernetes applications.

References