Kubernetes DaemonSet Update Strategy

In the Kubernetes ecosystem, DaemonSets play a crucial role in ensuring that a specific pod runs on every node or a selected set of nodes in a cluster. A common use - case is running monitoring agents, log collectors, or network daemons on each node. However, as applications evolve, there is a need to update the pods managed by a DaemonSet. Kubernetes provides different update strategies for DaemonSets to handle these updates gracefully. This blog post will explore the core concepts, typical usage examples, common practices, and best practices related to Kubernetes DaemonSet update strategies.

Table of Contents

  1. Core Concepts
  2. Typical Usage Example
  3. Common Practices
  4. Best Practices
  5. Conclusion
  6. References

Core Concepts

Update Strategies in DaemonSets

Kubernetes DaemonSets support two main update strategies:

  • OnDelete: This is the default strategy in older Kubernetes versions. When using the OnDelete strategy, Kubernetes will not automatically update the pods in the DaemonSet when the DaemonSet specification is changed. Instead, the administrator has to manually delete the existing pods, and Kubernetes will then create new pods based on the updated DaemonSet specification.
  • RollingUpdate: This strategy allows for a more seamless update process. When the DaemonSet specification is changed, Kubernetes will gradually replace the old pods with new ones. You can control the update process by specifying parameters such as maxUnavailable and maxSurge.
    • maxUnavailable: This parameter defines the maximum number of pods that can be unavailable during the update process. It can be an absolute number or a percentage. For example, if set to 1, it means that at most one pod can be unavailable at any given time during the update.
    • maxSurge: This parameter specifies the maximum number of pods that can be created above the desired number of pods during the update. Similar to maxUnavailable, it can be an absolute number or a percentage.

Typical Usage Example

Let’s assume we have a simple DaemonSet running a logging agent on each node in our cluster.

Step 1: Create a DaemonSet

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: logging-agent
spec:
  selector:
    matchLabels:
      name: logging-agent
  template:
    metadata:
      labels:
        name: logging-agent
    spec:
      containers:
      - name: logging-agent
        image: logging-agent:1.0

Step 2: Update the DaemonSet to use the RollingUpdate strategy

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: logging-agent
spec:
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  selector:
    matchLabels:
      name: logging-agent
  template:
    metadata:
      labels:
        name: logging-agent
    spec:
      containers:
      - name: logging-agent
        image: logging-agent:2.0

In this example, we first created a DaemonSet running the logging - agent:1.0 image. Then, we updated the DaemonSet to use the RollingUpdate strategy with maxUnavailable set to 1 and changed the image to logging - agent:2.0. Kubernetes will start replacing the old pods with new ones, ensuring that at most one pod is unavailable at a time.

Common Practices

Monitoring the Update Process

It is essential to monitor the update process of a DaemonSet. You can use Kubernetes commands like kubectl rollout status daemonset <daemonset - name> to check the status of the rollout. Additionally, you can use monitoring tools like Prometheus and Grafana to monitor the health of the pods and the overall performance of the DaemonSet during the update.

Testing Updates in a Staging Environment

Before rolling out updates to a production environment, it is a good practice to test the updates in a staging environment. This allows you to identify and fix any potential issues before they impact the production system.

Version Control for DaemonSet Specifications

Keep your DaemonSet specifications under version control. This makes it easier to track changes, roll back to a previous version if necessary, and collaborate with other team members.

Best Practices

Set Appropriate Update Parameters

When using the RollingUpdate strategy, carefully set the maxUnavailable and maxSurge parameters based on the nature of your application. For critical applications, you may want to set a lower maxUnavailable value to minimize the impact on the system.

Implement Pre - and Post - Update Hooks

You can use init containers and post - start hooks to perform tasks before and after the pod update. For example, you can use an init container to perform some pre - update checks, and a post - start hook to perform post - update validations.

Use Canary Releases

Consider using canary releases for DaemonSet updates. This involves updating a small subset of pods first and monitoring their performance before rolling out the update to the entire cluster. This helps to quickly identify and mitigate any issues with the new version.

Conclusion

Kubernetes DaemonSet update strategies provide a flexible way to manage the update process of pods running on every node in a cluster. By understanding the core concepts, following common practices, and implementing best practices, you can ensure a smooth and reliable update process for your DaemonSets. Whether you choose the OnDelete or RollingUpdate strategy, proper planning and monitoring are key to successful updates.

References