Skip to main content

Managing Resources with Server-Side Apply

20 min

Server-Side Apply (SSA) is a resource management strategy introduced in Kubernetes v1.13. Clients using SSA can safely share the management of Kubernetes resources by making the API Server responsible for computing diffs and resolving conflicts.

The v4 release of the Pulumi Kubernetes provider enables SSA by default. SSA lets more than one controller safely manage a resource, allows you to "upsert" resources, adds Patch resource types for patching resources, and removes the need for the last-applied-configuration annotation.

In this tutorial, you will enable SSA support and use these features to upsert resources, patch existing resources, and resolve field-manager conflicts.

What you'll learn
  • How to enable Server-Side Apply in the Pulumi Kubernetes provider
  • How to upsert a resource so it is created if missing or updated if it exists
  • How to patch existing resources with the SDK's Patch resource types
  • How to delete a field from an existing resource using patches
  • How to detect and resolve field-manager conflicts
Prerequisites

Server-Side Apply (SSA) is a resource management strategy that was introduced in Kubernetes v1.13. Clients using SSA can safely share the management of Kubernetes resources by making the API Server responsible for computing diffs and resolving conflicts.

The v4 release of the Pulumi Kubernetes provider enables SSA by default. Using SSA provides the following benefits:

  1. Kubernetes resources may be safely managed by more than one controller.
  2. It is now possible to “Upsert” resources; create the resource if it does not exist, or apply the configuration to an existing resource.
  3. It is now possible to patch resources with the Patch resource types in the SDK. Each resource type in the SDK has a corresponding Patch resource.
  4. The last-applied-configuration annotation is no longer used.

This guide will show you how to enable SSA support and use these features to manage Kubernetes resources.

Prerequisites#

Server-Side Apply support requires patch permission on the Kubernetes cluster for update operations. Previews are computed using input diffs, so they may be performed with restricted permissions.

Enable Server-Side Apply#

SSA is enabled by default in pulumi-kubernetes v4, but the enableServerSideApply Provider option may be set explicitly in the following ways:

  1. Set the stack config explicitly with pulumi config set kubernetes:enableServerSideApply true
  2. Set the stack config using an environment variable: PULUMI_K8S_ENABLE_SERVER_SIDE_APPLY=true
  3. Set the enableServerSideApply option on a first-class Provider as shown in the following example
import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";


const provider = new kubernetes.Provider("k8s", {enableServerSideApply: true});

Upsert a Resource#

With Server-Side Apply mode enabled, you can “upsert” a resource; that is, update a resource if it already exists, or create it if it does not yet exist. You can set the pulumi.com/patchForce annotation if you want to force an override for any conflicts with an existing version of the resource.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";


const provider = new kubernetes.Provider("k8s", {enableServerSideApply: true});
const example = new kubernetes.core.v1.ConfigMap("example", {
    metadata: {
        annotations: {
            "pulumi.com/patchForce": "true",
        },
        name: "example",
    },
    data: {
        foo: "bar",
    },
}, {
    provider: provider,
});

Patch a Resource#

With Server-Side Apply mode enabled, you can patch an existing resource to make changes to it. Every resource type in the SDK includes a corresponding Patch resource for this purpose. Patch resources have a few important differences from typical SDK resources:

  1. The resource name (.metadata.name) is required, and auto-naming is not supported. The patch operation applies to an existing resource, so this field is used to specify which resource to patch. Remember to specify the namespace as well if required.
  2. All other fields are optional, even if they would be required for a normal resource. The patch specification must be unambiguous, but it is not necessary to specify the complete resource.
  3. More than one Patch resource may modify the same underlying Kubernetes resource. Each Patch resource is assigned a unique FieldManager name by default, or you can manually specify one using the pulumi.com/patchFieldManager annotation.
  4. Server-Side Apply mode must be enabled to use Patch resources.
  5. Deleting a Patch resource will undo the patched changes to the Kubernetes resource, but the resource will not be deleted. Any fields that become unmanaged during this process will reset to their default values, so the resulting resource may have changes from its original state before patching. See the upstream docs for more information about this behavior.

You can set the pulumi.com/patchForce annotation if you want to force an override for any conflicts with an existing version of the resource.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";


const provider = new kubernetes.Provider("provider", {enableServerSideApply: true});
const patch1 = new kubernetes.core.v1.ConfigMapPatch("patch1", {
    metadata: {
        annotations: {
            "pulumi.com/patchForce": "true",
        },
        name: "example",
    },
    data: {
        foo: "bar",
    },
}, {
    provider: provider,
});
const patch2 = new kubernetes.core.v1.ConfigMapPatch("patch2", {
    metadata: {
        annotations: {
            "pulumi.com/patchForce": "true",
        },
        name: "example",
    },
    data: {
        oof: "rab",
    },
}, {
    provider: provider,
    dependsOn: [patch1],
});

Delete an existing field using Patch#

With Server-Side Apply mode enabled, you can also remove a field from an existing resource. This example demonstrates removing .labels.foo from an existing Namespace.

In most cases, it is not necessary to set the fieldManager name explicitly because the provider automatically assigns a unique manager name to each resource by default.

Deleting an existing field is a special case where this is needed. In this workflow, we first take ownership of the field to delete (using the patch1 resource in this example), and then set it to null/undefined in a subsequent step (using the patch2 in this example). Since both patches need to operate on the same field, they must use the same fieldManager rather than using the auto-assigned value. Also note the use of the patchForce option to explicitly take ownership of the field from another fieldManager.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";


const uuid = "ee0d1b94-e749-4de4-ae8a-e4319a7f3ef2"; // Arbitrary UUID
const provider = new kubernetes.Provider("provider", {enableServerSideApply: true});
const patch1 = new kubernetes.core.v1.NamespacePatch("patch1", {metadata: {
        name: "foo",
        annotations: {
            "pulumi.com/patchForce": "true",
            "pulumi.com/patchFieldManager": uuid,
        },
        labels: {
            foo: "",
        },
    }}, {
    provider: provider,
});
const patch2 = new kubernetes.core.v1.NamespacePatch("patch2", {metadata: {
        name: "foo",
        annotations: {
            "pulumi.com/patchForce": "true",
            "pulumi.com/patchFieldManager": uuid,
        },
        labels: {
            foo: undefined,
        },
    }}, {
    provider: provider,
    dependsOn: [patch1],
});

Handle Field Conflicts on Existing Resources#

After a resource is deployed to the cluster, it is possible for other controllers to start managing it. Thanks to the power of SSA, it is possible to see a detailed record of which controller last changed every field for the resource. An additional benefit of this fine-grained tracking is the ability to deliberately resolve conflicts rather than inadvertently overwriting changes made by another controller.

If you encounter a field conflict, there are several options for resolving the conflict:

  1. Overwrite conflicts by using the patchForce option. This option can be enabled either by setting an annotation, or by using the PULUMI_K8S_ENABLE_PATCH_FORCE environment variable.
  2. Use the ignoreChanges resource option to skip the update for the conflicting field.
  3. Transfer ownership of the conflicting field to another field manager. See the upstream documentation for additional information about this approach.

Use the PULUMI_K8S_ENABLE_PATCH_FORCE environment variable for a one-time override#

Terminal window
# Enable patch force for the target resource. In this example, we use the `**` wildcard to ignore most of the URN,
# and match based on the type and name of the resource. (type=DeploymentPatch, name=example)
PULUMI_K8S_ENABLE_PATCH_FORCE="true" pulumi up --target="**DeploymentPatch::example"

Set the pulumi.com/patchForce annotation to persistently overwrite any field conflicts for a resource#

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";


const provider = new kubernetes.Provider("k8s", {enableServerSideApply: true});
const example = new kubernetes.core.v1.ConfigMap("example", {
    metadata: {
        annotations: {
            "pulumi.com/patchForce": "true",
        },
        name: "example",
    },
    data: {
        foo: "bar",
    },
}, {
    provider: provider,
});

Use ignoreChanges to allow another controller to manage a conflicting field#

Cert-manager is a common example of an operator that mutates other cluster resources. Consider the following sequence of events:

  1. Deploy the cert-manager operator.
  2. Deploy a ValidatingWebhookConfiguration resource “X” using Pulumi with the cert-manager.io/inject-ca-from annotation set.
  3. The cert-manager operator updates “X” by setting the webhooks[*].clientConfig.caBundle field(s). This field is now managed by cert-manager rather than by Pulumi, and doesn’t match Pulumi’s configuration.
  4. Run another Pulumi update, and the field conflict is detected.

Pulumi provides a way to ignore changes for particular fields using the ignoreChanges resource option. Set this resource option on resource “X” to tell Pulumi to ignore changes to this field since another controller is managing it.

const res = new ValidatingWebhookConfiguration("res",
    { prop: "new-value" },
    { ignoreChanges: ["webhooks[*].clientConfig", "webhooks[*].namespaceSelector"] });

Helm Charts#

Although any resource can be updated by multiple controllers, it is particularly common for Helm charts that are deployed by Pulumi. Consider the following sequence of events:

  1. Deploy a chart using Pulumi. This chart contains a resource “X” plus an operator that will update “X”.
  2. Once the chart is deployed, the operator starts running and makes an update to resource “X”. The updated field is now managed by a different controller, and the value does not match Pulumi’s configuration.
  3. Run another Pulumi update, and the field conflict is detected.

Pulumi provides a way to ignore changes for particular fields using the ignoreChanges resource option. Normally we could set the resource option directly, but since the Helm component is managing the resources on our behalf, we need to use a transformation to accomplish this.

Note that you can also set the patchForce option to resolve conflicts using a transformation, but that this is likely to cause further conflicts if the operator expects to manage these fields.

import * as k8s from "@pulumi/kubernetes";


const kruise = new k8s.helm.v3.Chart("kruise", {
    chart: "kruise",
    version: "1.3.0",
    fetchOpts:{
        repo: "https://openkruise.github.io/charts/",
    },
}, {
    transformations: [
        // Ignore changes that will be overwritten by the kruise-manager deployment.
        args => {
            if (args.type === "kubernetes:admissionregistration.k8s.io/v1:ValidatingWebhookConfiguration" ||
                args.type === "kubernetes:admissionregistration.k8s.io/v1:MutatingWebhookConfiguration") {
                return {
                    props: args.props,
                    opts: pulumi.mergeOptions(args.opts, {
                        ignoreChanges: ["metadata.annotations.template", "webhooks[*].clientConfig"],
                    })
                }
            }
            return undefined;
        }
    ],
});
KubernetesTypeScriptPythonGo.NETJavaYAML

Related

Tutorial

Deleting Helm Chart Default Values

Learn how to remove a Helm chart default value in Pulumi by setting it to null.
10 min
Tutorial

Typed CustomResources with Provider Extensions

Generate typed SDKs for your Kubernetes CRDs with Pulumi provider extensions.
15 min

The infrastructure as code platform for any cloud.

Contact usGet started