The Easier Way to Create Pulumi Providers in Go

Kyle Dixler — Thu, 19 Jan 2023 00:00:00 +0000

We are excited to announce that we’ve updated the Pulumi Provider Boilerplate to make custom provider implementation easier!

This major update brings a wealth of usability improvements to the Pulumi Provider Boilerplate by incorporating our brand new Pulumi Go Provider SDK.

Note: This update will not have any impact on existing providers that are utilizing an older version of the repository.

Background

The Pulumi Provider Boilerplate (view on GitHub) is a template repository that contains a minimal example of a working provider and has served as a starting point for writing your own provider.

For those who may not be familiar with provider authoring, it involves three distinct parts:

Package schema definition
Provider implementation
SDK generation

Providers are implemented as gRPC servers that handle requests to perform CRUD (create, read, update, delete) operations on resources on behalf of the Pulumi engine. SDK generation, on the other hand, involves generating source code libraries for a Pulumi program to utilize the provided resources.

In the past, implementing a provider using the old provider boilerplate could be difficult because you had to implement:

the provider’s schema
the CRUD operations on a provider’s resources
the gRPC server
SDK code generation to consume these resources in a Pulumi program

Now, using our code-first approach, you can implement your provider in Go by defining the provider’s metadata, resources, and functions, and then compiling the provider binary. The library takes care of the provider schema and gRPC server for you and the Pulumi CLI takes care of the SDK code generation with the following command:

pulumi package gen-sdk <path/to/provider-binary>

You can leave the heavy lifting to us and focus on the implementation details that matter to your organization.

Boilerplate code

Note: The completed boilerplate is attached at the bottom of this document and in the pulumi-provider-boilerplate repository (view on GitHub).

Provider Entrypoint

The code below uses the new pulumi-go-provider v1 APIs. Make sure you are using the latest version of github.com/pulumi/pulumi-go-provider.

The pulumi provider boilerplate program is currently quite short and the main entrypoint is as follows.

package main

import (
 "context"
 "log"

 "github.com/pulumi/pulumi-go-provider/infer"
)

func main() {
 prov, err := infer.NewProviderBuilder().
 WithNamespace("my-team-name").
 WithResources(
 // We tell the provider what resources it needs to support.
 // In this case, a single custom resource.
 infer.Resource(Random{}),
 ).
 Build()
 if err != nil {
 log.Fatal(err.Error())
 }

 _ = prov.Run(context.Background(), "xyz", "v0.0.1")
}

Resource Implementation

Resources consist of arguments, state, and CRUD operations to control them.

// Each resource has in input struct, defining what arguments it accepts.
type RandomArgs struct {
 // Fields projected into Pulumi must be public and hava a `pulumi:"..."` tag.
 // The pulumi tag doesn't need to match the field name, but its generally a
 // good idea.
 Length int `pulumi:"length"`
}

// Each resource has a state, describing the fields that exist on the created resource.
type RandomState struct {
 // It is generally a good idea to embed args in outputs, but it isn't strictly necessary.
 RandomArgs
 // Here we define a required output called result.
 Result string `pulumi:"result"`
}

// Each resource has a controlling struct.
type Random struct{}

Resource behavior is determined by implementing methods on the controlling struct.

// All resources must implement Create at a minimum.
func (Random) Create(ctx p.Context, name string, input RandomArgs, preview bool) (string, RandomState, error) {
 state := RandomState{RandomArgs: input}
 if preview {
 return name, state, nil
 }
 state.Result = makeRandom(input.Length)
 return name, state, nil
}

The Create method is mandatory, but other methods are optional.

Check: Remap inputs before they are typed.
Diff: Change how instances of a resource are compared.
Update: Mutate a resource in place.
Read: Get the state of a resource from the backing provider.
Delete: Custom logic when the resource is deleted.
Annotate: Describe fields and set defaults for a resource.
WireDependencies: Control how outputs and secrets flows through values.

Wrapping up

We encourage everyone to try the new authoring experience, including devs who may have previously found it challenging in comparison to the ease of program authorship.

Additional Example: Command provider

If you want a more involved example, the command provider (view on GitHub) has been rewritten to use the Pulumi Go Provider library entirely and is heavily commented. You can actually compare the project before and after using it to see how it has streamlined provider implementation.

Completed example

The complete boilerplate provider looks as follows:

package main

import (
 "context"
 "log"

 "github.com/pulumi/pulumi-go-provider/infer"
)

// Version is initialized by the Go linker to contain the semver of this build.
var Version string

func main() {
 prov, err := infer.NewProviderBuilder().
 WithNamespace("my-team-name").
 WithResources(
 // We tell the provider what resources it needs to support.
 // In this case, a single custom resource.
 infer.Resource(Random{}),
 ).
 Build()
 if err != nil {
 log.Fatal(err.Error())
 }

 _ = prov.Run(context.Background(), "xyz", Version)
}

// Each resource has a controlling struct.
// Resource behavior is determined by implementing methods on the controlling struct.
// The `Create` method is mandatory, but other methods are optional.
// - Check: Remap inputs before they are typed.
// - Diff: Change how instances of a resource are compared.
// - Update: Mutate a resource in place.
// - Read: Get the state of a resource from the backing provider.
// - Delete: Custom logic when the resource is deleted.
// - Annotate: Describe fields and set defaults for a resource.
// - WireDependencies: Control how outputs and secrets flows through values.
type Random struct{}

// Each resource has in input struct, defining what arguments it accepts.
type RandomArgs struct {
 // Fields projected into Pulumi must be public and hava a `pulumi:"..."` tag.
 // The pulumi tag doesn't need to match the field name, but its generally a
 // good idea.
 Length int `pulumi:"length"`
}

// Each resource has a state, describing the fields that exist on the created resource.
type RandomState struct {
 // It is generally a good idea to embed args in outputs, but it isn't strictly necessary.
 RandomArgs
 // Here we define a required output called result.
 Result string `pulumi:"result"`
}

// All resources must implement Create at a minimum.
func (Random) Create(ctx p.Context, name string, input RandomArgs, preview bool) (string, RandomState, error) {
 state := RandomState{RandomArgs: input}
 if preview {
 return name, state, nil
 }
 state.Result = makeRandom(input.Length)
 return name, state, nil
}

func makeRandom(length int) string {
 seededRand := rand.New(rand.NewSource(time.Now().UnixNano()))
 charset := []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789")

 result := make([]rune, length)
 for i := range result {
 result[i] = charset[seededRand.Intn(len(charset))]
 }
 return string(result)
}

Pulumi CLI now displays time elapsed per resource

Kyle Dixler — Thu, 03 Nov 2022 11:19:51 -0600

If you’ve deployed resources to your favorite cloud provider, you have probably found yourself sitting in the console thinking: “I don’t know how long this is going to take.” Then you deploy the resource and think: “When did I even start this?” When using Pulumi, the CLI prints out how long the update took after it ran, but while you’re in the moment, it feels like ages.

We’re excited to announce a CLI usability enhancement

You can now see how long each of your resources are taking to deploy.

When you run pulumi up, you’ll see that individual resources have the time taken displayed during updates including up, destroy, import, and refresh.

Comparing resource deployment times

I have a contrived Pulumi program that deploys comparable resources in AWS and GCP and you can see the differences in deployment times between them. It provides a rough idea of the orders of magnitude in deployment time between different resource classes.

Here’s a screenrecording of the output of the program (it took fairly long):

For those interested, the code is located here.

This is far from a perfect example. These numbers have many sources of error, including the load on the cloud provider, the chosen availability zone, quotas and throttling, whether the resource is ready for requests, inefficient resource options, and size of the images (for compute resources). Nonetheless, it provides more perspective as instead of a feeling short or long, we now see concrete numbers to help us quantify it.

Thanks for reading. If you have any feedback on how we can further improve our CLI experience, let us know by commenting or upvoting open issues tracking enhancements on GitHub. If you can’t find an existing issue, feel free to open a new one!

Pulumi Blog: Kyle Dixler