Create a Custom Policy Pack

In this step, we’ll create a policy pack to enforce the following rules for AWS resources:

S3 buckets must be prefixed with the product name myproduct-.
EC2 instances must use the t2.micro instance type.
All AWS resources must have at least one tag defined.

Set up your policy pack project

First, create a new directory for your policy pack project:

mkdir custom-policy-pack
cd custom-policy-pack

Then, initialize your project. Choose Python or TypeScript based on your preferred language.

pulumi policy new aws-typescript

This will create the following files and directories:

PulumiPolicy.yaml: A Pulumi project file that indicates this a policy pack.
index.ts: The TypeScript entry point where the policies will be defined in code.
node_modules/: The NPM modules directory
package-lock.json: A list of the module dependencies used by npm.
package.json: The npm package description file.
tsconfig.json: The TypeScript configuration file.

In this example, we are using the aws-typescript policy pack template. To see the full list of available policy pack templates, check out the pulumi/templates-policy GitHub repository.

pulumi policy new aws-python

This will create the following files and directories:

PulumiPolicy.yaml: A Pulumi project file that indicates this a policy pack.
__main__.py: The Python entry point where the policies will be defined in code.
requirements.txt: A list of the module dependencies used by pip.
venv/: The Python virtual environment.

In this example, we are using the aws-python policy pack template. To see the full list of available policy pack templates, check out the pulumi/templates-policy GitHub repository.

This will initialize your project, creating the necessary files for Pulumi to use as a policy, including module dependencies to the providers that will let us interact with AWS resources.

Define Policies

Policies are written in Python or TypeScript. Like Pulumi Programs, you can use the full power of your preferred language, including standard features like leveraging third-party modules, using conditional logic and control flow, and can be validated with unit testing frameworks.

By default the template sets up an example resource policy that prevents S3 buckets from being publically readable:

File: custom-policy-pack/index.ts

import * as aws from "@pulumi/aws";
import { PolicyPack, validateResourceOfType } from "@pulumi/policy";

new PolicyPack("aws-typescript", {
    policies: [{
        name: "s3-no-public-read",
        description: "Prohibits setting the publicRead or publicReadWrite permission on AWS S3 buckets.",
        enforcementLevel: "mandatory",
        validateResource: validateResourceOfType(aws.s3.Bucket, (bucket, args, reportViolation) => {
            if (bucket.acl === "public-read" || bucket.acl === "public-read-write") {
                reportViolation(
                    "You cannot set public-read or public-read-write on an S3 bucket. " +
                    "Read more about ACLs here: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html");
            }
        }),
    }],
});

File: custom-policy-pack/__main__.py

from pulumi_policy import (
    EnforcementLevel,
    PolicyPack,
    ReportViolation,
    ResourceValidationArgs,
    ResourceValidationPolicy,
)

def s3_no_public_read_validator(args: ResourceValidationArgs, report_violation: ReportViolation):
    if args.resource_type == "aws:s3/bucket:Bucket" and "acl" in args.props:
        acl = args.props["acl"]
        if acl == "public-read" or acl == "public-read-write":
            report_violation(
                "You cannot set public-read or public-read-write on an S3 bucket. " +
                "Read more about ACLs here: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html")

s3_no_public_read = ResourceValidationPolicy(
    name="s3-no-public-read",
    description="Prohibits setting the publicRead or publicReadWrite permission on AWS S3 buckets.",
    enforcement_level=EnforcementLevel.MANDATORY,
    validate=s3_no_public_read_validator,
)

PolicyPack(
    name="aws-python",
    policies=[
        s3_no_public_read,
    ],
)

Here you can see the basic structure of a policy pack:

Some imports from the Pulumi Crossguard SDK
a function that implements the policy
a policy definition that wraps the implementation and describes the policy
a policy pack definition that packages the policies together

While this example is a useful policy, it’s not what we need right now. Let’s delete all of that code and create some new custom policies.

Replace the contents of index.ts with the following:

import * as aws from "@pulumi/aws";
import { PolicyPack, ReportViolation, ResourceValidationArgs, validateResourceOfType, ResourceValidationPolicy } from "@pulumi/policy";

export const REQUIRED_S3_PREFIX: string = "myproduct-";

// Policy: Ensure S3 buckets have product prefix enabled
export const s3ProductPrefixPolicy: ResourceValidationPolicy = {
    name: "s3-product-prefix",
    description: "Ensures S3 buckets have the correct product prefix.",
    enforcementLevel: "mandatory",
    validateResource: validateResourceOfType(aws.s3.Bucket, (bucket, args, reportViolation) => {
        const prefix = bucket.bucketPrefix || "";
        if (prefix != REQUIRED_S3_PREFIX) {
            reportViolation(`Invalid prefix: '${prefix}'. S3 buckets must use '${REQUIRED_S3_PREFIX}' prefix.`);
        }
    }),
};

export const REQUIRED_INSTANCE_TYPE: string = "t2.micro";

// Policy: Restrict EC2 instance types
export const ec2InstanceTypeRestrictedPolicy: ResourceValidationPolicy = {
    name: "ec2-instance-type-restricted",
    description: "Ensures EC2 instances use approved instance type.",
    enforcementLevel: "mandatory",
    validateResource: validateResourceOfType(aws.ec2.Instance, (instance, args, reportViolation) => {
        const instanceType = instance.instanceType || "";
        if (instanceType !== REQUIRED_INSTANCE_TYPE) {
            reportViolation(`Invalid instance type: '${instanceType}'. EC2 instances must use '${REQUIRED_INSTANCE_TYPE}' instance type.`);
        }
    }),
};

// Policy: Ensure all AWS resources have at least one tag
export const allAwsResourcesMustHaveTagsPolicy: ResourceValidationPolicy = {
    name: "all-aws-resources-must-have-tags",
    description: "Ensures all AWS resources have at least one tag.",
    enforcementLevel: "mandatory",
    validateResource: (args: ResourceValidationArgs, reportViolation: ReportViolation) => {
        if (args.type.startsWith("aws")) {
            const tags = args.props.tags || {};
            if (Object.keys(tags).length === 0) {
                reportViolation("All AWS resources must have at least one tag.");
            }
        }
    },
};

new PolicyPack("custom-policy-pack", {
    policies: [s3ProductPrefixPolicy, ec2InstanceTypeRestrictedPolicy, allAwsResourcesMustHaveTagsPolicy],
});

Here we define three different policies:

s3-product-prefix: Ensures S3 buckets are prefixed with the product name by checking the bucketPrefix property on all aws:s3:Bucket resources.
ec2-instance-type-restricted: Restricts EC2 instance types to only use the affordable t2.micro type, by checking the instanceType property on all aws:ec2/instance:Instance resources.
all-aws-resources-must-have-tags: Ensures all AWS resources have at least one tag by checking the tags property on all resources who’s type starts with aws.

Each of the policies uses the same pattern:

Define a ResourceValidationPolicy with a bit of metadata and a validation function. Each policy can have an individual enforcement level, name, and description.
The validation function is defined inline using the validateResourceOfType helper function. The way Crossguard resource policies work, each resource in the project will be passed to every policy in the pack. This strongly-typed helper function creates a filter that checks the resource type (e.g. aws.ec2.Instance from the pulumi-aws provider library) and will only run the validation function on resources that match the type.
The validation functions take an instance of the resource, an args property bag, and a function for reporting policy violations. Inside the validation function, we can check one or more properties on the strongly-typed resource instance.
If there is a problem with the property value, or some other aspect of the resource is out of compliance, we use the reportViolation function that was passed in to indicate that there’s a problem. The error message should be a full sentence and give useful information on how to remediate the problem.

What’s great about the strong typing in our SDK is that your IDE’s intellisense knows the type definition and can give you dot-completion for properties on the resource instance, so there’s no guessing about what the property is called, or what its type should be.

That said, sometimes you might need to write a policy that works against more than one resource type. For example, the all-aws-resources-must-have-tags policy checks every kind of AWS resource. In this case, instead of using the validateResourceOfType helper function, we just pass the anonymous function directly. Then, in the function we need to check the resource type as a string by inspecting the value of args.type.

If you’re not sure what the correct resource type string is for your particular set of resources, you can run pulumi stack to list the resources in your current stack. The TYPE column of the output contains the same resource types strings that you would use to filter on inside of a policy. In our above example, we apply the policy to any resource who’s type starts with aws.

$ pulumi stack
[...]
Current stack resources (5):
    TYPE                                    NAME
    pulumi:pulumi:Stack                     custom-policy-pack-integration-test-dev
    ├─ aws:s3:Bucket                        my-bucket
    ├─ aws:ec2/securityGroup:SecurityGroup  ssh-security-group
    ├─ aws:ec2/instance:Instance            web-server
    └─ pulumi:providers:aws                 default_6_65_0

Finally we assemble the policies into a policy pack object, giving it the name custom-policy-pack.

Writing Testable Code: We’ve structured this code to be a little bit more readable than the template example, and also more testable. Instead of defining everything in one big nested object, we break each policy out into its own definition and then assemble the policy pack at the end. We use the export keyword to make these policies available to the testing framework (although not technically necessary for the policy pack itself).

Have a look at the test directory in the full version of this example to see how you can write unit tests for each policy.

Replace the contents of __main__.py with the following:

from pulumi_policy import (
    EnforcementLevel,
    ReportViolation,
    ResourceValidationArgs,
    ResourceValidationPolicy,
)

# S3 Bucket Policy
REQUIRED_S3_PREFIX="myproduct-"

def s3_product_prefix(args: ResourceValidationArgs, report_violation: ReportViolation):
    if args.resource_type == "aws:s3/bucket:Bucket" and "bucketPrefix" in args.props:
        actualPrefix = args.props["bucketPrefix"]
        if actualPrefix != REQUIRED_S3_PREFIX:
            report_violation("Invalid prefix: '{}'. S3 buckets must use '{}' prefix.".format(actualPrefix, REQUIRED_S3_PREFIX))

s3_product_prefix_policy = ResourceValidationPolicy(
    name="s3-product-prefix",
    description="Ensures S3 buckets have the correct product prefix.",
    validate=s3_product_prefix,
    enforcement_level=EnforcementLevel.MANDATORY
)

# EC2 Instance Policy
REQUIRED_INSTANCE_TYPE="t2.micro"

def ec2_instance_type_restricted(args: ResourceValidationArgs, report_violation: ReportViolation):
    if args.resource_type == "aws:ec2/instance:Instance" and "instanceType" in args.props:
        actualInstanceType = args.props["instanceType"]
        if actualInstanceType != REQUIRED_INSTANCE_TYPE:
            report_violation("Invalid instance type: '{}'. E2 instances must use '{}' instance type.".format(actualInstanceType, REQUIRED_INSTANCE_TYPE))

ec2_instance_type_restricted_policy = ResourceValidationPolicy(
    name="ec2-instance-type-restricted",
    description="Ensures EC2 instances use approved instance type.",
    validate=ec2_instance_type_restricted,
    enforcement_level=EnforcementLevel.MANDATORY
)

# AWS Resource Tags Policy
def all_aws_resources_must_have_tags(args: ResourceValidationArgs, report_violation: ReportViolation):
    if args.resource_type.startswith("aws"):
        tags = {}
        if 'tags' in args.props:
            tags = args.props['tags']
        if len(tags) == 0:
            report_violation("All AWS resources must have at least one tag.")

all_aws_resources_must_have_tags_policy = ResourceValidationPolicy(
    name="all-aws-resources-must-have-tags",
    description="Ensures all AWS resources have at least one tag.",
    validate=all_aws_resources_must_have_tags,
    enforcement_level=EnforcementLevel.MANDATORY
)

Here we define three different policies:

s3-product-prefix: Ensures S3 buckets are prefixed with the product name by checking the bucketPrefix property on all aws:s3:Bucket resources.
ec2-instance-type-restricted: Restricts EC2 instance types to only use the affordable t2.micro type, by checking the instanceType property on all aws:ec2/instance:Instance resources.
all-aws-resources-must-have-tags: Ensures all AWS resources have at least one tag by checking the tags property on all resources who’s type starts with aws.

Each of the policies uses the same pattern:

Define a ResourceValidationPolicy with a bit of metadata and a validation function. Each policy can have an individual enforcement level, name, and description.
The validation function is defined separately and takes as its inputs an args object of type ResourceValidationArgs and report_violation, a function of type ReportViolation. The args object contains information about the resource to test, and the report_violation function is used to report policy violations.
The first step in a resource policy should always be to check the resource type using the args.resource_type property, to make sure it is something you want to act on. The way Crossguard resource policies work, each resource in the stack will be passed to every policy in the pack, so filtering out resources that don’t relate to this check allows the policy engine to move on to the next policy/resource quickly.
Inside the validation function, we can check one or more properties on via the args.props property bag.
If there is a problem with the a property value, or some other aspect of the resource is out of compliance, we use the reportViolation function that was passed in to indicate that there’s a problem. The error message should be a full sentence and give useful information on how to remediate the problem.

Sometimes you might need to write a policy that works against more than one type of resource. For example, the all-aws-resources-must-have-tags policy applies to every kind of AWS resource by checking if the resource type string starts with aws.

$ pulumi stack
[...]
Current stack resources (5):
    TYPE                                    NAME
    pulumi:pulumi:Stack                     custom-policy-pack-integration-test-dev
    ├─ aws:s3:Bucket                        my-bucket
    ├─ aws:ec2/securityGroup:SecurityGroup  ssh-security-group
    ├─ aws:ec2/instance:Instance            web-server
    └─ pulumi:providers:aws                 default_6_65_0

Finally we assemble the policies into a policy pack object in __main__.py, giving it the name custom-policy-pack.

from pulumi_policy import (
    PolicyPack,
)
import policies

# Create PolicyPack
PolicyPack(
    name="custom-policy-pack",
    policies=[
        policies.s3_product_prefix_policy, 
        policies.ec2_instance_type_restricted_policy,
        policies.all_aws_resources_must_have_tags_policy
    ]
)

Writing Testable Code: We’ve structured this code to be a little bit more readable and testable than the template example. In the full version of this example, the PolicyPack definition is in __main__.py while the policies are in policies.py. This allows us to import the policies into to the testing framework, without including the PolicyPack, which would cause a unit test run to hang. Have a look at the policy_tests.py file to see how you can write unit tests for each policy.

Next up, we’ll validate our custom policy pack by checking some cloud resources for compliance. Let’s go!

← Back to Creating a Custom Policy Pack Next: Validate a Custom Policy Pack →

Neo just got smarter about infrastructure policy automation