Create a Custom Policy Pack
In this step, we’ll create a policy pack to enforce the following rules for AWS resources:
- S3 buckets must be prefixed with the product name
myproduct-
. - EC2 instances must use the
t2.micro
instance type. - All AWS resources must have at least one tag defined.
Set up your policy pack project
First, create a new directory for your policy pack project:
mkdir custom-policy-pack
cd custom-policy-pack
Then, initialize your project. Choose Python or TypeScript based on your preferred language.
pulumi policy new aws-typescript
This will create the following files and directories:
PulumiPolicy.yaml
: A Pulumi project file that indicates this a policy pack.index.ts
: The TypeScript entry point where the policies will be defined in code.node_modules/
: The NPM modules directorypackage-lock.json
: A list of the module dependencies used bynpm
.package.json
: Thenpm
package description file.tsconfig.json
: The TypeScript configuration file.
aws-typescript
policy pack template. To see the full list of available policy pack templates, check out the pulumi/templates-policy
GitHub repository.pulumi policy new aws-python
This will create the following files and directories:
PulumiPolicy.yaml
: A Pulumi project file that indicates this a policy pack.__main__.py
: The Python entry point where the policies will be defined in code.requirements.txt
: A list of the module dependencies used bypip
.venv/
: The Python virtual environment.
aws-python
policy pack template. To see the full list of available policy pack templates, check out the pulumi/templates-policy
GitHub repository.This will initialize your project, creating the necessary files for Pulumi to use as a policy, including module dependencies to the providers that will let us interact with AWS resources.
Define Policies
Policies are written in Python or TypeScript. Like Pulumi Programs, you can use the full power of your preferred language, including standard features like leveraging third-party modules, using conditional logic and control flow, and can be validated with unit testing frameworks.
By default the template sets up an example resource policy that prevents S3 buckets from being publically readable:
File: custom-policy-pack/index.ts
import * as aws from "@pulumi/aws";
import { PolicyPack, validateResourceOfType } from "@pulumi/policy";
new PolicyPack("aws-typescript", {
policies: [{
name: "s3-no-public-read",
description: "Prohibits setting the publicRead or publicReadWrite permission on AWS S3 buckets.",
enforcementLevel: "mandatory",
validateResource: validateResourceOfType(aws.s3.Bucket, (bucket, args, reportViolation) => {
if (bucket.acl === "public-read" || bucket.acl === "public-read-write") {
reportViolation(
"You cannot set public-read or public-read-write on an S3 bucket. " +
"Read more about ACLs here: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html");
}
}),
}],
});
File: custom-policy-pack/__main__.py
from pulumi_policy import (
EnforcementLevel,
PolicyPack,
ReportViolation,
ResourceValidationArgs,
ResourceValidationPolicy,
)
def s3_no_public_read_validator(args: ResourceValidationArgs, report_violation: ReportViolation):
if args.resource_type == "aws:s3/bucket:Bucket" and "acl" in args.props:
acl = args.props["acl"]
if acl == "public-read" or acl == "public-read-write":
report_violation(
"You cannot set public-read or public-read-write on an S3 bucket. " +
"Read more about ACLs here: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html")
s3_no_public_read = ResourceValidationPolicy(
name="s3-no-public-read",
description="Prohibits setting the publicRead or publicReadWrite permission on AWS S3 buckets.",
enforcement_level=EnforcementLevel.MANDATORY,
validate=s3_no_public_read_validator,
)
PolicyPack(
name="aws-python",
policies=[
s3_no_public_read,
],
)
Here you can see the basic structure of a policy pack:
- Some imports from the Pulumi Crossguard SDK
- a function that implements the policy
- a policy definition that wraps the implementation and describes the policy
- a policy pack definition that packages the policies together
While this example is a useful policy, it’s not what we need right now. Let’s delete all of that code and create some new custom policies.
Replace the contents of index.ts
with the following:
import * as aws from "@pulumi/aws";
import { PolicyPack, ReportViolation, ResourceValidationArgs, validateResourceOfType, ResourceValidationPolicy } from "@pulumi/policy";
export const REQUIRED_S3_PREFIX: string = "myproduct-";
// Policy: Ensure S3 buckets have product prefix enabled
export const s3ProductPrefixPolicy: ResourceValidationPolicy = {
name: "s3-product-prefix",
description: "Ensures S3 buckets have the correct product prefix.",
enforcementLevel: "mandatory",
validateResource: validateResourceOfType(aws.s3.BucketV2, (bucket, args, reportViolation) => {
const prefix = bucket.bucketPrefix || "";
if (prefix != REQUIRED_S3_PREFIX) {
reportViolation(`Invalid prefix: '${prefix}'. S3 buckets must use '${REQUIRED_S3_PREFIX}' prefix.`);
}
}),
};
export const REQUIRED_INSTANCE_TYPE: string = "t2.micro";
// Policy: Restrict EC2 instance types
export const ec2InstanceTypeRestrictedPolicy: ResourceValidationPolicy = {
name: "ec2-instance-type-restricted",
description: "Ensures EC2 instances use approved instance type.",
enforcementLevel: "mandatory",
validateResource: validateResourceOfType(aws.ec2.Instance, (instance, args, reportViolation) => {
const instanceType = instance.instanceType || "";
if (instanceType !== REQUIRED_INSTANCE_TYPE) {
reportViolation(`Invalid instance type: '${instanceType}'. EC2 instances must use '${REQUIRED_INSTANCE_TYPE}' instance type.`);
}
}),
};
// Policy: Ensure all AWS resources have at least one tag
export const allAwsResourcesMustHaveTagsPolicy: ResourceValidationPolicy = {
name: "all-aws-resources-must-have-tags",
description: "Ensures all AWS resources have at least one tag.",
enforcementLevel: "mandatory",
validateResource: (args: ResourceValidationArgs, reportViolation: ReportViolation) => {
if (args.type.startsWith("aws")) {
const tags = args.props.tags || {};
if (Object.keys(tags).length === 0) {
reportViolation("All AWS resources must have at least one tag.");
}
}
},
};
new PolicyPack("custom-policy-pack", {
policies: [s3ProductPrefixPolicy, ec2InstanceTypeRestrictedPolicy, allAwsResourcesMustHaveTagsPolicy],
});
Here we define three different policies:
- s3-product-prefix: Ensures S3 buckets are prefixed with the product name by checking the
bucketPrefix
property on allaws:s3/bucket:BucketV2
resources. - ec2-instance-type-restricted: Restricts EC2 instance types to only use the affordable
t2.micro
type, by checking theinstanceType
property on allaws:ec2/instance:Instance
resources. - all-aws-resources-must-have-tags: Ensures all AWS resources have at least one tag by checking the
tags
property on all resources who’s type starts withaws
.
Each of the policies uses the same pattern:
- Define a
ResourceValidationPolicy
with a bit of metadata and a validation function. Each policy can have an individual enforcement level, name, and description. - The validation function is defined inline using the
validateResourceOfType
helper function. The way Crossguard resource policies work, each resource in the project will be passed to every policy in the pack. This strongly-typed helper function creates a filter that checks the resource type (e.g.aws.ec2.Instance
from thepulumi-aws
provider library) and will only run the validation function on resources that match the type. - The validation functions take an instance of the resource, an args property bag, and a function for reporting policy violations. Inside the validation function, we can check one or more properties on the strongly-typed resource instance.
- If there is a problem with the property value, or some other aspect of the resource is out of compliance, we use the
reportViolation
function that was passed in to indicate that there’s a problem. The error message should be a full sentence and give useful information on how to remediate the problem.
What’s great about the strong typing in our SDK is that your IDE’s intellisense knows the type definition and can give you dot-completion for properties on the resource instance, so there’s no guessing about what the property is called, or what its type should be.
That said, sometimes you might need to write a policy that works against more than one resource type. For example, the all-aws-resources-must-have-tags
policy checks every kind of AWS resource. In this case, instead of using the validateResourceOfType
helper function, we just pass the anonymous function directly. Then, in the function we need to check the resource type as a string by inspecting the value of args.type
.
If you’re not sure what the correct resource type string is for your particular set of resources, you can run pulumi stack
to list the resources in your current stack. The TYPE
column of the output contains the same resource types strings that you would use to filter on inside of a policy. In our above example, we apply the policy to any resource who’s type starts with aws
.
$ pulumi stack
[...]
Current stack resources (5):
TYPE NAME
pulumi:pulumi:Stack custom-policy-pack-integration-test-dev
├─ aws:s3/bucketV2:BucketV2 my-bucket
├─ aws:ec2/securityGroup:SecurityGroup ssh-security-group
├─ aws:ec2/instance:Instance web-server
└─ pulumi:providers:aws default_6_65_0
Finally we assemble the policies into a policy pack object, giving it the name custom-policy-pack
.
Writing Testable Code: We’ve structured this code to be a little bit more readable than the template example, and also more testable. Instead of defining everything in one big nested object, we break each policy out into its own definition and then assemble the policy pack at the end. We use the export
keyword to make these policies available to the testing framework (although not technically necessary for the policy pack itself).
Have a look at the test
directory in the full version of this example to see how you can write unit tests for each policy.
Replace the contents of __main__.py
with the following:
from pulumi_policy import (
EnforcementLevel,
ReportViolation,
ResourceValidationArgs,
ResourceValidationPolicy,
)
# S3 Bucket Policy
REQUIRED_S3_PREFIX="myproduct-"
def s3_product_prefix(args: ResourceValidationArgs, report_violation: ReportViolation):
if args.resource_type == "aws:s3/bucketV2:BucketV2" and "bucketPrefix" in args.props:
actualPrefix = args.props["bucketPrefix"]
if actualPrefix != REQUIRED_S3_PREFIX:
report_violation("Invalid prefix: '{}'. S3 buckets must use '{}' prefix.".format(actualPrefix, REQUIRED_S3_PREFIX))
s3_product_prefix_policy = ResourceValidationPolicy(
name="s3-product-prefix",
description="Ensures S3 buckets have the correct product prefix.",
validate=s3_product_prefix,
enforcement_level=EnforcementLevel.MANDATORY
)
# EC2 Instance Policy
REQUIRED_INSTANCE_TYPE="t2.micro"
def ec2_instance_type_restricted(args: ResourceValidationArgs, report_violation: ReportViolation):
if args.resource_type == "aws:ec2/instance:Instance" and "instanceType" in args.props:
actualInstanceType = args.props["instanceType"]
if actualInstanceType != REQUIRED_INSTANCE_TYPE:
report_violation("Invalid instance type: '{}'. E2 instances must use '{}' instance type.".format(actualInstanceType, REQUIRED_INSTANCE_TYPE))
ec2_instance_type_restricted_policy = ResourceValidationPolicy(
name="ec2-instance-type-restricted",
description="Ensures EC2 instances use approved instance type.",
validate=ec2_instance_type_restricted,
enforcement_level=EnforcementLevel.MANDATORY
)
# AWS Resource Tags Policy
def all_aws_resources_must_have_tags(args: ResourceValidationArgs, report_violation: ReportViolation):
if args.resource_type.startswith("aws"):
tags = {}
if 'tags' in args.props:
tags = args.props['tags']
if len(tags) == 0:
report_violation("All AWS resources must have at least one tag.")
all_aws_resources_must_have_tags_policy = ResourceValidationPolicy(
name="all-aws-resources-must-have-tags",
description="Ensures all AWS resources have at least one tag.",
validate=all_aws_resources_must_have_tags,
enforcement_level=EnforcementLevel.MANDATORY
)
Here we define three different policies:
- s3-product-prefix: Ensures S3 buckets are prefixed with the product name by checking the
bucketPrefix
property on allaws:s3/bucket:BucketV2
resources. - ec2-instance-type-restricted: Restricts EC2 instance types to only use the affordable
t2.micro
type, by checking theinstanceType
property on allaws:ec2/instance:Instance
resources. - all-aws-resources-must-have-tags: Ensures all AWS resources have at least one tag by checking the
tags
property on all resources who’s type starts withaws
.
Each of the policies uses the same pattern:
- Define a
ResourceValidationPolicy
with a bit of metadata and a validation function. Each policy can have an individual enforcement level, name, and description. - The validation function is defined separately and takes as its inputs an
args
object of typeResourceValidationArgs
andreport_violation
, a function of typeReportViolation
. Theargs
object contains information about the resource to test, and thereport_violation
function is used to report policy violations. - The first step in a resource policy should always be to check the resource type using the
args.resource_type
property, to make sure it is something you want to act on. The way Crossguard resource policies work, each resource in the stack will be passed to every policy in the pack, so filtering out resources that don’t relate to this check allows the policy engine to move on to the next policy/resource quickly. - Inside the validation function, we can check one or more properties on via the
args.props
property bag. - If there is a problem with the a property value, or some other aspect of the resource is out of compliance, we use the
reportViolation
function that was passed in to indicate that there’s a problem. The error message should be a full sentence and give useful information on how to remediate the problem.
If you’re not sure what the correct resource type string is for your particular set of resources, you can run pulumi stack
to list the resources in your current stack. The TYPE
column of the output contains the same resource types strings that you would use to filter on inside of a policy.
Sometimes you might need to write a policy that works against more than one type of resource. For example, the all-aws-resources-must-have-tags
policy applies to every kind of AWS resource by checking if the resource type string starts with aws
.
$ pulumi stack
[...]
Current stack resources (5):
TYPE NAME
pulumi:pulumi:Stack custom-policy-pack-integration-test-dev
├─ aws:s3/bucketV2:BucketV2 my-bucket
├─ aws:ec2/securityGroup:SecurityGroup ssh-security-group
├─ aws:ec2/instance:Instance web-server
└─ pulumi:providers:aws default_6_65_0
Finally we assemble the policies into a policy pack object in __main__.py
, giving it the name custom-policy-pack
.
from pulumi_policy import (
PolicyPack,
)
import policies
# Create PolicyPack
PolicyPack(
name="custom-policy-pack",
policies=[
policies.s3_product_prefix_policy,
policies.ec2_instance_type_restricted_policy,
policies.all_aws_resources_must_have_tags_policy
]
)
PolicyPack
definition is in __main__.py
while the policies are in policies.py
. This allows us to import the policies into to the testing framework, without including the PolicyPack
, which would cause a unit test run to hang. Have a look at the policy_tests.py
file to see how you can write unit tests for each policy.Next up, we’ll validate our custom policy pack by checking some cloud resources for compliance. Let’s go!