Create a Custom Policy Pack
In this step, we’ll create a policy pack to enforce the following rules for AWS resources:
- S3 buckets must be prefixed with the product name
myproduct-. - EC2 instances must use the
t2.microinstance type. - All AWS resources must have at least one tag defined.
Set up your policy pack project#
First, create a new directory for your policy pack project:
mkdir custom-policy-packcd custom-policy-packThen, initialize your project. Choose Python or TypeScript based on your preferred language.
pulumi policy new aws-typescriptThis will create the following files and directories:
PulumiPolicy.yaml: A Pulumi project file that indicates this a policy pack.index.ts: The TypeScript entry point where the policies will be defined in code.node_modules/: The NPM modules directorypackage-lock.json: A list of the module dependencies used bynpm.package.json: Thenpmpackage description file.tsconfig.json: The TypeScript configuration file.
pulumi policy new aws-pythonThis will create the following files and directories:
PulumiPolicy.yaml: A Pulumi project file that indicates this a policy pack.__main__.py: The Python entry point where the policies will be defined in code.requirements.txt: A list of the module dependencies used bypip.venv/: The Python virtual environment.
This will initialize your project, creating the necessary files for Pulumi to use as a policy, including module dependencies to the providers that will let us interact with AWS resources.
Define Policies#
Policies are written in Python or TypeScript. Like Pulumi Programs, you can use the full power of your preferred language, including standard features like leveraging third-party modules, using conditional logic and control flow, and can be validated with unit testing frameworks.
By default the template sets up an example resource policy that prevents S3 buckets from being publicly readable:
File: custom-policy-pack/index.ts
import * as aws from "@pulumi/aws";import { PolicyPack, validateResourceOfType } from "@pulumi/policy";
new PolicyPack("aws-typescript", { policies: [{ name: "s3-no-public-read", description: "Prohibits setting the publicRead or publicReadWrite permission on AWS S3 bucket ACLs.", enforcementLevel: "mandatory", validateResource: validateResourceOfType(aws.s3.BucketAcl, (bucketAcl, args, reportViolation) => { if (bucketAcl.acl === "public-read" || bucketAcl.acl === "public-read-write") { reportViolation( "You cannot set public-read or public-read-write on an S3 bucket ACL. " + "Read more about ACLs here: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html"); } }), }],});File: custom-policy-pack/__main__.py
from pulumi_policy import ( EnforcementLevel, PolicyPack, ReportViolation, ResourceValidationArgs, ResourceValidationPolicy,)
def s3_no_public_read_validator(args: ResourceValidationArgs, report_violation: ReportViolation): if args.resource_type == "aws:s3/bucketAcl:BucketAcl" and "acl" in args.props: acl = args.props["acl"] if acl == "public-read" or acl == "public-read-write": report_violation( "You cannot set public-read or public-read-write on an S3 bucket ACL. " + "Read more about ACLs here: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html")
s3_no_public_read = ResourceValidationPolicy( name="s3-no-public-read", description="Prohibits setting the publicRead or publicReadWrite permission on AWS S3 bucket ACLs.", enforcement_level=EnforcementLevel.MANDATORY, validate=s3_no_public_read_validator,)
PolicyPack( name="aws-python", policies=[ s3_no_public_read, ],)Here you can see the basic structure of a policy pack:
- Some imports from the Pulumi Crossguard SDK
- a function that implements the policy
- a policy definition that wraps the implementation and describes the policy
- a policy pack definition that packages the policies together
While this example is a useful policy, it’s not what we need right now. Let’s delete all of that code and create some new custom policies.
Replace the contents of index.ts with the following:
import * as aws from "@pulumi/aws";import { PolicyPack, ReportViolation, ResourceValidationArgs, validateResourceOfType, ResourceValidationPolicy } from "@pulumi/policy";
export const REQUIRED_S3_PREFIX: string = "myproduct-";
// Policy: Ensure S3 buckets have product prefix enabledexport const s3ProductPrefixPolicy: ResourceValidationPolicy = { name: "s3-product-prefix", description: "Ensures S3 buckets have the correct product prefix.", enforcementLevel: "mandatory", validateResource: validateResourceOfType(aws.s3.Bucket, (bucket, args, reportViolation) => { const prefix = bucket.bucketPrefix || ""; if (prefix != REQUIRED_S3_PREFIX) { reportViolation(`Invalid prefix: '${prefix}'. S3 buckets must use '${REQUIRED_S3_PREFIX}' prefix.`); } }),};
export const REQUIRED_INSTANCE_TYPE: string = "t2.micro";
// Policy: Restrict EC2 instance typesexport const ec2InstanceTypeRestrictedPolicy: ResourceValidationPolicy = { name: "ec2-instance-type-restricted", description: "Ensures EC2 instances use approved instance type.", enforcementLevel: "mandatory", validateResource: validateResourceOfType(aws.ec2.Instance, (instance, args, reportViolation) => { const instanceType = instance.instanceType || ""; if (instanceType !== REQUIRED_INSTANCE_TYPE) { reportViolation(`Invalid instance type: '${instanceType}'. EC2 instances must use '${REQUIRED_INSTANCE_TYPE}' instance type.`); } }),};
// Policy: Ensure all AWS resources have at least one tagexport const allAwsResourcesMustHaveTagsPolicy: ResourceValidationPolicy = { name: "all-aws-resources-must-have-tags", description: "Ensures all AWS resources have at least one tag.", enforcementLevel: "mandatory", validateResource: (args: ResourceValidationArgs, reportViolation: ReportViolation) => { if (args.type.startsWith("aws")) { const tags = args.props.tags || {}; if (Object.keys(tags).length === 0) { reportViolation("All AWS resources must have at least one tag."); } } },};
new PolicyPack("custom-policy-pack", { policies: [s3ProductPrefixPolicy, ec2InstanceTypeRestrictedPolicy, allAwsResourcesMustHaveTagsPolicy],});Here we define three different policies:
- s3-product-prefix: Ensures S3 buckets are prefixed with the product name by checking the
bucketPrefixproperty on allaws:s3:Bucketresources. - ec2-instance-type-restricted: Restricts EC2 instance types to only use the affordable
t2.microtype, by checking theinstanceTypeproperty on allaws:ec2/instance:Instanceresources. - all-aws-resources-must-have-tags: Ensures all AWS resources have at least one tag by checking the
tagsproperty on all resources who’s type starts withaws.
Each of the policies uses the same pattern:
- Define a
ResourceValidationPolicywith a bit of metadata and a validation function. Each policy can have an individual enforcement level, name, and description. - The validation function is defined inline using the
validateResourceOfTypehelper function. The way Crossguard resource policies work, each resource in the project will be passed to every policy in the pack. This strongly-typed helper function creates a filter that checks the resource type (e.g.aws.ec2.Instancefrom thepulumi-awsprovider library) and will only run the validation function on resources that match the type. - The validation functions take an instance of the resource, an args property bag, and a function for reporting policy violations. Inside the validation function, we can check one or more properties on the strongly-typed resource instance.
- If there is a problem with the property value, or some other aspect of the resource is out of compliance, we use the
reportViolationfunction that was passed in to indicate that there’s a problem. The error message should be a full sentence and give useful information on how to remediate the problem.
What’s great about the strong typing in our SDK is that your IDE’s intellisense knows the type definition and can give you dot-completion for properties on the resource instance, so there’s no guessing about what the property is called, or what its type should be.
That said, sometimes you might need to write a policy that works against more than one resource type. For example, the all-aws-resources-must-have-tags policy checks every kind of AWS resource. In this case, instead of using the validateResourceOfType helper function, we just pass the anonymous function directly. Then, in the function we need to check the resource type as a string by inspecting the value of args.type.
If you’re not sure what the correct resource type string is for your particular set of resources, you can run pulumi stack to list the resources in your current stack. The TYPE column of the output contains the same resource types strings that you would use to filter on inside of a policy. In our above example, we apply the policy to any resource who’s type starts with aws.
pulumi stack[...]Current stack resources (5): TYPE NAME pulumi:pulumi:Stack custom-policy-pack-integration-test-dev ├─ aws:s3:Bucket my-bucket ├─ aws:ec2/securityGroup:SecurityGroup ssh-security-group ├─ aws:ec2/instance:Instance web-server └─ pulumi:providers:aws default_6_65_0Finally we assemble the policies into a policy pack object, giving it the name custom-policy-pack.
Replace the contents of __main__.py with the following:
from pulumi_policy import ( PolicyPack,)import policies
# Create PolicyPackPolicyPack( name="custom-policy-pack", policies=[ policies.s3_product_prefix_policy, policies.ec2_instance_type_restricted_policy, policies.all_aws_resources_must_have_tags_policy ])Here we define three different policies:
- s3-product-prefix: Ensures S3 buckets are prefixed with the product name by checking the
bucketPrefixproperty on allaws:s3:Bucketresources. - ec2-instance-type-restricted: Restricts EC2 instance types to only use the affordable
t2.microtype, by checking theinstanceTypeproperty on allaws:ec2/instance:Instanceresources. - all-aws-resources-must-have-tags: Ensures all AWS resources have at least one tag by checking the
tagsproperty on all resources who’s type starts withaws.
Each of the policies uses the same pattern:
- Define a
ResourceValidationPolicywith a bit of metadata and a validation function. Each policy can have an individual enforcement level, name, and description. - The validation function is defined separately and takes as its inputs an
argsobject of typeResourceValidationArgsandreport_violation, a function of typeReportViolation. Theargsobject contains information about the resource to test, and thereport_violationfunction is used to report policy violations. - The first step in a resource policy should always be to check the resource type using the
args.resource_typeproperty, to make sure it is something you want to act on. The way Crossguard resource policies work, each resource in the stack will be passed to every policy in the pack, so filtering out resources that don’t relate to this check allows the policy engine to move on to the next policy/resource quickly. - Inside the validation function, we can check one or more properties on via the
args.propsproperty bag. - If there is a problem with the a property value, or some other aspect of the resource is out of compliance, we use the
reportViolationfunction that was passed in to indicate that there’s a problem. The error message should be a full sentence and give useful information on how to remediate the problem.
If you’re not sure what the correct resource type string is for your particular set of resources, you can run pulumi stack to list the resources in your current stack. The TYPE column of the output contains the same resource types strings that you would use to filter on inside of a policy.
Sometimes you might need to write a policy that works against more than one type of resource. For example, the all-aws-resources-must-have-tags policy applies to every kind of AWS resource by checking if the resource type string starts with aws.
pulumi stack[...]Current stack resources (5): TYPE NAME pulumi:pulumi:Stack custom-policy-pack-integration-test-dev ├─ aws:s3:Bucket my-bucket ├─ aws:ec2/securityGroup:SecurityGroup ssh-security-group ├─ aws:ec2/instance:Instance web-server └─ pulumi:providers:aws default_6_65_0Finally we assemble the policies into a policy pack object in __main__.py, giving it the name custom-policy-pack.
from pulumi_policy import ( PolicyPack,)import policies
# Create PolicyPackPolicyPack( name="custom-policy-pack", policies=[ policies.s3_product_prefix_policy, policies.ec2_instance_type_restricted_policy, policies.all_aws_resources_must_have_tags_policy ])Next up, we’ll validate our custom policy pack by checking some cloud resources for compliance. Let’s go!