1. Tutorials
  2. Creating a Custom Policy Pack
  3. Validate a Custom Policy Pack

Validate a Custom Policy Pack

Test the Policy Pack

In the previous step we created a custom policy pack. Now, let’s see it in action. We’ll create some AWS resources that violate the policies, run our custom policy pack to check compliance, and then fix the resources to adhere to the policies.

Step 1: Create Non-Compliant Resources

First, create a new Pulumi project from a template:

$ mkdir custom-policy-pack-integration-test-typescript && cd custom-policy-pack-integration-test-typescript
$ pulumi new aws-typescript

Follow the prompts as usual to set up your project.

Below are examples of non-compliant resources defined in Pulumi. Replace the contents of index.ts with this code.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create an AWS S3 Bucket
const bucket = new aws.s3.BucketV2("my-bucket", {
    bucketPrefix: "something-unexpected-",
    tags: {},
});

// Export the name of the bucket
export const bucketName = bucket.id;

// Create an AWS EC2 Instance

// Find an appropriate AMI
const ubuntu = aws.ec2.getAmi({
    mostRecent: true,
    filters: [
        {
            name: "name",
            values: ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"],
        },
        {
            name: "virtualization-type",
            values: ["hvm"],
        },
    ],
    owners: ["099720109477"], // Canonical
});

// Define a security group
// Create a new security group that permits SSH access.
const ssh_security_group = new aws.ec2.SecurityGroup("ssh-security-group", {
    description: "Enable SSH access",
    ingress: [{ protocol: "tcp", fromPort: 22, toPort: 22, cidrBlocks: ["0.0.0.0/0"] }],
});

// Define the EC2 instance
const instance = new aws.ec2.Instance("web-server", {
    instanceType: aws.ec2.InstanceType.T3_Micro, // Instance type
    ami: ubuntu.then(ubuntu => ubuntu.id),
    vpcSecurityGroupIds: [ssh_security_group.id],
    tags: {
        Name: "web-server",
    },
});

// Export the instance's public IP address
export const publicIp = instance.publicIp;
$ mkdir custom-policy-pack-integration-test-python && cd custom-policy-pack-integration-test-python
$ pulumi new aws-python

Follow the prompts as usual to set up your project.

Below are examples of non-compliant resources defined in Pulumi. Replace the contents of __main__.py with this code.

import pulumi
import pulumi_aws as aws

# Create an AWS S3 Bucket
bucket = aws.s3.BucketV2("my-bucket",
    bucket_prefix="something-unexpected-",
    tags={}
)

# Export the name of the bucket
pulumi.export('bucket_name', bucket.id)

# Find an appropriate AMI
ubuntu = aws.ec2.get_ami(
    most_recent=True,
    filters=[
        {
            'name': 'name', 
            'values': ['ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*']
        },
        {
            "name": "virtualization-type",
            "values": ["hvm"],
        }
    ],
    owners=['099720109477'], # Canonical
)

# Define a security group that permits SSH access
ssh_security_group = aws.ec2.SecurityGroup('ssh-security-group',
    description='Enable SSH access',
    ingress=[
        aws.ec2.SecurityGroupIngressArgs(
            protocol='tcp', from_port=22, to_port=22, cidr_blocks=['0.0.0.0/0'],
        )
    ]
)

# Define the EC2 instance
instance = aws.ec2.Instance('web-server',
    instance_type=aws.ec2.InstanceType.T3_MICRO, # Instance type
    ami=ubuntu.id,
    vpc_security_group_ids=[ssh_security_group.id],
    tags={
        "Name": "web-server",
    }
)

# Export the instance's public IP address
pulumi.export('public_ip', instance.public_ip)

This Pulumi project defines an S3 Bucket, a Security Group, and an EC2 instance. As written, these violate our custom policies in the following ways:

  • The Bucket has a tag property, but it’s empty.
  • The Bucket has a non-compliant prefix.
  • The Security Group has no tags property.
  • The EC2 instance uses a non-compliant instance type.

So, if we run pulumi preview on these with our custom policy pack applied, we should see four policy violations. Let’s try it!

Step 2: Run the Policy Pack

From the root of the Pulumi project, run pulumi preview with the --policy-pack option, pointing to the directory containing our custom policies. Pulumi will evaluate the policy pack against these resources and report any violations.

$ pulumi preview --policy-pack ../custom-policy-pack-typescript

Loading policy packs...

     Type                      Name                                                Plan
 +   pulumi:pulumi:Stack       custom-policy-pack-integration-test-typescript-dev  create
 +   ├─ aws:s3:BucketV2        my-bucket                                           create
 +   ├─ aws:ec2:Instance       web-server                                          create
 +   └─ aws:ec2:SecurityGroup  ssh-security-group                                  create

Policies:
    ❌ custom-policy-pack@v0.0.1 (local: ../custom-policy-pack-typescript)
        - [mandatory]  all-aws-resources-must-have-tags  (aws:ec2/securityGroup:SecurityGroup: ssh-security-group)
          Ensures all AWS resources have at least one tag.
          All AWS resources must have at least one tag.
        - [mandatory]  all-aws-resources-must-have-tags  (aws:s3/bucketV2:BucketV2: my-bucket)
          Ensures all AWS resources have at least one tag.
          All AWS resources must have at least one tag.
        - [mandatory]  ec2-instance-type-restricted  (aws:ec2/instance:Instance: web-server)
          Ensures EC2 instances use approved instance type.
          Invalid instance type: 't3.micro'. EC2 instances must use 't2.micro' instance type.
        - [mandatory]  s3-product-prefix  (aws:s3/bucketV2:BucketV2: my-bucket)
          Ensures S3 buckets have the correct product prefix.
          Invalid prefix: 'something-unexpected-'. S3 buckets must use 'myproduct-' prefix.
$ pulumi preview --policy-pack ../custom-policy-pack-python

Loading policy packs...

     Type                      Name                                            Plan
 +   pulumi:pulumi:Stack       custom-policy-pack-integration-test-python-dev  create
 +   ├─ aws:s3:BucketV2        my-bucket                                       create
 +   ├─ aws:ec2:Instance       web-server                                      create
 +   └─ aws:ec2:SecurityGroup  ssh-security-group                              create

Policies:
    ❌ custom-policy-pack@v0.0.1 (local: ../custom-policy-pack-python)
        - [mandatory]  all-aws-resources-must-have-tags  (aws:ec2/securityGroup:SecurityGroup: ssh-security-group)
          Ensures all AWS resources have at least one tag.
          All AWS resources must have at least one tag.
        - [mandatory]  all-aws-resources-must-have-tags  (aws:s3/bucketV2:BucketV2: my-bucket)
          Ensures all AWS resources have at least one tag.
          All AWS resources must have at least one tag.
        - [mandatory]  ec2-instance-type-restricted  (aws:ec2/instance:Instance: web-server)
          Ensures EC2 instances use approved instance type.
          Invalid instance type: 't3.micro'. EC2 instances must use 't2.micro' instance type.
        - [mandatory]  s3-product-prefix  (aws:s3/bucketV2:BucketV2: my-bucket)
          Ensures S3 buckets have the correct product prefix.
          Invalid prefix: 'something-unexpected-'. S3 buckets must use 'myproduct-' prefix.

Step 3: Fix the Resources

Ok, now that we see the expected violations, let’s update the resources to comply with the policies.

We need to:

  • Update the Bucket to use the correct prefix: myproduct-
  • Update the Bucket to have at least one tag
  • Update the Security Group to have at least one tag
  • Update the EC2 instance to use the correct instance type: t2.micro

Replace the contents of index.ts with this code.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create an AWS S3 Bucket
const bucket = new aws.s3.BucketV2("my-bucket", {
    bucketPrefix: "myproduct-",
    tags: {
        example: "tag value",
    },
});

// Export the name of the bucket
export const bucketName = bucket.id;

// Create an AWS EC2 Instance

// Find an appropriate AMI
const ubuntu = aws.ec2.getAmi({
    mostRecent: true,
    filters: [
        {
            name: "name",
            values: ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"],
        },
        {
            name: "virtualization-type",
            values: ["hvm"],
        },
    ],
    owners: ["099720109477"], // Canonical
});

// Define a security group
// Create a new security group that permits SSH access.
const ssh_security_group = new aws.ec2.SecurityGroup("ssh-security-group", {
    description: "Enable SSH access",
    ingress: [{ protocol: "tcp", fromPort: 22, toPort: 22, cidrBlocks: ["0.0.0.0/0"] }],
    tags: {
        example: "tag value",
    },
});

// Define the EC2 instance
const instance = new aws.ec2.Instance("web-server", {
    instanceType: aws.ec2.InstanceType.T2_Micro, // Instance type
    ami: ubuntu.then(ubuntu => ubuntu.id),
    vpcSecurityGroupIds: [ssh_security_group.id],
    tags: {
        Name: "web-server",
    },
});

// Export the instance's public IP address
export const publicIp = instance.publicIp;

Replace the contents of __main__.py with this code.

import pulumi
import pulumi_aws as aws

# Create an AWS S3 Bucket
bucket = aws.s3.BucketV2("my-bucket",
    bucket_prefix="myproduct-",
    tags={
        "example": "tag value",
    })

# Export the name of the bucket
pulumi.export('bucket_name', bucket.id)

# Find an appropriate AMI
ubuntu = aws.ec2.get_ami(
    most_recent=True,
    filters=[
        {
            'name': 'name', 
            'values': ['ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*']
        },
        {
            "name": "virtualization-type",
            "values": ["hvm"],
        }
    ],
    owners=['099720109477'], # Canonical
)

# Define a security group that permits SSH access
ssh_security_group = aws.ec2.SecurityGroup('ssh-security-group',
    description='Enable SSH access',
    ingress=[
        aws.ec2.SecurityGroupIngressArgs(
            protocol='tcp', from_port=22, to_port=22, cidr_blocks=['0.0.0.0/0'],
        )
    ],
    tags={
        "example": "tag value",
    }
)

# Define the EC2 instance
instance = aws.ec2.Instance('web-server',
    instance_type=aws.ec2.InstanceType.T2_MICRO, # Instance type
    ami=ubuntu.id,
    vpc_security_group_ids=[ssh_security_group.id],
    tags={
        "Name": "web-server",
    }
)

# Export the instance's public IP address
pulumi.export('public_ip', instance.public_ip)

Step 4: Verify Compliance

Run pulumi preview again. This time, no policy violations should be reported.

$ pulumi preview --policy-pack ../custom-policy-pack-typescript

Loading policy packs...

     Type                      Name                                                Plan
 +   pulumi:pulumi:Stack       custom-policy-pack-integration-test-typescript-dev  create
 +   ├─ aws:s3:BucketV2        my-bucket                                           create
 +   ├─ aws:ec2:Instance       web-server                                          create
 +   └─ aws:ec2:SecurityGroup  ssh-security-group                                  create

Policies:
    ✅ custom-policy-pack@v0.0.1 (local: ../custom-policy-pack-typescript)
$ pulumi preview --policy-pack ../custom-policy-pack-python

Loading policy packs...

     Type                      Name                                            Plan
 +   pulumi:pulumi:Stack       custom-policy-pack-integration-test-python-dev  create
 +   ├─ aws:s3:BucketV2        my-bucket                                       create
 +   ├─ aws:ec2:Instance       web-server                                      create
 +   └─ aws:ec2:SecurityGroup  ssh-security-group                              create

Policies:
    ✅ custom-policy-pack@v0.0.1 (local: ../custom-policy-pack-python)

Congratulations! You’ve successfully created and tested a policy pack with Pulumi CrossGuard.

Next Steps

To learn more about using Pulumi CrossGuard and policies, explore the following:

If you are using Pulumi Business Critical edition, you can also publish this policy to your Pulumi Cloud organization, which enables server-side policy enforcement on all Pulumi projects in your organization. See pricing for more details.