Auto-scaling Redis users for load adjustment

Question

Pulumi · Accepted Answer

Auto-scaling a Redis instance to adjust the number of users based on the load implies the need for an auto-scaling arrangement that dynamically changes the capacity of the Redis instance cluster based on demand or load metrics. However, Redis itself does not have a native auto-scaling feature; instead, you will typically need to use the auto-scaling capabilities of the cloud provider where the Redis service is hosted.

Assuming you are using AWS, you can utilize AWS ElastiCache for Redis along with AWS Application Auto Scaling to manage the scaling. This combination enables you to adjust the number of replicas in your Redis (cluster mode enabled) replication groups in response to load.

Here’s a basic outline of how you could set this up with Pulumi in TypeScript:

1. Deploy an ElastiCache for Redis cluster with the desired initial size.
2. Configure an Application Auto Scaling target and policy for the ElastiCache Redis cluster.

The following example sets up an autoscaling policy for an AWS ElastiCache for Redis cluster. It assumes that you have already configured your AWS credentials and Pulumi setup to interact with your AWS account.

## Pulumi TypeScript Program for Auto-Scaling AWS ElastiCache for Redis

```typescript
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create an AWS ElastiCache replication group for Redis
const redisReplicationGroup = new aws.elasticache.ReplicationGroup("my-redis", {
    replicationGroupDescription: "example",
    automaticFailoverEnabled: true,
    engine: "redis",
    engineVersion: "6.x", // Use appropriate engine version
    nodeType: "cache.t3.micro",
    numberCacheClusters: 2, // Initial number of cache clusters
    parameterGroupName: "default.redis6.x",
    port: 6379,
});

// Reference to the Replication Group ID, to be used in scaling policy
const replicationGroupId = redisReplicationGroup.id;

// Create Application Auto Scaling Target
const redisScalableTarget = new aws.appautoscaling.Target("redisTarget", {
    maxCapacity: 5,
    minCapacity: 2,
    resourceId: pulumi.interpolate`replication-group/${replicationGroupId}`,
    scalableDimension: "elasticache:replication-group:ReplicaCount",
    serviceNamespace: "elasticache",
});

// Create Application Auto Scaling Policy
const redisScalingPolicy = new aws.appautoscaling.Policy("redisPolicy", {
    policyType: "TargetTrackingScaling",
    resourceId: pulumi.interpolate`replication-group/${replicationGroupId}`,
    scalableDimension: "elasticache:replication-group:ReplicaCount",
    serviceNamespace: "elasticache",
    targetTrackingScalingPolicyConfiguration: {
        predefinedMetricSpecification: {
            predefinedMetricType: "ElastiCachePrimaryEngineCPUUtilization",
        },
        scaleInCooldown: 300,
        scaleOutCooldown: 300,
        targetValue: 50.0,
    },
});

// Export the Redis endpoint
export const redisEndpoint = redisReplicationGroup.primaryEndpointAddress;
```

Explanation:

- We begin by importing the `pulumi` and `aws` modules so that we can interact with AWS services.
- We instantiate an `aws.elasticache.ReplicationGroup` to create a Redis replication group. You can configure the properties like `engineVersion`, `nodeType`, etc., according to your requirements.
- With the replication group's ID, we create a scalable target using `aws.appautoscaling.Target`. This object represents our Redis replication group concerning Application Auto Scaling.
- The `aws.appautoscaling.Policy` is configured to use a target tracking scaling policy based on a predefined metric type, which is `ElastiCachePrimaryEngineCPUUtilization` in this case. This policy will adjust the number of replicas to keep the CPU utilization around the target value (50% in the given example). The `scaleInCooldown` and `scaleOutCooldown` parameters configuration ensures that there is a buffer period before scaling actions take place after the previous action to allow the metrics to stabilize.
- At the end, we export the Redis endpoint which you can use to connect your applications to the Redis cluster.

Remember that the values for `scaleInCooldown`, `scaleOutCooldown`, and `targetValue` should be adjusted based on the actual load characteristics of your application and your performance goals.

This program sets up a basic autoscaling scenario. Depending on your use case, you might need to adjust the Redis configurations, include other related resources, or add additional policies.