Real-Time Analytics with MongoDB Atlas for AI Insights

Question

Pulumi · Accepted Answer

To set up a real-time analytics platform with MongoDB Atlas that can power AI insights, you will need to provision and configure a MongoDB Atlas cluster, where your data will be stored and analyzed. Pulumi makes it easy to declare and manage cloud infrastructure, and we can use the `pulumi_mongodbatlas` package to work with MongoDB Atlas.

Below is a Pulumi program in Python that demonstrates how to create a MongoDB Atlas project, configure a cluster with MongoDB Atlas for real-time analytics, and add auditing support for the cluster to track access and changes. The cluster is set up with a specific instance size and the capability to scale as per demand, which is important for real-time analytics workloads.

We will also enable encryption at rest to ensure data security, which is critical for real-time analytics platforms, especially when working with sensitive data for AI insights.

Here's a step-by-step guide on what the following Pulumi program is doing:

1. We will start by importing the necessary Pulumi and `pulumi_mongodbatlas` libraries.
2. We'll then create a MongoDB Atlas project which acts as a container for the cluster and other MongoDB Atlas resources.
3. We will provision a MongoDB Atlas cluster within the project, tailored for high performance, which includes setting the instance size, enabling auto-scaling, and configuring backups and encryption at rest.
4. We will set up auditing for the cluster to ensure all access and operations on the cluster are tracked, which is important for maintaining compliance and monitoring the platform's usage.
5. Finally, we'll export the connection string for the Atlas cluster, which can be used by your applications to connect to the database.

Please remember that to run this Pulumi program, you need to have the Pulumi CLI installed and configured with the appropriate access credentials for MongoDB Atlas. You also need to have Python installed along with the `pulumi` and `pulumi_mongodbatlas` packages.

Here's the Pulumi program:

```python
import pulumi
import pulumi_mongodbatlas as mongodbatlas

# Configure MongoDB Atlas Provider with API Keys using Pulumi configuration or environment variables
# API keys need to be generated from the MongoDB Atlas user interface

# Create a new MongoDB Atlas Project
project = mongodbatlas.Project("my-project",
                               org_id="your_org_id") # Replace your_org_id with your MongoDB Atlas organization ID.

# Create a MongoDB Atlas Cluster within the project
cluster = mongodbatlas.Cluster("my-cluster",
                               project_id=project.id,
                               provider_name="AWS",  # Specify the cloud provider for the cluster.
                               provider_region_name="us_east_1",  # Set the cluster region location.
                               provider_instance_size_name="M10",  # Choose the instance size.
                               cluster_type="REPLICASET",  # Cluster type set as replica set.
                               backup_enabled=True,  # Enable cloud provider snapshots.
                               provider_backup_enabled=True,  # Enable MongoDB's cloud backup system.
                               disk_size_gb=pulumi.Input(10),  # Set the disk size in GB.
                               encryption_at_rest_provider="AWS",  # Enable encryption at rest using AWS KMS.
                               auto_scaling_compute_enabled=True,  # Enable auto-scaling for the cluster's compute resources.
                               auto_scaling_compute_scale_down_enabled=True)  # Allow auto-scaling to scale down if needed.

# Set up auditing for the cluster
auditing = mongodbatlas.Auditing("my-auditing-setup",
                                 project_id=project.id,
                                 enabled=True,  # Enable auditing.
                                 audit_filter="""{
                                                "atype": {"$in": ["authenticate", "createIndex", "dropCollection"]}
                                              }""")  # Define filters for auditing logs.

# Export the connection strings for the MongoDB Atlas cluster
# The connection strings can be used in your application to connect to the MongoDB Atlas cluster
pulumi.export("mongodb_cluster_uri", cluster.connection_strings["standard"])
pulumi.export("mongodb_cluster_srv_uri", cluster.connection_strings["standard_srv"])
```

Replace `'your_org_id'` with the appropriate Organization ID from MongoDB Atlas. The `auto_scaling_compute_enabled` and `auto_scaling_compute_scale_down_enabled` properties allow the cluster to automatically scale its compute resources, which is ideal for real-time analytics workloads that may have variable demand.

In the audit filter, we are currently tracking `authenticate`, `createIndex`, and `dropCollection` actions. You can modify this to include other events based on your needs.

After applying this configuration, you will have a MongoDB Atlas cluster ready to receive data and perform real-time analytics, which can be used to garner insights for AI applications. The exposed connection strings in the output can be leveraged by your applications or analytics tools to interact with the database cluster.