Automating GitLab backup to an external cloud storage service

Automating GitLab Backups using Google Cloud Storage

To automate GitLab backup to an external cloud storage service, we’ll walk through a Pulumi program in TypeScript. Specifically, we’ll create a Google Cloud Storage bucket where GitLab backups will be stored. The backup process itself would typically involve a scheduled task that creates GitLab backups and uses gsutil or the Google Cloud SDK to transfer them to the bucket. This Pulumi program will only handle the infrastructure setup, and you would need to configure GitLab and any necessary CI/CD pipelines to run the actual backup commands.

In the program below, we define a Google Cloud Storage bucket for storing the backups. The bucket is configured with versioning to keep a history of backups and lifecycle rules to automatically delete older backups, based on your retention policy.

Let's go step-by-step to set up a Google Cloud Storage (GCS) bucket where GitLab backups will be stored:

Prerequisites

Pulumi Account: You need a Pulumi account to manage the state of your infrastructure. Sign up for one here.
Google Cloud Project: You also need a Google Cloud project where the resources will be provisioned. If you don't have one, you can create it on the Google Cloud Console.
Pulumi Stack Configuration: Ensure your Pulumi stack is configured with the correct GCP project and region.

Pulumi Program:

import * as gcp from "@pulumi/gcp";

// Create a Google Cloud Storage bucket
const bucket = new gcp.storage.Bucket("gitlab-backup-bucket", {
    location: "US", // Choose the region that is close to your GitLab server
    forceDestroy: true, // This allows the bucket to be deleted even if it still contains objects
    versioning: {
        enabled: true, // Enable versioning to keep a history of backups
    },
    lifecycleRules: [
        {
            action: { type: "Delete" }, // Automatically delete objects
            condition: {
                // Conditions under which to delete. Adjust to your desired retention policy
                age: 30, // Number of days after which to auto-delete backups
                withState: "ANY", // Apply rule to both live and archived versions
            },
        },
    ],
});

// Export the bucket URL for easy access
export const bucketUrl = bucket.url;

// Additional: Create an IAM binding for the storage object viewer role
// This is optional and is meant for allowing read access to a particular entity
const bucketIamBinding = new gcp.storage.BucketIAMBinding("gitlab-backup-viewer", {
    bucket: bucket.name,
    role: "roles/storage.objectViewer",
    members: ["serviceAccount:your-backup-service-account@your-project-id.iam.gserviceaccount.com"],
});

// Note: Replace 'your-backup-service-account' and 'your-project-id' with your actual service account and project ID

This Pulumi program does the following:

Imports the @pulumi/gcp package to interact with Google Cloud resources.
Creates a new storage bucket with "US" as the location; you should replace this with the region closest to where your GitLab server is hosted.
Enables versioning on the bucket, so every time a backup is replaced, the old version is kept, which can be very useful for backup integrity and recovery.
Defines lifecycle rules with one condition set to delete objects older than 30 days, which helps you automate the cleanup process based on your backup retention policy.
Includes an optional BucketIAMBinding resource creation to grant read access to a service account. This should be adjusted to match the entity you wish to have read access to the backups.
Exports the bucket URL so that you can easily access it.

Integration with GitLab:

Once the bucket is created, you’ll need to set up GitLab for backups.
You'll configure GitLab with an appropriate backup command using GitLab's backup tools and schedule this through CRON jobs or GitLab's built-in backup scheduling.
The actual backup operation will need a script using the gcloud CLI or gsutil to copy the backup tarballs to the GCS bucket.

Security Note:

Be sure to manage the permissions and access correctly. In the code provided, we've added an IAM policy to allow a service account to view the objects in the bucket. Be careful with the privileges you grant and restrict the bucket access only to the authorized users or service accounts that require it.

This program sets up the necessary infrastructure, but the operational aspect of creating and transferring backups would be outside Pulumi's scope and is part of your operational procedures and scripting.