GitLab Variables for Conditional AI Pipeline Execution

Conditioning CI/CD with GitLab Variables

In GitLab CI/CD, variables are essential components that can control the flow of the pipeline execution. They can be defined at various levels such as the project, group, or instance level. These variables can be used within your .gitlab-ci.yml file to conditionally execute jobs, allowing for dynamic and flexible pipeline configurations.

Let's go through the steps to create GitLab variables using Pulumi, with a focus on setting up project-level variables. We will use the gitlab.ProjectVariable resource from the Pulumi GitLab package (pulumi_gitlab) for this. These variables will be the key to conditionally executing pipeline jobs.

Here's a Python program that sets up project variables in GitLab using Pulumi:

Install the pulumi_gitlab provider, if you don't have it already:

pip install pulumi_gitlab
In your Pulumi program, import the required packages.
Define project variables using gitlab.ProjectVariable. You need to provide the key, value, and project ID.

The following program defines two project variables:

AI_ENABLED: A boolean variable that we can use to determine whether AI-related jobs should run.
AI_MODEL_PATH: A string variable that defines the path to the AI model, which can be used conditionally based on the AI_ENABLED variable.

import pulumi
import pulumi_gitlab as gitlab

# Define your GitLab project ID here
project_id = "your-gitlab-project-id"

# Create a project variable to control the AI pipeline execution
ai_enabled_variable = gitlab.ProjectVariable("ai-enabled-variable",
    project=project_id,
    key="AI_ENABLED",
    value="true",  # "true" to enable AI jobs, "false" to disable them
    protected=False,  # Set to `True` to protect the variable (only available in protected branches/tags)
    masked=False  # Set to `True` to mask the variable's value in the job logs
)

# Create another project variable for the AI model path
ai_model_path_variable = gitlab.ProjectVariable("ai-model-path-variable",
    project=project_id,
    key="AI_MODEL_PATH",
    value="/models/my-ai-model",  # The path to the AI model
    protected=False,
    masked=False
)

# Export the variable IDs for easy retrieval
pulumi.export('ai_enabled_variable_id', ai_enabled_variable.id)
pulumi.export('ai_model_path_variable_id', ai_model_path_variable.id)

These exported IDs allow you to reference or modify the variables outside of Pulumi if needed.

To use these variables in your GitLab CI/CD pipeline, reference them in your .gitlab-ci.yml file:

stages:
  - build
  - test
  - deploy

run_ai_job:
  stage: test
  script:
    - echo "Running AI job with model path $AI_MODEL_PATH"
  only:
    variables:
      - $AI_ENABLED == "true"  # This job runs only if AI_ENABLED is "true"

In this example, the run_ai_job will only execute if AI_ENABLED is set to "true". The script accesses the AI model path using the variable AI_MODEL_PATH, which we set in the Pulumi program.

By following these steps, you've seen how to use Pulumi to set up GitLab variables for conditional pipeline execution. Adjust the boolean value of AI_ENABLED to true or false based on whether you want to enable or disable AI-related jobs. This approach brings infrastructure as code principles to your CI/CD pipeline, making it more maintainable and manageable.