Gitlab CI/CD Basics

GitLab CI/CD allows you to automate workflows directly from your GitLab repository. It enables you to build, test, and deploy your code based on specific events (such as code pushes, merge requests)

Key Concepts of GitLab CI/CD

• Pipelines: Automated processes that you define in your repository.

• Stages: Groups of jobs that run sequentially (e.g., build, test, deploy).

• Jobs: Tasks that run in parallel within a stage.

• Scripts: Commands that run in the job's environment.

• Runners: Servers that execute the jobs in your pipeline.

Adding a GitLab CI/CD Pipeline to Your Repository

Create a .gitlab-ci.yml File:

• Create a .gitlab-ci.yml file in the root directory of your repository.

• Use YAML syntax to define your pipeline.

stages:
  - test

variables:
  # These variables need to be defined in GitLab CI/CD settings
  # The $ syntax is how GitLab references variables
  OPENAI_API_KEY: $OPENAI_API_KEY
  ARIZE_API_KEY: $ARIZE_API_KEY
  SPACE_ID: $SPACE_ID
  DATASET_ID: $DATASET_ID

llm-experiment-job:
  stage: test
  image: python:3.10
  # The 'only' directive specifies when this job should run
  # This will run for merge requests that change files in copilot/search
  only:
    refs:
      - merge_requests
    changes:
      - copilot/search/**/*
  script:
    - pip install -q arize==7.36.0 arize-phoenix==4.29.0 nest_asyncio packaging openai 'gql[all]'
    - python ./copilot/experiments/ai_search_test.py
  artifacts:
    paths:
      - experiment_results.json
    expire_in: 1 week

Breakdown:

• stages: The stages that will run as part of the pipeline.

• variables: Environment variables for your pipeline.

• llm-experiment-job: The name of the job that will run.

• stage: Specifies which stage the job belongs to.

• image: The Docker image to use for the job.

• only: Conditions that determine when the job will run.

• script: Commands to run in the job.

• artifacts: Files to save after the job completes.

Common only Event Options:

1. Merge Requests: Triggers the pipeline when a merge request is created or updated.

only:
  - merge_requests

1. Specific Branches: Triggers the pipeline when code is pushed to specific branches.

only:
  - main
  - develop

1. Changes to Specific Files: Triggers the pipeline when specific files are changed.

only:
  changes:
    - copilot/search/**/*

1. Scheduled Pipelines: Triggers the pipeline on a scheduled time.

only:
  - schedules

1. Tags: Triggers the pipeline when a tag is created.

only:
  - tags

1. Combining Multiple Conditions: You can combine multiple conditions for more granular control.

only:
  refs:
    - merge_requests
    - main
  changes:
    - copilot/search/**/*