GCP Artifact Registry for GitHub Organization (Part 1)

Today I want to describe how anyone can easily use GCP “Workload Identity Federation” feature to provide a secure way to push docker images from github actions CI to the GCP Artifact Registry

This topic is a kind of tutorial for people how have an organization in GitHub, but for a some reason want to store docker images in GCP Artifact Registry (f.e. speed up pulls inside GCP)

So, what we have and what we want to achieve?

We have:

an organization in GitHub.com, let’s say mnacharov-eu
a GCP project, let’s say mnacharov-eu-docker

Goal:

Have a private GCP Artifact Registry repository for each GitHub repository in our organization
Be able to push docker images from GitHub Actions CI to private GCP Artifact Registry repository without adding secrets to the github repository.
Easily manage people’s access to GCP Artifact Registry repositories

Authorize GitHubCI job to GCP via short-lived tokens

Let’s start from simple things: authorization via Workload Identity Federation. This feature is must be used in your GitHub CI Pipelines!

If you have ever used quite popular way to authorize github actions CI in google - google google-github-actions/auth action, you must noticed that the preferred way to use it is the “Direct Workload Identity Federation”. The setup instructions there is quite good, but in case you would like to have it in terraform spec, here is my setup:

variable "github_organization" {
  description = "The GitHub organization for the attribute condition."
  default     = "mnacharov-eu"
}

resource "google_iam_workload_identity_pool" "github_pool" {
  project                   = var.project_id
  workload_identity_pool_id = "github-com"
  display_name              = "GitHub identity pool"
}

resource "google_iam_workload_identity_pool_provider" "github_mnacharov-eu_repos" {
  project      = var.project_id
  display_name = "GitHub for mnacharov-eu repos"

  workload_identity_pool_id          = google_iam_workload_identity_pool.github_pool.workload_identity_pool_id
  workload_identity_pool_provider_id = "mnacharov-eu"

  attribute_mapping = {
    "google.subject"             = "assertion.sub"
    "attribute.repository"       = "assertion.repository"
    "attribute.repository_owner" = "assertion.repository_owner"
  }

  # Condition: specific github organization
  attribute_condition = <<EOT
  assertion.repository_owner == '${var.github_organization}'
  EOT

  oidc {
    issuer_uri = "https://token.actions.githubusercontent.com"
  }
}

after that I able to authenticate any github actions pipeline in my organization in google. But how can I add a permission to push images to the Artifact Registry? It will look like this:

locals {
  github_repos = [
    "push-to-gcp",
  ]
}

resource "google_artifact_registry_repository" "registries" {
  for_each      = toset(local.github_repos)
  repository_id = each.value
  description   = "github.com/mnacharov-eu/${each.value} images"
  format        = "DOCKER"
  location      = var.region
  project       = var.project_id

  docker_config {
    immutable_tags = true
  }
  cleanup_policies {
    id     = "keep-release-images"
    action = "KEEP"
    condition {
      tag_state    = "TAGGED"
      tag_prefixes = ["v"]
    }
  }
  cleanup_policies {
    id     = "delete-untagged"
    action = "DELETE"
    condition {
      tag_state    = "UNTAGGED"
      older_than   = "86400s"    # 1 day
    }
  }
  cleanup_policies {
    id     = "delete after 30d"
    action = "DELETE"
    condition {
      older_than   = "2592000s"  # 30 days
    }
  }
}

resource "google_artifact_registry_repository_iam_member" "allow_to_push_from_github" {
  for_each   = toset(local.github_repos)
  project    = var.project_id
  location   = google_artifact_registry_repository.registries[each.value].location
  repository = google_artifact_registry_repository.registries[each.value].name
  role       = "roles/artifactregistry.writer"
  member     = "principalSet://iam.googleapis.com/projects/${var.project_number}/locations/global/workloadIdentityPools/github-com/attribute.repository/mnacharov-eu/${each.value}"
}

so, github actions pipelines in repository mnacharov-eu/push-to-gcp will be able to push to the private Artifact Registry europe-docker.pkg.dev/mnacharov-eu-docker/push-to-gcp. Let’s test it via simple pipeline

on:
  schedule:
  - cron: '26 4 * * 4'
  push:
    branches:
    - main
    tags:
    - 'v*'

env:
  REGISTRY: europe-docker.pkg.dev

jobs:
  docker-build:
    runs-on: ubuntu-latest
    permissions:
      contents: 'read'
      id-token: 'write'

    steps:
    - uses: 'actions/checkout@v4'
    - uses: 'google-github-actions/auth@v2'
      with:
        project_id: 'mnacharov-eu-docker'
        workload_identity_provider: 'projects/152460024713/locations/global/workloadIdentityPools/github-com/providers/mnacharov-eu'
    - name: Docker meta
      id: meta
      uses: docker/metadata-action@v5
      with:
        images: europe-docker.pkg.dev/mnacharov-eu-docker/push-to-gcp/main
        flavor: |
          latest=false
        tags: |
          type=semver,pattern={{version}}
          type=sha,enable=${{ github.ref == format('refs/heads/{0}', 'main') && github.event.schedule == null }}
          type=raw,enable=${{ github.event.schedule != null }},value={{date 'cron-YY-MM-DD'}}
    - name: Configure GC docker
      run: |
        gcloud auth configure-docker europe-docker.pkg.dev
    - name: Build & Push Airflow
      uses: docker/build-push-action@v6
      with:
        push: true
        tags: ${{ steps.meta.outputs.tags }}
        labels: ${{ steps.meta.outputs.labels }}

it works, here is the link to the latest build https://github.com/mnacharov-eu/push-to-gcp/actions

That’s it for today

So, it looks like we already have a secure way to authorize and push images to Artifact Registry from GitHub Actions CI, but only for one repository

Next time I’ll scale up this approach to have an Artifact Registry repository with all required permissions in it right after GitHub repository creation! See you next time..