Skip to content

Runner Development Environment

Follow:

Preparations

git clone https://gitlab.com/gitlab-org/gitlab-runner.git && cd gitlab-runner

go get gitlab.com/gitlab-org/gitlab-runner

make deps

Manually install gox using Homebrew, go get did not install the binary into $GOPATH/bin for some reason. 

brew install gox

TODO: Figure out why and provide a workaround for the Makefile upstream, specifically for macOS builds.

Build

make help                                                                                                    
# Commands:
# make all => install deps and build Runner binaries and Helper images
# make version - show information about current version
#
# Development commands:
# make development_setup - setup needed environment for tests
# make runner-bin-host - build executable for your arch and OS
# make runner-and-helper-bin-host - build executable for your arch and OS, including docker dependencies
# make runner-and-helper-bin-linux - build executable for all supported architectures for linux OS, including docker dependencies
# make runner-and-helper-bin - build executable for all supported platforms, including docker dependencies
# make runner-and-helper-docker-host - build Alpine and Ubuntu Docker images with the runner executable and helper
# make helper-dockerarchive - build Runner Helper docker dependencies for all supported platforms
# make helper-dockerarchive-host - build Runner Helper docker dependencies for your oarch and OS
#
# Testing commands:
# make test - run project tests
# make lint - run code quality analysis
# make lint-docs - run documentation linting
#
# Deployment commands:
# make deps - install all dependencies
# make runner-bin - build project for all supported platforms
# make package - package project using FPM
# make packagecloud - send all packages to packagecloud
# make packagecloud-yank - remove specific version from packagecloud

Binaries and Helper Images

make runner-and-helper-bin-host

Fails.

make runner-and-helper-bin-host                                           ─╯
# Building gitlab-runner in version 14.11.0~beta.4.g9c8e5b22 for host platform
/Library/Developer/CommandLineTools/usr/bin/make runner-bin BUILD_PLATFORMS="-osarch=darwin/amd64"
go get github.com/mitchellh/gox
# Building gitlab-runner in version 14.11.0~beta.4.g9c8e5b22 for -osarch=darwin/amd64
gox -osarch=darwin/amd64 \
           -ldflags "-X gitlab.com/gitlab-org/gitlab-runner/common.NAME=gitlab-runner -X gitlab.com/gitlab-org/gitlab-runner/common.VERSION=14.11.0~beta.4.g9c8e5b22 -X gitlab.com/gitlab-org/gitlab-runner/common.REVISION=9c8e5b22 -X gitlab.com/gitlab-org/gitlab-runner/common.BUILT=2022-04-20T15:37:05+0000 -X gitlab.com/gitlab-org/gitlab-runner/common.BRANCH=otel -w" \
           -output="out/binaries/gitlab-runner-{{.OS}}-{{.Arch}}" \
           gitlab.com/gitlab-org/gitlab-runner
Number of parallel builds: 11

-->    darwin/amd64: gitlab.com/gitlab-org/gitlab-runner

1 errors occurred:
--> darwin/amd64 error: exit status 2
Stderr: # k8s.io/klog/v2
../../../go/pkg/mod/k8s.io/klog/v2@v2.8.0/klog.go:703:13: invalid operation: logr != nil (mismatched types logr.Logger and untyped nil)
../../../go/pkg/mod/k8s.io/klog/v2@v2.8.0/klog.go:722:13: invalid operation: logr != nil (mismatched types logr.Logger and untyped nil)
../../../go/pkg/mod/k8s.io/klog/v2@v2.8.0/klog.go:740:13: invalid operation: logr != nil (mismatched types logr.Logger and untyped nil)
../../../go/pkg/mod/k8s.io/klog/v2@v2.8.0/klog.go:761:13: invalid operation: logr != nil (mismatched types logr.Logger and untyped nil)
../../../go/pkg/mod/k8s.io/klog/v2@v2.8.0/klog.go:780:14: invalid operation: loggr != nil (mismatched types logr.Logger and untyped nil)
../../../go/pkg/mod/k8s.io/klog/v2@v2.8.0/klog.go:792:14: invalid operation: loggr != nil (mismatched types logr.Logger and untyped nil)
../../../go/pkg/mod/k8s.io/klog/v2@v2.8.0/klog.go:910:12: invalid operation: log != nil (mismatched types logr.Logger and untyped nil)
../../../go/pkg/mod/k8s.io/klog/v2@v2.8.0/klog.go:1272:21: invalid operation: logging.logr == nil (mismatched types logr.Logger and untyped nil)
../../../go/pkg/mod/k8s.io/klog/v2@v2.8.0/klog.go:1273:21: cannot use nil as logr.Logger value in struct literal

make[1]: *** [runner-bin] Error 1
make: *** [runner-bin-host] Error 2

https://github.com/kubernetes/klog/issues/253 suggests to bump the klog dependency.

go get -u k8s.io/klog/v2

Binaries

make runner-bin-host

./out/binaries/gitlab-runner run --help                                                                      ─╯
Runtime platform                                    arch=amd64 os=darwin pid=42287 revision=9c8e5b22 version=14.11.0~beta.4.g9c8e5b22
NAME:
   gitlab-runner run - run multi runner service

USAGE:
   gitlab-runner run [command options] [arguments...]

OPTIONS:
   -c value, --config value             Config file (default: "/Users/mfriedrich/.gitlab-runner/config.toml") [$CONFIG_FILE]
   --listen-address value               Metrics / pprof server listening address [$LISTEN_ADDRESS]
   -n value, --service value            Use different names for different services (default: "gitlab-runner")
   -d value, --working-directory value  Specify custom working directory
   -u value, --user value               Use specific user to execute shell scripts
   --syslog                             Log to system service logger [$LOG_SYSLOG]

./out/binaries/gitlab-runner --debug run

Tests

Virtualbox installed with Homebrew before.

make development_setup

make test

Configuration

Navigate into the project settings in Settings > CI/CD > Runners and copy the runner registration token. Export its value into the environment variable RUNNER_TOKEN.

Install curl, jq and ensure that a GitLab auth token is exported into the environment variable GITLAB_TOKEN. Register a new runner using the GitLab API endpoint.

export RUNNER_TOKEN=XXX
export GITLAB_TOKEN=XXX

curl -s -H "Authorization: Bearer $GITLAB_TOKEN" -X POST 'https://gitlab.com/api/v4/runners' --form "token=$RUNNER_TOKEN" --form "description=otel-dev-runner-michi" --form "tag_list=otel" | jq -c '.token'

This can be parsed into a new variable used in the runner configuration:

export RUNNER_AUTH_TOKEN=$(curl -s -H "Authorization: Bearer $GITLAB_TOKEN" -X POST 'https://gitlab.com/api/v4/runners' --form "token=$RUNNER_TOKEN" --form "description=otel-dev-runner-michi" --form "tag_list=otel" | jq -c '.token')

Next, copy the dev config example file into dev-config.toml. This file is defined in .gitignore to avoid accidental commits and token leaks in this repository.

cp -f dev-config.toml.example dev-config.toml

sed -i "s/token =.*/token = $RUNNER_AUTH_TOKEN/g" dev-config.toml

Note that the parsed RUNNER_AUTH_TOKEN value already contains trailing string quotes.

Run the compiled runner binary with configuration.

./out/binaries/gitlab-runner --debug run -c dev-config.toml

./out/binaries/gitlab-runner --debug run -c $HOME/dev/everyonecancontribute/observability/opentelemetry-gitlab-runner-playground/dev-config.toml
Runtime platform                                    arch=amd64 os=darwin pid=47158 revision=9c8e5b22 version=14.11.0~beta.4.g9c8e5b22
Starting multi-runner from /Users/mfriedrich/dev/everyonecancontribute/observability/opentelemetry-gitlab-runner-playground/dev-config.toml...  builds=0
Checking runtime mode                               GOOS=darwin uid=501
WARNING: Running in user-mode.
WARNING: Use sudo for system-mode:
WARNING: $ sudo gitlab-runner...

Configuration loaded                                builds=0

Verify the register runner in the project settings in Settings > CI/CD > Runners.

TODO: Add these instructions to the development docs.

Back to top