BigW Consortium Gitlab

Skip to content

G
gitlab-ce
Commit 4099b560 by Achilleas Pipinellis
Options

Merge branch 'docs/refactor-ci-variables' into 'master'

Refactor CI variables docs See merge request !7751
parents 615eb1f0 4c7c66a9
Showing with 218 additions and 61 deletions
doc/ci/README.md
......@@ -11,7 +11,8 @@
- [Configure a Runner, the application that runs your builds](runners/README.md)
- [Use Docker images with GitLab Runner](docker/using_docker_images.md)
- [Use CI to build Docker images](docker/using_docker_build.md)
- [Use variables in your `.gitlab-ci.yml`](variables/README.md)
- [CI Variables](variables/README.md) - Learn how to use variables defined in
your `.gitlab-ci.yml` or secured ones defined in your project's settings
- [Use SSH keys in your build environment](ssh_keys/README.md)
- [Trigger builds through the API](triggers/README.md)
- [Build artifacts](../user/project/builds/artifacts.md)
......@@ -25,4 +26,6 @@
## Breaking changes
- [New CI build permissions model](../user/project/new_ci_build_permissions_model.md) Read about what changed in GitLab 8.12 and how that affects your builds. There's a new way to access your Git submodules and LFS objects in builds.
- [New CI build permissions model](../user/project/new_ci_build_permissions_model.md)
Read about what changed in GitLab 8.12 and how that affects your builds.
There's a new way to access your Git submodules and LFS objects in builds.
doc/ci/variables/README.md
## Variables
# Variables
When receiving a build from GitLab CI, the runner prepares the build environment.
It starts by setting a list of **predefined variables** (Environment Variables) and a list of **user-defined variables**
When receiving a build from GitLab CI, the [Runner] prepares the build environment.
It starts by setting a list of **predefined variables** (environment variables)
and a list of **user-defined variables**.
The variables can be overwritten. They take precedence over each other in this order:
1. Trigger variables
1. Secure variables
1. YAML-defined job-level variables
1. YAML-defined global variables
1. Predefined variables
## Priority of variables
For example, if you define:
1. `API_TOKEN=SECURE` as Secure Variable
1. `API_TOKEN=YAML` as YAML-defined variable
The variables can be overwritten and they take precedence over each other in
this order:
The `API_TOKEN` will take the Secure Variable value: `SECURE`.
1. [Trigger variables][triggers] (take precedence over all)
1. [Secure variables](#secure-variables)
1. YAML-defined [job-level variables](../yaml/README.md#job-variables)
1. YAML-defined [global variables](../yaml/README.md#variables)
1. [Predefined variables](#predefined-variables-environment-variables) (are the
lowest in the chain)
### Predefined variables (Environment Variables)
For example, if you define `API_TOKEN=secure` as a secure variable and
`API_TOKEN=yaml` in your `.gitlab-ci.yml`, the `API_TOKEN` will take the value
`secure` as the secure variables are higher in the chain.
## Predefined variables (Environment variables)
Some of the predefined environment variables are available only if a minimum
version of [GitLab Runner][runner] is used. Consult the table below to find the
version of Runner required.
| Variable | GitLab | Runner | Description |
|-------------------------|--------|--------|-------------|
......@@ -52,7 +60,6 @@ The `API_TOKEN` will take the Secure Variable value: `SECURE`.
| **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the build |
| **GITLAB_USER_EMAIL** | 8.12 | all | The email of the user who started the build |
**Some of the variables are only available when using runner with at least defined version.**
Example values:
......@@ -60,7 +67,7 @@ Example values:
export CI_BUILD_ID="50"
export CI_BUILD_REF="1ecfd275763eff1d6b4844ea3168962458c9f27a"
export CI_BUILD_REF_NAME="master"
export CI_BUILD_REPO="https://gitab-ci-token:abcde-1234ABCD5678ef@gitlab.com/gitlab-org/gitlab-ce.git"
export CI_BUILD_REPO="https://gitab-ci-token:abcde-1234ABCD5678ef@example.com/gitlab-org/gitlab-ce.git"
export CI_BUILD_TAG="1.0.0"
export CI_BUILD_NAME="spec:other"
export CI_BUILD_STAGE="test"
......@@ -73,9 +80,9 @@ export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce"
export CI_PROJECT_NAME="gitlab-ce"
export CI_PROJECT_NAMESPACE="gitlab-org"
export CI_PROJECT_PATH="gitlab-org/gitlab-ce"
export CI_PROJECT_URL="https://gitlab.com/gitlab-org/gitlab-ce"
export CI_REGISTRY="registry.gitlab.com"
export CI_REGISTRY_IMAGE="registry.gitlab.com/gitlab-org/gitlab-ce"
export CI_PROJECT_URL="https://example.com/gitlab-org/gitlab-ce"
export CI_REGISTRY="registry.example.com"
export CI_REGISTRY_IMAGE="registry.example.com/gitlab-org/gitlab-ce"
export CI_RUNNER_ID="10"
export CI_RUNNER_DESCRIPTION="my runner"
export CI_RUNNER_TAGS="docker, linux"
......@@ -84,30 +91,64 @@ export CI_SERVER_NAME="GitLab"
export CI_SERVER_REVISION="70606bf"
export CI_SERVER_VERSION="8.9.0"
export GITLAB_USER_ID="42"
export GITLAB_USER_EMAIL="alexzander@sporer.com"
export GITLAB_USER_EMAIL="user@example.com"
```
### YAML-defined variables
**This feature requires GitLab Runner 0.5.0 or higher and GitLab CI 7.14 or higher **
## `.gitlab-ci.yaml` defined variables
>**Note:**
This feature requires GitLab Runner 0.5.0 or higher and GitLab CI 7.14 or higher.
GitLab CI allows you to add to `.gitlab-ci.yml` variables that are set in the
build environment. The variables are hence saved in the repository, and they
are meant to store non-sensitive project configuration, e.g., `RAILS_ENV` or
`DATABASE_URL`.
GitLab CI allows you to add to `.gitlab-ci.yml` variables that are set in build environment.
The variables are stored in repository and are meant to store non-sensitive project configuration, ie. RAILS_ENV or DATABASE_URL.
For example, if you set the variable below globally (not inside a job), it will
be used in all executed commands and scripts:
```yaml
variables:
DATABASE_URL: "postgres://postgres@postgres/my_database"
```
These variables can be later used in all executed commands and scripts.
The YAML-defined variables are also set to all created
[service containers](../docker/using_docker_images.md), thus allowing to fine
tune them.
Variables can be defined at a global level, but also at a job level. To turn off
global defined variables in your job, define an empty array:
```yaml
job_name:
variables: []
```
## Secure variables
The YAML-defined variables are also set to all created service containers, thus allowing to fine tune them.
>**Notes:**
- This feature requires GitLab Runner 0.4.0 or higher.
- Be aware that secure variables are not masked, and their values can be shown
in the build logs if explicitly asked to do so. If your project is public or
internal, you can set the pipelines private from your project's Pipelines
settings. Follow the discussion in issue [#13784][ce-13784] for masking the
secure variables.
Variables can be defined at a global level, but also at a job level.
GitLab CI allows you to define per-project **Secure variables** that are set in
the build environment. The secure variables are stored out of the repository
(`.gitlab-ci.yml`) and are securely passed to GitLab Runner making them
available in the build environment. It's the recommended method to use for
storing things like passwords, secret keys and credentials.
More information about Docker integration can be found in [Using Docker Images](../docker/using_docker_images.md).
Secure variables can be added by going to your project's
**Settings ➔ Variables ➔ Add variable**.
#### Debug tracing
Once you set them, they will be available for all subsequent builds.
## Debug tracing
> Introduced in GitLab Runner 1.7.
>
> **WARNING:** Enabling debug tracing can have severe security implications. The
output **will** contain the content of all your secure variables and any other
secrets! The output **will** be uploaded to the GitLab server and made visible
......@@ -124,55 +165,162 @@ trace, resulting in a verbose build trace listing all commands that were run,
variables that were set, etc.
Before enabling this, you should ensure builds are visible to
[team members only](../../../user/permissions.md#project-features). You should
also [erase](../pipelines.md#seeing-build-traces) all generated build traces
[team members only](../../user/permissions.md#project-features). You should
also [erase](../pipelines.md#seeing-build-status) all generated build traces
before making them visible again.
To enable debug traces, set the `CI_DEBUG_TRACE` variable to `true`:
```yaml
job1:
job_name:
variables:
CI_DEBUG_TRACE: "true"
```
The [example project](https://gitlab.com/gitlab-examples/ci-debug-trace)
demonstrates a working configuration, including build trace examples.
Example truncated output with debug trace set to true:
### User-defined variables (Secure Variables)
**This feature requires GitLab Runner 0.4.0 or higher**
GitLab CI allows you to define per-project **Secure Variables** that are set in
the build environment.
The secure variables are stored out of the repository (the `.gitlab-ci.yml`).
The variables are securely passed to GitLab Runner and are available in the
build environment.
It's desired method to use them for storing passwords, secret keys or whatever
you want.
```bash
...
export CI_SERVER_TLS_CA_FILE="/builds/gitlab-examples/ci-debug-trace.tmp/CI_SERVER_TLS_CA_FILE"
if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then
echo $'\''\x1b[32;1mFetching changes...\x1b[0;m'\''
$'\''cd'\'' "/builds/gitlab-examples/ci-debug-trace"
$'\''git'\'' "config" "fetch.recurseSubmodules" "false"
$'\''rm'\'' "-f" ".git/index.lock"
$'\''git'\'' "clean" "-ffdx"
$'\''git'\'' "reset" "--hard"
$'\''git'\'' "remote" "set-url" "origin" "https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@example.com/gitlab-examples/ci-debug-trace.git"
$'\''git'\'' "fetch" "origin" "--prune" "+refs/heads/*:refs/remotes/origin/*" "+refs/tags/*:refs/tags/*"
else
$'\''mkdir'\'' "-p" "/builds/gitlab-examples/ci-debug-trace.tmp/git-template"
$'\''rm'\'' "-r" "-f" "/builds/gitlab-examples/ci-debug-trace"
$'\''git'\'' "config" "-f" "/builds/gitlab-examples/ci-debug-trace.tmp/git-template/config" "fetch.recurseSubmodules" "false"
echo $'\''\x1b[32;1mCloning repository...\x1b[0;m'\''
$'\''git'\'' "clone" "--no-checkout" "https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@example.com/gitlab-examples/ci-debug-trace.git" "/builds/gitlab-examples/ci-debug-trace" "--template" "/builds/gitlab-examples/ci-debug-trace.tmp/git-template"
$'\''cd'\'' "/builds/gitlab-examples/ci-debug-trace"
fi
echo $'\''\x1b[32;1mChecking out dd648b2e as master...\x1b[0;m'\''
$'\''git'\'' "checkout" "-f" "-q" "dd648b2e48ce6518303b0bb580b2ee32fadaf045"
'
+++ hostname
++ echo 'Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-machine-1480971377-317a7d0f-digital-ocean-4gb...'
Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-machine-1480971377-317a7d0f-digital-ocean-4gb...
++ export CI=true
++ CI=true
++ export CI_DEBUG_TRACE=false
++ CI_DEBUG_TRACE=false
++ export CI_BUILD_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045
++ CI_BUILD_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045
++ export CI_BUILD_BEFORE_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045
++ CI_BUILD_BEFORE_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045
++ export CI_BUILD_REF_NAME=master
++ CI_BUILD_REF_NAME=master
++ export CI_BUILD_ID=7046507
++ CI_BUILD_ID=7046507
++ export CI_BUILD_REPO=https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@example.com/gitlab-examples/ci-debug-trace.git
++ CI_BUILD_REPO=https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@example.com/gitlab-examples/ci-debug-trace.git
++ export CI_BUILD_TOKEN=xxxxxxxxxxxxxxxxxxxx
++ CI_BUILD_TOKEN=xxxxxxxxxxxxxxxxxxxx
++ export CI_PROJECT_ID=1796893
++ CI_PROJECT_ID=1796893
++ export CI_PROJECT_DIR=/builds/gitlab-examples/ci-debug-trace
++ CI_PROJECT_DIR=/builds/gitlab-examples/ci-debug-trace
++ export CI_SERVER=yes
++ CI_SERVER=yes
++ export 'CI_SERVER_NAME=GitLab CI'
++ CI_SERVER_NAME='GitLab CI'
++ export CI_SERVER_VERSION=
++ CI_SERVER_VERSION=
++ export CI_SERVER_REVISION=
++ CI_SERVER_REVISION=
++ export GITLAB_CI=true
++ GITLAB_CI=true
++ export CI=true
++ CI=true
++ export GITLAB_CI=true
++ GITLAB_CI=true
++ export CI_BUILD_ID=7046507
++ CI_BUILD_ID=7046507
++ export CI_BUILD_TOKEN=xxxxxxxxxxxxxxxxxxxx
++ CI_BUILD_TOKEN=xxxxxxxxxxxxxxxxxxxx
++ export CI_BUILD_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045
++ CI_BUILD_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045
++ export CI_BUILD_BEFORE_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045
++ CI_BUILD_BEFORE_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045
++ export CI_BUILD_REF_NAME=master
++ CI_BUILD_REF_NAME=master
++ export CI_BUILD_NAME=debug_trace
++ CI_BUILD_NAME=debug_trace
++ export CI_BUILD_STAGE=test
++ CI_BUILD_STAGE=test
++ export CI_SERVER_NAME=GitLab
++ CI_SERVER_NAME=GitLab
++ export CI_SERVER_VERSION=8.14.3-ee
++ CI_SERVER_VERSION=8.14.3-ee
++ export CI_SERVER_REVISION=82823
++ CI_SERVER_REVISION=82823
++ export CI_PROJECT_ID=17893
++ CI_PROJECT_ID=17893
++ export CI_PROJECT_NAME=ci-debug-trace
++ CI_PROJECT_NAME=ci-debug-trace
++ export CI_PROJECT_PATH=gitlab-examples/ci-debug-trace
++ CI_PROJECT_PATH=gitlab-examples/ci-debug-trace
++ export CI_PROJECT_NAMESPACE=gitlab-examples
++ CI_PROJECT_NAMESPACE=gitlab-examples
++ export CI_PROJECT_URL=https://example.com/gitlab-examples/ci-debug-trace
++ CI_PROJECT_URL=https://example.com/gitlab-examples/ci-debug-trace
++ export CI_PIPELINE_ID=52666
++ CI_PIPELINE_ID=52666
++ export CI_RUNNER_ID=1337
++ CI_RUNNER_ID=1337
++ export CI_RUNNER_DESCRIPTION=shared-runners-manager-1.example.com
++ CI_RUNNER_DESCRIPTION=shared-runners-manager-1.example.com
++ export 'CI_RUNNER_TAGS=shared, docker, linux, ruby, mysql, postgres, mongo, git-annex'
++ CI_RUNNER_TAGS='shared, docker, linux, ruby, mysql, postgres, mongo, git-annex'
++ export CI_REGISTRY=registry.example.com
++ CI_REGISTRY=registry.example.com
++ export CI_DEBUG_TRACE=true
++ CI_DEBUG_TRACE=true
++ export GITLAB_USER_ID=42
++ GITLAB_USER_ID=42
++ export GITLAB_USER_EMAIL=user@example.com
++ GITLAB_USER_EMAIL=axilleas@axilleas.me
++ export VERY_SECURE_VARIABLE=imaverysecurevariable
++ VERY_SECURE_VARIABLE=imaverysecurevariable
++ mkdir -p /builds/gitlab-examples/ci-debug-trace.tmp
++ echo -n '-----BEGIN CERTIFICATE-----
MIIFQzCCBCugAwIBAgIRAL/ElDjuf15xwja1ZnCocWAwDQYJKoZIhvcNAQELBQAw'
...
```
**The value of the variable can be shown in build log if explicitly asked to do so.**
If your project is public or internal you can make the builds private.
## Using the CI variables in your job scripts
Secure Variables can added by going to `Project > Variables > Add Variable`.
All variables are set as environment variables in the build environment, and
they are accessible with normal methods that are used to access such variables.
In most cases `bash` or `sh` is used to execute the build script.
They will be available for all subsequent builds.
To access the variables (predefined and user-defined) in a `bash`/`sh` environment,
prefix the variable name with the dollar sign (`$`):
### Use variables
The variables are set as environment variables in build environment and are accessible with normal methods that are used to access such variables.
In most cases the **bash** is used to execute build script.
To access variables (predefined and user-defined) in bash environment, prefix the variable name with `$`:
```
job_name:
script:
- echo $CI_BUILD_ID
```
You can also list all environment variables with `export` command,
but be aware that this will also expose value of all **Secure Variables** in build log:
You can also list all environment variables with the `export` command,
but be aware that this will also expose the values of all the secure variables
you set, in the build log:
```
job_name:
script:
- export
```
[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784
[runner]: https://docs.gitlab.com/runner/
[triggered]: ../triggers/README.md
[triggers]: ../triggers/README.md#pass-build-variables-to-a-trigger
doc/ci/yaml/README.md
......@@ -406,14 +406,20 @@ except master.
### job variables
It is possible to define build variables using a `variables` keyword on a job
level. It works basically the same way as its global-level equivalent but
allows you to define job-specific build variables.
level. It works basically the same way as its [global-level equivalent](#variables)
but allows you to define job-specific build variables.
When the `variables` keyword is used on a job level, it overrides global YAML
build variables and predefined variables.
build variables and predefined variables. To turn off global defined variables
in your job, define an empty array:
Build variables priority is defined in
[variables documentation](../variables/README.md).
```yaml
job_name:
variables: []
```
Build variables priority is defined in the
[variables documentation][variables].
### tags
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment