BigW Consortium Gitlab

Skip to content

G
gitlab-ce
Commit 770504ba by Tomasz Maczukin Committed by Achilleas Pipinellis
Options

Add documentation about variables usage in GitLab CI

parent a30e6b81
Showing with 135 additions and 52 deletions
doc/ci/README.md
......@@ -19,7 +19,7 @@ Here's some info we've gathered to get you started.
The first steps towards your GitLab CI/CD journey.
- [Getting started with GitLab CI/CD](quick_start/README.md): understand how GitLab CI/CD works.
- GitLab CI/CD configuration file: [`.gitlab-ci.yml`](yaml/README.md) - Learn all about the ins and outs of `.gitlab-ci.yml`.
- [GitLab CI/CD configuration file: `.gitlab-ci.yml`](yaml/README.md) - Learn all about the ins and outs of `.gitlab-ci.yml`.
- [Pipelines and jobs](pipelines.md): configure your GitLab CI/CD pipelines to build, test, and deploy your application.
- Runners: The [GitLab Runner](https://docs.gitlab.com/runner/) is responsible by running the jobs in your CI/CD pipeline. On GitLab.com, Shared Runners are enabled by default, so
you don't need to set up anything to start to use them with GitLab CI/CD.
......@@ -46,7 +46,9 @@ you don't need to set up anything to start to use them with GitLab CI/CD.
## Exploring GitLab CI/CD
- [CI/CD Variables](variables/README.md) - Learn how to use variables defined in
your `.gitlab-ci.yml` or secured ones defined in your project's settings
your `.gitlab-ci.yml` or the ones defined in your project's settings
- [Where variables can be used](variables/where_variables_can_be_used.md) - A
deeper look on where and how the CI/CD variables can be used
- **The permissions model** - Learn about the access levels a user can have for
performing certain CI actions
- [User permissions](../user/permissions.md#gitlab-ci)
......
doc/ci/environments.md
......@@ -246,23 +246,14 @@ As the name suggests, it is possible to create environments on the fly by just
declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is
the basis of [Review apps](review_apps/index.md).
>**Note:**
The `name` and `url` parameters can use most of the defined CI variables,
including predefined, secure variables and `.gitlab-ci.yml`
[`variables`](yaml/README.md#variables). You however cannot use variables
defined under `script` or on the Runner's side. There are other variables that
are unsupported in environment name context:
- `CI_PIPELINE_ID`
- `CI_JOB_ID`
- `CI_JOB_TOKEN`
- `CI_BUILD_ID`
- `CI_BUILD_TOKEN`
- `CI_REGISTRY_USER`
- `CI_REGISTRY_PASSWORD`
- `CI_REPOSITORY_URL`
- `CI_ENVIRONMENT_URL`
- `CI_DEPLOY_USER`
- `CI_DEPLOY_PASSWORD`
NOTE: **Note:**
The `name` and `url` parameters can use most of the CI/CD variables,
including [predefined](variables/README.md#predefined-variables-environment-variables),
[secret](variables/README.md#secret-variables) and
[`.gitlab-ci.yml` variables](yaml/README.md#variables). You however cannot use variables
defined under `script` or on the Runner's side. There are also other variables that
are unsupported in the context of `environment:name`. You can read more about
[where variables can be used](variables/where_variables_can_be_used.md).
GitLab Runner exposes various [environment variables][variables] when a job runs,
and as such, you can use them as environment names. Let's add another job in
......
doc/ci/variables/README.md
......@@ -22,6 +22,12 @@ For example, if you define `API_TOKEN=secure` as a secret variable and
`API_TOKEN=yaml` in your `.gitlab-ci.yml`, the `API_TOKEN` will take the value
`secure` as the secret variables are higher in the chain.
## Unsupported variables
There are cases where some variables cannot be used in the context of a
`.gitlab-ci.yml` definition (for example under `script`). Read more
about which variables are [not supported](where_variables_can_be_used.md).
## Predefined variables (Environment variables)
Some of the predefined environment variables are available only if a minimum
......@@ -36,6 +42,7 @@ future GitLab releases.**
| Variable | GitLab | Runner | Description |
|-------------------------------- |--------|--------|-------------|
| **ARTIFACT_DOWNLOAD_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to download artifacts running a job |
| **CI** | all | 0.4 | Mark that job is executed in CI environment |
| **CI_COMMIT_REF_NAME** | 9.0 | all | The branch or tag name for which project is built |
| **CI_COMMIT_REF_SLUG** | 9.0 | all | `$CI_COMMIT_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. |
......@@ -46,6 +53,8 @@ future GitLab releases.**
| **CI_COMMIT_DESCRIPTION** | 10.8 | all | The description of the commit: the message without first line, if the title is shorter than 100 characters; full message in other case. |
| **CI_CONFIG_PATH** | 9.4 | 0.5 | The path to CI config file. Defaults to `.gitlab-ci.yml` |
| **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled |
| **CI_DEPLOY_USER** | 10.8 | all | Authentication username of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.|
| **CI_DEPLOY_PASSWORD** | 10.8 | all | Authentication password of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.|
| **CI_DISPOSABLE_ENVIRONMENT** | all | 10.1 | Marks that the job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). If the environment is disposable, it is set to true, otherwise it is not defined at all. |
| **CI_ENVIRONMENT_NAME** | 8.15 | all | The name of the environment for this job |
| **CI_ENVIRONMENT_SLUG** | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. |
......@@ -82,16 +91,13 @@ future GitLab releases.**
| **CI_SERVER_REVISION** | all | all | GitLab revision that is used to schedule jobs |
| **CI_SERVER_VERSION** | all | all | GitLab version that is used to schedule jobs |
| **CI_SHARED_ENVIRONMENT** | all | 10.1 | Marks that the job is executed in a shared environment (something that is persisted across CI invocations like `shell` or `ssh` executor). If the environment is shared, it is set to true, otherwise it is not defined at all. |
| **ARTIFACT_DOWNLOAD_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to download artifacts running a job |
| **GET_SOURCES_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to fetch sources running a job |
| **GITLAB_CI** | all | all | Mark that job is executed in GitLab CI environment |
| **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the job |
| **GITLAB_USER_EMAIL** | 8.12 | all | The email of the user who started the job |
| **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the job |
| **GITLAB_USER_LOGIN** | 10.0 | all | The login username of the user who started the job |
| **GITLAB_USER_NAME** | 10.0 | all | The real name of the user who started the job |
| **RESTORE_CACHE_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to restore the cache running a job |
| **CI_DEPLOY_USER** | 10.8 | all | Authentication username of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.|
| **CI_DEPLOY_PASSWORD** | 10.8 | all | Authentication password of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.|
## 9.0 Renaming
......@@ -540,34 +546,6 @@ Below you can find supported syntax reference:
Pattern matching is case-sensitive by default. Use `i` flag modifier, like
`/pattern/i` to make a pattern case-insensitive.
### Unsupported predefined variables
Because GitLab evaluates variables before creating jobs, we do not support a
few variables that depend on persistence layer, like `$CI_JOB_ID`.
Environments (like `production` or `staging`) are also being created based on
what jobs pipeline consists of, thus some environment-specific variables are
not supported as well.
We do not support variables containing tokens because of security reasons.
You can find a full list of unsupported variables below:
- `CI_PIPELINE_ID`
- `CI_JOB_ID`
- `CI_JOB_TOKEN`
- `CI_BUILD_ID`
- `CI_BUILD_TOKEN`
- `CI_REGISTRY_USER`
- `CI_REGISTRY_PASSWORD`
- `CI_REPOSITORY_URL`
- `CI_ENVIRONMENT_URL`
- `CI_DEPLOY_USER`
- `CI_DEPLOY_PASSWORD`
These variables are also not supported in a context of a
[dynamic environment name][dynamic-environments].
[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI secret variables"
[eep]: https://about.gitlab.com/products/ "Available only in GitLab Premium"
[envs]: ../environments.md
......@@ -579,5 +557,4 @@ These variables are also not supported in a context of a
[triggers]: ../triggers/README.md#pass-job-variables-to-a-trigger
[subgroups]: ../../user/group/subgroups/index.md
[builds-policies]: ../yaml/README.md#only-and-except-complex
[dynamic-environments]: ../environments.md#dynamic-environments
[gitlab-deploy-token]: ../../user/project/deploy_tokens/index.md#gitlab-deploy-token
doc/ci/variables/where_variables_can_be_used.md 0 → 100644
# Where variables can be used
As it's described in the [CI/CD variables](README.md) docs, you can
define many different variables. Some of them can be used for all GitLab CI/CD
features, but some of them are more or less limited.
This document describes where and how the different types of variables can be used.
## Variables usage
There are basically two places where you can use any defined variables:
1. On GitLab's side there's `.gitlab-ci.yml`
1. On the Runner's side there's `config.toml`
### `.gitlab-ci.yml` file
| Definition | Can be expanded? | Expansion place | Description |
|--------------------------------------|-------------------|-----------------|--------------|
| `environment:url` | yes | GitLab | The variable expansion is made by GitLab's [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism).<ul><li>**Supported:** all variables defined for a job (secret variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules)</li><li>**Not suported:** variables defined in Runner's `config.toml` and variables created in job's `script`</li></ul> |
| `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion **doesn't support**: <ul><li>variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`)</li><li>any other variables related to environment (currently only `CI_ENVIRONMENT_URL`)</li><li>[persisted variables](#persisted-variables)</li></ul> |
| `variables` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `image` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `services:[]` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `services:[]:name` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `cache:key` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `artifacts:name` | yes | Runner | The variable expansion is made by GitLab Runner's shell environment |
| `script`, `before_script`, `after_script` | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment) |
| `only:variables:[]`, `except:variables:[]` | no | n/a | The variable must be in the form of `$variable`.<br/>**Not supported:**<ul><li>variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`)</li><li>any other variables related to environment (currently only `CI_ENVIRONMENT_URL`)</li><li>[persisted variables](#persisted-variables)</li></ul> |
### `config.toml` file
NOTE: **Note:**
You can read more about `config.toml` in the [Runner's docs](https://docs.gitlab.com/runner/configuration/advanced-configuration.html).
| Definition | Can be expanded? | Description |
|--------------------------------------|------------------|-------------|
| `runners.environment` | yes | The variable expansion is made by the Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `runners.kubernetes.pod_labels` | yes | The Variable expansion is made by the Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `runners.kubernetes.pod_annotations` | yes | The Variable expansion is made by the Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
## Expansion mechanisms
There are three expansion mechanisms:
- GitLab
- GitLab Runner
- Execution shell environment
### GitLab internal variable expansion mechanism
The expanded part needs to be in a form of `$variable`, or `${variable}` or `%variable%`.
Each form is handled in the same way, no matter which OS/shell will finally handle the job,
since the expansion is done in GitLab before any Runner will get the job.
### GitLab Runner internal variable expansion mechanism
- **Supported:** secret variables, `.gitlab-ci.yml` variables, `config.toml` variables, and
variables from triggers and pipeline schedules
- **Not supported:** variables defined inside of scripts (e.g., `export MY_VARIABLE="test"`)
The Runner uses Go's `os.Expand()` method for variable expansion. It means that it will handle
only variables defined as `$variable` and `${variable}`. What's also important, is that
the expansion is done only once, so nested variables may or may not work, depending on the
ordering of variables definitions.
### Execution shell environment
This is an expansion that takes place during the `script` execution.
How it works depends on the used shell (bash/sh/cmd/PowerShell). For example, if the job's
`script` contains a line `echo $MY_VARIABLE-${MY_VARIABLE_2}`, it should be properly handled
by bash/sh (leaving empty strings or some values depending whether the variables were
defined or not), but will not work with Windows' cmd/PowerShell, since these shells
are using a different variables syntax.
**Supported:**
- The `script` may use all available variables that are default for the shell (e.g., `$PATH` which
should be present in all bash/sh shells) and all variables defined by GitLab CI/CD (secret variables,
`.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers and pipeline schedules).
- The `script` may also use all variables defined in the lines before. So, for example, if you define
a variable `export MY_VARIABLE="test"`:
- in `before_script`, it will work in the following lines of `before_script` and
all lines of the related `script`
- in `script`, it will work in the following lines of `script`
- in `after_script`, it will work in following lines of `after_script`
## Persisted variables
NOTE: **Note:**
Some of the persisted variables contain tokens and cannot be used by some definitions
due to security reasons.
The following variables are known as "persisted":
- `CI_PIPELINE_ID`
- `CI_JOB_ID`
- `CI_JOB_TOKEN`
- `CI_BUILD_ID`
- `CI_BUILD_TOKEN`
- `CI_REGISTRY_USER`
- `CI_REGISTRY_PASSWORD`
- `CI_REPOSITORY_URL`
- `CI_DEPLOY_USER`
- `CI_DEPLOY_PASSWORD`
They are:
- **supported** for all definitions as [described in the table](#gitlab-ci-yml-file) where the "Expansion place" is "Runner"
- **not supported:**
- by the definitions [described in the table](#gitlab-ci-yml-file) where the "Expansion place" is "GitLab"
- in the `only` and `except` [variables expressions](README.md#variables-expressions)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment