BigW Consortium Gitlab

Skip to content

G
gitlab-ce
Commit 36f3e206 by Kamil Trzciński
Options

Merge branch 'docker-single' into 'master'

Update docker documentation to use only one docker container I updated documentation to use only one docker container with manually specified path to the persistent data. The container is named `gitlab` and the image will be put in GitLab organisation: https://registry.hub.docker.com/u/gitlab/gitlab-ce/ - [x] Add VOLUMEs to Dockerfile - [x] Removed all other Dockerfiles (moved everything from docker/single/ to docker/) - [x] Create DockerHub repository for GitLab CE - [x] Reword README to use single image - [x] Push images (manually) to the repository - [ ] Push to GitHub - [ ] Enable Automated Build - [ ] Integrate tagging and pushing of the image with GitLab CI @JobV @sytses Please review. See merge request !909
parents 6e0c596a b8c9c69e
Showing with 131 additions and 233 deletions
docker/single/Dockerfile → docker/Dockerfile
......@@ -7,7 +7,9 @@ RUN apt-get update -q \
ca-certificates \
openssh-server \
wget \
apt-transport-https
apt-transport-https \
vim \
nano
# Download & Install GitLab
# If you run GitLab Enterprise Edition point it to a location where you have downloaded it.
......@@ -23,12 +25,21 @@ RUN mkdir -p /opt/gitlab/sv/sshd/supervise \
&& ln -s /opt/gitlab/sv/sshd /opt/gitlab/service \
&& mkdir -p /var/run/sshd
# Prepare default configuration
RUN ( \
echo "" && \
echo "# Docker options" && \
echo "# Prevent Postgres from trying to allocate 25% of total memory" && \
echo "postgresql['shared_buffers'] = '1MB'" ) >> /etc/gitlab/gitlab.rb
# Expose web & ssh
EXPOSE 80 22
EXPOSE 443 80 22
# Define data volumes
VOLUME ["/etc/gitlab", "/var/opt/gitlab", "/var/log/gitlab"]
# Copy assets
COPY assets/wrapper /usr/local/bin/
COPY assets/gitlab.rb /etc/gitlab/
# Wrapper to handle signal, trigger runit and reconfigure GitLab
CMD ["/usr/local/bin/wrapper"]
docker/README.md
......@@ -11,150 +11,156 @@ After starting a container you can go to [http://localhost:8080/](http://localho
It might take a while before the docker container is responding to queries.
You can check the status with something like `sudo docker logs -f 7c10172d7705`.
You can check the status with something like `sudo docker logs -f gitlab`.
You can login to the web interface with username `root` and password `password`.
Next time, you can just use docker start and stop to run the container.
## How to build the docker images
## Run the image
This guide will also let you know how to build docker images yourself.
Please run all the commands from the GitLab repo root directory.
People using boot2docker should run all the commands without sudo.
Run the image:
```bash
sudo docker run --detach \
--publish 8443:443 --publish 8080:80 --publish 2222:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest
```
## Choosing between the single and the app and data images
This will download and start GitLab CE container and publish ports needed to access SSH, HTTP and HTTPS.
All GitLab data will be stored as subdirectories of `/srv/gitlab/`.
The container will automatically `restart` after system reboot.
Normally docker uses a single image for one applications.
But GitLab stores repositories and uploads in the filesystem.
This means that upgrades of a single image are hard.
That is why we recommend using separate app and data images.
We'll first describe how to use a single image.
After that we'll describe how to use the app and data images.
After this you can login to the web interface as explained above in 'After starting a container'.
## Single image
## Where is the data stored?
Get a published image from Dockerhub:
The GitLab container uses host mounted volumes to store persistent data:
- `/srv/gitlab/data` mounted as `/var/opt/gitlab` in the container is used for storing *application data*
- `/srv/gitlab/logs` mounted as `/var/log/gitlab` in the container is used for storing *logs*
- `/srv/gitlab/config` mounted as `/etc/gitlab` in the container is used for storing *configuration*
```bash
sudo docker pull sytse/gitlab-ce:7.10.1
```
You can fine tune these directories to meet your requirements.
Run the image:
### Configure GitLab
This container uses the official Omnibus GitLab distribution, so all configuration is done in the unique configuration file `/etc/gitlab/gitlab.rb`.
To access GitLab configuration, you can start an bash in a new the context of running container, you will be able to browse all directories and use your favorite text editor:
```bash
sudo docker run --detach --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1
sudo docker exec -it gitlab /bin/bash
```
After this you can login to the web interface as explained above in 'After starting a container'.
Build the image:
You can also edit just `/etc/gitlab/gitlab.rb`:
```bash
sudo docker build --tag sytse/gitlab-ce:7.10.1 docker/single/
sudo docker exec -it gitlab vi /etc/gitlab/gitlab.rb
```
Publish the image to Dockerhub:
**You should set the `external_url` to point to a valid URL.**
```bash
sudo docker push sytse/gitlab-ce
```
**You may also be interesting in [Enabling HTTPS](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/nginx.md#enable-https).**
**To receive e-mails from GitLab you have to configure the [SMTP settings](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/smtp.md),
because Docker image doesn't have a SMTP server.**
Diagnosing commands:
**Note** that GitLab will reconfigure itself **at each container start.** You will need to restart the container to reconfigure your GitLab:
```bash
sudo docker run -i -t sytse/gitlab-ce:7.10.1
sudo docker run -ti -e TERM=linux --name gitlab-ce-troubleshoot --publish 8080:80 --publish 2222:22 sytse/gitlab-ce:7.10.1 bash /usr/local/bin/wrapper
sudo docker restart gitlab
```
## App and data images
For more options for configuring the container please check [Omnibus GitLab documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#configuration).
### Get published images from Dockerhub
## Diagnose potential problems
Read container logs:
```bash
sudo docker pull sytse/gitlab-data
sudo docker pull sytse/gitlab-app:7.10.1
sudo docker logs gitlab
```
### Run the images
Enter running container:
```bash
sudo docker run --name gitlab-data sytse/gitlab-data /bin/true
sudo docker run --detach --name gitlab-app --publish 8080:80 --publish 2222:22 --volumes-from gitlab-data sytse/gitlab-app:7.10.1
sudo docker exec -it gitlab /bin/bash
```
After this you can login to the web interface as explained above in 'After starting a container'.
### Build images
From within container you can administrer GitLab container as you would normally administer Omnibus installation: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md.
Build your own based on the Omnibus packages with the following commands.
### Upgrade GitLab to newer version
To upgrade GitLab to new version you have to do:
1. pull new image,
```bash
sudo docker build --tag gitlab-data docker/data/
sudo docker build --tag gitlab-app:7.10.1 docker/app/
sudo docker stop gitlab
```
After this run the images:
1. stop running container,
```bash
sudo docker run --name gitlab-data gitlab-data /bin/true
sudo docker run --detach --name gitlab-app --publish 8080:80 --publish 2222:22 --volumes-from gitlab-data gitlab-app:7.10.1
sudo docker rm gitlab
```
We assume using a data volume container, this will simplify migrations and backups.
This empty container will exist to persist as volumes the 3 directories used by GitLab, so remember not to delete it.
The directories on data container are:
- `/var/opt/gitlab` for application data
- `/var/log/gitlab` for logs
- `/etc/gitlab` for configuration
### Configure GitLab
This container uses the official Omnibus GitLab distribution, so all configuration is done in the unique configuration file `/etc/gitlab/gitlab.rb`.
To access GitLab configuration, you can start an interactive command line in a new container using the shared data volume container, you will be able to browse the 3 directories and use your favorite text editor:
1. remove existing container,
```bash
sudo docker pull gitlab/gitlab-ce:latest
```
1. create the container once again with previously specified options.
```bash
sudo docker run -ti -e TERM=linux --rm --volumes-from gitlab-data ubuntu
vi /etc/gitlab/gitlab.rb
sudo docker run --detach \
--publish 8443:443 --publish 8080:80 --publish 2222:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest
```
**Note** that GitLab will reconfigure itself **at each container start.** You will need to restart the container to reconfigure your GitLab.
On the first run GitLab will reconfigure and update itself.
You can find all available options in [Omnibus GitLab documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#configuration).
### Run GitLab CE on public IP address
### Upgrade GitLab with app and data images
You can make Docker to use your IP address and forward all traffic to the GitLab CE container.
You can do that by modifying the `--publish` ([Binding container ports to the host](https://docs.docker.com/articles/networking/#binding-ports)):
To upgrade GitLab to new versions, stop running container, create new docker image and container from that image.
> --publish=[] : Publish a container᾿s port or a range of ports to the host format: ip:hostPort:containerPort | ip::containerPort | hostPort:containerPort | containerPort
It Assumes that you're upgrading from 7.8.1 to 7.10.1 and you're in the updated GitLab repo root directory:
To expose GitLab CE on IP 1.1.1.1:
```bash
sudo docker stop gitlab-app
sudo docker rm gitlab-app
sudo docker build --tag gitlab-app:7.10.1 docker/app/
sudo docker run --detach --name gitlab-app --publish 8080:80 --publish 2222:22 --volumes-from gitlab-data gitlab-app:7.10.1
sudo docker run --detach \
--publish 1.1.1.1:443:443 --publish 1.1.1.1:80:80 --publish 1.1.1.1:22:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest
```
On the first run GitLab will reconfigure and update itself. If everything runs OK don't forget to cleanup the app image:
You can then access GitLab instance at http://1.1.1.1/ and https://1.1.1.1/.
### Build the image
This guide will also let you know how to build docker image yourself.
Please run the command from the GitLab repo root directory.
People using boot2docker should run all the commands without sudo.
```bash
sudo docker rmi gitlab-app:7.8.1
sudo docker build --tag gitlab/gitlab-ce:latest docker/
```
### Publish images to Dockerhub
### Publish the image to Dockerhub
- Ensure the containers are running
- Login to Dockerhub with `sudo docker login`
- Run the following (replace '7.10.1' with the version you're using and 'Sytse Sijbrandij' with your name):
```bash
sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab-app sytse/gitlab-app:7.10.1
sudo docker push sytse/gitlab-app:7.10.1
sudo docker commit -m "Initial commit" -a "Sytse Sijbrandij" gitlab-data sytse/gitlab-data
sudo docker push sytse/gitlab-data
sudo docker login
sudo docker push gitlab/gitlab-ce:latest
```
## Troubleshooting
......
docker/app/Dockerfile deleted 100644 → 0
FROM ubuntu:14.04
# Install required packages
RUN apt-get update -q \
&& DEBIAN_FRONTEND=noninteractive apt-get install -qy --no-install-recommends \
ca-certificates \
openssh-server \
wget \
apt-transport-https
# Download & Install GitLab
# If you run GitLab Enterprise Edition point it to a location where you have downloaded it.
RUN echo "deb https://packages.gitlab.com/gitlab/gitlab-ce/ubuntu/ `lsb_release -cs` main" > /etc/apt/sources.list.d/gitlab_gitlab-ce.list
RUN wget -q -O - https://packages.gitlab.com/gpg.key | apt-key add -
RUN apt-get update && apt-get install -yq --no-install-recommends gitlab-ce
# Manage SSHD through runit
RUN mkdir -p /opt/gitlab/sv/sshd/supervise \
&& mkfifo /opt/gitlab/sv/sshd/supervise/ok \
&& printf "#!/bin/sh\nexec 2>&1\numask 077\nexec /usr/sbin/sshd -D" > /opt/gitlab/sv/sshd/run \
&& chmod a+x /opt/gitlab/sv/sshd/run \
&& ln -s /opt/gitlab/sv/sshd /opt/gitlab/service \
&& mkdir -p /var/run/sshd
# Expose web & ssh
EXPOSE 80 22
# Copy assets
COPY assets/wrapper /usr/local/bin/
# Wrapper to handle signal, trigger runit and reconfigure GitLab
CMD ["/usr/local/bin/wrapper"]
\ No newline at end of file
docker/app/assets/wrapper deleted 100755 → 0
#!/bin/bash
function sigterm_handler() {
echo "SIGTERM signal received, try to gracefully shutdown all services..."
gitlab-ctl stop
}
trap "sigterm_handler; exit" TERM
function entrypoint() {
# Default is to run runit and reconfigure GitLab
gitlab-ctl reconfigure &
/opt/gitlab/embedded/bin/runsvdir-start &
wait
}
entrypoint
docker/data/Dockerfile deleted 100644 → 0
FROM busybox
# Declare volumes
VOLUME ["/var/opt/gitlab", "/var/log/gitlab", "/etc/gitlab"]
# Copy assets
COPY assets/gitlab.rb /etc/gitlab/
CMD /bin/sh
docker/data/assets/gitlab.rb deleted 100644 → 0
# External URL should be your Docker instance.
# By default, this example is the "standard" boot2docker IP.
# Always use port 80 here to force the internal nginx to bind port 80,
# even if you intend to use another port in Docker.
external_url "http://192.168.59.103/"
# Prevent Postgres from trying to allocate 25% of total memory
postgresql['shared_buffers'] = '1MB'
# Configure GitLab to redirect PostgreSQL logs to the data volume
postgresql['log_directory'] = '/var/log/gitlab/postgresql'
# Some configuration of GitLab
# You can find more at https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#configuration
gitlab_rails['gitlab_email_from'] = 'gitlab@example.com'
gitlab_rails['gitlab_support_email'] = 'support@example.com'
gitlab_rails['time_zone'] = 'Europe/Paris'
# SMTP settings
# You must use an external server, the Docker container does not install an SMTP server
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.example.com"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "user"
gitlab_rails['smtp_password'] = "password"
gitlab_rails['smtp_domain'] = "example.com"
gitlab_rails['smtp_authentication'] = "plain"
gitlab_rails['smtp_enable_starttls_auto'] = true
# Enable LDAP authentication
# gitlab_rails['ldap_enabled'] = true
# gitlab_rails['ldap_host'] = 'ldap.example.com'
# gitlab_rails['ldap_port'] = 389
# gitlab_rails['ldap_method'] = 'plain' # 'ssl' or 'plain'
# gitlab_rails['ldap_allow_username_or_email_login'] = false
# gitlab_rails['ldap_uid'] = 'uid'
# gitlab_rails['ldap_base'] = 'ou=users,dc=example,dc=com'
docker/fig.yml 0 → 100644
app:
build: .
docker/single/assets/gitlab.rb deleted 100644 → 0
# External URL should be your Docker instance.
# By default, GitLab will use the Docker container hostname.
# Always use port 80 here to force the internal nginx to bind port 80,
# even if you intend to use another port in Docker.
# external_url "http://192.168.59.103/"
# Prevent Postgres from trying to allocate 25% of total memory
postgresql['shared_buffers'] = '1MB'
# Configure GitLab to redirect PostgreSQL logs to the data volume
postgresql['log_directory'] = '/var/log/gitlab/postgresql'
# Some configuration of GitLab
# You can find more at https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#configuration
gitlab_rails['gitlab_email_from'] = 'gitlab@example.com'
gitlab_rails['gitlab_support_email'] = 'support@example.com'
gitlab_rails['time_zone'] = 'Europe/Paris'
# SMTP settings
# You must use an external server, the Docker container does not install an SMTP server
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.example.com"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "user"
gitlab_rails['smtp_password'] = "password"
gitlab_rails['smtp_domain'] = "example.com"
gitlab_rails['smtp_authentication'] = "plain"
gitlab_rails['smtp_enable_starttls_auto'] = true
# Enable LDAP authentication
# gitlab_rails['ldap_enabled'] = true
# gitlab_rails['ldap_host'] = 'ldap.example.com'
# gitlab_rails['ldap_port'] = 389
# gitlab_rails['ldap_method'] = 'plain' # 'ssl' or 'plain'
# gitlab_rails['ldap_allow_username_or_email_login'] = false
# gitlab_rails['ldap_uid'] = 'uid'
# gitlab_rails['ldap_base'] = 'ou=users,dc=example,dc=com'
docker/troubleshooting.md
......@@ -9,24 +9,19 @@ postgresql['log_directory'] = '/var/log/gitlab/postgresql'
# Commands
```bash
sudo docker build --tag gitlab_image docker/
sudo docker build --tag gitlab/gitlab-ce:latest docker/
sudo docker rm -f gitlab_app
sudo docker rm -f gitlab_data
sudo docker rm -f gitlab
sudo docker run --name gitlab_data gitlab_image /bin/true
sudo docker exec -it gitlab vim /etc/gitlab/gitlab.rb
sudo docker run -ti --rm --volumes-from gitlab_data ubuntu apt-get update && sudo apt-get install -y vim && sudo vim /etc/gitlab/gitlab.rb
sudo docker exec gitlab tail -f /var/log/gitlab/reconfigure.log
sudo docker run --detach --name gitlab_app --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data gitlab_image
sudo docker exec gitlab tail -f /var/log/gitlab/postgresql/current
sudo docker run -t --rm --volumes-from gitlab_data ubuntu tail -f /var/log/gitlab/reconfigure.log
sudo docker exec gitlab cat /var/opt/gitlab/postgresql/data/postgresql.conf | grep shared_buffers
sudo docker run -t --rm --volumes-from gitlab_data ubuntu tail -f /var/log/gitlab/postgresql/current
sudo docker run -t --rm --volumes-from gitlab_data ubuntu cat /var/opt/gitlab/postgresql/data/postgresql.conf | grep shared_buffers
sudo docker run -t --rm --volumes-from gitlab_data ubuntu cat /etc/gitlab/gitlab.rb
sudo docker exec gitlab cat /etc/gitlab/gitlab.rb
```
# Interactively
......@@ -37,7 +32,16 @@ sudo docker run -t --rm --volumes-from gitlab_data ubuntu cat /etc/gitlab/gitlab
# - we run interactively (-t -i)
# - we define TERM=linux because it allows to use arrow keys in vi (!!!)
# - we choose another startup command (bash)
sudo docker run -ti -e TERM=linux --name gitlab_app --publish 8080:80 --publish 2222:22 --volumes-from gitlab_data gitlab_image bash
sudo docker run --ti \
-e TERM=linux
--publish 80443:443 --publish 8080:80 --publish 2222:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest \
bash
# Configure GitLab to redirect PostgreSQL logs
echo "postgresql['log_directory'] = '/var/log/gitlab/postgresql'" >> /etc/gitlab/gitlab.rb
......@@ -64,10 +68,17 @@ free -m
# Cleanup
Remove ALL docker containers and images (also non GitLab ones):
Remove ALL docker containers and images (also non GitLab ones).
**Be careful, because the `-v` also removes volumes attached to the images.**
```
docker rm $(docker ps -a -q)
```bash
# Remove all containers with attached volumes
docker rm -v $(docker ps -a -q)
# Remove all images
docker rmi $(docker images -q)
# Remove GitLab persistent data
rm -rf /srv/gitlab
```
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment