BigW Consortium Gitlab

Skip to content

G
gitlab-ce
Commit 36175709 by Achilleas Pipinellis
Options

Merge branch 'docs-article-devops-gamedev' into 'master'

Docs: new article devops + game dev with GitLab Closes #43948 See merge request gitlab-org/gitlab-ce!17577
parents 138ae501 46f3e541
Showing with 536 additions and 6 deletions
doc/ci/examples/README.md
......@@ -32,34 +32,38 @@ There's also a collection of repositories with [example projects](https://gitlab
- **Debian**: [Continuous Deployment with GitLab: how to build and deploy a Debian Package with GitLab CI](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/)
- **Maven**: [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md)
### Game development
- [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md)
### Miscellaneous
- [Using `dpl` as deployment tool](deployment/README.md)
- [The `.gitlab-ci.yml` file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml)
### Code quality analysis
## Code quality analysis
[Analyze code quality with the Code Climate CLI](code_climate.md).
### Static Application Security Testing (SAST)
## Static Application Security Testing (SAST)
- **(Ultimate)** [Scan your code for vulnerabilities](https://docs.gitlab.com/ee/ci/examples/sast.html)
- [Scan your Docker images for vulnerabilities](sast_docker.md)
### Dynamic Application Security Testing (DAST)
## Dynamic Application Security Testing (DAST)
Scan your app for vulnerabilities with GitLab [Dynamic Application Security Testing (DAST)](dast.md).
### Browser Performance Testing with Sitespeed.io
## Browser Performance Testing with Sitespeed.io
Analyze your [browser performance with Sitespeed.io](browser_performance.md).
### GitLab CI/CD for Review Apps
## GitLab CI/CD for Review Apps
- [Example project](https://gitlab.com/gitlab-examples/review-apps-nginx/) that shows how to use GitLab CI/CD for [Review Apps](../review_apps/index.html).
- [Dockerizing GitLab Review Apps](https://about.gitlab.com/2017/07/11/dockerizing-review-apps/)
### GitLab CI/CD for GitLab Pages
## GitLab CI/CD for GitLab Pages
See the documentation on [GitLab Pages](../../user/project/pages/index.md) for a complete overview.
......
doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md 0 → 100644
---
author: Ryan Hall
author_gitlab: blitzgren
level: intermediary
article_type: tutorial
date: 2018-03-07
---
# DevOps and Game Dev with GitLab CI/CD
With advances in WebGL and WebSockets, browsers are extremely viable as game development
platforms without the use of plugins like Adobe Flash. Furthermore, by using GitLab and [AWS](https://aws.amazon.com/),
single game developers, as well as game dev teams, can easily host browser-based games online.
In this tutorial, we'll focus on DevOps, as well as testing and hosting games with Continuous
Integration/Deployment methods. We assume you are familiar with GitLab, javascript,
and the basics of game development.
## The game
Our [demo game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/) consists of a simple spaceship traveling in space that shoots by clicking the mouse in a given direction.
Creating a strong CI/CD pipeline at the beginning of developing another game, [Dark Nova](http://darknova.io/about),
was essential for the fast pace the team worked at. This tutorial will build upon my
[previous introductory article](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) and go through the following steps:
1. Using code from the previous article to start with a barebones [Phaser](https://phaser.io) game built by a gulp file
1. Adding and running unit tests
1. Creating a `Weapon` class that can be triggered to spawn a `Bullet` in a given direction
1. Adding a `Player` class that uses this weapon and moves around the screen
1. Adding the sprites we will use for the `Player` and `Weapon`
1. Testing and deploying with Continuous Integration and Continuous Deployment methods
By the end, we'll have the core of a [playable game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/)
that's tested and deployed on every push to the `master` branch of the [codebase](https://gitlab.com/blitzgren/gitlab-game-demo).
This will also provide
boilerplate code for starting a browser-based game with the following components:
- Written in [Typescript](https://www.typescriptlang.org/) and [PhaserJs](https://phaser.io)
- Building, running, and testing with [Gulp](http://gulpjs.com/)
- Unit tests with [Chai](http://chaijs.com/) and [Mocha](https://mochajs.org/)
- CI/CD with GitLab
- Hosting the codebase on GitLab.com
- Hosting the game on AWS
- Deploying to AWS
## Requirements and setup
Please refer to my previous article [DevOps and Game Dev](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) to learn the foundational
development tools, running a Hello World-like game, and building this game using GitLab
CI/CD from every new push to master. The `master` branch for this game's [repository](https://gitlab.com/blitzgren/gitlab-game-demo)
contains a completed version with all configurations. If you would like to follow along
with this article, you can clone and work from the `devops-article` branch:
```sh
git clone git@gitlab.com:blitzgren/gitlab-game-demo.git
git checkout devops-article
```
Next, we'll create a small subset of tests that exemplify most of the states I expect
this `Weapon` class to go through. To get started, create a folder called `lib/tests`
and add the following code to a new file `weaponTests.ts`:
```ts
import { expect } from 'chai';
import { Weapon, BulletFactory } from '../lib/weapon';
describe('Weapon', () => {
var subject: Weapon;
var shotsFired: number = 0;
// Mocked bullet factory
var bulletFactory: BulletFactory = <BulletFactory>{
generate: function(px, py, vx, vy, rot) {
shotsFired++;
}
};
var parent: any = { x: 0, y: 0 };
beforeEach(() => {
shotsFired = 0;
subject = new Weapon(bulletFactory, parent, 0.25, 1);
});
it('should shoot if not in cooldown', () => {
subject.trigger(true);
subject.update(0.1);
expect(shotsFired).to.equal(1);
});
it('should not shoot during cooldown', () => {
subject.trigger(true);
subject.update(0.1);
subject.update(0.1);
expect(shotsFired).to.equal(1);
});
it('should shoot after cooldown ends', () => {
subject.trigger(true);
subject.update(0.1);
subject.update(0.3); // longer than timeout
expect(shotsFired).to.equal(2);
});
it('should not shoot if not triggered', () => {
subject.update(0.1);
subject.update(0.1);
expect(shotsFired).to.equal(0);
});
});
```
To build and run these tests using gulp, let's also add the following gulp functions
to the existing `gulpfile.js` file:
```ts
gulp.task('build-test', function () {
return gulp.src('src/tests/**/*.ts', { read: false })
.pipe(tap(function (file) {
// replace file contents with browserify's bundle stream
file.contents = browserify(file.path, { debug: true })
.plugin(tsify, { project: "./tsconfig.test.json" })
.bundle();
}))
.pipe(buffer())
.pipe(sourcemaps.init({loadMaps: true}) )
.pipe(gulp.dest('built/tests'));
});
gulp.task('run-test', function() {
gulp.src(['./built/tests/**/*.ts']).pipe(mocha());
});
```
We will start implementing the first part of our game and get these `Weapon` tests to pass.
The `Weapon` class will expose a method to trigger the generation of a bullet at a given
direction and speed. Later we will implement a `Player` class that ties together the user input
to trigger the weapon. In the `src/lib` folder create a `weapon.ts` file. We'll add two classes
to it: `Weapon` and `BulletFactory` which will encapsulate Phaser's **sprite** and
**group** objects, and the logic specific to our game.
```ts
export class Weapon {
private isTriggered: boolean = false;
private currentTimer: number = 0;
constructor(private bulletFactory: BulletFactory, private parent: Phaser.Sprite, private cooldown: number, private bulletSpeed: number) {
}
public trigger(on: boolean): void {
this.isTriggered = on;
}
public update(delta: number): void {
this.currentTimer -= delta;
if (this.isTriggered && this.currentTimer <= 0) {
this.shoot();
}
}
private shoot(): void {
// Reset timer
this.currentTimer = this.cooldown;
// Get velocity direction from player rotation
var parentRotation = this.parent.rotation + Math.PI / 2;
var velx = Math.cos(parentRotation);
var vely = Math.sin(parentRotation);
// Apply a small forward offset so bullet shoots from head of ship instead of the middle
var posx = this.parent.x - velx * 10
var posy = this.parent.y - vely * 10;
this.bulletFactory.generate(posx, posy, -velx * this.bulletSpeed, -vely * this.bulletSpeed, this.parent.rotation);
}
}
export class BulletFactory {
constructor(private bullets: Phaser.Group, private poolSize: number) {
// Set all the defaults for this BulletFactory's bullet object
this.bullets.enableBody = true;
this.bullets.physicsBodyType = Phaser.Physics.ARCADE;
this.bullets.createMultiple(30, 'bullet');
this.bullets.setAll('anchor.x', 0.5);
this.bullets.setAll('anchor.y', 0.5);
this.bullets.setAll('outOfBoundsKill', true);
this.bullets.setAll('checkWorldBounds', true);
}
public generate(posx: number, posy: number, velx: number, vely: number, rot: number): Phaser.Sprite {
// Pull a bullet from Phaser's Group pool
var bullet = this.bullets.getFirstExists(false);
// Set the few unique properties about this bullet: rotation, position, and velocity
if (bullet) {
bullet.reset(posx, posy);
bullet.rotation = rot;
bullet.body.velocity.x = velx;
bullet.body.velocity.y = vely;
}
return bullet;
}
}
```
Lastly, we'll redo our entry point, `game.ts`, to tie together both `Player` and `Weapon` objects
as well as add them to the update loop. Here is what the updated `game.ts` file looks like:
```ts
import { Player } from "./player";
import { Weapon, BulletFactory } from "./weapon";
window.onload = function() {
var game = new Phaser.Game(800, 600, Phaser.AUTO, 'gameCanvas', { preload: preload, create: create, update: update });
var player: Player;
var weapon: Weapon;
// Import all assets prior to loading the game
function preload () {
game.load.image('player', 'assets/player.png');
game.load.image('bullet', 'assets/bullet.png');
}
// Create all entities in the game, after Phaser loads
function create () {
// Create and position the player
var playerSprite = game.add.sprite(400, 550, 'player');
playerSprite.anchor.setTo(0.5);
player = new Player(game.input, playerSprite, 150);
var bulletFactory = new BulletFactory(game.add.group(), 30);
weapon = new Weapon(bulletFactory, player.sprite, 0.25, 1000);
player.loadWeapon(weapon);
}
// This function is called once every tick, default is 60fps
function update() {
var deltaSeconds = game.time.elapsedMS / 1000; // convert to seconds
player.update(deltaSeconds);
weapon.update(deltaSeconds);
}
}
```
Run `gulp serve` and you can run around and shoot. Wonderful! Let's update our CI
pipeline to include running the tests along with the existing build job.
## Continuous Integration
To ensure our changes don't break the build and all tests still pass, we utilize
Continuous Integration (CI) to run these checks automatically for every push.
Read through this article to understand [Continuous Integration, Continuous Delivery, and Continuous Deployment](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/),
and how these methods are leveraged by GitLab.
From the [last tutorial](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) we already have a `gitlab-ci.yml` file set up for building our app from
every push. We need to set up a new CI job for testing, which GitLab CI/CD will run after the build job using our generated artifacts from gulp.
Please read through the [documentation on CI/CD configuration](../../../ci/yaml/README.md) file to explore its contents and adjust it to your needs.
### Build your game with GitLab CI/CD
We need to update our build job to ensure tests get run as well. Add `gulp build-test`
to the end of the `script` array for the existing `build` job. Once these commands run,
we know we will need to access everything in the `built` folder, given by GitLab CI/CD's `artifacts`.
We'll also cache `node_modules` to avoid having to do a full re-pull of those dependencies:
just pack them up in the cache. Here is the full `build` job:
```yml
build:
stage: build
script:
- npm i gulp -g
- npm i
- gulp
- gulp build-test
cache:
policy: push
paths:
- node_modules
artifacts:
paths:
- built
```
### Test your game with GitLab CI/CD
For testing locally, we simply run `gulp run-tests`, which requires gulp to be installed
globally like in the `build` job. We pull `node_modules` from the cache, so the `npm i`
command won't have to do much. In preparation for deployment, we know we will still need
the `built` folder in the artifacts, which will be brought over as default behavior from
the previous job. Lastly, by convention, we let GitLab CI/CD know this needs to be run after
the `build` job by giving it a `test` [stage](../../../ci/yaml/README.md#stages).
Following the YAML structure, the `test` job should look like this:
```yml
test:
stage: test
script:
- npm i gulp -g
- npm i
- gulp run-test
cache:
policy: push
paths:
- node_modules/
artifacts:
paths:
- built/
```
We have added unit tests for a `Weapon` class that shoots on a specified interval.
The `Player` class implements `Weapon` along with the ability to move around and shoot. Also,
we've added test artifacts and a test stage to our GitLab CI/CD pipeline using `.gitlab-ci.yml`,
allowing us to run our tests by every push.
Our entire `.gitlab-ci.yml` file should now look like this:
```yml
image: node:6
build:
stage: build
script:
- npm i gulp -g
- npm i
- gulp
- gulp build-test
cache:
policy: push
paths:
- node_modules/
artifacts:
paths:
- built/
test:
stage: test
script:
- npm i gulp -g
- npm i
- gulp run-test
cache:
policy: pull
paths:
- node_modules/
artifacts:
paths:
- built/
```
### Run your CI/CD pipeline
That's it! Add all your new files, commit, and push. For a reference of what our repo should
look like at this point, please refer to the [final commit related to this article on my sample repository](https://gitlab.com/blitzgren/gitlab-game-demo/commit/8b36ef0ecebcf569aeb251be4ee13743337fcfe2).
By applying both build and test stages, GitLab will run them sequentially at every push to
our repository. If all goes well you'll end up with a green check mark on each job for the pipeline:
![Passing Pipeline](img/test_pipeline_pass.png)
You can confirm that the tests passed by clicking on the `test` job to enter the full build logs.
Scroll to the bottom and observe, in all its passing glory:
```sh
$ gulp run-test
[18:37:24] Using gulpfile /builds/blitzgren/gitlab-game-demo/gulpfile.js
[18:37:24] Starting 'run-test'...
[18:37:24] Finished 'run-test' after 21 ms
Weapon
✓ should shoot if not in cooldown
✓ should not shoot during cooldown
✓ should shoot after cooldown ends
✓ should not shoot if not triggered
4 passing (18ms)
Uploading artifacts...
built/: found 17 matching files
Uploading artifacts to coordinator... ok id=17095874 responseStatus=201 Created token=aaaaaaaa Job succeeded
```
## Continuous Deployment
We have our codebase built and tested on every push. To complete the full pipeline with Continuous Deployment,
let's set up [free web hosting with AWS S3](https://aws.amazon.com/s/dm/optimization/server-side-test/free-tier/free_np/) and a job through which our build artifacts get
deployed. GitLab also has a free static site hosting service we could use, [GitLab Pages](https://about.gitlab.com/features/pages/),
however Dark Nova specifically uses other AWS tools that necessitates using `AWS S3`.
Read through this article that describes [deploying to both S3 and GitLab Pages](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/)
and further delves into the principles of GitLab CI/CD than discussed in this article.
### Set up S3 Bucket
1. Log into your AWS account and go to [S3](https://console.aws.amazon.com/s3/home)
1. Click the **Create Bucket** link at the top
1. Enter a name of your choosing and click next
1. Keep the default **Properties** and click next
1. Click the **Manage group permissions** and allow **Read** for the **Everyone** group, click next
1. Create the bucket, and select it in your S3 bucket list
1. On the right side, click **Properties** and enable the **Static website hosting** category
1. Update the radio button to the **Use this bucket to host a website** selection. Fill in `index.html` and `error.html` respectively
### Set up AWS Secrets
We need to be able to deploy to AWS with our AWS account credentials, but we certainly
don't want to put secrets into source code. Luckily GitLab provides a solution for this
with [Secret Variables](../../../ci/variables/README.md). This can get complicated
due to [IAM](https://aws.amazon.com/iam/) management. As a best practice, you shouldn't
use root security credentials. Proper IAM credential management is beyond the scope of this
article, but AWS will remind you that using root credentials is unadvised and against their
best practices, as they should. Feel free to follow best practices and use a custom IAM user's
credentials, which will be the same two credentials (Key ID and Secret). It's a good idea to
fully understand [IAM Best Practices in AWS](http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html). We need to add these credentials to GitLab:
1. Log into your AWS account and go to the [Security Credentials page](https://console.aws.amazon.com/iam/home#/security_credential)
1. Click the **Access Keys** section and **Create New Access Key**. Create the key and keep the id and secret around, you'll need them later
![AWS Access Key Config](img/aws_config_window.png)
1. Go to your GitLab project, click **Settings > CI/CD** on the left sidebar
1. Expand the **Secret Variables** section
![GitLab Secret Config](img/gitlab_config.png)
1. Add a key named `AWS_KEY_ID` and copy the key id from Step 2 into the **Value** textbox
1. Add a key named `AWS_KEY_SECRET` and copy the key secret from Step 2 into the **Value** textbox
### Deploy your game with GitLab CI/CD
To deploy our build artifacts, we need to install the [AWS CLI](https://aws.amazon.com/cli/) on
the Shared Runner. The Shared Runner also needs to be able to authenticate with your AWS
account to deploy the artifacts. By convention, AWS CLI will look for `AWS_ACCESS_KEY_ID`
and `AWS_SECRET_ACCESS_KEY`. GitLab's CI gives us a way to pass the secret variables we
set up in the prior section using the `variables` portion of the `deploy` job. At the end,
we add directives to ensure deployment `only` happens on pushes to `master`. This way, every
single branch still runs through CI, and only merging (or committing directly) to master will
trigger the `deploy` job of our pipeline. Put these together to get the following:
```yml
deploy:
stage: deploy
variables:
AWS_ACCESS_KEY_ID: "$AWS_KEY_ID"
AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET"
script:
- apt-get update
- apt-get install -y python3-dev python3-pip
- easy_install3 -U pip
- pip3 install --upgrade awscli
- aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete
only:
- master
```
Be sure to update the region and S3 URL in that last script command to fit your setup.
Our final configuration file `.gitlab-ci.yml` looks like:
```yml
image: node:6
build:
stage: build
script:
- npm i gulp -g
- npm i
- gulp
- gulp build-test
cache:
policy: push
paths:
- node_modules/
artifacts:
paths:
- built/
test:
stage: test
script:
- npm i gulp -g
- gulp run-test
cache:
policy: pull
paths:
- node_modules/
artifacts:
paths:
- built/
deploy:
stage: deploy
variables:
AWS_ACCESS_KEY_ID: "$AWS_KEY_ID"
AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET"
script:
- apt-get update
- apt-get install -y python3-dev python3-pip
- easy_install3 -U pip
- pip3 install --upgrade awscli
- aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete
only:
- master
```
## Conclusion
Within the [demo repository](https://gitlab.com/blitzgren/gitlab-game-demo) you can also find a handful of boilerplate code to get
[Typescript](https://www.typescriptlang.org/), [Mocha](https://mochajs.org/), [Gulp](http://gulpjs.com/) and [Phaser](https://phaser.io) all playing
together nicely with GitLab CI/CD, which is the result of lessons learned while making [Dark Nova](http://darknova.io/).
Using a combination of free and open source software, we have a full CI/CD pipeline, a game foundation,
and unit tests, all running and deployed at every push to master - with shockingly little code.
Errors can be easily debugged through GitLab's build logs, and within minutes of a successful commit,
you can see the changes live on your game.
Setting up Continous Integration and Continuous Deployment from the start with Dark Nova enables
rapid but stable development. We can easily test changes in a separate [environment](../../../ci/environments.md#introduction-to-environments-and-deployments),
or multiple environments if needed. Balancing and updating a multiplayer game can be ongoing
and tedious, but having faith in a stable deployment with GitLab CI/CD allows
a lot of breathing room in quickly getting changes to players.
## Further settings
Here are some ideas to further investigate that can speed up or improve your pipeline:
- [Yarn](https://yarnpkg.com) instead of npm
- Setup a custom [Docker](../../../ci/docker/using_docker_images.md#define-image-and-services-from-gitlab-ci-yml) image that can preload dependencies and tools (like AWS CLI)
- Forward a [custom domain](http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) to your game's S3 static website
- Combine jobs if you find it unnecessary for a small project
- Avoid the queues and set up your own [custom GitLab CI/CD runner](https://about.gitlab.com/2016/03/01/gitlab-runner-with-docker/)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment