BigW Consortium Gitlab

testing.md 8.26 KB
Newer Older
1 2
# Frontend Testing

3 4 5
There are two types of test suites you'll encounter while developing frontend code
at GitLab.  We use Karma and Jasmine for JavaScript unit and integration testing, and RSpec
feature tests with Capybara for e2e (end-to-end) integration testing.
6

7 8 9
Unit and feature tests need to be written for all new features.
Most of the time, you should use rspec for your feature tests.
There are cases where the behaviour you are testing is not worth the time spent running the full application,
10
for example, if you are testing styling, animation, edge cases or small actions that don't involve the backend,
11 12 13 14 15 16 17
you should write an integration test using Jasmine.

![Testing priority triangle](img/testing_triangle.png)

_This diagram demonstrates the relative priority of each test type we use_

Regression tests should be written for bug fixes to prevent them from recurring in the future.
18

19
See [the Testing Standards and Style Guidelines](../testing.md)
20 21 22 23 24
for more information on general testing practices at GitLab.

## Karma test suite

GitLab uses the [Karma][karma] test runner with [Jasmine][jasmine] as its test
25 26
framework for our JavaScript unit and integration tests. For integration tests,
we generate HTML files using RSpec (see `spec/javascripts/fixtures/*.rb` for examples).
27
Some fixtures are still HAML templates that are translated to HTML files using the same mechanism (see `static_fixtures.rb`).
28 29 30
Adding these static fixtures should be avoided as they are harder to keep up to date with real views.
The existing static fixtures will be migrated over time.
Please see [gitlab-org/gitlab-ce#24753](https://gitlab.com/gitlab-org/gitlab-ce/issues/24753) to track our progress.
31
Fixtures are served during testing by the [jasmine-jquery][jasmine-jquery] plugin.
32

33 34 35 36 37 38 39 40 41
JavaScript tests live in `spec/javascripts/`, matching the folder structure
of `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js`
has a corresponding `spec/javascripts/behaviors/autosize_spec.js` file.

Keep in mind that in a CI environment, these tests are run in a headless
browser and you will not have access to certain APIs, such as
[`Notification`](https://developer.mozilla.org/en-US/docs/Web/API/notification),
which will have to be stubbed.

42 43 44
### Best practice

#### Naming unit tests
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70

When writing describe test blocks to test specific functions/methods,
please use the method name as the describe block name.

```javascript
// Good
describe('methodName', () => {
  it('passes', () => {
    expect(true).toEqual(true);
  });
});

// Bad
describe('#methodName', () => {
  it('passes', () => {
    expect(true).toEqual(true);
  });
});

// Bad
describe('.methodName', () => {
  it('passes', () => {
    expect(true).toEqual(true);
  });
});
```
71 72 73 74 75 76
#### Testing Promises

When testing Promises you should always make sure that the test is asynchronous and rejections are handled.
Your Promise chain should therefore end with a call of the `done` callback and `done.fail` in case an error occurred.

```javascript
77
// Good
78 79 80 81 82 83 84 85 86
it('tests a promise', (done) => {
  promise
    .then((data) => {
      expect(data).toBe(asExpected);
    })
    .then(done)
    .catch(done.fail);
});

87
// Good
88 89
it('tests a promise rejection', (done) => {
  promise
90
    .then(done.fail)
91 92 93 94 95 96 97
    .catch((error) => {
      expect(error).toBe(expectedError);
    })
    .then(done)
    .catch(done.fail);
});

98
// Bad (missing done callback)
99 100 101 102 103 104 105
it('tests a promise', () => {
  promise
    .then((data) => {
      expect(data).toBe(asExpected);
    })
});

106
// Bad (missing catch)
107 108 109 110 111 112 113 114
it('tests a promise', (done) => {
  promise
    .then((data) => {
      expect(data).toBe(asExpected);
    })
    .then(done)
});

115
// Bad (use done.fail in asynchronous tests)
116 117 118 119 120 121 122 123 124
it('tests a promise', (done) => {
  promise
    .then((data) => {
      expect(data).toBe(asExpected);
    })
    .then(done)
    .catch(fail)
});

125
// Bad (missing catch)
126 127 128 129 130 131 132 133
it('tests a promise rejection', (done) => {
  promise
    .catch((error) => {
      expect(error).toBe(expectedError);
    })
    .then(done)
});
```
134

135 136 137 138 139 140 141 142
#### Stubbing

For unit tests, you should stub methods that are unrelated to the current unit you are testing.
If you need to use a prototype method, instantiate an instance of the class and call it there instead of mocking the instance completely.

For integration tests, you should stub methods that will effect the stability of the test if they
execute their original behaviour. i.e. Network requests.

143 144 145
### Vue.js unit tests
See this [section][vue-test].

146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176
### Running frontend tests

`rake karma` runs the frontend-only (JavaScript) tests.
It consists of two subtasks:

- `rake karma:fixtures` (re-)generates fixtures
- `rake karma:tests` actually executes the tests

As long as the fixtures don't change, `rake karma:tests` (or `yarn karma`)
is sufficient (and saves you some time).

### Live testing and focused testing

While developing locally, it may be helpful to keep karma running so that you
can get instant feedback on as you write tests and modify code.  To do this
you can start karma with `npm run karma-start`.  It will compile the javascript
assets and run a server at `http://localhost:9876/` where it will automatically
run the tests on any browser which connects to it.  You can enter that url on
multiple browsers at once to have it run the tests on each in parallel.

While karma is running, any changes you make will instantly trigger a recompile
and retest of the entire test suite, so you can see instantly if you've broken
a test with your changes.  You can use [jasmine focused][jasmine-focus] or
excluded tests (with `fdescribe` or `xdescribe`) to get karma to run only the
tests you want while you're working on a specific feature, but make sure to
remove these directives when you commit your code.

## RSpec Feature Integration Tests

Information on setting up and running RSpec integration tests with
[Capybara][capybara] can be found in the
177
[general testing guide](../testing.md).
178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208

## Gotchas

### Errors due to use of unsupported JavaScript features

Similar errors will be thrown if you're using JavaScript features not yet
supported by the PhantomJS test runner which is used for both Karma and RSpec
tests.  We polyfill some JavaScript objects for older browsers, but some
features are still unavailable:

- Array.from
- Array.first
- Async functions
- Generators
- Array destructuring
- For..Of
- Symbol/Symbol.iterator
- Spread

Until these are polyfilled appropriately, they should not be used.  Please
update this list with additional unsupported features.

### RSpec errors due to JavaScript

By default RSpec unit tests will not run JavaScript in the headless browser
and will simply rely on inspecting the HTML generated by rails.

If an integration test depends on JavaScript to run correctly, you need to make
sure the spec is configured to enable JavaScript when the tests are run. If you
don't do this you'll see vague error messages from the spec runner.

209
To enable a JavaScript driver in an `rspec` test, add `:js` to the
210 211 212 213 214
individual spec or the context block containing multiple specs that need
JavaScript enabled:

```ruby
# For one spec
215
it 'presents information about abuse report', :js do
216
  # assertions...
217 218
end

219
describe "Admin::AbuseReports", :js do
220 221 222 223 224 225
  it 'presents information about abuse report' do
    # assertions...
  end
  it 'shows buttons for adding to abuse report' do
    # assertions...
  end
226 227 228 229 230 231 232 233 234 235 236 237 238 239 240
end
```

### Spinach errors due to missing JavaScript

> **Note:** Since we are discouraging the use of Spinach when writing new
> feature tests, you shouldn't ever need to use this.  This information is kept
> available for legacy purposes only.

In Spinach, the JavaScript driver is enabled differently. In the `*.feature`
file for the failing spec, add the `@javascript` flag above the Scenario:

```
@javascript
Scenario: Developer can approve merge request
241 242 243 244 245 246
  Given I am a "Shop" developer
  And I visit project "Shop" merge requests page
  And merge request 'Bug NS-04' must be approved
  And I click link "Bug NS-04"
  When I click link "Approve"
  Then I should see approved merge request "Bug NS-04"
247 248 249 250 251 252 253
```

[capybara]: http://teamcapybara.github.io/capybara/
[jasmine]: https://jasmine.github.io/
[jasmine-focus]: https://jasmine.github.io/2.5/focused_specs.html
[jasmine-jquery]: https://github.com/velesin/jasmine-jquery
[karma]: http://karma-runner.github.io/
254
[vue-test]:https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components