Forbid scripting for wiki files

Wiki files (not pages - files in the repo) are just sent to the browser
with whatever content-type the mime_types gem assigns to them based on
their extension. As this is from the same domain as the GitLab
application, this is an XSS vulnerability.

Set a CSP forbidding all sources for scripting, CSS, XHR, etc. on these
files.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/17298.

See merge request !1969

Instrument Grape API endpoints

See merge request !4587

Instrument private methods and instance private methods

See merge request !4639

Fix displaying of project settings links the user cannot access.

## What does this MR do?

It fixes the Project Settings dropdown displaying project settings links that the user cannot actually access.

## Are there points in the code the reviewer needs to double check?

I've tested combinations I can think of, feel free to mess around with and see if it breaks?

## Why was this MR needed?

Users were seeing links in the Project Settings dropdown that they shouldn't have seen, if they clicked them they would be shown permission errors.

## What are the relevant issue numbers?

#18294 

## Screenshots (if relevant)

Users without any permissions:

![Screen_Shot_2016-06-10_at_10.41.27_AM](/uploads/b70ca18a36b5f774b85694d8f1728882/Screen_Shot_2016-06-10_at_10.41.27_AM.png)

Guest members of the project:

![Screen_Shot_2016-06-10_at_10.48.36_AM](/uploads/a37986b4daa789063661c2fa8cf59d43/Screen_Shot_2016-06-10_at_10.48.36_AM.png)

Full permissions:

![Screen_Shot_2016-06-10_at_10.41.57_AM](/uploads/4c5cc97962e69a1a72ee8e237591ec22/Screen_Shot_2016-06-10_at_10.41.57_AM.png)

cc: @dzaporozhets @annabeldunstone @jschatz1 

See merge request !4599

Artifacts expire date

What do you think @grzesiek?

The syntax will be simple:
```
job:
  artifacts:
    expire_in: 7d
```

- [x] Implement `expire_in`
- [x] Check current design of expiry information with @jschatz1 and @markpundsack 
- [x] Add tests in GitLab application for a `ExpireBuildArtifactsWorker` and for `ArtifactsController::keep`
- [x] Add user documentation how to use `artifacts:expire_in`
- [x] Prepare GitLab Runner changes to pass `expire_in`: gitlab-org/gitlab-ci-multi-runner!191
- [x] Fix `timeago` with help of @jschatz1
- [x] Merge latest master after builds view changes @iamphill
- [ ] Add Omnibus support for `expire_build_artifacts_worker` cron job
- [ ] Add documentation how to configure `expire_build_artifacts_worker`

This is based on https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4201.

See merge request !4200

Update CI API docs

Since CI is now part of GitLab, it makes sense to group all API related endpoints under
one location. This is a step towards the much awaited issue of doc restructure
https://gitlab.com/gitlab-org/gitlab-ce/issues/3349

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/18404

See merge request !4567

By default instrumentation will instrument public,
protected and private methods, because usually
heavy work is done on private method or at least
that’s what facts is showing

Fix timing issues on convertion migration award emoji

See merge request !4581

- Move ci/api under api/ci
- Clean up builds.md and runners.md
- Replace old links with new ones
- Add CI API links in ci/README.md

Measure CPU time for instrumented methods

See merge request !4640

Generating the following tags

Grape#GET /projects/:id/archive

from Grape::Route objects like

{ :path => /:version/projects/:id/archive(.:format)
  :version => “v3”,
  :method => “GET” }

Use an instance variable to cache raw_path transformations.
This variable is only going to growth to the number of 
endpoints of the API, not with exact different requests

We can store this cache as an instance variable because 
middleware are initialised only once

Update CI Docker docs

## What does this MR do?

Updates documentation with:

* instructions for bind-mounting for docker runners (fixes #17769 and #13898)
* examples of using the GitLab Container Registry in docker-based builds (fixes #17968 and #17967)
* update runner instructions for `gitlab-ci-multi-runner` (fixes https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1337)
* some grammar fixes

## Are there points in the code the reviewer needs to double check?

The markdown in general. The runner commands. I haven't been able to verify the socket version works yet.

## Why was this MR needed?

Better documentation for users.

## What are the relevant issue numbers?
#17769, #13898, #17968, #17967 

## Screenshots (if relevant)

See merge request !4524

Add global entry with before script to new CI config

## What does this MR do?

This MR adds a new entries to a new CI config class. It is next refactoring step after !4462.

See #15060

See merge request !4482

Wiki files (not pages - files in the repo) are just sent to the browser
with whatever content-type the mime_types gem assigns to them based on
their extension. As this is from the same domain as the GitLab
application, this is an XSS vulnerability.

Set a CSP forbidding all sources for scripting, CSS, XHR, etc. on these
files.

Add whitelisted elements correctly in sanitization

Add whitelisted elements correctly in sanitization

Consider this command:

    bundle exec rails r "include GitlabMarkdownHelper
    puts markdown('<span>this is a span</span>', pipeline: :description)
    puts markdown('<span>this is a span</span>')"

And the same in the opposite order:

    bundle exec rails r "include GitlabMarkdownHelper
    puts markdown('<span>this is a span</span>')
    puts markdown('<span>this is a span</span>', pipeline: :description)"

Before this change, they would both output:

    <p><span>this is a span</span></p>
    <p>this is a span</p>

That's because `span` is added to the list of whitelisted elements in
the `SanitizationFilter`, but this method tries not to make the same
changes multiple times. Unfortunately,
`HTML::Pipeline::SanitizationFilter::LIMITED`, which is used by the
`DescriptionPipeline`, uses the same Ruby objects for all of its hash
values _except_ `:elements`.

That means that whichever of `DescriptionPipeline` and `GfmPipeline` is
called first would have `span` in its whitelisted elements, and the
second wouldn't.

Fix this by adding a special check for modifying `:elements` twice, then
checking `:transformers` as before.


See merge request !4588

MySQL apparently doesn't support executing multiple queries in the same
`execute` call so we have to use a separate one for the "LOCK TABLES"
statement.

As suggested by @yorrickpeterse in
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4581#note_12373882
the locking of the MySQL database wasn't correct.

This commit does two things:
1. It adds logic which prevents timing issues when running the
migration. During the migration, notes can be created which _should_
be award emoji and thus migrated. To prevent these timing issues, a
lock is obtained on the table (MySQL) or on Transaction level (PG).
2. There was no down migration before as you'd probably lose some data.
Data effected is all awards on notes. These could be migrated back, as
the noteable type would just be Note, though this would litter the DB
with data which should not be there. This down migration does not yet
delete the table.

Allow users to create confidential issues in private projects

Closes #14787

## What does this MR do?

Allow users to create confidential issues in private projects, and exclude access to them to project members with `Guest` role.

## Are there points in the code the reviewer needs to double check?

The query generated by the `User#authorized_projects` method.

## Why was this MR needed?

Community have been requesting this feature.

## What are the relevant issue numbers?

https://gitlab.com/gitlab-org/gitlab-ce/issues/14787

https://gitlab.com/gitlab-org/gitlab-ce/issues/3678

## Screenshots (if relevant)

Not relevant.

## Todo

- [x] Allow users to create confidential issues in private projects
- [x] Project members with `Guest` role should not have access to confidential issues
- [ ] ~~Apply changes in EE + Elasticsearch~~ Will be done in another MR, when this got merged

See merge request !3471

Bamboo & TeamCity Services: Fix missing credentials & URL handling

_Note: Originally opened at !4367 by @bentolor_

I've also fixed the URL handling for TeamCity which is very similar to Bamboo implementation-wise.

-----

*Note:* This is a port from my [original pull request on GitHub](https://github.com/gitlabhq/gitlabhq/pull/9428)

## What does this MR do?
This improves the Bamboo Service and provides two fixes:

1. One for the situation, where the build trigger won't work because Bamboo is requiring authentication credentials for the trigger GET: 8f25aca307b49ee006172b8c2985a878800aa6b6
2. One which fixes the way how the configured Bamboo base URL is assembled to the final REST URL. fe9eb30d7ebe4a83eefea7e06f8b69b135dad15d

### Regarding credentials
The change now does provide additional HTTP Basic Auth parameters if user credentials were provided and appends an request parameter indicating the HTTP Basic Authentication should be used. This aligns interaction with Bamboo with the other calls this service executes.

### Regarding URL handling
If one had configured a `bamboo_url` like http://foo.bar/bamboo in the previous implementation the plugin directed it's request i.e. to http://foo.bar/rest/... instead of http://foo.bar/bamboo/rest/...


## Are there points in the code the reviewer needs to double check?
The second issues was probably an unwanted side effect of how Ruby's `URI.join` is working. It will only work correctly, if 
- ... the prefix URL has at least one or more  trailing `/`
- .. the appendix parts are _not_ prefixed with `/`

I need try & figure it out using the rather lacking, official stdlib documentation and playing around in `irb`. As I'm an absolute Ruby novice I'm unable to add/provide new tests.

## Why was this MR needed?
Because Gitlab does not work in our Bamboo-Environment at all: Neither it is able to trigger Bamboo runs nor does the Merge status check work. This MR at least fixes the trigger issues.

## What are the relevant issue numbers?
This MR originates from my [original pull request on GitHub](https://github.com/gitlabhq/gitlabhq/pull/9428).
Sadly the issue, that the merge status is still not working correctly for branches will still not work. But at least the trigger works. 

There happened to be very much discussion about the branch status issue in #1355 and  #2562 though that one is lost as the author retracted his branch. 

See merge request !4408

Add more information into RSS feed for issues

## What does this MR do?

This MR adds issue text, labels , milestone, assignee and due date into issues RSS feed.

## Are there points in the code the reviewer needs to double check?

#14572 requests to add 'weight' among other fields. Seems like issue weight is available
in enterprise edition only so it is not implemented in this MR. Please correct me if I'm wrong.

## Why was this MR needed?

This MR is needed because it extends issues RSS feed with useful information requested in
#14572.

## What are the relevant issue numbers?

https://gitlab.com/gitlab-org/gitlab-ce/issues/14572

See merge request !4158

If one had configured a `teamcity_url` like http://foo.bar/teamcity in
the previous implementation the plugin directed it's request i.e. to
http://foo.bar/httpAuth/... instead of http://foo.bar/teamcity/httpAuth/...

`URI.join` only works correctly, if the prefix URL has
  - at least one or more  trailing '/'
  - the appended parts are _not_ prefixed with '/'

The current implementation should work with all sorts of TeamCity base
URLs.
Signed-off-by: Rémy Coutable <remy@rymai.me>

Signed-off-by: Rémy Coutable <remy@rymai.me>

If one had configured a `bamboo_url` like http://foo.bar/bamboo in the
previous implementation the plugin directed it's request i.e. to
http://foo.bar/rest/... instead of http://foo.bar/bamboo/rest/...

`URI.join` only works correctly, if the prefix URL has
  - at least one or more  trailing '/'
  - the appended parts are _not_ prefixed with '/'

The current implementation should work with all sorts of Bamboo base URLs.
Signed-off-by: Rémy Coutable <remy@rymai.me>

This improves the Bamboo Service and provides a fix for situations,
where the build trigger won't work, because Bamboo is requiring
authentication also for the trigger GET.

The change now does provide additional HTTP Basic Auth parameters
if user credentials were provided and appends an request parameter
indicating the HTTP Basic Authentication should be used.
This aligns interaction with Bamboo with the other calls this service
executes.
Signed-off-by: Rémy Coutable <remy@rymai.me>

Retry spinach tests in case of failure using rerun reporter

## What does this MR do?

Fixes Spinach tests to retry on tests on master


See merge request !4539