BigW Consortium Gitlab

sidekiq_job_throttling.md 1.38 KB
Newer Older
1 2 3 4 5
# Sidekiq Job throttling

> Note: Introduced with GitLab 8.14

When your GitLab installation needs to handle tens of thousands of background
6 7 8
jobs, it can be convenient to throttle queues that do not need to be executed
immediately, e.g. long running jobs like Pipelines, thus allowing jobs that do
need to be executed immediately to have access to more resources.
9 10

In order to accomplish this, you can limit the amount of workers that certain
11
slow running queues can have available. This is what we call Sidekiq Job
12
Throttling. Depending on your infrastructure, you might have different slow
13 14
running queues, which is why you can choose which queues you want to throttle 
and by how much you want to throttle them.
15 16 17 18 19 20 21 22 23 24 25 26 27

These settings are available in the Application Settings of your GitLab
installation.

![Sidekiq Job Throttling](img/sidekiq_job_throttling.png)

The throttle factor determines the maximum number of workers a queue can run on.
This value gets multiplied by `:concurrency` value set in the Sidekiq settings
and rounded up to the closest full integer.

So, for example, you set the `:concurrency` to 25 and the `Throttling factor` to
0.1, the maximum workers assigned to the selected queues would be 3.

28 29
```ruby
queue_limit = (factor * Sidekiq.options[:concurrency]).ceil
30 31 32 33
```

After enabling the job throttling, you will need to restart your GitLab
instance, in order for the changes to take effect.