BigW Consortium Gitlab
Batch size >1000 does not pay off We did a small experiment to see how a full scan of the Redis keys on gitlab.com speeds up as we increase the batch size. The values on the right are time in seconds for a full scan (no delete operations). ``` count: 10); 284.500529021 count: 100); 86.21216934 count: 1_000); 60.931676195 count: 10_000); 60.96355610 count: 100_000); 62.378172541 ``` It looks like 1,000 is a good number. See merge request !2970
Name |
Last commit
|
Last update |
---|---|---|
.. | ||
api | Loading commit data... | |
assets | Loading commit data... | |
backup | Loading commit data... | |
banzai | Loading commit data... | |
ci | Loading commit data... | |
gitlab | Loading commit data... | |
omni_auth | Loading commit data... | |
rouge/formatters | Loading commit data... | |
support | Loading commit data... | |
tasks | Loading commit data... | |
award_emoji.rb | Loading commit data... | |
banzai.rb | Loading commit data... | |
disable_email_interceptor.rb | Loading commit data... | |
event_filter.rb | Loading commit data... | |
extracts_path.rb | Loading commit data... | |
file_size_validator.rb | Loading commit data... | |
file_streamer.rb | Loading commit data... | |
gitlab.rb | Loading commit data... | |
gt_one_coercion.rb | Loading commit data... | |
repository_cache.rb | Loading commit data... | |
static_model.rb | Loading commit data... | |
unfold_form.rb | Loading commit data... | |
uploaded_file.rb | Loading commit data... | |
version_check.rb | Loading commit data... |