Avoid hitting statement timeout finding MR pipelines

For MRs with many thousands of commits, `SELECT DISTINCT(sha)` will be very slow. What we can't do to fix this: 1. Add an index. Postgres won't use it for DISTINCT without a lot of ceremony. 2. Do the `uniq` in Ruby. That can still be very slow with hundreds of thousands of commits. 3. Use a subquery. We haven't removed the `st_commits` column yet, but we will soon. Until 3 is available to us, we can just do 2, but also add a limit clause. There is no ordering, so this may return different results, but our goal with these MRs is just to get them to load, so it's not a huge deal.

Avoid hitting statement timeout finding MR pipelines
ba9b4c4d · Sean McGivern · e9d35261 · ba9b4c4d · ba9b4c4d
Commit ba9b4c4d authored Oct 27, 2017 by Sean McGivern
2 changed files
--- a/app/models/merge_request.rb
+++ b/app/models/merge_request.rb
@@ -874,7 +874,7 @@ class MergeRequest < ActiveRecord::Base
  #
  def all_commit_shas
    if persisted?
-      column_shas = MergeRequestDiffCommit.where(merge_request_diff: merge_request_diffs).pluck('DISTINCT(sha)')
+      column_shas = MergeRequestDiffCommit.where(merge_request_diff: merge_request_diffs).limit(10_000).pluck('sha')
      serialised_shas = merge_request_diffs.where.not(st_commits: nil).flat_map(&:commit_shas)

      (column_shas + serialised_shas).uniq

--- a/changelogs/unreleased/39054-activerecord-statementinvalid-pg-querycanceled-error-canceling-statement-due-to-statement-timeout.yml
+++ b/changelogs/unreleased/39054-activerecord-statementinvalid-pg-querycanceled-error-canceling-statement-due-to-statement-timeout.yml
+---
+title: Stop merge requests with thousands of commits from timing out
+merge_request: 15063
+author:
+type: performance