BigW Consortium Gitlab

  1. 26 Sep, 2016 1 commit
  2. 20 Sep, 2016 3 commits
    • Implement a second round of review comments from @DouweM. · 918e589c
      Timothy Andrew authored
      - Don't use `TableReferences` - using `.arel_table` is shorter!
      - Move some database-related code to `Gitlab::Database`
      - Remove the `MergeRequest#issues_closed` and
        `Issue#closed_by_merge_requests`  associations. They were either
        shadowing or were too similar to existing methods. They are not being
        used anywhere, so it's better to remove them to reduce confusion.
      - Use Rails 3-style validations
      - Index for `MergeRequest::Metrics#first_deployed_to_production_at`
      - Only include `CycleAnalyticsHelpers::TestGeneration` for specs that
        need it.
      - Other minor refactorings.
    • Implement review comments from @yorickpeterse · 8957293d
      Timothy Andrew authored
      1. Change multiple updates to a single `update_all`
      
      2. Use cascading deletes
      
      3. Extract an average function for the database median.
      
      4. Move database median to `lib/gitlab/database`
      
      5. Use `delete_all` instead of `destroy_all`
      
      6. Minor refactoring
  3. 19 Sep, 2016 1 commit
  4. 17 Sep, 2016 1 commit
    • Move cycle analytics calculations to SQL. · 7d69ff3d
      Timothy Andrew authored
      1. Use Arel for composable queries.
      2. For a project with ~10k issues, the page loads in around 600ms.
         Previously, a project with ~5k issues would have a ~20s page load
         time.
  5. 15 Sep, 2016 1 commit
    • Improve performance of the cycle analytics page. · ba25e2f1
      Timothy Andrew authored
      1. These changes bring down page load time for 100 issues from more than
         a minute to about 1.5 seconds.
      
      2. This entire commit is composed of these types of performance
         enhancements:
      
           - Cache relevant data in `IssueMetrics` wherever possible.
           - Cache relevant data in `MergeRequestMetrics` wherever possible.
           - Preload metrics
      
      3. Given these improvements, we now only need to make 4 SQL calls:
      
          - Load all issues
          - Load all merge requests
          - Load all metrics for the issues
          - Load all metrics for the merge requests
      
      4. A list of all the data points that are now being pre-calculated:
      
          a. The first time an issue is mentioned in a commit
      
            - In `GitPushService`, find all issues mentioned by the given commit
              using `ReferenceExtractor`. Set the `first_mentioned_in_commit_at`
              flag for each of them.
      
            - There seems to be a (pre-existing) bug here - files (and
              therefore commits) created using the Web CI don't have
              cross-references created, and issues are not closed even when
              the commit title is "Fixes #xx".
      
          b. The first time a merge request is deployed to production
      
            When a `Deployment` is created, find all merge requests that
            were merged in before the deployment, and set the
            `first_deployed_to_production_at` flag for each of them.
      
          c. The start / end time for a merge request pipeline
      
            Hook into the `Pipeline` state machine. When the `status` moves to
            `running`, find the merge requests whose tip commit matches the
            pipeline, and record the `latest_build_started_at` time for each
            of them. When the `status` moves to `success`, record the
            `latest_build_finished_at` time.
      
          d. The merge requests that close an issue
      
            - This was a big cause of the performance problems we were having
              with Cycle Analytics. We need to use `ReferenceExtractor` to make
              this calculation, which is slow when we have to run it on a large
              number of merge requests.
      
            - When a merge request is created, updated, or refreshed, find the
              issues it closes, and create an instance of
              `MergeRequestsClosingIssues`, which acts as a join model between
              merge requests and issues.
      
            - If a `MergeRequestsClosingIssues` instance links a merge request
              and an issue, that issue closes that merge request.
      
      5. The `Queries` module was changed into a class, so we can cache the
         results of `issues` and `merge_requests_closing_issues` across
         various cycle analytics stages.
      
      6. The code added in this commit is untested. Tests will be added in the
         next commit.
  6. 14 Sep, 2016 1 commit
  7. 07 Sep, 2016 2 commits
    • Test the `test` cycle analytics phase. · 9cff3f8f
      Timothy Andrew authored
    • Test the `staging` cycle analytics phase. · dd112ef1
      Timothy Andrew authored
      Remove overlap from the "start + end" durations in the happy test
      case. For the `staging` phase, the end time is the _first_ deployment
      that happens after the MR merge.
      
      If we have 5 MRs where the `start_time`s (merge time) are the
      same, and all the `end_time`s (deploy to production) a few days from
      now, only the earliest deploy will get picked up, because that
      consitutes a deploy for _all_ the MRs.
      
      We fix this by removing overlap. Every `start_time` is now generated to
      be _after_ the preceding `end_time`.
  8. 02 Sep, 2016 1 commit
    • Tweak cycle analytics query to match the current requirements. · 0910868d
      Timothy Andrew authored
      - The `review` phase ends when a MR is merged, not "merged OR closed".
      - The `code` phase starts when a MR is first mentioned in a commit, and
        ends when a merge request closing the issue is created.
      - The `plan` phase ends when the issue first mentioned in a commit.
      
      ---
      
      - Fix the `median` function so it sorts the incoming data points.
      - A data point where `end_time` is prior to `start_time` is invalid.
  9. 26 Aug, 2016 8 commits