BigW Consortium Gitlab

  1. 25 Nov, 2016 1 commit
  2. 23 Nov, 2016 1 commit
    • Remove event caching code · 5371da34
      Yorick Peterse authored
      Flushing the events cache worked by updating a recent number of rows in
      the "events" table. This has the result that on PostgreSQL a lot of dead
      tuples are produced on a regular basis. This in turn means that
      PostgreSQL will spend considerable amounts of time vacuuming this table.
      This in turn can lead to an increase of database load.
      
      For GitLab.com we measured the impact of not using events caching and
      found no measurable increase in response timings. Meanwhile not flushing
      the events cache lead to the "events" table having no more dead tuples
      as now rows are only inserted into this table.
      
      As a result of this we are hereby removing events caching as it does not
      appear to help and only increases database load.
      
      For more information see the following comment:
      https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6578#note_18864037
  3. 16 Nov, 2016 1 commit
  4. 09 Nov, 2016 1 commit
  5. 27 Oct, 2016 1 commit
  6. 20 Oct, 2016 1 commit
    • Differentiate the expire from leave event · 9124310f
      Callum Dryden authored
      At the moment we cannot see weather a user left a project due to their
      membership expiring of if they themselves opted to leave the project.
      This adds a new event type that allows us to make this differentiation.
      Note that is not really feasable to go back and reliably fix up the
      previous events. As a result the events for previous expire removals
      will remain the same however events of this nature going forward will be
      correctly represented.
  7. 11 Oct, 2016 1 commit
  8. 07 Oct, 2016 1 commit
    • Fix Event#reset_project_activity updates · 16626409
      Stan Hu authored
      !6678 removed the lease from Event#reset_project_activity, but it wasn't
      actually updating the project's last_activity_at timestamp properly.
      The WHERE clause would always return no matching projects. The spec
      passed occasionally because the created_at timestamp was automatically
      set to last_activity_at.
  9. 04 Oct, 2016 1 commit
    • Remove lease from Event#reset_project_activity · c9bcfc63
      Yorick Peterse authored
      Per GitLab.com's performance metrics this method could take up to 5
      seconds of wall time to complete, while only taking 1-2 milliseconds of
      CPU time. Removing the Redis lease in favour of conditional updates
      allows us to work around this.
      
      A slight drawback is that this allows for multiple threads/processes to
      try and update the same row. However, only a single thread/process will
      ever win since the UPDATE query uses a WHERE condition to only update
      rows that were not updated in the last hour.
      
      Fixes gitlab-org/gitlab-ce#22473
  10. 20 Sep, 2016 1 commit
  11. 19 Sep, 2016 1 commit
  12. 30 Aug, 2016 1 commit
  13. 06 Jul, 2016 1 commit
  14. 04 Jul, 2016 1 commit
  15. 28 Jun, 2016 1 commit
  16. 03 Jun, 2016 2 commits
  17. 16 May, 2016 5 commits
  18. 09 May, 2016 1 commit
    • Remove the annotate gem and delete old annotations · f1479b56
      Jeroen van Baarsen authored
      In 8278b763 the default behaviour of annotation
      has changes, which was causing a lot of noise in diffs. We decided in #17382
      that it is better to get rid of the whole annotate gem, and instead let people
      look at schema.rb for the columns in a table.
      
      Fixes: #17382
  19. 06 May, 2016 1 commit
  20. 25 Apr, 2016 1 commit
  21. 24 Mar, 2016 2 commits
  22. 17 Mar, 2016 1 commit
  23. 04 Feb, 2016 1 commit
  24. 27 Jan, 2016 1 commit
    • Use Atom update times of the first event · de7c9c7a
      Yorick Peterse authored
      By simply loading the first event from the already sorted set we save
      ourselves extra (slow) queries just to get the latest update timestamp.
      This removes the need for Event.latest_update_time and significantly
      reduces the time needed to build an Atom feed.
      
      Fixes gitlab-org/gitlab-ce#12415
  25. 26 Jan, 2016 1 commit
  26. 02 Dec, 2015 1 commit
  27. 18 Nov, 2015 2 commits
    • Added Event.limit_recent · 01620dd7
      Yorick Peterse authored
      This will be used to move some querying logic from the users controller
      to the Event model (where it belongs).
    • Faster way of obtaining latest event update time · 054f2f98
      Yorick Peterse authored
      Instead of using MAX(events.updated_at) we can simply sort the events in
      descending order by the "id" column and grab the first row. In other
      words, instead of this:
      
          SELECT max(events.updated_at) AS max_id
          FROM events
          LEFT OUTER JOIN projects   ON projects.id   = events.project_id
          LEFT OUTER JOIN namespaces ON namespaces.id = projects.namespace_id
          WHERE events.author_id IS NOT NULL
          AND events.project_id IN (13083);
      
      we can use this:
      
          SELECT events.updated_at AS max_id
          FROM events
          LEFT OUTER JOIN projects   ON projects.id   = events.project_id
          LEFT OUTER JOIN namespaces ON namespaces.id = projects.namespace_id
          WHERE events.author_id IS NOT NULL
          AND events.project_id IN (13083)
          ORDER BY events.id DESC
          LIMIT 1;
      
      This has the benefit that on PostgreSQL a backwards index scan can be
      used, which due to the "LIMIT 1" will at most process only a single row.
      This in turn greatly speeds up the process of grabbing the latest update
      time. This can be confirmed by looking at the query plans. The first
      query produces the following plan:
      
          Aggregate  (cost=43779.84..43779.85 rows=1 width=12) (actual time=2142.462..2142.462 rows=1 loops=1)
            ->  Index Scan using index_events_on_project_id on events  (cost=0.43..43704.69 rows=30060 width=12) (actual time=0.033..2138.086 rows=32769 loops=1)
                  Index Cond: (project_id = 13083)
                  Filter: (author_id IS NOT NULL)
          Planning time: 1.248 ms
          Execution time: 2142.548 ms
      
      The second query in turn produces the following plan:
      
          Limit  (cost=0.43..41.65 rows=1 width=16) (actual time=1.394..1.394 rows=1 loops=1)
            ->  Index Scan Backward using events_pkey on events  (cost=0.43..1238907.96 rows=30060 width=16) (actual time=1.394..1.394 rows=1 loops=1)
                  Filter: ((author_id IS NOT NULL) AND (project_id = 13083))
                  Rows Removed by Filter: 2104
          Planning time: 0.166 ms
          Execution time: 1.408 ms
      
      According to the above plans the 2nd query is around 1500 times faster.
      However, re-running the first query produces timings of around 80 ms,
      making the 2nd query "only" around 55 times faster.
  28. 11 Nov, 2015 1 commit
    • Change "recent" scopes to sort by "id" · 7eb502c0
      Yorick Peterse authored
      These scopes can just sort by the "id" column in descending order to
      achieve the same result. An added benefit is being able to perform a
      backwards index scan (depending on the rest of the final query) instead
      of having to actually sort data.
  29. 15 Sep, 2015 1 commit
  30. 02 Jul, 2015 1 commit
    • 'created_at DESC' is performed twice · 87ac5900
      catatsuy authored
      If you are already sorting in descending order in the created_at,
      it is run twice when you run the .recent.
      It has passed in the string 'created_at DESC'.
      Ruby on Rails is directly given to the SQL.
      It is a slow query in MySQL.
  31. 06 Apr, 2015 1 commit
  32. 22 Mar, 2015 1 commit
  33. 18 Mar, 2015 1 commit