BigW Consortium Gitlab

  1. 06 Nov, 2017 1 commit
  2. 07 Sep, 2017 1 commit
  3. 24 Aug, 2017 1 commit
  4. 06 Jul, 2017 1 commit
  5. 19 Jun, 2017 1 commit
  6. 16 Jun, 2017 1 commit
    • Refactor ProjectsFinder#init_collection · d2934722
      Yorick Peterse authored
      This changes ProjectsFinder#init_collection so it no longer relies on a
      UNION. For example, to get starred projects of a user we used to run:
      
          SELECT projects.*
          FROM projects
          WHERE projects.pending_delete = 'f'
          AND (
              projects.id IN (
                  SELECT projects.id
                  FROM projects
                  INNER JOIN users_star_projects
                      ON users_star_projects.project_id = projects.id
                  INNER JOIN project_authorizations
                      ON projects.id = project_authorizations.project_id
                  WHERE projects.pending_delete = 'f'
                  AND project_authorizations.user_id = 1
                  AND users_star_projects.user_id = 1
      
                  UNION
      
                  SELECT projects.id
                  FROM projects
                  INNER JOIN users_star_projects
                      ON users_star_projects.project_id = projects.id
                  WHERE projects.visibility_level IN (20, 10)
                  AND users_star_projects.user_id = 1
              )
          )
          ORDER BY projects.id DESC;
      
      With these changes the above query is turned into the following instead:
      
          SELECT projects.*
          FROM projects
          INNER JOIN users_star_projects
              ON users_star_projects.project_id = projects.id
          WHERE projects.pending_delete = 'f'
          AND (
              EXISTS (
                  SELECT 1
                  FROM project_authorizations
                  WHERE project_authorizations.user_id = 1
                  AND (project_id = projects.id)
              )
              OR projects.visibility_level IN (20,10)
          )
          AND users_star_projects.user_id = 1
          ORDER BY projects.id DESC;
      
      This query in turn produces a better execution plan and takes less time,
      though the difference is only a few milliseconds (this however depends
      on the amount of data involved and additional conditions that may be
      added).
  7. 30 May, 2017 4 commits
    • Add :owned param to ProjectFinder · db679788
      Toon Claes authored
      And use it in the API.
    • Make it possible to combine :trending with other params · 5654ac87
      Toon Claes authored
      Now it is possible to combine the :non_public parameter. This might be useful
      when a user wants to know the trending projects they are member of.
    • UNION of SELECT/WHERE is faster than WHERE on UNION · 01c6323d
      Toon Claes authored
      Instead of applying WHERE on a UNION, apply the WHERE on each of the seperate
      SELECT statements, and do UNION on that.
      
      Local tests with about 2_000_000 projects:
       - 1_500_000 private projects
       -    40_000 internal projects
       -   400_000 public projects
      
      For the API endpoint `/api/v4/projects?visibility=private` the slowest query was:
      
      ```sql
      SELECT "projects".*
      FROM "projects"
      WHERE ...
      ```
      
      The original query took 1073.8ms.
      The query refactored to UNION of SELECT/WHERE took 2.3ms.
      
      The original query was:
      
      ```sql
      SELECT "projects".*
      FROM "projects"
      WHERE "projects"."pending_delete" = $1
        AND (projects.id IN
               (SELECT "projects"."id"
                FROM "projects"
                INNER JOIN "project_authorizations" ON "projects"."id" = "project_authorizations"."project_id"
                WHERE "projects"."pending_delete" = 'f'
                  AND "project_authorizations"."user_id" = 23
                UNION SELECT "projects"."id"
                FROM "projects"
                WHERE "projects"."visibility_level" IN (20,
                                                        10)))
        AND "projects"."visibility_level" = $2
        AND "projects"."archived" = $3
      ORDER BY "projects"."created_at" DESC
      LIMIT 20
      OFFSET 0 [["pending_delete", "f"],
             ["visibility_level", 0],
             ["archived", "f"]]
      ```
      
      The refactored query:
      ```sql
      SELECT "projects".*
      FROM "projects"
      WHERE "projects"."pending_delete" = $1
        AND (projects.id IN
               (SELECT "projects"."id"
                FROM "projects"
                INNER JOIN "project_authorizations" ON "projects"."id" = "project_authorizations"."project_id"
                WHERE "projects"."pending_delete" = 'f'
                  AND "project_authorizations"."user_id" = 23
                  AND "projects"."visibility_level" = 0
                  AND "projects"."archived" = 'f'
                UNION SELECT "projects"."id"
                FROM "projects"
                WHERE "projects"."visibility_level" IN (20,
                                                        10)
                  AND "projects"."visibility_level" = 0
                  AND "projects"."archived" = 'f'))
      ORDER BY "projects"."created_at" DESC
      LIMIT 20
      OFFSET 0 [["pending_delete", "f"]]
      ```
    • Change ProjectFinder so starred can be combined with other filters · 07250508
      Toon Claes authored
      The `starred` parameter couldn't be used in combination with `trending` or
      `non_public`. But this is changed now.
  8. 06 Apr, 2017 1 commit
    • ProjectsFinder should handle more options · b996a82f
      Jacopo authored
      Extended ProjectFinder in order to handle the following options:
       - current_user - which user use
       - project_ids_relation: int[] - project ids to use
       - params:
         -  trending: boolean
         -  non_public: boolean
         -  starred: boolean
         -  sort: string
         -  visibility_level: int
         -  tags: string[]
         -  personal: boolean
         -  search: string
         -  non_archived: boolean
      
      GroupProjectsFinder now inherits from ProjectsFinder.
      Changed the code in order to use the new available options.
  9. 08 Feb, 2017 1 commit
  10. 15 Aug, 2016 2 commits
  11. 20 Mar, 2016 1 commit
  12. 14 Mar, 2016 1 commit
  13. 13 Mar, 2016 2 commits
  14. 12 Mar, 2016 1 commit
    • Removed User#project_relations · 3b76b73a
      Yorick Peterse authored
      GitLab EE adds an extra relation that selects a "project_id" column
      instead of an "id" column, making it very hard for this method to be
      re-used in EE. Since using User#authorized_groups in
      ProjectsFinder#all_groups apparently has no performance impact we can
      just use it and keep everything compatible with EE.
  15. 11 Mar, 2016 2 commits
  16. 20 Nov, 2015 1 commit
    • Port GitLab EE ProjectsFinder changes · fed059a1
      Yorick Peterse authored
      These changes were added in GitLab EE commit
      d39de0ea91b26b8840195e5674b92c353cc16661. The tests were a bit bugged
      (they used a non existing group, thus not testing a crucial part) which
      I only noticed when porting CE changes to EE.
  17. 18 Nov, 2015 1 commit
    • Refactor ProjectsFinder to not pluck IDs · fbcf3bd3
      Yorick Peterse authored
      This class now uses a UNION (when needed) instead of plucking tens of
      thousands of project IDs into memory. The tests have also been
      re-written to ensure all different use cases are tested properly
      (assuming I didn't forget any cases).
      
      The finder has also been broken up into 3 different finder classes:
      
      * ContributedProjectsFinder: class for getting the projects a user
        contributed to.
      * PersonalProjectsFinder: class for getting the personal projects of a
        user.
      * ProjectsFinder: class for getting generic projects visible to a given
        user.
      
      Previously a lot of the logic of these finders was handled directly in
      the users controller.
  18. 15 Sep, 2014 1 commit
  19. 14 Sep, 2014 1 commit
  20. 05 Jun, 2014 1 commit
  21. 25 Feb, 2014 1 commit