Merge branch 'docs/ca-feedback' into 'master'

doc/user/project/cycle_analytics.md

@@ -6,7 +6,7 @@
 This the first iteration of Cycle Analytics, you can follow the following issue
 to track the changes that are coming to this feature: [#20975][ce-20975].
 
-Cycle Analytics measures the time it takes to go from [an idea to production] for
+Cycle Analytics measures the time it takes to go from an [idea to production] for
 each project you have. This is achieved by not only indicating the total time it
 takes to reach at that point, but the total time is broken down into the
 multiple stages an idea has to pass through to be shipped.
@@ -28,9 +28,10 @@ You can see that there are seven stages in total:
       (first assignment, any milestone, milestone date or assignee is not required)
 - **Plan** (Board)
     - Median time from giving an issue a milestone or label until pushing the
-      first commit
+      first commit to the branch
 - **Code** (IDE)
-    - Median time from the first commit until the merge request is created
+    - Median time from the first commit to the branch until the merge request is
+      created
 - **Test** (CI)
     - Median total test time for all commits/merges
 - **Review** (Merge Request/MR)
@@ -40,7 +41,10 @@ You can see that there are seven stages in total:
     - Median time from when the merge request got merged until the deploy to
       production (production is last stage/environment)
 - **Production** (Total)
-   - Sum of all the above stages excluding the Test (CI) time
+   - Sum of all the above stages' times excluding the Test (CI) time. To clarify,
+     it's not so much that CI time is "excluded", but rather CI time is already
+     counted in the review stage since CI is done automatically. Most of the
+     other stages are purely sequential, but **Test** is not.
 
 ## How the data is measured
 
@@ -57,25 +61,24 @@ Below you can see in more detail what the various stages of Cycle Analytics mean
 | **Stage** | **Description** |
 | --------- | --------------- |
 | Issue     | Measures the median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whatever comes first. The label will be tracked only if it already has an [Issue Board list][board] created for it. |
-| Plan      | Measures the median time between the action you took for the previous stage, and pushing the first commit to the repository. To make this change tracked, the pushed commit needs to contain the [issue closing pattern], for example `Closes #xxx`, where `xxx` is the number of the issue related to this commit. If the commit does not contain the issue closing pattern, it is not considered to the measurement time of the stage. |
-| Code      | Measures the median time between pushing a first commit (previous stage) and creating a merge request related to that commit. The key to keep the process tracked is include the [issue closing pattern] to the description of the merge request. |
+| Plan      | Measures the median time between the action you took for the previous stage, and pushing the first commit to the branch. The very first commit of the branch is the one that triggers the separation between **Plan** and **Code**, and at least one of the commits in the branch needs to contain the related issue number (e.g., `#42`). If none of the commits in the branch mention the related issue number, it is not considered to the measurement time of the stage. |
+| Code      | Measures the median time between pushing a first commit (previous stage) and creating a merge request (MR) related to that commit. The key to keep the process tracked is to include the [issue closing pattern] to the description of the merge request (for example, `Closes #xxx`, where `xxx` is the number of the issue related to this merge request). If the issue closing pattern is not present in the merge request description, the MR is not considered to the measurement time of the stage. |
 | Test      | Measures the median time to run the entire pipeline for that project. It's related to the time GitLab CI takes to run every job for the commits pushed to that merge request defined in the previous stage. It is basically the start->finish time for all pipelines. `master` is not excluded. It does not attempt to track time for any particular stages. |
 | Review    | Measures the median time taken to review the merge request, between its creation and until it's merged. |
-| Staging   | Measures the median time between merging the merge request until the very first deployment to production. It's tracked by the [environment] set to `production` in your GitLab CI configuration. If there isn't a `production` environment, this is not tracked. |
-| Production| The sum of all time taken to run the entire process, from issue creation to deploying the code to production. |
+| Staging   | Measures the median time between merging the merge request until the very first deployment to production. It's tracked by the [environment] set to `production` (case-sensitive, `Production` won't work) in your GitLab CI configuration. If there isn't a `production` environment, this is not tracked. |
+| Production| The sum of all time (medians) taken to run the entire process, from issue creation to deploying the code to production. |
 
 ---
 
 Here's a little explanation of how this works behind the scenes:
 
 1. Issues and merge requests are grouped together in pairs, such that for each
-   `<issue, merge request>` pair, the merge request has `Fixes #xxx` for the
-   corresponding issue. All other issues and merge requests are **not** considered.
-
+   `<issue, merge request>` pair, the merge request has the [issue closing pattern]
+   for the corresponding issue. All other issues and merge requests are **not**
+   considered.
 1. Then the <issue, merge request> pairs are filtered out. Any merge request
    that has **not** been deployed to production in the last XX days (specified
    by the UI - default is 90 days) prohibits these pairs from being considered.
-
 1. For the remaining `<issue, merge request>` pairs, we check the information that
    we need for the stages, like issue creation date, merge request merge time,
    etc.
@@ -86,6 +89,60 @@ label present in the Issue Board or assigned a milestone or a project has no
 `production` environment, the Cycle Analytics dashboard won't present any data
 at all.
 
+## Example workflow
+
+Below is a simple fictional workflow of a single cycle that happens in a
+single day passing through all seven stages. Note that if a stage does not have
+a start/stop mark, it is not measured and hence not calculated in the median
+time. It is assumed that milestones are created and CI for testing and setting
+environments is configured.
+
+1. Issue is created at 09:00 (start of **Issue** stage).
+1. Issue is added to a milestone at 11:00 (stop of **Issue** stage / start of
+   **Plan** stage).
+1. Start working on the issue, create a branch locally and make one commit at
+   12:00.
+1. Make a second commit to the branch which mentions the issue number at 12.30
+   (stop of **Plan** stage / start of **Code** stage).
+1. Push branch and create a merge request that contains the [issue closing pattern]
+   in its description at 14:00 (stop of **Code** stage / start of **Test** and
+   **Review** stages).
+1. The CI starts running your scripts defined in [`.gitlab-ci.yml`][yml] and
+   takes 5min (stop of **Test** stage).
+1. Review merge request, ensure that everything is OK and merge the merge
+   request at 19:00. (stop of **Review** stage / start of **Staging** stage).
+1. Now that the merge request is merged, a deployment to the `production`
+   environment starts and finishes at 19:30 (stop of **Staging** stage).
+1. The cycle completes and the sum of the median times of the previous stages
+   is recorded to the **Production** stage. That is the time between creating an
+   issue and deploying its relevant merge request to production.
+
+From the above example you can conclude the time it took each stage to complete
+as long as their total time:
+
+- **Issue**:  2h (11:00 - 09:00)
+- **Plan**:   1h (12:00 - 11:00)
+- **Code**:   2h (14:00 - 12:00)
+- **Test**:   5min
+- **Review**: 5h (19:00 - 14:00)
+- **Staging**:  30min (19:30 - 19:00)
+- **Production**: Since this stage measures the sum of median time off all
+  previous stages, we cannot calculate it if we don't know the status of the
+  stages before. In case this is the very first cycle that is run in the project,
+  then the **Production** time is 10h 30min (19:30 - 09:00)
+
+A few notes:
+
+- In the above example we demonstrated that it doesn't matter if your first
+  commit doesn't mention the issue number, you can do this later in any commit
+  of the branch you are working on.
+- You can see that the **Test** stage is not calculated to the overall time of
+  the cycle since it is included in the **Review** process (every MR should be
+  tested).
+- The example above was just **one cycle** of the seven stages. Add multiple
+  cycles, calculate their median time and the result is what the dashboard of
+  Cycle Analytics is showing.
+
 ## Permissions
 
 The current permissions on the Cycle Analytics dashboard are:
@@ -104,11 +161,12 @@ Learn more about Cycle Analytics in the following resources:
 - [Cycle Analytics feature highlight](https://about.gitlab.com/2016/09/21/cycle-analytics-feature-highlight/)
 
 
+[board]: issue_board.md#creating-a-new-list
 [ce-5986]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5986
 [ce-20975]: https://gitlab.com/gitlab-org/gitlab-ce/issues/20975
-[GitLab flow]: ../../workflow/gitlab_flow.md
-[permissions]: ../permissions.md
 [environment]: ../../ci/yaml/README.md#environment
-[board]: issue_board.md#creating-a-new-list
+[GitLab flow]: ../../workflow/gitlab_flow.md
 [idea to production]: https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab
 [issue closing pattern]: issues/automatic_issue_closing.md
+[permissions]: ../permissions.md
+[yml]: ../../ci/yaml/README.md