BigW Consortium Gitlab

Commit 73f27db2 by Rémy Coutable

Update CI parallelization description

Signed-off-by: 's avatarRémy Coutable <remy@rymai.me>
parent da7353d3
......@@ -493,24 +493,24 @@ Here are some things to keep in mind regarding test performance:
Our current CI parallelization setup is as follows:
1. The `knapsack` job in the prepare stage that is supposed to ensure we have a
`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file:
1. The `retrieve-tests-metadata` job in the `prepare` stage ensures that we have
a `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file:
- The `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file is fetched
from S3, if it's not here we initialize the file with `{}`.
1. Each `rspec x y` job are run with `knapsack rspec` and should have an evenly
distributed share of tests:
1. Each `rspec-pg x y`/`rspec-mysql x y` job is run with `knapsack rspec` and
should have an evenly distributed share of tests:
- It works because the jobs have access to the
`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` since the "artifacts
from all previous stages are passed by default". [^1]
- the jobs set their own report path to
- The jobs set their own report path to
`KNAPSACK_REPORT_PATH=knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json`.
- if knapsack is doing its job, test files that are run should be listed under
- If knapsack is doing its job, test files that are run should be listed under
`Report specs`, not under `Leftover specs`.
1. The `update-knapsack` job takes all the
1. The `update-tests-metadata` job takes all the
`knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json`
files from the `rspec x y` jobs and merge them all together into a single
`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file that is then
uploaded to S3.
files from the `rspec-pg x y`/`rspec-mysql x y`jobs and merge them all together
into a single `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file that
is then uploaded to S3.
After that, the next pipeline will use the up-to-date
`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file. The same strategy
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment