Development Guide

Prerequisites

The OpenLab CI is built based on NodePool and Zuul tools of the OpenStack infrastructure. The CI jobs definition uses Ansible playbooks. Learn more about the tools first:

  • Zuul documentation is hosted on Zuul

  • NodePool documentation is hosted on NodePool

  • Ansible documentation is hosted on Ansible Docs

Testing workflow

Working on test request with OpenLab CI

The accepted test requests can be found in TODO list, developers can get start with following steps:

  1. Pick one test request you wish to work on, move it to IN PROGRESS list, edit it to add the development plan into comments

  2. The target project of the request should install the OpenLab-CI github APP, so that OpenLab CI system and target project can communicate with each other. See the details in connecting guideline

  3. Fork your own copy of the CI job repository openlab-zuul-jobs to your account, develop the test job and submit a PR(Pull Request) to openlab-zuul-jobs. In order to manage the test request well, please add change details and link the test request number in PR's commit message, the format like:

    Related-Bug: theopenlab/openlab#number

  4. OpenLab development team @theopenlab/dev will review the PR and give some suggestion, development core team @theopenlab/dev-core have permission to merge it if there is no issue, at the meanwhile, you should contact @theopenlab/dev-core to add the target project into main.yaml in zuul node if it is not in project list of zuul

  5. When your PR is merged into master of openlab-zuul-jobs, trigger to test the integration job by trigger comments, see the build status, click result FAILURE enter the build web page and then click the log url to get the log details if the job fail. Maybe you can find some problems of job, feel free to submit new pull request to openlab-zuul-jobs to fix job issue, link the issue number of related test request in PR's commit message as we do on step3

  6. Close the In Progress test request by yourself when the work completely, notify @theopenlab/governance and @theopenlab/docs to update website and documents in RP comments.

Working on test request with OpenLab physical machine

  1. If your test request need some physical machines, please contact @theopenlab/governance members in IRC channel or send them an email, they will help you to get OpenLab dedicated resources to launch testing

  2. When you can access the physical machines, move your test request to IN PROGRESS list, edit it to add the development plan into comments

  3. When the work is completely, close the In Progress test request by yourself, please commit the test result into comments of test request related issue. A blog also is a good choice to show more details of testing. Notify @theopenlab/governance and @theopenlab/docs to update website and documents in RP comments.

Working on feature and bug

  1. Features and bugs for OpenLab are tracked in issue list

  2. Perhaps the merged CI job will fail due to some reasons after it has been running for some time, or if you find any other enhancement points or problems of OpenLab, welcome to new an issue

  3. The development core team @theopenlab/dev-core will confirm the feature or bug, add them into TODO list

  4. You can assign one feature or bug from TODO list, move it to IN PROGRESS list. Feel free to pick up the issues with label need volunteer, these issues are not claimed by anyone yet

  5. Submit a PR(Pull Request) to openlab-zuul-jobs to implement feature or fix bug, please add change details and link the issue number in PR's commit message:

    Closes: theopenlab/openlab#{number}

    Related-Bug: theopenlab/openlab#{number}

  6. OpenLab development team @theopenlab/dev will review the PR and give some suggestion, development core team @theopenlab/dev-core have permission to merge it if there is no problem. When the PR is merged into openlab-zuul-jobs, you can test jobs by trigger comments

  7. Close the In Progress feature or bug by yourself when feature is implemented or bug is fixed.

Job naming notations

When we implement an integration test request, usually we need to add new job into openlab-zuul-jobs. To unify the job name format, we have the following naming notations:

{project}-{test type}-{backend}-{service}-{version}
  • The project usually is the name of the git repository which contains tests to run.

  • The test type is the type of tests to run, e.g. acceptance test, integration test, functional test, unit test.

  • The backend is the test environment provider, include: deploying tools, public cloud, private cloud,

    e.g. devstack, kubeadm, minikube, VEXXHOST, optionally, default is devstack.

  • The service is the specific backend service which this job will run against, optionally, default is core services.

  • The version is the specific backend version which this job will run against, optionally, default is master.

For an example, the job definition about running the specific Trove related acceptance tests of terraform-provider-openstack repository can be named as:

terraform-provider-openstack-acceptance-test-trove

And running Spark integration test against Kubernetes 1.13.0 that is deployed by minikube:

spark-integration-test-minikube-k8s-1.13.0

Periodic job and pipeline schedule

OpenLab manage lots of periodic jobs for many projects, these jobs will be triggered by OpenLab CI to execute testing and collect result in everyday. For decreasing OpenLab resource usage, we should split all of periodic jobs into each pipeline as evenly as possible, and avoiding concurrent jobs with same test target affect each other, like: public cloud tenant, for example, lots of periodic jobs run to create instance in same public cloud tenant, that will hit tenant quota limit. So there are three basic principles to help you to decide which pipeline when you want to add new periodic job:

  1. Split the jobs have same backend(public cloud account), like: VEXXHOST, OpenTelekomCloud or HuaweiCloud into different pipeline to avoid influencing each other. Job backend is local deployed OpenStack, Kubernetes and other, that is OK in same pipeline.

  2. Split all of periodic jobs into each pipeline as evenly as possible.

  3. Choose the trigger time you like.

Current pipeline and job mapping: