Separate the extra tests to run in a separate job
Recently we are seeing a lot of job timeout(bug#2004780)
and we see many tests taking time and also number of tests
increasing over time. This is to prepare the list of extra tests
(here extra tests means the tests which are covered by the other
API or scenario tests) which we do not need to run in every
integrated jobs. Instead, we can run them in a separete job(s).
Currently I am adding admin (except keystone) and negative tests
in the 'extra tests' list but we can add more tests here which
we think are covered in some other tests.
As negative tests are important for interop, adding those extra
tests coverage for stable branch job also but running them in
periodic run only.
Related-Bug: #2004780
Change-Id: Id02221df0d6180519751c63e890851bd59fdafa0
diff --git a/zuul.d/project.yaml b/zuul.d/project.yaml
index 126119b..4281d6b 100644
--- a/zuul.d/project.yaml
+++ b/zuul.d/project.yaml
@@ -26,6 +26,8 @@
- ^.gitignore$
- ^.gitreview$
- ^.mailmap$
+ - tempest-extra-tests:
+ irrelevant-files: *tempest-irrelevant-files
- tempest-full-ubuntu-focal:
irrelevant-files: *tempest-irrelevant-files
- glance-multistore-cinder-import:
@@ -63,6 +65,7 @@
- ^tools/tempest-integrated-gate-placement-exclude-list.txt
- ^tools/tempest-integrated-gate-storage-blacklist.txt
- ^tools/tempest-integrated-gate-storage-exclude-list.txt
+ - ^tools/tempest-extra-tests-list.txt
- ^tools/verify-ipv6-only-deployments.sh
- ^tools/with_venv.sh
# tools/ is not here since this relies on a script in tools/.
@@ -86,6 +89,7 @@
- ^tools/tempest-integrated-gate-placement-exclude-list.txt
- ^tools/tempest-integrated-gate-storage-blacklist.txt
- ^tools/tempest-integrated-gate-storage-exclude-list.txt
+ - ^tools/tempest-extra-tests-list.txt
- ^tools/tempest-plugin-sanity.sh
- ^tools/with_venv.sh
- ^.coveragerc$
@@ -131,6 +135,8 @@
irrelevant-files: *tempest-irrelevant-files
- tempest-full-py3:
irrelevant-files: *tempest-irrelevant-files
+ - tempest-extra-tests:
+ irrelevant-files: *tempest-irrelevant-files
- grenade:
irrelevant-files: *tempest-irrelevant-files
- tempest-ipv6-only:
@@ -177,6 +183,9 @@
- tempest-slow-zed
- tempest-slow-yoga
- tempest-slow-xena
+ - tempest-full-zed-extra-tests
+ - tempest-full-yoga-extra-tests
+ - tempest-full-xena-extra-tests
periodic:
jobs:
- tempest-all