| Martin Kopec | d76178e | 2024-01-16 20:43:56 +0100 | [diff] [blame] | 1 | .. _serial_tests_guide: |
| 2 | |
| 3 | Tempest Field Guide to Serial tests |
| 4 | =================================== |
| 5 | |
| 6 | |
| 7 | What are these tests? |
| 8 | --------------------- |
| 9 | |
| 10 | Tempest can run tests serially as well as in parallel, depending on the |
| 11 | configuration that is fully up to the user. However, sometimes you need to |
| 12 | make sure that tests are not interfering with each other via OpenStack |
| 13 | resources with the other tests running in parallel. Tempest creates separate |
| 14 | projects for each test class to separate project based resources between test |
| 15 | cases. |
| 16 | |
| 17 | If your tests use resources outside of projects, e.g. host aggregates then |
| 18 | you might need to explicitly separate interfering test cases. If you only need |
| 19 | to separate a small set of test cases from each other then you can use the |
| 20 | ``LockFixture``. |
| 21 | |
| 22 | However, in some cases, a small set of tests needs to be run serially. For |
| 23 | example, some of the host aggregate and availability zone testing needs |
| 24 | compute nodes without any running nova server to be able to move compute hosts |
| 25 | between availability zones. But many tempest tests start one or more nova |
| 26 | servers. |
| 27 | |
| 28 | |
| 29 | Why are these tests in Tempest? |
| 30 | ------------------------------- |
| 31 | |
| 32 | This is one of Tempest's core purposes, testing the integration between |
| 33 | projects. |
| 34 | |
| 35 | |
| 36 | Scope of these tests |
| 37 | -------------------- |
| 38 | |
| 39 | The tests should always use the Tempest implementation of the OpenStack API, |
| 40 | as we want to ensure that bugs aren't hidden by the official clients. |
| 41 | |
| 42 | Tests should be tagged with which services they exercise, as |
| 43 | determined by which client libraries are used directly by the test. |
| 44 | |
| 45 | |
| 46 | Example of a good test |
| 47 | ---------------------- |
| 48 | |
| 49 | While we are looking for interaction of 2 or more services, be specific in |
| 50 | your interactions. A giant "this is my data center" smoke test is hard to |
| 51 | debug when it goes wrong. |
| 52 | |
| 53 | The tests that need to be run serially need to be marked with the |
| 54 | ``@serial`` class decorator. This will make sure that even if tempest is |
| 55 | configured to run the tests in parallel, these tests will always be executed |
| 56 | separately from the rest of the test cases. |
| 57 | |
| 58 | Please note that due to test ordering optimization reasons test cases marked |
| 59 | for ``@serial`` execution need to be put under ``tempest/serial_tests`` |
| 60 | directory. This will ensure that the serial tests will block the parallel tests |
| 61 | in the least amount of time. |