| Attila Fazekas | 58d2330 | 2013-07-24 10:25:02 +0200 | [diff] [blame] | 1 | ============================ |
| Sean Dague | 7679b48 | 2013-05-24 08:49:44 -0400 | [diff] [blame] | 2 | Tempest Field Guide Overview |
| Attila Fazekas | 58d2330 | 2013-07-24 10:25:02 +0200 | [diff] [blame] | 3 | ============================ |
| Sean Dague | 31a57ad | 2013-05-07 15:24:05 -0400 | [diff] [blame] | 4 | |
| 5 | Tempest is designed to be useful for a large number of different |
| 6 | environments. This includes being useful for gating commits to |
| 7 | OpenStack core projects, being used to validate OpenStack cloud |
| 8 | implementations for both correctness, as well as a burn in tool for |
| 9 | OpenStack clouds. |
| 10 | |
| 11 | As such Tempest tests come in many flavors, each with their own rules |
| gaobin | af6d18a | 2018-12-09 19:59:12 +0800 | [diff] [blame] | 12 | and guidelines. Below is the overview of the Tempest repository structure |
| Sean Dague | 31a57ad | 2013-05-07 15:24:05 -0400 | [diff] [blame] | 13 | to make this clear. |
| 14 | |
| David Rabel | 0a83c71 | 2018-05-24 08:45:26 +0200 | [diff] [blame] | 15 | .. code-block:: console |
| Masayuki Igawa | 989f911 | 2017-11-14 16:29:35 +0900 | [diff] [blame] | 16 | |
| David Rabel | 0a83c71 | 2018-05-24 08:45:26 +0200 | [diff] [blame] | 17 | tempest/ |
| 18 | api/ - API tests |
| 19 | scenario/ - complex scenario tests |
| 20 | tests/ - unit tests for Tempest internals |
| Sean Dague | 31a57ad | 2013-05-07 15:24:05 -0400 | [diff] [blame] | 21 | |
| 22 | Each of these directories contains different types of tests. What |
| 23 | belongs in each directory, the rules and examples for good tests, are |
| 24 | documented in a README.rst file in the directory. |
| 25 | |
| Matthew Treinish | a9aef87 | 2014-07-25 16:03:08 -0400 | [diff] [blame] | 26 | :ref:`api_field_guide` |
| 27 | ---------------------- |
| Sean Dague | 31a57ad | 2013-05-07 15:24:05 -0400 | [diff] [blame] | 28 | |
| 29 | API tests are validation tests for the OpenStack API. They should not |
| Jordan Pittier | 74a56ab | 2017-04-26 16:46:20 +0200 | [diff] [blame] | 30 | use the existing Python clients for OpenStack, but should instead use |
| 31 | the Tempest implementations of clients. Having raw clients let us |
| guo yunxian | a221647 | 2016-08-01 16:34:43 +0800 | [diff] [blame] | 32 | pass invalid JSON to the APIs and see the results, something we could |
| 33 | not get with the native clients. |
| Sean Dague | 31a57ad | 2013-05-07 15:24:05 -0400 | [diff] [blame] | 34 | |
| 35 | When it makes sense, API testing should be moved closer to the |
| 36 | projects themselves, possibly as functional tests in their unit test |
| 37 | frameworks. |
| 38 | |
| Sean Dague | 31a57ad | 2013-05-07 15:24:05 -0400 | [diff] [blame] | 39 | |
| Matthew Treinish | a9aef87 | 2014-07-25 16:03:08 -0400 | [diff] [blame] | 40 | :ref:`scenario_field_guide` |
| 41 | --------------------------- |
| Sean Dague | 31a57ad | 2013-05-07 15:24:05 -0400 | [diff] [blame] | 42 | |
| 43 | Scenario tests are complex "through path" tests for OpenStack |
| 44 | functionality. They are typically a series of steps where complicated |
| 45 | state requiring multiple services is set up exercised, and torn down. |
| 46 | |
| Jordan Pittier | 74a56ab | 2017-04-26 16:46:20 +0200 | [diff] [blame] | 47 | Scenario tests should not use the existing Python clients for OpenStack, |
| 48 | but should instead use the Tempest implementations of clients. |
| Sean Dague | 31a57ad | 2013-05-07 15:24:05 -0400 | [diff] [blame] | 49 | |
| Sean Dague | 31a57ad | 2013-05-07 15:24:05 -0400 | [diff] [blame] | 50 | |
| Matthew Treinish | 817a808 | 2014-07-25 16:07:20 -0400 | [diff] [blame] | 51 | :ref:`unit_tests_field_guide` |
| 52 | ----------------------------- |
| 53 | |
| 54 | Unit tests are the self checks for Tempest. They provide functional |
| Jordan Pittier | 74a56ab | 2017-04-26 16:46:20 +0200 | [diff] [blame] | 55 | verification and regression checking for the internal components of Tempest. |
| 56 | They should be used to just verify that the individual pieces of Tempest are |
| Matthew Treinish | 817a808 | 2014-07-25 16:07:20 -0400 | [diff] [blame] | 57 | working as expected. |