Sean Dague | 7679b48 | 2013-05-24 08:49:44 -0400 | [diff] [blame] | 1 | Tempest Field Guide to Stress Tests |
Attila Fazekas | 23fdf1d | 2013-06-09 16:35:23 +0200 | [diff] [blame] | 2 | =================================== |
David Kranz | b9d9750 | 2013-05-01 15:55:04 -0400 | [diff] [blame] | 3 | |
| 4 | Nova is a distributed, asynchronous system that is prone to race condition |
| 5 | bugs. These bugs will not be easily found during |
| 6 | functional testing but will be encountered by users in large deployments in a |
| 7 | way that is hard to debug. The stress test tries to cause these bugs to happen |
| 8 | in a more controlled environment. |
| 9 | |
| 10 | |
| 11 | Environment |
Attila Fazekas | 23fdf1d | 2013-06-09 16:35:23 +0200 | [diff] [blame] | 12 | ----------- |
Sean Dague | 7679b48 | 2013-05-24 08:49:44 -0400 | [diff] [blame] | 13 | This particular framework assumes your working Nova cluster understands Nova |
David Kranz | b9d9750 | 2013-05-01 15:55:04 -0400 | [diff] [blame] | 14 | API 2.0. The stress tests can read the logs from the cluster. To enable this |
| 15 | you have to provide the hostname to call 'nova-manage' and |
| 16 | the private key and user name for ssh to the cluster in the |
| 17 | [stress] section of tempest.conf. You also need to provide the |
| 18 | location of the log files: |
| 19 | |
| 20 | target_logfiles = "regexp to all log files to be checked for errors" |
| 21 | target_private_key_path = "private ssh key for controller and log file nodes" |
| 22 | target_ssh_user = "username for controller and log file nodes" |
| 23 | target_controller = "hostname or ip of controller node (for nova-manage) |
| 24 | log_check_interval = "time between checking logs for errors (default 60s)" |
| 25 | |
Marc Koderer | b714de5 | 2013-08-08 09:21:46 +0200 | [diff] [blame] | 26 | To activate logging on your console please make sure that you activate `use_stderr` |
| 27 | in tempest.conf or use the default `logging.conf.sample` file. |
David Kranz | b9d9750 | 2013-05-01 15:55:04 -0400 | [diff] [blame] | 28 | |
Marc Koderer | 31fe483 | 2013-11-06 17:02:03 +0100 | [diff] [blame] | 29 | Running default stress test set |
| 30 | ------------------------------- |
| 31 | |
| 32 | The stress test framework can automatically discover test inside the tempest |
| 33 | test suite. All test flag with the `@stresstest` decorator will be executed. |
| 34 | In order to use this discovery you have to be in the tempest root directory |
| 35 | and execute the following: |
| 36 | |
| 37 | tempest/stress/run_stress.py -a -d 30 |
| 38 | |
David Kranz | b9d9750 | 2013-05-01 15:55:04 -0400 | [diff] [blame] | 39 | Running the sample test |
| 40 | ----------------------- |
| 41 | |
| 42 | To test installation, do the following (from the tempest/stress directory): |
| 43 | |
DennyZhang | d7a51b4 | 2013-09-23 12:10:12 -0500 | [diff] [blame] | 44 | ./run_stress.py -t etc/server-create-destroy-test.json -d 30 |
David Kranz | b9d9750 | 2013-05-01 15:55:04 -0400 | [diff] [blame] | 45 | |
| 46 | This sample test tries to create a few VMs and kill a few VMs. |
| 47 | |
| 48 | |
| 49 | Additional Tools |
| 50 | ---------------- |
| 51 | |
| 52 | Sometimes the tests don't finish, or there are failures. In these |
| 53 | cases, you may want to clean out the nova cluster. We have provided |
| 54 | some scripts to do this in the ``tools`` subdirectory. |
| 55 | You can use the following script to destroy any keypairs, |
| 56 | floating ips, and servers: |
| 57 | |
| 58 | tempest/stress/tools/cleanup.py |