Merge "Add scenario test of instance/volume snapshot"
diff --git a/.testr.conf b/.testr.conf
index a0262d8..9a72d29 100644
--- a/.testr.conf
+++ b/.testr.conf
@@ -2,3 +2,4 @@
test_command=${PYTHON:-python} -m subunit.run discover -t ./ ./tempest $LISTOPT $IDOPTION
test_id_option=--load-list $IDFILE
test_list_option=--list
+group_regex=([^\.]*\.)*
diff --git a/HACKING.rst b/HACKING.rst
index d69f935..1eb2d4f 100644
--- a/HACKING.rst
+++ b/HACKING.rst
@@ -1,6 +1,25 @@
Tempest Coding Guide
====================
+- Step 1: Read the OpenStack Style Commandments
+ https://github.com/openstack-dev/hacking/blob/master/HACKING.rst
+- Step 2: Read on
+
+Tempest Specific Commandments
+------------------------------
+
+[T101] If a test is broken because of a bug it is appropriate to skip the test until
+bug has been fixed. However, the skip message should be formatted so that
+Tempest's skip tracking tool can watch the bug status. The skip message should
+contain the string 'Bug' immediately followed by a space. Then the bug number
+should be included in the message '#' in front of the number.
+
+Example::
+
+ @testtools.skip("Skipped until the Bug #980688 is resolved")
+
+- [T102] Cannot import OpenStack python clients in tempest/api tests
+- [T103] tempest/tests is deprecated
Test Data/Configuration
-----------------------
@@ -10,157 +29,10 @@
- Use configuration files for values that will vary by environment
-General
--------
-- Put two newlines between top-level code (funcs, classes, etc)
-- Put one newline between methods in classes and anywhere else
-- Long lines should be wrapped in parentheses
- in preference to using a backslash for line continuation.
-- Do not write "except:", use "except Exception:" at the very least
-- Include your name with TODOs as in "#TODO(termie)"
-- Do not name anything the same name as a built-in or reserved word Example::
-
- def list():
- return [1, 2, 3]
-
- mylist = list() # BAD, shadows `list` built-in
-
- class Foo(object):
- def list(self):
- return [1, 2, 3]
-
- mylist = Foo().list() # OKAY, does not shadow built-in
-
-Imports
--------
-- Do not import objects, only modules (*)
-- Do not import more than one module per line (*)
-- Do not make relative imports
-- Order your imports by the full module path
-- Organize your imports according to the following template
-
-Example::
-
- # vim: tabstop=4 shiftwidth=4 softtabstop=4
- {{stdlib imports in human alphabetical order}}
- \n
- {{third-party lib imports in human alphabetical order}}
- \n
- {{tempest imports in human alphabetical order}}
- \n
- \n
- {{begin your code}}
-
-
-Human Alphabetical Order Examples
----------------------------------
-Example::
-
- import httplib
- import logging
- import random
- import StringIO
- import testtools
- import time
-
- import eventlet
- import webob.exc
-
- import tempest.config
- from tempest.services.compute.json.limits_client import LimitsClientJSON
- from tempest.services.compute.xml.limits_client import LimitsClientXML
- from tempest.services.volume.volumes_client import VolumesClientJSON
- import tempest.test
-
-
-Docstrings
-----------
-Example::
-
- """A one line docstring looks like this and ends in a period."""
-
-
- """A multi line docstring has a one-line summary, less than 80 characters.
-
- Then a new paragraph after a newline that explains in more detail any
- general information about the function, class or method. Example usages
- are also great to have here if it is a complex class for function.
-
- When writing the docstring for a class, an extra line should be placed
- after the closing quotations. For more in-depth explanations for these
- decisions see http://www.python.org/dev/peps/pep-0257/
-
- If you are going to describe parameters and return values, use Sphinx, the
- appropriate syntax is as follows.
-
- :param foo: the foo parameter
- :param bar: the bar parameter
- :returns: return_type -- description of the return value
- :returns: description of the return value
- :raises: AttributeError, KeyError
- """
-
-
-Dictionaries/Lists
-------------------
-If a dictionary (dict) or list object is longer than 80 characters, its items
-should be split with newlines. Embedded iterables should have their items
-indented. Additionally, the last item in the dictionary should have a trailing
-comma. This increases readability and simplifies future diffs.
-
-Example::
-
- my_dictionary = {
- "image": {
- "name": "Just a Snapshot",
- "size": 2749573,
- "properties": {
- "user_id": 12,
- "arch": "x86_64",
- },
- "things": [
- "thing_one",
- "thing_two",
- ],
- "status": "ACTIVE",
- },
- }
-
-
-Calling Methods
----------------
-Calls to methods 80 characters or longer should format each argument with
-newlines. This is not a requirement, but a guideline::
-
- unnecessarily_long_function_name('string one',
- 'string two',
- kwarg1=constants.ACTIVE,
- kwarg2=['a', 'b', 'c'])
-
-
-Rather than constructing parameters inline, it is better to break things up::
-
- list_of_strings = [
- 'what_a_long_string',
- 'not as long',
- ]
-
- dict_of_numbers = {
- 'one': 1,
- 'two': 2,
- 'twenty four': 24,
- }
-
- object_one.call_a_method('string three',
- 'string four',
- kwarg1=list_of_strings,
- kwarg2=dict_of_numbers)
-
-
Exception Handling
------------------
According to the ``The Zen of Python`` the
- ``Errors should never pass silently.``
+``Errors should never pass silently.``
Tempest usually runs in special environment (jenkins gate jobs), in every
error or failure situation we should provide as much error related
information as possible, because we usually do not have the chance to
@@ -185,6 +57,10 @@
exception at least logged. When the exception is logged you usually need
to ``raise`` the same or a different exception anyway.
+Use of ``self.addCleanup`` is often a good way to avoid having to catch
+exceptions and still ensure resources are correctly cleaned up if the
+test fails part way through.
+
Use the ``self.assert*`` methods provided by the unit test framework
the signal failures early.
@@ -202,72 +78,10 @@
This and the service logs are your only guide to find the root cause of flaky
issue.
-
-Test Skips
+Guidelines
----------
-If a test is broken because of a bug it is appropriate to skip the test until
-bug has been fixed. However, the skip message should be formatted so that
-Tempest's skip tracking tool can watch the bug status. The skip message should
-contain the string 'Bug' immediately followed by a space. Then the bug number
-should be included in the message '#' in front of the number.
-
-Example::
-
- @testtools.skip("Skipped until the Bug #980688 is resolved")
-
-
-openstack-common
-----------------
-
-A number of modules from openstack-common are imported into the project.
-
-These modules are "incubating" in openstack-common and are kept in sync
-with the help of openstack-common's update.py script. See:
-
- http://wiki.openstack.org/CommonLibrary#Incubation
-
-The copy of the code should never be directly modified here. Please
-always update openstack-common first and then run the script to copy
-the changes across.
-
-
-OpenStack Trademark
--------------------
-
-OpenStack is a registered trademark of the OpenStack Foundation, and uses the
-following capitalization:
-
- OpenStack
-
-
-Commit Messages
----------------
-Using a common format for commit messages will help keep our git history
-readable. Follow these guidelines:
-
- First, provide a brief summary (it is recommended to keep the commit title
- under 50 chars).
-
- The first line of the commit message should provide an accurate
- description of the change, not just a reference to a bug or
- blueprint. It must be followed by a single blank line.
-
- If the change relates to a specific driver (libvirt, xenapi, qpid, etc...),
- begin the first line of the commit message with the driver name, lowercased,
- followed by a colon.
-
- Following your brief summary, provide a more detailed description of
- the patch, manually wrapping the text at 72 characters. This
- description should provide enough detail that one does not have to
- refer to external resources to determine its high-level functionality.
-
- Once you use 'git review', two lines will be appended to the commit
- message: a blank line followed by a 'Change-Id'. This is important
- to correlate this commit with a specific review in Gerrit, and it
- should not be modified.
-
-For further information on constructing high quality commit messages,
-and how to split up commits into a series of changes, consult the
-project wiki:
-
- http://wiki.openstack.org/GitCommitMessages
+- Do not submit changesets with only testcases which are skipped as
+ they will not be merged.
+- Consistently check the status code of responses in testcases. The
+ earlier a problem is detected the easier it is to debug, especially
+ where there is complicated setup required.
diff --git a/README.rst b/README.rst
index da0f5f3..f18628a 100644
--- a/README.rst
+++ b/README.rst
@@ -1,5 +1,3 @@
-::
-
Tempest - The OpenStack Integration Test Suite
==============================================
@@ -37,9 +35,11 @@
Tempest is not tied to any single test runner, but Nose been the most commonly
used tool. After setting up your configuration file, you can execute
the set of Tempest tests by using ``nosetests`` ::
+
$> nosetests tempest
To run one single test ::
+
$> nosetests -sv tempest.api.compute.servers.test_server_actions.py:
ServerActionsTestJSON.test_rebuild_nonexistent_server
diff --git a/etc/tempest.conf.sample b/etc/tempest.conf.sample
index f5e51cd..033bc82 100644
--- a/etc/tempest.conf.sample
+++ b/etc/tempest.conf.sample
@@ -91,6 +91,9 @@
# IP version of the address used for SSH
ip_version_for_ssh = 4
+# Number of seconds to wait to ping to an instance
+ping_timeout = 60
+
# Number of seconds to wait to authenticate to an instance
ssh_timeout = 300
@@ -209,8 +212,6 @@
# for each tenant to have their own router.
public_router_id = {$PUBLIC_ROUTER_ID}
-# Whether or not neutron is expected to be available
-neutron_available = false
[volume]
# This section contains the configuration options used when executing tests
@@ -308,9 +309,6 @@
# tests spawn full VMs, which could be slow if the test is already in a VM.
build_timeout = 300
-# Whether or not Heat is expected to be available
-heat_available = false
-
# Instance type for tests. Needs to be big enough for a
# full OS plus the test workload
instance_type = m1.micro
@@ -323,6 +321,13 @@
# any key, which will generate a keypair for each test class
#keypair_name = heat_key
+[dashboard]
+# URL where to find the dashboard home page
+dashboard_url = 'http://localhost/'
+
+# URL where to submit the login form
+login_url = 'http://localhost/auth/login/'
+
[scenario]
# Directory containing image files
img_dir = /opt/stack/new/devstack/files/images/cirros-0.3.1-x86_64-uec
@@ -348,3 +353,19 @@
enabled = True
# directory where python client binaries are located
cli_dir = /usr/local/bin
+
+[service_available]
+# Whether or not cinder is expected to be available
+cinder = True
+# Whether or not neutron is expected to be available
+neutron = false
+# Whether or not glance is expected to be available
+glance = True
+# Whether or not swift is expected to be available
+swift = True
+# Whether or not nova is expected to be available
+nova = True
+# Whether or not Heat is expected to be available
+heat = false
+# Whether or not horizon is expected to be available
+horizon = True
diff --git a/run_tests.sh b/run_tests.sh
index 366564e..a645b22 100755
--- a/run_tests.sh
+++ b/run_tests.sh
@@ -11,6 +11,7 @@
echo " -u, --update Update the virtual environment with any newer package versions"
echo " -s, --smoke Only run smoke tests"
echo " -w, --whitebox Only run whitebox tests"
+ echo " -t, --with-testr Run using testr instead of nose"
echo " -c, --nova-coverage Enable Nova coverage collection"
echo " -C, --config Config file location"
echo " -p, --pep8 Just run pep8"
@@ -26,6 +27,7 @@
just_pep8=0
venv=.venv
with_venv=tools/with_venv.sh
+with_testr=0
always_venv=0
never_venv=0
no_site_packages=0
@@ -37,7 +39,7 @@
logging=0
logging_config=etc/logging.conf
-if ! options=$(getopt -o VNnfuswcphdSC:lL: -l virtual-env,no-virtual-env,no-site-packages,force,update,smoke,whitebox,nova-coverage,pep8,help,debug,stdout,config:,logging,logging-config: -- "$@")
+if ! options=$(getopt -o VNnfuswtcphdSC:lL: -l virtual-env,no-virtual-env,no-site-packages,force,update,smoke,whitebox,with-testr,nova-coverage,pep8,help,debug,stdout,config:,logging,logging-config: -- "$@")
then
# parse error
usage
@@ -60,6 +62,7 @@
-p|--pep8) let just_pep8=1;;
-s|--smoke) noseargs="$noseargs --attr=type=smoke";;
-w|--whitebox) noseargs="$noseargs --attr=type=whitebox";;
+ -t|--with-testr) with_testr=1;;
-S|--stdout) noseargs="$noseargs -s";;
-l|--logging) logging=1;;
-L|--logging-config) logging_config=$2; shift;;
@@ -105,8 +108,20 @@
noseargs="$noseargs tempest"
fi
+function testr_init {
+ if [ ! -d .testrepository ]; then
+ ${wrapper} testr init
+ fi
+}
+
function run_tests {
- ${wrapper} $NOSETESTS
+ if [ $with_testr -eq 1 ]; then
+ testr_init
+ ${wrapper} find . -type f -name "*.pyc" -delete
+ ${wrapper} testr run --parallel --subunit $noseargs | ${wrapper} subunit-2to1 | ${wrapper} tools/colorizer.py
+ else
+ ${wrapper} $NOSETESTS
+ fi
}
function run_pep8 {
diff --git a/tempest/README.rst b/tempest/README.rst
index 8f07a07..33021c8 100644
--- a/tempest/README.rst
+++ b/tempest/README.rst
@@ -1,6 +1,6 @@
-============
+============================
Tempest Field Guide Overview
-============
+============================
Tempest is designed to be useful for a large number of different
environments. This includes being useful for gating commits to
@@ -26,7 +26,7 @@
api
-------------
+---
API tests are validation tests for the OpenStack API. They should not
use the existing python clients for OpenStack, but should instead use
@@ -41,7 +41,7 @@
cli
-------------
+---
CLI tests use the openstack CLI to interact with the OpenStack
cloud. CLI testing in unit tests is somewhat difficult because unlike
@@ -51,7 +51,7 @@
scenario
-------------
+--------
Scenario tests are complex "through path" tests for OpenStack
functionality. They are typically a series of steps where complicated
@@ -61,7 +61,7 @@
stress
------------
+------
Stress tests are designed to stress an OpenStack environment by
running a high workload against it and seeing what breaks. Tools may
@@ -72,7 +72,7 @@
thirdparty
-------------
+----------
Many openstack components include 3rdparty API support. It is
completely legitimate for Tempest to include tests of 3rdparty APIs,
@@ -81,7 +81,7 @@
whitebox
-----------
+--------
Whitebox tests are tests which require access to the database of the
target OpenStack machine to verify internal state after operations
diff --git a/tempest/api/compute/admin/test_fixed_ips.py b/tempest/api/compute/admin/test_fixed_ips.py
index 2eaf3b0..8b96370 100644
--- a/tempest/api/compute/admin/test_fixed_ips.py
+++ b/tempest/api/compute/admin/test_fixed_ips.py
@@ -56,7 +56,7 @@
CONF = config.TempestConfig()
- @testtools.skipIf(CONF.network.neutron_available, "This feature is not" +
+ @testtools.skipIf(CONF.service_available.neutron, "This feature is not" +
"implemented by Neutron. See bug: #1194569")
@attr(type='gate')
def test_list_fixed_ip_details(self):
diff --git a/tempest/api/compute/admin/test_services.py b/tempest/api/compute/admin/test_services.py
index 78dac21..3b3c6ce 100644
--- a/tempest/api/compute/admin/test_services.py
+++ b/tempest/api/compute/admin/test_services.py
@@ -1,6 +1,7 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 NEC Corporation
+# Copyright 2013 IBM Corp.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
@@ -36,17 +37,91 @@
@attr(type='gate')
def test_list_services(self):
- # List Compute services
resp, services = self.client.list_services()
self.assertEqual(200, resp.status)
- self.assertTrue(len(services) >= 2)
+ self.assertNotEqual(0, len(services))
@attr(type=['negative', 'gate'])
def test_list_services_with_non_admin_user(self):
- # List Compute service with non admin user
self.assertRaises(exceptions.Unauthorized,
self.non_admin_client.list_services)
+ @attr(type='gate')
+ def test_get_service_by_service_binary_name(self):
+ binary_name = 'nova-compute'
+ params = {'binary': binary_name}
+ resp, services = self.client.list_services(params)
+ self.assertEqual(200, resp.status)
+ self.assertNotEqual(0, len(services))
+ for service in services:
+ self.assertEqual(binary_name, service['binary'])
+
+ @attr(type='gate')
+ def test_get_service_by_host_name(self):
+ resp, services = self.client.list_services()
+ host_name = services[0]['host']
+ services_on_host = [service for service in services if
+ service['host'] == host_name]
+ params = {'host': host_name}
+ resp, services = self.client.list_services(params)
+ self.assertEqual(services_on_host, services)
+
+ @attr(type=['negative', 'gate'])
+ def test_get_service_by_invalid_params(self):
+ #return all services if send the request with invalid parameter
+ resp, services = self.client.list_services()
+ params = {'xxx': 'nova-compute'}
+ resp, services_xxx = self.client.list_services(params)
+ self.assertEqual(200, resp.status)
+ self.assertEqual(len(services), len(services_xxx))
+
+ @attr(type='gate')
+ def test_get_service_by_service_and_host_name(self):
+ resp, services = self.client.list_services()
+ host_name = services[0]['host']
+ binary_name = services[0]['binary']
+ params = {'host': host_name, 'binary': binary_name}
+ resp, services = self.client.list_services(params)
+ self.assertEqual(200, resp.status)
+ self.assertEqual(1, len(services))
+ self.assertEqual(host_name, services[0]['host'])
+ self.assertEqual(binary_name, services[0]['binary'])
+
+ @attr(type=['negative', 'gate'])
+ def test_get_service_by_invalid_service_and_valid_host(self):
+ resp, services = self.client.list_services()
+ host_name = services[0]['host']
+ params = {'host': host_name, 'binary': 'xxx'}
+ resp, services = self.client.list_services(params)
+ self.assertEqual(200, resp.status)
+ self.assertEqual(0, len(services))
+
+ @attr(type=['negative', 'gate'])
+ def test_get_service_with_valid_service_and_invalid_host(self):
+ resp, services = self.client.list_services()
+ binary_name = services[0]['binary']
+ params = {'host': 'xxx', 'binary': binary_name}
+ resp, services = self.client.list_services(params)
+ self.assertEqual(200, resp.status)
+ self.assertEqual(0, len(services))
+
+ @attr(type='gate')
+ def test_service_enable_disable(self):
+ resp, services = self.client.list_services()
+ host_name = services[0]['host']
+ binary_name = services[0]['binary']
+
+ resp, service = self.client.disable_service(host_name, binary_name)
+ self.assertEqual(200, resp.status)
+ params = {'host': host_name, 'binary': binary_name}
+ resp, services = self.client.list_services(params)
+ self.assertEqual('disabled', services[0]['status'])
+
+ resp, service = self.client.enable_service(host_name, binary_name)
+ self.assertEqual(200, resp.status)
+ resp, services = self.client.list_services(params)
+ self.assertEqual('enabled', services[0]['status'])
+
class ServicesAdminTestXML(ServicesAdminTestJSON):
_interface = 'xml'
diff --git a/tempest/api/compute/base.py b/tempest/api/compute/base.py
index abc5899..8ba074e 100644
--- a/tempest/api/compute/base.py
+++ b/tempest/api/compute/base.py
@@ -22,7 +22,6 @@
from tempest.common import log as logging
from tempest.common.utils.data_utils import parse_image_id
from tempest.common.utils.data_utils import rand_name
-from tempest import exceptions
import tempest.test
@@ -36,6 +35,9 @@
@classmethod
def setUpClass(cls):
+ if not cls.config.service_available.nova:
+ skip_msg = ("%s skipped as nova is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
cls.isolated_creds = []
if cls.config.compute.allow_tenant_isolation:
@@ -79,89 +81,6 @@
cls.servers_client_v3_auth = os.servers_client_v3_auth
@classmethod
- def _get_identity_admin_client(cls):
- """
- Returns an instance of the Identity Admin API client
- """
- os = clients.AdminManager(interface=cls._interface)
- admin_client = os.identity_client
- return admin_client
-
- @classmethod
- def _get_client_args(cls):
-
- return (
- cls.config,
- cls.config.identity.admin_username,
- cls.config.identity.admin_password,
- cls.config.identity.uri
- )
-
- @classmethod
- def _get_isolated_creds(cls):
- """
- Creates a new set of user/tenant/password credentials for a
- **regular** user of the Compute API so that a test case can
- operate in an isolated tenant container.
- """
- admin_client = cls._get_identity_admin_client()
- password = "pass"
-
- while True:
- try:
- rand_name_root = rand_name(cls.__name__)
- if cls.isolated_creds:
- # Main user already created. Create the alt one...
- rand_name_root += '-alt'
- tenant_name = rand_name_root + "-tenant"
- tenant_desc = tenant_name + "-desc"
-
- resp, tenant = admin_client.create_tenant(
- name=tenant_name, description=tenant_desc)
- break
- except exceptions.Duplicate:
- if cls.config.compute.allow_tenant_reuse:
- tenant = admin_client.get_tenant_by_name(tenant_name)
- LOG.info('Re-using existing tenant %s', tenant)
- break
-
- while True:
- try:
- rand_name_root = rand_name(cls.__name__)
- if cls.isolated_creds:
- # Main user already created. Create the alt one...
- rand_name_root += '-alt'
- username = rand_name_root + "-user"
- email = rand_name_root + "@example.com"
- resp, user = admin_client.create_user(username,
- password,
- tenant['id'],
- email)
- break
- except exceptions.Duplicate:
- if cls.config.compute.allow_tenant_reuse:
- user = admin_client.get_user_by_username(tenant['id'],
- username)
- LOG.info('Re-using existing user %s', user)
- break
- # Store the complete creds (including UUID ids...) for later
- # but return just the username, tenant_name, password tuple
- # that the various clients will use.
- cls.isolated_creds.append((user, tenant))
-
- return username, tenant_name, password
-
- @classmethod
- def clear_isolated_creds(cls):
- if not cls.isolated_creds:
- return
- admin_client = cls._get_identity_admin_client()
-
- for user, tenant in cls.isolated_creds:
- admin_client.delete_user(user['id'])
- admin_client.delete_tenant(tenant['id'])
-
- @classmethod
def clear_servers(cls):
for server in cls.servers:
try:
@@ -189,7 +108,7 @@
def tearDownClass(cls):
cls.clear_images()
cls.clear_servers()
- cls.clear_isolated_creds()
+ cls._clear_isolated_creds()
@classmethod
def create_server(cls, **kwargs):
@@ -263,10 +182,16 @@
admin_username = cls.config.compute_admin.username
admin_password = cls.config.compute_admin.password
admin_tenant = cls.config.compute_admin.tenant_name
-
if not (admin_username and admin_password and admin_tenant):
msg = ("Missing Compute Admin API credentials "
"in configuration.")
raise cls.skipException(msg)
-
- cls.os_adm = clients.ComputeAdminManager(interface=cls._interface)
+ if cls.config.compute.allow_tenant_isolation:
+ creds = cls._get_isolated_creds(admin=True)
+ admin_username, admin_tenant_name, admin_password = creds
+ cls.os_adm = clients.Manager(username=admin_username,
+ password=admin_password,
+ tenant_name=admin_tenant_name,
+ interface=cls._interface)
+ else:
+ cls.os_adm = clients.ComputeAdminManager(interface=cls._interface)
diff --git a/tempest/api/compute/images/test_image_metadata.py b/tempest/api/compute/images/test_image_metadata.py
index 7b8e1cc..52239cd 100644
--- a/tempest/api/compute/images/test_image_metadata.py
+++ b/tempest/api/compute/images/test_image_metadata.py
@@ -27,6 +27,10 @@
@classmethod
def setUpClass(cls):
super(ImagesMetadataTestJSON, cls).setUpClass()
+ if not cls.config.service_available.glance:
+ skip_msg = ("%s skipped as glance is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
+
cls.servers_client = cls.servers_client
cls.client = cls.images_client
diff --git a/tempest/api/compute/images/test_images.py b/tempest/api/compute/images/test_images.py
index a74bb68..4f9364b 100644
--- a/tempest/api/compute/images/test_images.py
+++ b/tempest/api/compute/images/test_images.py
@@ -1,7 +1,6 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2012 OpenStack, LLC
-# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
@@ -30,6 +29,9 @@
@classmethod
def setUpClass(cls):
super(ImagesTestJSON, cls).setUpClass()
+ if not cls.config.service_available.glance:
+ skip_msg = ("%s skipped as glance is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
cls.client = cls.images_client
cls.servers_client = cls.servers_client
@@ -89,24 +91,29 @@
'!@#$%^&*()', name, meta)
@attr(type=['negative', 'gate'])
- def test_create_image_when_server_is_terminating(self):
- # Return an error when creating image of server that is terminating
+ def test_create_image_from_stopped_server(self):
resp, server = self.create_server(wait_until='ACTIVE')
- self.servers_client.delete_server(server['id'])
-
+ self.servers_client.stop(server['id'])
+ self.servers_client.wait_for_server_status(server['id'],
+ 'SHUTOFF')
+ self.addCleanup(self.servers_client.delete_server, server['id'])
snapshot_name = rand_name('test-snap-')
- self.assertRaises(exceptions.Duplicate, self.client.create_image,
- server['id'], snapshot_name)
+ resp, image = self.create_image_from_server(server['id'],
+ name=snapshot_name,
+ wait_until='ACTIVE')
+ self.addCleanup(self.client.delete_image, image['id'])
+ self.assertEqual(snapshot_name, image['name'])
- @attr(type=['negative', 'gate'])
- def test_create_image_when_server_is_rebooting(self):
- # Return error when creating an image of server that is rebooting
+ @attr(type='gate')
+ def test_delete_saving_image(self):
+ snapshot_name = rand_name('test-snap-')
resp, server = self.create_server(wait_until='ACTIVE')
- self.servers_client.reboot(server['id'], 'HARD')
-
- snapshot_name = rand_name('test-snap-')
- self.assertRaises(exceptions.Duplicate, self.client.create_image,
- server['id'], snapshot_name)
+ self.addCleanup(self.servers_client.delete_server, server['id'])
+ resp, image = self.create_image_from_server(server['id'],
+ name=snapshot_name,
+ wait_until='SAVING')
+ resp, body = self.client.delete_image(image['id'])
+ self.assertEqual('204', resp['status'])
@attr(type=['negative', 'gate'])
def test_create_image_specify_uuid_35_characters_or_less(self):
diff --git a/tempest/api/compute/images/test_images_oneserver.py b/tempest/api/compute/images/test_images_oneserver.py
index 7740cfc..64f1854 100644
--- a/tempest/api/compute/images/test_images_oneserver.py
+++ b/tempest/api/compute/images/test_images_oneserver.py
@@ -40,6 +40,9 @@
def setUpClass(cls):
super(ImagesOneServerTestJSON, cls).setUpClass()
cls.client = cls.images_client
+ if not cls.config.service_available.glance:
+ skip_msg = ("%s skipped as glance is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
try:
resp, cls.server = cls.create_server(wait_until='ACTIVE')
diff --git a/tempest/api/compute/images/test_list_image_filters.py b/tempest/api/compute/images/test_list_image_filters.py
index 5c6b630..b27d710 100644
--- a/tempest/api/compute/images/test_list_image_filters.py
+++ b/tempest/api/compute/images/test_list_image_filters.py
@@ -31,6 +31,9 @@
@classmethod
def setUpClass(cls):
super(ListImageFiltersTestJSON, cls).setUpClass()
+ if not cls.config.service_available.glance:
+ skip_msg = ("%s skipped as glance is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
cls.client = cls.images_client
cls.image_ids = []
diff --git a/tempest/api/compute/images/test_list_images.py b/tempest/api/compute/images/test_list_images.py
index fddad14..c7e23b1 100644
--- a/tempest/api/compute/images/test_list_images.py
+++ b/tempest/api/compute/images/test_list_images.py
@@ -25,6 +25,9 @@
@classmethod
def setUpClass(cls):
super(ListImagesTestJSON, cls).setUpClass()
+ if not cls.config.service_available.glance:
+ skip_msg = ("%s skipped as glance is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
cls.client = cls.images_client
@attr(type='smoke')
diff --git a/tempest/api/compute/security_groups/test_security_groups.py b/tempest/api/compute/security_groups/test_security_groups.py
index e105121..ab100a3 100644
--- a/tempest/api/compute/security_groups/test_security_groups.py
+++ b/tempest/api/compute/security_groups/test_security_groups.py
@@ -158,7 +158,7 @@
self.client.create_security_group, s_name,
s_description)
- @testtools.skipIf(config.TempestConfig().network.neutron_available,
+ @testtools.skipIf(config.TempestConfig().service_available.neutron,
"Neutron allows duplicate names for security groups")
@attr(type=['negative', 'gate'])
def test_security_group_create_with_duplicate_name(self):
diff --git a/tempest/api/compute/servers/test_attach_interfaces.py b/tempest/api/compute/servers/test_attach_interfaces.py
index de095c5..9f66a6c 100644
--- a/tempest/api/compute/servers/test_attach_interfaces.py
+++ b/tempest/api/compute/servers/test_attach_interfaces.py
@@ -24,7 +24,7 @@
@classmethod
def setUpClass(cls):
- if not cls.config.network.neutron_available:
+ if not cls.config.service_available.neutron:
raise cls.skipException("Neutron is required")
super(AttachInterfacesTestJSON, cls).setUpClass()
cls.client = cls.os.interfaces_client
diff --git a/tempest/api/compute/servers/test_list_servers_negative.py b/tempest/api/compute/servers/test_list_servers_negative.py
index db9bdc1..c03c43e 100644
--- a/tempest/api/compute/servers/test_list_servers_negative.py
+++ b/tempest/api/compute/servers/test_list_servers_negative.py
@@ -59,8 +59,9 @@
if num_servers > 0:
username = cls.os.username
tenant_name = cls.os.tenant_name
- msg = ("User/tenant %(username)s/%(tenant_name)s already have "
- "existing server instances. Skipping test.") % locals()
+ msg = ("User/tenant %(u)s/%(t)s already have "
+ "existing server instances. Skipping test." %
+ {'u': username, 't': tenant_name})
raise cls.skipException(msg)
resp, body = cls.alt_client.list_servers()
@@ -69,8 +70,9 @@
if num_servers > 0:
username = cls.alt_manager.username
tenant_name = cls.alt_manager.tenant_name
- msg = ("Alt User/tenant %(username)s/%(tenant_name)s already have "
- "existing server instances. Skipping test.") % locals()
+ msg = ("Alt User/tenant %(u)s/%(t)s already have "
+ "existing server instances. Skipping test." %
+ {'u': username, 't': tenant_name})
raise cls.skipException(msg)
# The following servers are created for use
diff --git a/tempest/api/compute/servers/test_server_rescue.py b/tempest/api/compute/servers/test_server_rescue.py
index 8225a4c..13c2f74 100644
--- a/tempest/api/compute/servers/test_server_rescue.py
+++ b/tempest/api/compute/servers/test_server_rescue.py
@@ -126,6 +126,13 @@
self.rescue_id, 'HARD')
@attr(type=['negative', 'gate'])
+ def test_rescue_non_existent_server(self):
+ # Rescue a non-existing server
+ self.assertRaises(exceptions.NotFound,
+ self.servers_client.rescue_server,
+ '999erra43')
+
+ @attr(type=['negative', 'gate'])
def test_rescued_vm_rebuild(self):
self.assertRaises(exceptions.Duplicate,
self.servers_client.rebuild,
diff --git a/tempest/api/compute/servers/test_servers_negative.py b/tempest/api/compute/servers/test_servers_negative.py
index 5cc8dc6..af58b5f 100644
--- a/tempest/api/compute/servers/test_servers_negative.py
+++ b/tempest/api/compute/servers/test_servers_negative.py
@@ -99,6 +99,17 @@
self.server_id, 'SOFT')
@attr(type=['negative', 'gate'])
+ def test_pause_paused_server(self):
+ # Pause a paused server.
+ resp, server = self.create_server(wait_until='ACTIVE')
+ self.server_id = server['id']
+ self.client.pause_server(self.server_id)
+ self.client.wait_for_server_status(self.server_id, 'PAUSED')
+ self.assertRaises(exceptions.Duplicate,
+ self.client.pause_server,
+ self.server_id)
+
+ @attr(type=['negative', 'gate'])
def test_rebuild_deleted_server(self):
# Rebuild a deleted server
diff --git a/tempest/api/compute/servers/test_virtual_interfaces.py b/tempest/api/compute/servers/test_virtual_interfaces.py
index 35f0fc0..2a5be8c 100644
--- a/tempest/api/compute/servers/test_virtual_interfaces.py
+++ b/tempest/api/compute/servers/test_virtual_interfaces.py
@@ -37,7 +37,7 @@
resp, server = cls.create_server(wait_until='ACTIVE')
cls.server_id = server['id']
- @testtools.skipIf(CONF.network.neutron_available, "This feature is not " +
+ @testtools.skipIf(CONF.service_available.neutron, "This feature is not " +
"implemented by Neutron. See bug: #1183436")
@attr(type='gate')
def test_list_virtual_interfaces(self):
diff --git a/tempest/api/compute/volumes/test_attach_volume.py b/tempest/api/compute/volumes/test_attach_volume.py
index b507e03..6571491 100644
--- a/tempest/api/compute/volumes/test_attach_volume.py
+++ b/tempest/api/compute/volumes/test_attach_volume.py
@@ -37,6 +37,9 @@
def setUpClass(cls):
super(AttachVolumeTestJSON, cls).setUpClass()
cls.device = 'vdb'
+ if not cls.config.service_available.cinder:
+ skip_msg = ("%s skipped as Cinder is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
def _detach(self, server_id, volume_id):
self.servers_client.detach_volume(server_id, volume_id)
diff --git a/tempest/api/compute/volumes/test_volumes_get.py b/tempest/api/compute/volumes/test_volumes_get.py
index 1acc57d..363cd6a 100644
--- a/tempest/api/compute/volumes/test_volumes_get.py
+++ b/tempest/api/compute/volumes/test_volumes_get.py
@@ -28,6 +28,9 @@
def setUpClass(cls):
super(VolumesGetTestJSON, cls).setUpClass()
cls.client = cls.volumes_extensions_client
+ if not cls.config.service_available.cinder:
+ skip_msg = ("%s skipped as Cinder is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
@attr(type='smoke')
def test_volume_create_get_delete(self):
diff --git a/tempest/api/compute/volumes/test_volumes_list.py b/tempest/api/compute/volumes/test_volumes_list.py
index d52349e..02cc4e1 100644
--- a/tempest/api/compute/volumes/test_volumes_list.py
+++ b/tempest/api/compute/volumes/test_volumes_list.py
@@ -36,6 +36,9 @@
def setUpClass(cls):
super(VolumesTestJSON, cls).setUpClass()
cls.client = cls.volumes_extensions_client
+ if not cls.config.service_available.cinder:
+ skip_msg = ("%s skipped as Cinder is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
# Create 3 Volumes
cls.volume_list = []
cls.volume_id_list = []
diff --git a/tempest/api/compute/volumes/test_volumes_negative.py b/tempest/api/compute/volumes/test_volumes_negative.py
index de214fc..f1ef5a4 100644
--- a/tempest/api/compute/volumes/test_volumes_negative.py
+++ b/tempest/api/compute/volumes/test_volumes_negative.py
@@ -28,6 +28,9 @@
def setUpClass(cls):
super(VolumesNegativeTest, cls).setUpClass()
cls.client = cls.volumes_extensions_client
+ if not cls.config.service_available.cinder:
+ skip_msg = ("%s skipped as Cinder is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
@attr(type='gate')
def test_volume_get_nonexistant_volume_id(self):
diff --git a/tempest/api/image/base.py b/tempest/api/image/base.py
index e62d84b..e27ec13 100644
--- a/tempest/api/image/base.py
+++ b/tempest/api/image/base.py
@@ -30,6 +30,9 @@
def setUpClass(cls):
cls.os = clients.Manager()
cls.created_images = []
+ if not cls.config.service_available.glance:
+ skip_msg = ("%s skipped as glance is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
@classmethod
def tearDownClass(cls):
diff --git a/tempest/api/network/base.py b/tempest/api/network/base.py
index 3b7f9dd..142ad7d 100644
--- a/tempest/api/network/base.py
+++ b/tempest/api/network/base.py
@@ -44,7 +44,7 @@
def setUpClass(cls):
os = clients.Manager()
cls.network_cfg = os.config.network
- if not cls.network_cfg.neutron_available:
+ if not cls.config.service_available.neutron:
raise cls.skipException("Neutron support is required")
cls.client = os.network_client
cls.networks = []
diff --git a/tempest/api/object_storage/base.py b/tempest/api/object_storage/base.py
index bf013ec..5a1fb5a 100644
--- a/tempest/api/object_storage/base.py
+++ b/tempest/api/object_storage/base.py
@@ -26,6 +26,9 @@
@classmethod
def setUpClass(cls):
+ if not cls.config.service_available.swift:
+ skip_msg = ("%s skipped as swift is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
cls.os = clients.Manager()
cls.object_client = cls.os.object_client
cls.container_client = cls.os.container_client
@@ -42,12 +45,6 @@
cls.data = DataGenerator(cls.identity_admin_client)
- try:
- cls.account_client.list_account_containers()
- except exceptions.EndpointNotFound:
- skip_msg = "No OpenStack Object Storage API endpoint"
- raise cls.skipException(skip_msg)
-
@classmethod
def delete_containers(cls, containers, container_client=None,
object_client=None):
diff --git a/tempest/api/orchestration/base.py b/tempest/api/orchestration/base.py
index ffa534a..a0b248c 100644
--- a/tempest/api/orchestration/base.py
+++ b/tempest/api/orchestration/base.py
@@ -31,7 +31,7 @@
os = clients.OrchestrationManager()
cls.orchestration_cfg = os.config.orchestration
- if not cls.orchestration_cfg.heat_available:
+ if not os.config.service_available.heat:
raise cls.skipException("Heat support is required")
cls.build_timeout = cls.orchestration_cfg.build_timeout
cls.build_interval = cls.orchestration_cfg.build_interval
diff --git a/tempest/api/utils.py b/tempest/api/utils.py
index 0738201..69ab7fb 100644
--- a/tempest/api/utils.py
+++ b/tempest/api/utils.py
@@ -17,7 +17,7 @@
"""Common utilities used in testing."""
-from testtools import TestCase
+from tempest.test import BaseTestCase
class skip_unless_attr(object):
@@ -32,7 +32,7 @@
"""Wrapped skipper function."""
testobj = args[0]
if not getattr(testobj, self.attr, False):
- raise TestCase.skipException(self.message)
+ raise BaseTestCase.skipException(self.message)
func(*args, **kw)
_skipper.__name__ = func.__name__
_skipper.__doc__ = func.__doc__
diff --git a/tempest/api/volume/base.py b/tempest/api/volume/base.py
index fc510cb..a84f9e8 100644
--- a/tempest/api/volume/base.py
+++ b/tempest/api/volume/base.py
@@ -19,8 +19,6 @@
from tempest import clients
from tempest.common import log as logging
-from tempest.common.utils.data_utils import rand_name
-from tempest import exceptions
import tempest.test
LOG = logging.getLogger(__name__)
@@ -34,6 +32,10 @@
def setUpClass(cls):
cls.isolated_creds = []
+ if not cls.config.service_available.cinder:
+ skip_msg = ("%s skipped as Cinder is not available" % cls.__name__)
+ raise cls.skipException(skip_msg)
+
if cls.config.compute.allow_tenant_isolation:
creds = cls._get_isolated_creds()
username, tenant_name, password = creds
@@ -55,72 +57,17 @@
cls.snapshots = []
cls.volumes = []
- skip_msg = ("%s skipped as Cinder endpoint is not available" %
- cls.__name__)
- try:
- cls.volumes_client.keystone_auth(cls.os.username,
- cls.os.password,
- cls.os.auth_url,
- cls.volumes_client.service,
- cls.os.tenant_name)
- except exceptions.EndpointNotFound:
- cls.clear_isolated_creds()
- raise cls.skipException(skip_msg)
-
- @classmethod
- def _get_identity_admin_client(cls):
- """
- Returns an instance of the Identity Admin API client
- """
- os = clients.ComputeAdminManager()
- return os.identity_client
-
- @classmethod
- def _get_isolated_creds(cls):
- """
- Creates a new set of user/tenant/password credentials for a
- **regular** user of the Volume API so that a test case can
- operate in an isolated tenant container.
- """
- admin_client = cls._get_identity_admin_client()
- rand_name_root = rand_name(cls.__name__)
- if cls.isolated_creds:
- # Main user already created. Create the alt one...
- rand_name_root += '-alt'
- username = rand_name_root + "-user"
- email = rand_name_root + "@example.com"
- tenant_name = rand_name_root + "-tenant"
- tenant_desc = tenant_name + "-desc"
- password = "pass"
-
- resp, tenant = admin_client.create_tenant(name=tenant_name,
- description=tenant_desc)
- resp, user = admin_client.create_user(username,
- password,
- tenant['id'],
- email)
- # Store the complete creds (including UUID ids...) for later
- # but return just the username, tenant_name, password tuple
- # that the various clients will use.
- cls.isolated_creds.append((user, tenant))
-
- return username, tenant_name, password
-
- @classmethod
- def clear_isolated_creds(cls):
- if not cls.isolated_creds:
- return
- admin_client = cls._get_identity_admin_client()
-
- for user, tenant in cls.isolated_creds:
- admin_client.delete_user(user['id'])
- admin_client.delete_tenant(tenant['id'])
+ cls.volumes_client.keystone_auth(cls.os.username,
+ cls.os.password,
+ cls.os.auth_url,
+ cls.volumes_client.service,
+ cls.os.tenant_name)
@classmethod
def tearDownClass(cls):
cls.clear_snapshots()
cls.clear_volumes()
- cls.clear_isolated_creds()
+ cls._clear_isolated_creds()
@classmethod
def create_snapshot(cls, volume_id=1, **kwargs):
@@ -201,6 +148,13 @@
msg = ("Missing Volume Admin API credentials "
"in configuration.")
raise cls.skipException(msg)
-
- cls.os_adm = clients.AdminManager(interface=cls._interface)
+ if cls.config.compute.allow_tenant_isolation:
+ creds = cls._get_isolated_creds(admin=True)
+ admin_username, admin_tenant_name, admin_password = creds
+ cls.os_adm = clients.Manager(username=admin_username,
+ password=admin_password,
+ tenant_name=admin_tenant_name,
+ interface=cls._interface)
+ else:
+ cls.os_adm = clients.AdminManager(interface=cls._interface)
cls.client = cls.os_adm.volume_types_client
diff --git a/tempest/cli/README.rst b/tempest/cli/README.rst
index 3eae492..f86adf3 100644
--- a/tempest/cli/README.rst
+++ b/tempest/cli/README.rst
@@ -12,7 +12,7 @@
Why are these tests in tempest?
-------------------------------
These tests exist here because it is extremely difficult to build a
-functional enough environment in the python-*client unit tests to
+functional enough environment in the python-\*client unit tests to
provide this kind of testing. Because we already put up a cloud in the
gate with devstack + tempest it was decided it was better to have
these as a side tree in tempest instead of another QA effort which
diff --git a/tempest/cli/simple_read_only/test_compute.py b/tempest/cli/simple_read_only/test_compute.py
index 561fd00..5dadbeb 100644
--- a/tempest/cli/simple_read_only/test_compute.py
+++ b/tempest/cli/simple_read_only/test_compute.py
@@ -23,7 +23,6 @@
import tempest.cli
from tempest.common import log as logging
-
CONF = cfg.CONF
@@ -69,6 +68,8 @@
def test_admin_credentials(self):
self.nova('credentials')
+ @testtools.skipIf(CONF.service_available.neutron,
+ "Neutron does not provide this feature")
def test_admin_dns_domains(self):
self.nova('dns-domains')
diff --git a/tempest/clients.py b/tempest/clients.py
index 5efce98..2154f8b 100644
--- a/tempest/clients.py
+++ b/tempest/clients.py
@@ -274,8 +274,9 @@
if None in (self.username, self.password, self.tenant_name):
msg = ("Missing required credentials. "
- "username: %(username)s, password: %(password)s, "
- "tenant_name: %(tenant_name)s") % locals()
+ "username: %(u)s, password: %(p)s, "
+ "tenant_name: %(t)s" %
+ {'u': username, 'p': password, 't': tenant_name})
raise exceptions.InvalidConfiguration(msg)
self.auth_url = self.config.identity.uri
diff --git a/tempest/common/glance_http.py b/tempest/common/glance_http.py
index cd33a22..4045430 100644
--- a/tempest/common/glance_http.py
+++ b/tempest/common/glance_http.py
@@ -125,11 +125,12 @@
conn.request(method, conn_url, **kwargs)
resp = conn.getresponse()
except socket.gaierror as e:
- message = "Error finding address for %(url)s: %(e)s" % locals()
+ message = ("Error finding address for %(url)s: %(e)s" %
+ {'url': url, 'e': e})
raise exc.EndpointNotFound(message)
except (socket.error, socket.timeout) as e:
- endpoint = self.endpoint
- message = "Error communicating with %(endpoint)s %(e)s" % locals()
+ message = ("Error communicating with %(endpoint)s %(e)s" %
+ {'endpoint': self.endpoint, 'e': e})
raise exc.TimeoutException(message)
body_iter = ResponseBodyIterator(resp)
diff --git a/tempest/common/utils/linux/remote_client.py b/tempest/common/utils/linux/remote_client.py
index fd5d3d0..de2bf43 100644
--- a/tempest/common/utils/linux/remote_client.py
+++ b/tempest/common/utils/linux/remote_client.py
@@ -1,3 +1,17 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
import re
import time
diff --git a/tempest/config.py b/tempest/config.py
index 2e56628..a918d0b 100644
--- a/tempest/config.py
+++ b/tempest/config.py
@@ -160,6 +160,10 @@
cfg.StrOpt('ssh_user',
default='root',
help="User name used to authenticate to an instance."),
+ cfg.IntOpt('ping_timeout',
+ default=60,
+ help="Timeout in seconds to wait for ping to "
+ "succeed."),
cfg.IntOpt('ssh_timeout',
default=300,
help="Timeout in seconds to wait for authentication to "
@@ -288,7 +292,7 @@
default="10.100.0.0/16",
help="The cidr block to allocate tenant networks from"),
cfg.IntOpt('tenant_network_mask_bits',
- default=29,
+ default=28,
help="The mask bits for tenant networks"),
cfg.BoolOpt('tenant_networks_reachable',
default=False,
@@ -302,9 +306,6 @@
default="",
help="Id of the public router that provides external "
"connectivity"),
- cfg.BoolOpt('neutron_available',
- default=False,
- help="Whether or not neutron is expected to be available"),
]
@@ -394,9 +395,6 @@
cfg.IntOpt('build_timeout',
default=300,
help="Timeout in seconds to wait for a stack to build."),
- cfg.BoolOpt('heat_available',
- default=False,
- help="Whether or not Heat is expected to be available"),
cfg.StrOpt('instance_type',
default='m1.micro',
help="Instance type for tests. Needs to be big enough for a "
@@ -416,6 +414,26 @@
for opt in OrchestrationGroup:
conf.register_opt(opt, group='orchestration')
+
+dashboard_group = cfg.OptGroup(name="dashboard",
+ title="Dashboard options")
+
+DashboardGroup = [
+ cfg.StrOpt('dashboard_url',
+ default='http://localhost/',
+ help="Where the dashboard can be found"),
+ cfg.StrOpt('login_url',
+ default='http://localhost/auth/login/',
+ help="Login page for the dashboard"),
+]
+
+
+def register_dashboard_opts(conf):
+ conf.register_group(scenario_group)
+ for opt in DashboardGroup:
+ conf.register_opt(opt, group='dashboard')
+
+
boto_group = cfg.OptGroup(name='boto',
title='EC2/S3 options')
BotoConfig = [
@@ -538,6 +556,40 @@
conf.register_opt(opt, group='scenario')
+service_available_group = cfg.OptGroup(name="service_available",
+ title="Available OpenStack Services")
+
+ServiceAvailableGroup = [
+ cfg.BoolOpt('cinder',
+ default=True,
+ help="Whether or not cinder is expected to be available"),
+ cfg.BoolOpt('neutron',
+ default=False,
+ help="Whether or not neutron is expected to be available"),
+ cfg.BoolOpt('glance',
+ default=True,
+ help="Whether or not glance is expected to be available"),
+ cfg.BoolOpt('swift',
+ default=True,
+ help="Whether or not swift is expected to be available"),
+ cfg.BoolOpt('nova',
+ default=True,
+ help="Whether or not nova is expected to be available"),
+ cfg.BoolOpt('heat',
+ default=False,
+ help="Whether or not Heat is expected to be available"),
+ cfg.BoolOpt('horizon',
+ default=True,
+ help="Whether or not Horizon is expected to be available"),
+]
+
+
+def register_service_available_opts(conf):
+ conf.register_group(scenario_group)
+ for opt in ServiceAvailableGroup:
+ conf.register_opt(opt, group='service_available')
+
+
@singleton
class TempestConfig:
"""Provides OpenStack configuration information."""
@@ -569,7 +621,7 @@
LOG.info("Using tempest config file %s" % path)
if not os.path.exists(path):
- msg = "Config file %(path)s not found" % locals()
+ msg = "Config file %s not found" % path
print(RuntimeError(msg), file=sys.stderr)
else:
config_files.append(path)
@@ -584,10 +636,12 @@
register_volume_opts(cfg.CONF)
register_object_storage_opts(cfg.CONF)
register_orchestration_opts(cfg.CONF)
+ register_dashboard_opts(cfg.CONF)
register_boto_opts(cfg.CONF)
register_compute_admin_opts(cfg.CONF)
register_stress_opts(cfg.CONF)
register_scenario_opts(cfg.CONF)
+ register_service_available_opts(cfg.CONF)
self.compute = cfg.CONF.compute
self.whitebox = cfg.CONF.whitebox
self.identity = cfg.CONF.identity
@@ -596,10 +650,12 @@
self.volume = cfg.CONF.volume
self.object_storage = cfg.CONF['object-storage']
self.orchestration = cfg.CONF.orchestration
+ self.dashboard = cfg.CONF.dashboard
self.boto = cfg.CONF.boto
self.compute_admin = cfg.CONF['compute-admin']
self.stress = cfg.CONF.stress
self.scenario = cfg.CONF.scenario
+ self.service_available = cfg.CONF.service_available
if not self.compute_admin.username:
self.compute_admin.username = self.identity.admin_username
self.compute_admin.password = self.identity.admin_password
diff --git a/tempest/manager.py b/tempest/manager.py
index 4a447f3..187e2c6 100644
--- a/tempest/manager.py
+++ b/tempest/manager.py
@@ -65,6 +65,15 @@
self.config = tempest.config.TempestConfig()
self.client_attr_names = []
+ # we do this everywhere, have it be part of the super class
+ def _validate_credentials(self, username, password, tenant_name):
+ if None in (username, password, tenant_name):
+ msg = ("Missing required credentials. "
+ "username: %(u)s, password: %(p)s, "
+ "tenant_name: %(t)s" %
+ {'u': username, 'p': password, 't': tenant_name})
+ raise exceptions.InvalidConfiguration(msg)
+
class FuzzClientManager(Manager):
@@ -103,11 +112,7 @@
password = password or self.config.identity.password
tenant_name = tenant_name or self.config.identity.tenant_name
- if None in (username, password, tenant_name):
- msg = ("Missing required credentials. "
- "username: %(username)s, password: %(password)s, "
- "tenant_name: %(tenant_name)s") % locals()
- raise exceptions.InvalidConfiguration(msg)
+ self._validate_credentials(username, password, tenant_name)
auth_url = self.config.identity.uri
diff --git a/tempest/scenario/manager.py b/tempest/scenario/manager.py
index fe6fbf5..8b24b2e 100644
--- a/tempest/scenario/manager.py
+++ b/tempest/scenario/manager.py
@@ -32,7 +32,6 @@
from tempest.common import log as logging
from tempest.common import ssh
from tempest.common.utils.data_utils import rand_name
-from tempest import exceptions
import tempest.manager
import tempest.test
@@ -76,11 +75,7 @@
if not tenant_name:
tenant_name = self.config.identity.tenant_name
- if None in (username, password, tenant_name):
- msg = ("Missing required credentials for compute client. "
- "username: %(username)s, password: %(password)s, "
- "tenant_name: %(tenant_name)s") % locals()
- raise exceptions.InvalidConfiguration(msg)
+ self._validate_credentials(username, password, tenant_name)
auth_url = self.config.identity.uri
dscv = self.config.identity.disable_ssl_certificate_validation
@@ -131,11 +126,7 @@
if not tenant_name:
tenant_name = self.config.identity.admin_tenant_name
- if None in (username, password, tenant_name):
- msg = ("Missing required credentials for identity client. "
- "username: %(username)s, password: %(password)s, "
- "tenant_name: %(tenant_name)s") % locals()
- raise exceptions.InvalidConfiguration(msg)
+ self._validate_credentials(username, password, tenant_name)
auth_url = self.config.identity.uri
dscv = self.config.identity.disable_ssl_certificate_validation
@@ -157,11 +148,7 @@
password = self.config.identity.admin_password
tenant_name = self.config.identity.admin_tenant_name
- if None in (username, password, tenant_name):
- msg = ("Missing required credentials for network client. "
- "username: %(username)s, password: %(password)s, "
- "tenant_name: %(tenant_name)s") % locals()
- raise exceptions.InvalidConfiguration(msg)
+ self._validate_credentials(username, password, tenant_name)
auth_url = self.config.identity.uri
dscv = self.config.identity.disable_ssl_certificate_validation
@@ -236,7 +223,7 @@
@classmethod
def check_preconditions(cls):
- if (cls.config.network.neutron_available):
+ if (cls.config.service_available.neutron):
cls.enabled = True
#verify that neutron_available is telling the truth
try:
@@ -438,24 +425,24 @@
if proc.returncode == 0:
return True
- # TODO(mnewby) Allow configuration of execution and sleep duration.
- return tempest.test.call_until_true(ping, 20, 1)
+ return tempest.test.call_until_true(
+ ping, self.config.compute.ping_timeout, 1)
def _is_reachable_via_ssh(self, ip_address, username, private_key,
- timeout=120):
+ timeout):
ssh_client = ssh.Client(ip_address, username,
pkey=private_key,
timeout=timeout)
return ssh_client.test_connection_auth()
- def _check_vm_connectivity(self, ip_address, username, private_key,
- timeout=120):
+ def _check_vm_connectivity(self, ip_address, username, private_key):
self.assertTrue(self._ping_ip_address(ip_address),
"Timed out waiting for %s to become "
"reachable" % ip_address)
- self.assertTrue(self._is_reachable_via_ssh(ip_address,
- username,
- private_key,
- timeout=timeout),
- 'Auth failure in connecting to %s@%s via ssh' %
- (username, ip_address))
+ self.assertTrue(self._is_reachable_via_ssh(
+ ip_address,
+ username,
+ private_key,
+ timeout=self.config.compute.ssh_timeout),
+ 'Auth failure in connecting to %s@%s via ssh' %
+ (username, ip_address))
diff --git a/tempest/scenario/test_dashboard_basic_ops.py b/tempest/scenario/test_dashboard_basic_ops.py
new file mode 100644
index 0000000..9a45572
--- /dev/null
+++ b/tempest/scenario/test_dashboard_basic_ops.py
@@ -0,0 +1,72 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+import urllib
+import urllib2
+
+from lxml import html
+
+from tempest.scenario import manager
+
+
+class TestDashboardBasicOps(manager.OfficialClientTest):
+
+ """
+ This is a basic scenario test:
+ * checks that the login page is available
+ * logs in as a regular user
+ * checks that the user home page loads without error
+ """
+
+ @classmethod
+ def setUpClass(cls):
+ super(TestDashboardBasicOps, cls).setUpClass()
+
+ if not cls.config.service_available.horizon:
+ raise cls.skipException("Horizon support is required")
+
+ def check_login_page(self):
+ response = urllib2.urlopen(self.config.dashboard.dashboard_url)
+ self.assertIn("<h3>Log In</h3>", response.read())
+
+ def user_login(self):
+ self.opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
+ response = self.opener.open(self.config.dashboard.dashboard_url).read()
+
+ # Grab the CSRF token and default region
+ csrf_token = html.fromstring(response).xpath(
+ '//input[@name="csrfmiddlewaretoken"]/@value')[0]
+ region = html.fromstring(response).xpath(
+ '//input[@name="region"]/@value')[0]
+
+ # Prepare login form request
+ req = urllib2.Request(self.config.dashboard.login_url)
+ req.add_header('Content-type', 'application/x-www-form-urlencoded')
+ req.add_header('Referer', self.config.dashboard.dashboard_url)
+ params = {'username': self.config.identity.username,
+ 'password': self.config.identity.password,
+ 'region': region,
+ 'csrfmiddlewaretoken': csrf_token}
+ self.opener.open(req, urllib.urlencode(params))
+
+ def check_home_page(self):
+ response = self.opener.open(self.config.dashboard.dashboard_url)
+ self.assertIn('Overview', response.read())
+
+ def test_basic_scenario(self):
+ self.check_login_page()
+ self.user_login()
+ self.check_home_page()
diff --git a/tempest/services/botoclients.py b/tempest/services/botoclients.py
index 628151a..32ec109 100644
--- a/tempest/services/botoclients.py
+++ b/tempest/services/botoclients.py
@@ -132,6 +132,7 @@
ALLOWED_METHODS = set(('create_key_pair', 'get_key_pair',
'delete_key_pair', 'import_key_pair',
'get_all_key_pairs',
+ 'get_all_tags',
'create_image', 'get_image',
'register_image', 'deregister_image',
'get_all_images', 'get_image_attribute',
diff --git a/tempest/services/compute/json/services_client.py b/tempest/services/compute/json/services_client.py
index d054f72..4db7596 100644
--- a/tempest/services/compute/json/services_client.py
+++ b/tempest/services/compute/json/services_client.py
@@ -1,6 +1,7 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 NEC Corporation
+# Copyright 2013 IBM Corp.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
@@ -16,6 +17,7 @@
# under the License.
import json
+import urllib
from tempest.common.rest_client import RestClient
@@ -27,7 +29,33 @@
auth_url, tenant_name)
self.service = self.config.compute.catalog_type
- def list_services(self):
- resp, body = self.get("os-services")
+ def list_services(self, params=None):
+ url = 'os-services'
+ if params:
+ url += '?%s' % urllib.urlencode(params)
+
+ resp, body = self.get(url)
body = json.loads(body)
return resp, body['services']
+
+ def enable_service(self, host_name, binary):
+ """
+ Enable service on a host
+ host_name: Name of host
+ binary: Service binary
+ """
+ post_body = json.dumps({'binary': binary, 'host': host_name})
+ resp, body = self.put('os-services/enable', post_body, self.headers)
+ body = json.loads(body)
+ return resp, body['service']
+
+ def disable_service(self, host_name, binary):
+ """
+ Disable service on a host
+ host_name: Name of host
+ binary: Service binary
+ """
+ post_body = json.dumps({'binary': binary, 'host': host_name})
+ resp, body = self.put('os-services/disable', post_body, self.headers)
+ body = json.loads(body)
+ return resp, body['service']
diff --git a/tempest/services/compute/xml/servers_client.py b/tempest/services/compute/xml/servers_client.py
index f2cca72..ea8b0e0 100644
--- a/tempest/services/compute/xml/servers_client.py
+++ b/tempest/services/compute/xml/servers_client.py
@@ -437,6 +437,12 @@
def revert_resize(self, server_id, **kwargs):
return self.action(server_id, 'revertResize', None, **kwargs)
+ def stop(self, server_id, **kwargs):
+ return self.action(server_id, 'os-stop', None, **kwargs)
+
+ def start(self, server_id, **kwargs):
+ return self.action(server_id, 'os-start', None, **kwargs)
+
def create_image(self, server_id, name):
return self.action(server_id, 'createImage', None, name=name)
diff --git a/tempest/services/compute/xml/services_client.py b/tempest/services/compute/xml/services_client.py
index ce23403..ac304e2 100644
--- a/tempest/services/compute/xml/services_client.py
+++ b/tempest/services/compute/xml/services_client.py
@@ -1,6 +1,7 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 NEC Corporation
+# Copyright 2013 IBM Corp.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
@@ -15,8 +16,12 @@
# License for the specific language governing permissions and limitations
# under the License.
+import urllib
+
from lxml import etree
from tempest.common.rest_client import RestClientXML
+from tempest.services.compute.xml.common import Document
+from tempest.services.compute.xml.common import Element
from tempest.services.compute.xml.common import xml_to_json
@@ -27,8 +32,42 @@
auth_url, tenant_name)
self.service = self.config.compute.catalog_type
- def list_services(self):
- resp, body = self.get("os-services", self.headers)
+ def list_services(self, params=None):
+ url = 'os-services'
+ if params:
+ url += '?%s' % urllib.urlencode(params)
+
+ resp, body = self.get(url, self.headers)
node = etree.fromstring(body)
body = [xml_to_json(x) for x in node.getchildren()]
return resp, body
+
+ def enable_service(self, host_name, binary):
+ """
+ Enable service on a host
+ host_name: Name of host
+ binary: Service binary
+ """
+ post_body = Element("service")
+ post_body.add_attr('binary', binary)
+ post_body.add_attr('host', host_name)
+
+ resp, body = self.put('os-services/enable', str(Document(post_body)),
+ self.headers)
+ body = xml_to_json(etree.fromstring(body))
+ return resp, body
+
+ def disable_service(self, host_name, binary):
+ """
+ Disable service on a host
+ host_name: Name of host
+ binary: Service binary
+ """
+ post_body = Element("service")
+ post_body.add_attr('binary', binary)
+ post_body.add_attr('host', host_name)
+
+ resp, body = self.put('os-services/disable', str(Document(post_body)),
+ self.headers)
+ body = xml_to_json(etree.fromstring(body))
+ return resp, body
diff --git a/tempest/services/identity/json/identity_client.py b/tempest/services/identity/json/identity_client.py
index a216b55..90e64e7 100644
--- a/tempest/services/identity/json/identity_client.py
+++ b/tempest/services/identity/json/identity_client.py
@@ -1,3 +1,17 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
import httplib2
import json
diff --git a/tempest/services/network/json/network_client.py b/tempest/services/network/json/network_client.py
index c4fe6b1..446a674 100644
--- a/tempest/services/network/json/network_client.py
+++ b/tempest/services/network/json/network_client.py
@@ -1,4 +1,19 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
import json
+
from tempest.common.rest_client import RestClient
diff --git a/tempest/stress/actions/create_destroy_server.py b/tempest/stress/actions/create_destroy_server.py
index 44b149f..68dc148 100644
--- a/tempest/stress/actions/create_destroy_server.py
+++ b/tempest/stress/actions/create_destroy_server.py
@@ -13,22 +13,27 @@
# limitations under the License.
from tempest.common.utils.data_utils import rand_name
+import tempest.stress.stressaction as stressaction
-def create_destroy(manager, logger):
- image = manager.config.compute.image_ref
- flavor = manager.config.compute.flavor_ref
- while True:
+class CreateDestroyServerTest(stressaction.StressAction):
+
+ def setUp(self, **kwargs):
+ self.image = self.manager.config.compute.image_ref
+ self.flavor = self.manager.config.compute.flavor_ref
+
+ def run(self):
name = rand_name("instance")
- logger.info("creating %s" % name)
- resp, server = manager.servers_client.create_server(
- name, image, flavor)
+ self.logger.info("creating %s" % name)
+ resp, server = self.manager.servers_client.create_server(
+ name, self.image, self.flavor)
server_id = server['id']
assert(resp.status == 202)
- manager.servers_client.wait_for_server_status(server_id, 'ACTIVE')
- logger.info("created %s" % server_id)
- logger.info("deleting %s" % name)
- resp, _ = manager.servers_client.delete_server(server_id)
+ self.manager.servers_client.wait_for_server_status(server_id,
+ 'ACTIVE')
+ self.logger.info("created %s" % server_id)
+ self.logger.info("deleting %s" % name)
+ resp, _ = self.manager.servers_client.delete_server(server_id)
assert(resp.status == 204)
- manager.servers_client.wait_for_server_termination(server_id)
- logger.info("deleted %s" % server_id)
+ self.manager.servers_client.wait_for_server_termination(server_id)
+ self.logger.info("deleted %s" % server_id)
diff --git a/tempest/stress/actions/volume_create_delete.py b/tempest/stress/actions/volume_create_delete.py
index e0c95b5..184f870 100644
--- a/tempest/stress/actions/volume_create_delete.py
+++ b/tempest/stress/actions/volume_create_delete.py
@@ -11,20 +11,23 @@
# limitations under the License.
from tempest.common.utils.data_utils import rand_name
+import tempest.stress.stressaction as stressaction
-def create_delete(manager, logger):
- while True:
+class CreateDeleteTest(stressaction.StressAction):
+
+ def run(self):
name = rand_name("volume")
- logger.info("creating %s" % name)
- resp, volume = manager.volumes_client.create_volume(size=1,
- display_name=name)
+ self.logger.info("creating %s" % name)
+ resp, volume = self.manager.volumes_client.\
+ create_volume(size=1, display_name=name)
assert(resp.status == 200)
- manager.volumes_client.wait_for_volume_status(volume['id'],
- 'available')
- logger.info("created %s" % volume['id'])
- logger.info("deleting %s" % name)
- resp, _ = manager.volumes_client.delete_volume(volume['id'])
+ vol_id = volume['id']
+ status = 'available'
+ self.manager.volumes_client.wait_for_volume_status(vol_id, status)
+ self.logger.info("created %s" % volume['id'])
+ self.logger.info("deleting %s" % name)
+ resp, _ = self.manager.volumes_client.delete_volume(vol_id)
assert(resp.status == 202)
- manager.volumes_client.wait_for_resource_deletion(volume['id'])
- logger.info("deleted %s" % volume['id'])
+ self.manager.volumes_client.wait_for_resource_deletion(vol_id)
+ self.logger.info("deleted %s" % vol_id)
diff --git a/tempest/stress/cleanup.py b/tempest/stress/cleanup.py
index 3b1c871..bfcf34f 100644
--- a/tempest/stress/cleanup.py
+++ b/tempest/stress/cleanup.py
@@ -19,10 +19,11 @@
from tempest import clients
-def cleanup():
+def cleanup(logger):
admin_manager = clients.AdminManager()
_, body = admin_manager.servers_client.list_servers({"all_tenants": True})
+ logger.debug("Cleanup::remove %s servers" % len(body['servers']))
for s in body['servers']:
try:
admin_manager.servers_client.delete_server(s['id'])
@@ -36,6 +37,7 @@
pass
_, keypairs = admin_manager.keypairs_client.list_keypairs()
+ logger.debug("Cleanup::remove %s keypairs" % len(keypairs))
for k in keypairs:
try:
admin_manager.keypairs_client.delete_keypair(k['name'])
@@ -43,6 +45,7 @@
pass
_, floating_ips = admin_manager.floating_ips_client.list_floating_ips()
+ logger.debug("Cleanup::remove %s floating ips" % len(floating_ips))
for f in floating_ips:
try:
admin_manager.floating_ips_client.delete_floating_ip(f['id'])
@@ -50,18 +53,43 @@
pass
_, users = admin_manager.identity_client.get_users()
+ logger.debug("Cleanup::remove %s users" % len(users))
for user in users:
if user['name'].startswith("stress_user"):
admin_manager.identity_client.delete_user(user['id'])
_, tenants = admin_manager.identity_client.list_tenants()
+ logger.debug("Cleanup::remove %s tenants" % len(tenants))
for tenant in tenants:
if tenant['name'].startswith("stress_tenant"):
admin_manager.identity_client.delete_tenant(tenant['id'])
+ # We have to delete snapshots first or
+ # volume deletion may block
+
+ _, snaps = admin_manager.snapshots_client.\
+ list_snapshots({"all_tenants": True})
+ logger.debug("Cleanup::remove %s snapshots" % len(snaps))
+ for v in snaps:
+ try:
+ admin_manager.snapshots_client.\
+ wait_for_snapshot_status(v['id'], 'available')
+ admin_manager.snapshots_client.delete_snapshot(v['id'])
+ except Exception:
+ pass
+
+ for v in snaps:
+ try:
+ admin_manager.snapshots_client.wait_for_resource_deletion(v['id'])
+ except Exception:
+ pass
+
_, vols = admin_manager.volumes_client.list_volumes({"all_tenants": True})
+ logger.debug("Cleanup::remove %s volumes" % len(vols))
for v in vols:
try:
+ admin_manager.volumes_client.\
+ wait_for_volume_status(v['id'], 'available')
admin_manager.volumes_client.delete_volume(v['id'])
except Exception:
pass
diff --git a/tempest/stress/driver.py b/tempest/stress/driver.py
index 51f159d..d170eb8 100644
--- a/tempest/stress/driver.py
+++ b/tempest/stress/driver.py
@@ -93,12 +93,12 @@
return None
-def get_action_function(path):
- (module_part, _, function) = path.rpartition('.')
- return getattr(importlib.import_module(module_part), function)
+def get_action_object(path):
+ (module_part, _, obj_name) = path.rpartition('.')
+ return getattr(importlib.import_module(module_part), obj_name)
-def stress_openstack(tests, duration):
+def stress_openstack(tests, duration, max_runs=None):
"""
Workload driver. Executes an action function against a nova-cluster.
@@ -116,7 +116,7 @@
manager = admin_manager
else:
manager = clients.Manager()
- for _ in xrange(test.get('threads', 1)):
+ for p_number in xrange(test.get('threads', 1)):
if test.get('use_isolated_tenants', False):
username = rand_name("stress_user")
tenant_name = rand_name("stress_tenant")
@@ -130,18 +130,48 @@
manager = clients.Manager(username=username,
password="pass",
tenant_name=tenant_name)
- target = get_action_function(test['action'])
- p = multiprocessing.Process(target=target,
- args=(manager, logger),
- kwargs=test.get('kwargs', {}))
- processes.append(p)
+
+ test_obj = get_action_object(test['action'])
+ test_run = test_obj(manager, logger, max_runs)
+
+ kwargs = test.get('kwargs', {})
+ test_run.setUp(**dict(kwargs.iteritems()))
+
+ logger.debug("calling Target Object %s" %
+ test_run.__class__.__name__)
+
+ mp_manager = multiprocessing.Manager()
+ shared_statistic = mp_manager.dict()
+ shared_statistic['runs'] = 0
+ shared_statistic['fails'] = 0
+
+ p = multiprocessing.Process(target=test_run.execute,
+ args=(shared_statistic,))
+
+ process = {'process': p,
+ 'p_number': p_number,
+ 'action': test['action'],
+ 'statistic': shared_statistic}
+
+ processes.append(process)
p.start()
end_time = time.time() + duration
had_errors = False
while True:
- remaining = end_time - time.time()
- if remaining <= 0:
- break
+ if max_runs is None:
+ remaining = end_time - time.time()
+ if remaining <= 0:
+ break
+ else:
+ remaining = log_check_interval
+ all_proc_term = True
+ for process in processes:
+ if process['process'].is_alive():
+ all_proc_term = False
+ break
+ if all_proc_term:
+ break
+
time.sleep(min(remaining, log_check_interval))
if not logfiles:
continue
@@ -149,8 +179,34 @@
if errors:
had_errors = True
break
- for p in processes:
- p.terminate()
+
+ for process in processes:
+ if process['process'].is_alive():
+ process['process'].terminate()
+ process['process'].join()
+
+ sum_fails = 0
+ sum_runs = 0
+
+ logger.info("Statistics (per process):")
+ for process in processes:
+ if process['statistic']['fails'] > 0:
+ had_errors = True
+ sum_runs += process['statistic']['runs']
+ sum_fails += process['statistic']['fails']
+ logger.info(" Process %d (%s): Run %d actions (%d failed)" %
+ (process['p_number'],
+ process['action'],
+ process['statistic']['runs'],
+ process['statistic']['fails']))
+ logger.info("Summary:")
+ logger.info("Run %d actions (%d failed)" %
+ (sum_runs, sum_fails))
+
if not had_errors:
logger.info("cleaning up")
- cleanup.cleanup()
+ cleanup.cleanup(logger)
+ if had_errors:
+ return 1
+ else:
+ return 0
diff --git a/tempest/stress/etc/sample-test.json b/tempest/stress/etc/sample-test.json
index 5a0189c..494c823 100644
--- a/tempest/stress/etc/sample-test.json
+++ b/tempest/stress/etc/sample-test.json
@@ -1,4 +1,4 @@
-[{"action": "tempest.stress.actions.create_destroy_server.create_destroy",
+[{"action": "tempest.stress.actions.create_destroy_server.CreateDestroyServerTest",
"threads": 8,
"use_admin": false,
"use_isolated_tenants": false,
diff --git a/tempest/stress/etc/stress-tox-job.json b/tempest/stress/etc/stress-tox-job.json
new file mode 100644
index 0000000..159794b
--- /dev/null
+++ b/tempest/stress/etc/stress-tox-job.json
@@ -0,0 +1,13 @@
+[{"action": "tempest.stress.actions.create_destroy_server.CreateDestroyServerTest",
+ "threads": 8,
+ "use_admin": false,
+ "use_isolated_tenants": false,
+ "kwargs": {}
+ },
+ {"action": "tempest.stress.actions.volume_create_delete.CreateDeleteTest",
+ "threads": 4,
+ "use_admin": false,
+ "use_isolated_tenants": false,
+ "kwargs": {}
+ }
+]
diff --git a/tempest/stress/etc/volume-create-delete-test.json b/tempest/stress/etc/volume-create-delete-test.json
index ed0aaeb..6325bdc 100644
--- a/tempest/stress/etc/volume-create-delete-test.json
+++ b/tempest/stress/etc/volume-create-delete-test.json
@@ -1,4 +1,4 @@
-[{"action": "tempest.stress.actions.volume_create_delete.create_delete",
+[{"action": "tempest.stress.actions.volume_create_delete.CreateDeleteTest",
"threads": 4,
"use_admin": false,
"use_isolated_tenants": false,
diff --git a/tempest/stress/run_stress.py b/tempest/stress/run_stress.py
index ef0ec8e..106049d 100755
--- a/tempest/stress/run_stress.py
+++ b/tempest/stress/run_stress.py
@@ -18,17 +18,35 @@
import argparse
import json
-
-from tempest.stress import driver
+import sys
def main(ns):
+ #NOTE(kodererm): moved import to make "-h" possible without OpenStack
+ from tempest.stress import driver
+ result = 0
tests = json.load(open(ns.tests, 'r'))
- driver.stress_openstack(tests, ns.duration)
+ if ns.serial:
+ for test in tests:
+ step_result = driver.stress_openstack([test],
+ ns.duration,
+ ns.number)
+ #NOTE(kodererm): we just save the last result code
+ if (step_result != 0):
+ result = step_result
+ else:
+ driver.stress_openstack(tests, ns.duration, ns.number)
+ return result
parser = argparse.ArgumentParser(description='Run stress tests. ')
parser.add_argument('-d', '--duration', default=300, type=int,
- help="Duration of test.")
+ help="Duration of test in secs.")
+parser.add_argument('-s', '--serial', action='store_true',
+ help="Trigger running tests serially.")
+parser.add_argument('-n', '--number', type=int,
+ help="How often an action is executed for each process.")
parser.add_argument('tests', help="Name of the file with test description.")
-main(parser.parse_args())
+
+if __name__ == "__main__":
+ sys.exit(main(parser.parse_args()))
diff --git a/tempest/stress/stressaction.py b/tempest/stress/stressaction.py
new file mode 100644
index 0000000..77ddd1c
--- /dev/null
+++ b/tempest/stress/stressaction.py
@@ -0,0 +1,69 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# (c) Copyright 2013 Hewlett-Packard Development Company, L.P.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+import signal
+import sys
+
+
+class StressAction(object):
+
+ def __init__(self, manager, logger, max_runs=None):
+ self.manager = manager
+ self.logger = logger
+ self.max_runs = max_runs
+
+ def _shutdown_handler(self, signal, frame):
+ self.tearDown()
+ sys.exit(0)
+
+ def setUp(self, **kwargs):
+ """This method is called before the run method
+ to help the test initiatlize any structures.
+ kwargs contains arguments passed in from the
+ configuration json file.
+
+ setUp doesn't count against the time duration.
+ """
+ self.logger.debug("setUp")
+
+ def tearDown(self):
+ """This method is called to do any cleanup
+ after the test is complete.
+ """
+ self.logger.debug("tearDown")
+
+ def execute(self, shared_statistic):
+ """This is the main execution entry point called
+ by the driver. We register a signal handler to
+ allow us to gracefull tearDown, and then exit.
+ We also keep track of how many runs we do.
+ """
+ signal.signal(signal.SIGHUP, self._shutdown_handler)
+ signal.signal(signal.SIGTERM, self._shutdown_handler)
+
+ while self.max_runs is None or (shared_statistic['runs'] <
+ self.max_runs):
+ try:
+ self.run()
+ except Exception:
+ shared_statistic['fails'] += 1
+ self.logger.exception("Failure in run")
+ finally:
+ shared_statistic['runs'] += 1
+
+ def run(self):
+ """This method is where the stress test code runs."""
+ raise NotImplemented()
diff --git a/tempest/stress/tools/cleanup.py b/tempest/stress/tools/cleanup.py
index 7139d6c..b6a26cd 100755
--- a/tempest/stress/tools/cleanup.py
+++ b/tempest/stress/tools/cleanup.py
@@ -14,7 +14,15 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import logging
+
from tempest.stress import cleanup
+_console = logging.StreamHandler()
+_console.setLevel(logging.DEBUG)
+# add the handler to the root logger
+logger = logging.getLogger('tempest.stress.cleanup')
+logger.addHandler(logging.StreamHandler())
+logger.setLevel(logging.DEBUG)
-cleanup.cleanup()
+cleanup.cleanup(logger)
diff --git a/tempest/test.py b/tempest/test.py
index d7008a7..5040f34 100644
--- a/tempest/test.py
+++ b/tempest/test.py
@@ -15,14 +15,18 @@
# License for the specific language governing permissions and limitations
# under the License.
+import os
import time
import nose.plugins.attrib
import testresources
import testtools
+from tempest import clients
from tempest.common import log as logging
+from tempest.common.utils.data_utils import rand_name
from tempest import config
+from tempest import exceptions
from tempest import manager
LOG = logging.getLogger(__name__)
@@ -54,6 +58,42 @@
return decorator
+# there is a mis-match between nose and testtools for older pythons.
+# testtools will set skipException to be either
+# unittest.case.SkipTest, unittest2.case.SkipTest or an internal skip
+# exception, depending on what it can find. Python <2.7 doesn't have
+# unittest.case.SkipTest; so if unittest2 is not installed it falls
+# back to the internal class.
+#
+# The current nose skip plugin will decide to raise either
+# unittest.case.SkipTest or its own internal exception; it does not
+# look for unittest2 or the internal unittest exception. Thus we must
+# monkey-patch testtools.TestCase.skipException to be the exception
+# the nose skip plugin expects.
+#
+# However, with the switch to testr nose may not be available, so we
+# require you to opt-in to this fix with an environment variable.
+#
+# This is temporary until upstream nose starts looking for unittest2
+# as testtools does; we can then remove this and ensure unittest2 is
+# available for older pythons; then nose and testtools will agree
+# unittest2.case.SkipTest is the one-true skip test exception.
+#
+# https://review.openstack.org/#/c/33056
+# https://github.com/nose-devs/nose/pull/699
+if 'TEMPEST_PY26_NOSE_COMPAT' in os.environ:
+ try:
+ import unittest.case.SkipTest
+ # convince pep8 we're using the import...
+ if unittest.case.SkipTest:
+ pass
+ raise RuntimeError("You have unittest.case.SkipTest; "
+ "no need to override")
+ except ImportError:
+ LOG.info("Overriding skipException to nose SkipTest")
+ testtools.TestCase.skipException = nose.plugins.skip.SkipTest
+
+
class BaseTestCase(testtools.TestCase,
testtools.testcase.WithAttributes,
testresources.ResourcedTestCase):
@@ -65,6 +105,104 @@
if hasattr(super(BaseTestCase, cls), 'setUpClass'):
super(BaseTestCase, cls).setUpClass()
+ @classmethod
+ def _get_identity_admin_client(cls):
+ """
+ Returns an instance of the Identity Admin API client
+ """
+ os = clients.AdminManager(interface=cls._interface)
+ admin_client = os.identity_client
+ return admin_client
+
+ @classmethod
+ def _get_client_args(cls):
+
+ return (
+ cls.config,
+ cls.config.identity.admin_username,
+ cls.config.identity.admin_password,
+ cls.config.identity.uri
+ )
+
+ @classmethod
+ def _get_isolated_creds(cls, admin=False):
+ """
+ Creates a new set of user/tenant/password credentials for a
+ **regular** user of the Compute API so that a test case can
+ operate in an isolated tenant container.
+ """
+ admin_client = cls._get_identity_admin_client()
+ password = "pass"
+
+ while True:
+ try:
+ rand_name_root = rand_name(cls.__name__)
+ if cls.isolated_creds:
+ # Main user already created. Create the alt or admin one...
+ if admin:
+ rand_name_root += '-admin'
+ else:
+ rand_name_root += '-alt'
+ tenant_name = rand_name_root + "-tenant"
+ tenant_desc = tenant_name + "-desc"
+
+ resp, tenant = admin_client.create_tenant(
+ name=tenant_name, description=tenant_desc)
+ break
+ except exceptions.Duplicate:
+ if cls.config.compute.allow_tenant_reuse:
+ tenant = admin_client.get_tenant_by_name(tenant_name)
+ LOG.info('Re-using existing tenant %s', tenant)
+ break
+
+ while True:
+ try:
+ rand_name_root = rand_name(cls.__name__)
+ if cls.isolated_creds:
+ # Main user already created. Create the alt one...
+ rand_name_root += '-alt'
+ username = rand_name_root + "-user"
+ email = rand_name_root + "@example.com"
+ resp, user = admin_client.create_user(username,
+ password,
+ tenant['id'],
+ email)
+ break
+ except exceptions.Duplicate:
+ if cls.config.compute.allow_tenant_reuse:
+ user = admin_client.get_user_by_username(tenant['id'],
+ username)
+ LOG.info('Re-using existing user %s', user)
+ break
+ # Store the complete creds (including UUID ids...) for later
+ # but return just the username, tenant_name, password tuple
+ # that the various clients will use.
+ cls.isolated_creds.append((user, tenant))
+
+ # Assign admin role if this is for admin creds
+ if admin:
+ _, roles = admin_client.list_roles()
+ role = None
+ try:
+ _, roles = admin_client.list_roles()
+ role = next(r for r in roles if r['name'] == 'admin')
+ except StopIteration:
+ msg = "No admin role found"
+ raise exceptions.NotFound(msg)
+ admin_client.assign_user_role(tenant['id'], user['id'], role['id'])
+
+ return username, tenant_name, password
+
+ @classmethod
+ def _clear_isolated_creds(cls):
+ if not cls.isolated_creds:
+ return
+ admin_client = cls._get_identity_admin_client()
+
+ for user, tenant in cls.isolated_creds:
+ admin_client.delete_user(user['id'])
+ admin_client.delete_tenant(tenant['id'])
+
def call_until_true(func, duration, sleep_for):
"""
diff --git a/tempest/thirdparty/README.rst b/tempest/thirdparty/README.rst
index 41d31f3..b775817 100644
--- a/tempest/thirdparty/README.rst
+++ b/tempest/thirdparty/README.rst
@@ -1,9 +1,9 @@
Tempest Guide to Third Party API tests
-========
+======================================
What are these tests?
---------
+---------------------
Third party tests are tests for non native OpenStack APIs that are
part of OpenStack projects. If we ship an API, we're really required
@@ -14,14 +14,14 @@
Why are these tests in tempest?
---------
+-------------------------------
If we ship an API in an OpenStack component, there should be tests in
tempest to exercise it in some way.
Scope of these tests
---------
+--------------------
Third party API testing should be limited to the functional testing of
third party API compliance. Complex scenarios should be avoided, and
diff --git a/tempest/thirdparty/boto/test_ec2_instance_run.py b/tempest/thirdparty/boto/test_ec2_instance_run.py
index 89891d2..1201866 100644
--- a/tempest/thirdparty/boto/test_ec2_instance_run.py
+++ b/tempest/thirdparty/boto/test_ec2_instance_run.py
@@ -113,6 +113,53 @@
self.cancelResourceCleanUp(rcuk)
@attr(type='smoke')
+ def test_run_stop_terminate_instance_with_tags(self):
+ # EC2 run, stop and terminate instance with tags
+ image_ami = self.ec2_client.get_image(self.images["ami"]
+ ["image_id"])
+ reservation = image_ami.run(kernel_id=self.images["aki"]["image_id"],
+ ramdisk_id=self.images["ari"]["image_id"],
+ instance_type=self.instance_type)
+ rcuk = self.addResourceCleanUp(self.destroy_reservation, reservation)
+
+ for instance in reservation.instances:
+ LOG.info("state: %s", instance.state)
+ if instance.state != "running":
+ self.assertInstanceStateWait(instance, "running")
+ instance.add_tag('key1', value='value1')
+
+ tags = self.ec2_client.get_all_tags()
+ self.assertEquals(tags[0].name, 'key1')
+ self.assertEquals(tags[0].value, 'value1')
+
+ tags = self.ec2_client.get_all_tags(filters={'key': 'key1'})
+ self.assertEquals(tags[0].name, 'key1')
+ self.assertEquals(tags[0].value, 'value1')
+
+ tags = self.ec2_client.get_all_tags(filters={'value': 'value1'})
+ self.assertEquals(tags[0].name, 'key1')
+ self.assertEquals(tags[0].value, 'value1')
+
+ tags = self.ec2_client.get_all_tags(filters={'key': 'value2'})
+ self.assertEquals(len(tags), 0)
+
+ for instance in reservation.instances:
+ instance.remove_tag('key1', value='value1')
+
+ tags = self.ec2_client.get_all_tags()
+ self.assertEquals(len(tags), 0)
+
+ for instance in reservation.instances:
+ instance.stop()
+ LOG.info("state: %s", instance.state)
+ if instance.state != "stopped":
+ self.assertInstanceStateWait(instance, "stopped")
+
+ for instance in reservation.instances:
+ instance.terminate()
+ self.cancelResourceCleanUp(rcuk)
+
+ @attr(type='smoke')
@testtools.skip("Skipped until the Bug #1098891 is resolved")
def test_run_terminate_instance(self):
# EC2 run, terminate immediately
diff --git a/test-requirements.txt b/test-requirements.txt
index 2185997..236a473 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -2,7 +2,7 @@
pep8==1.4.5
pyflakes==0.7.2
flake8==2.0
-hacking>=0.5.3,<0.6
+hacking>=0.5.6,<0.7
# needed for doc build
+docutils==0.9.1
sphinx>=1.1.2
-
diff --git a/tools/colorizer.py b/tools/colorizer.py
new file mode 100755
index 0000000..76a3bd3
--- /dev/null
+++ b/tools/colorizer.py
@@ -0,0 +1,333 @@
+#!/usr/bin/env python
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright (c) 2013, Nebula, Inc.
+# Copyright 2010 United States Government as represented by the
+# Administrator of the National Aeronautics and Space Administration.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# Colorizer Code is borrowed from Twisted:
+# Copyright (c) 2001-2010 Twisted Matrix Laboratories.
+#
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be
+# included in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+"""Display a subunit stream through a colorized unittest test runner."""
+
+import heapq
+import subunit
+import sys
+import unittest
+
+import testtools
+
+
+class _AnsiColorizer(object):
+ """
+ A colorizer is an object that loosely wraps around a stream, allowing
+ callers to write text to the stream in a particular color.
+
+ Colorizer classes must implement C{supported()} and C{write(text, color)}.
+ """
+ _colors = dict(black=30, red=31, green=32, yellow=33,
+ blue=34, magenta=35, cyan=36, white=37)
+
+ def __init__(self, stream):
+ self.stream = stream
+
+ def supported(cls, stream=sys.stdout):
+ """
+ A class method that returns True if the current platform supports
+ coloring terminal output using this method. Returns False otherwise.
+ """
+ if not stream.isatty():
+ return False # auto color only on TTYs
+ try:
+ import curses
+ except ImportError:
+ return False
+ else:
+ try:
+ try:
+ return curses.tigetnum("colors") > 2
+ except curses.error:
+ curses.setupterm()
+ return curses.tigetnum("colors") > 2
+ except Exception:
+ # guess false in case of error
+ return False
+ supported = classmethod(supported)
+
+ def write(self, text, color):
+ """
+ Write the given text to the stream in the given color.
+
+ @param text: Text to be written to the stream.
+
+ @param color: A string label for a color. e.g. 'red', 'white'.
+ """
+ color = self._colors[color]
+ self.stream.write('\x1b[%s;1m%s\x1b[0m' % (color, text))
+
+
+class _Win32Colorizer(object):
+ """
+ See _AnsiColorizer docstring.
+ """
+ def __init__(self, stream):
+ import win32console
+ red, green, blue, bold = (win32console.FOREGROUND_RED,
+ win32console.FOREGROUND_GREEN,
+ win32console.FOREGROUND_BLUE,
+ win32console.FOREGROUND_INTENSITY)
+ self.stream = stream
+ self.screenBuffer = win32console.GetStdHandle(
+ win32console.STD_OUT_HANDLE)
+ self._colors = {'normal': red | green | blue,
+ 'red': red | bold,
+ 'green': green | bold,
+ 'blue': blue | bold,
+ 'yellow': red | green | bold,
+ 'magenta': red | blue | bold,
+ 'cyan': green | blue | bold,
+ 'white': red | green | blue | bold}
+
+ def supported(cls, stream=sys.stdout):
+ try:
+ import win32console
+ screenBuffer = win32console.GetStdHandle(
+ win32console.STD_OUT_HANDLE)
+ except ImportError:
+ return False
+ import pywintypes
+ try:
+ screenBuffer.SetConsoleTextAttribute(
+ win32console.FOREGROUND_RED |
+ win32console.FOREGROUND_GREEN |
+ win32console.FOREGROUND_BLUE)
+ except pywintypes.error:
+ return False
+ else:
+ return True
+ supported = classmethod(supported)
+
+ def write(self, text, color):
+ color = self._colors[color]
+ self.screenBuffer.SetConsoleTextAttribute(color)
+ self.stream.write(text)
+ self.screenBuffer.SetConsoleTextAttribute(self._colors['normal'])
+
+
+class _NullColorizer(object):
+ """
+ See _AnsiColorizer docstring.
+ """
+ def __init__(self, stream):
+ self.stream = stream
+
+ def supported(cls, stream=sys.stdout):
+ return True
+ supported = classmethod(supported)
+
+ def write(self, text, color):
+ self.stream.write(text)
+
+
+def get_elapsed_time_color(elapsed_time):
+ if elapsed_time > 1.0:
+ return 'red'
+ elif elapsed_time > 0.25:
+ return 'yellow'
+ else:
+ return 'green'
+
+
+class NovaTestResult(testtools.TestResult):
+ def __init__(self, stream, descriptions, verbosity):
+ super(NovaTestResult, self).__init__()
+ self.stream = stream
+ self.showAll = verbosity > 1
+ self.num_slow_tests = 10
+ self.slow_tests = [] # this is a fixed-sized heap
+ self.colorizer = None
+ # NOTE(vish): reset stdout for the terminal check
+ stdout = sys.stdout
+ sys.stdout = sys.__stdout__
+ for colorizer in [_Win32Colorizer, _AnsiColorizer, _NullColorizer]:
+ if colorizer.supported():
+ self.colorizer = colorizer(self.stream)
+ break
+ sys.stdout = stdout
+ self.start_time = None
+ self.last_time = {}
+ self.results = {}
+ self.last_written = None
+
+ def _writeElapsedTime(self, elapsed):
+ color = get_elapsed_time_color(elapsed)
+ self.colorizer.write(" %.2f" % elapsed, color)
+
+ def _addResult(self, test, *args):
+ try:
+ name = test.id()
+ except AttributeError:
+ name = 'Unknown.unknown'
+ test_class, test_name = name.rsplit('.', 1)
+
+ elapsed = (self._now() - self.start_time).total_seconds()
+ item = (elapsed, test_class, test_name)
+ if len(self.slow_tests) >= self.num_slow_tests:
+ heapq.heappushpop(self.slow_tests, item)
+ else:
+ heapq.heappush(self.slow_tests, item)
+
+ self.results.setdefault(test_class, [])
+ self.results[test_class].append((test_name, elapsed) + args)
+ self.last_time[test_class] = self._now()
+ self.writeTests()
+
+ def _writeResult(self, test_name, elapsed, long_result, color,
+ short_result, success):
+ if self.showAll:
+ self.stream.write(' %s' % str(test_name).ljust(66))
+ self.colorizer.write(long_result, color)
+ if success:
+ self._writeElapsedTime(elapsed)
+ self.stream.writeln()
+ else:
+ self.colorizer.write(short_result, color)
+
+ def addSuccess(self, test):
+ super(NovaTestResult, self).addSuccess(test)
+ self._addResult(test, 'OK', 'green', '.', True)
+
+ def addFailure(self, test, err):
+ if test.id() == 'process-returncode':
+ return
+ super(NovaTestResult, self).addFailure(test, err)
+ self._addResult(test, 'FAIL', 'red', 'F', False)
+
+ def addError(self, test, err):
+ super(NovaTestResult, self).addFailure(test, err)
+ self._addResult(test, 'ERROR', 'red', 'E', False)
+
+ def addSkip(self, test, reason=None, details=None):
+ super(NovaTestResult, self).addSkip(test, reason, details)
+ self._addResult(test, 'SKIP', 'blue', 'S', True)
+
+ def startTest(self, test):
+ self.start_time = self._now()
+ super(NovaTestResult, self).startTest(test)
+
+ def writeTestCase(self, cls):
+ if not self.results.get(cls):
+ return
+ if cls != self.last_written:
+ self.colorizer.write(cls, 'white')
+ self.stream.writeln()
+ for result in self.results[cls]:
+ self._writeResult(*result)
+ del self.results[cls]
+ self.stream.flush()
+ self.last_written = cls
+
+ def writeTests(self):
+ time = self.last_time.get(self.last_written, self._now())
+ if not self.last_written or (self._now() - time).total_seconds() > 2.0:
+ diff = 3.0
+ while diff > 2.0:
+ classes = self.results.keys()
+ oldest = min(classes, key=lambda x: self.last_time[x])
+ diff = (self._now() - self.last_time[oldest]).total_seconds()
+ self.writeTestCase(oldest)
+ else:
+ self.writeTestCase(self.last_written)
+
+ def done(self):
+ self.stopTestRun()
+
+ def stopTestRun(self):
+ for cls in list(self.results.iterkeys()):
+ self.writeTestCase(cls)
+ self.stream.writeln()
+ self.writeSlowTests()
+
+ def writeSlowTests(self):
+ # Pare out 'fast' tests
+ slow_tests = [item for item in self.slow_tests
+ if get_elapsed_time_color(item[0]) != 'green']
+ if slow_tests:
+ slow_total_time = sum(item[0] for item in slow_tests)
+ slow = ("Slowest %i tests took %.2f secs:"
+ % (len(slow_tests), slow_total_time))
+ self.colorizer.write(slow, 'yellow')
+ self.stream.writeln()
+ last_cls = None
+ # sort by name
+ for elapsed, cls, name in sorted(slow_tests,
+ key=lambda x: x[1] + x[2]):
+ if cls != last_cls:
+ self.colorizer.write(cls, 'white')
+ self.stream.writeln()
+ last_cls = cls
+ self.stream.write(' %s' % str(name).ljust(68))
+ self._writeElapsedTime(elapsed)
+ self.stream.writeln()
+
+ def printErrors(self):
+ if self.showAll:
+ self.stream.writeln()
+ self.printErrorList('ERROR', self.errors)
+ self.printErrorList('FAIL', self.failures)
+
+ def printErrorList(self, flavor, errors):
+ for test, err in errors:
+ self.colorizer.write("=" * 70, 'red')
+ self.stream.writeln()
+ self.colorizer.write(flavor, 'red')
+ self.stream.writeln(": %s" % test.id())
+ self.colorizer.write("-" * 70, 'red')
+ self.stream.writeln()
+ self.stream.writeln("%s" % err)
+
+
+test = subunit.ProtocolTestCase(sys.stdin, passthrough=None)
+
+if sys.version_info[0:2] <= (2, 6):
+ runner = unittest.TextTestRunner(verbosity=2)
+else:
+ runner = unittest.TextTestRunner(verbosity=2, resultclass=NovaTestResult)
+
+if runner.run(test).wasSuccessful():
+ exit_code = 0
+else:
+ exit_code = 1
+sys.exit(exit_code)
diff --git a/tools/pretty_tox.sh b/tools/pretty_tox.sh
new file mode 100755
index 0000000..a5a6076
--- /dev/null
+++ b/tools/pretty_tox.sh
@@ -0,0 +1,4 @@
+#!/bin/sh
+
+TESTRARGS=$1
+python setup.py testr --slowest --testr-args="--subunit $TESTRARGS" | subunit2pyunit
diff --git a/tox.ini b/tox.ini
index 964dbca..eb1ef4b 100644
--- a/tox.ini
+++ b/tox.ini
@@ -3,24 +3,16 @@
[testenv]
setenv = VIRTUAL_ENV={envdir}
- NOSE_WITH_OPENSTACK=1
- NOSE_OPENSTACK_COLOR=1
- NOSE_OPENSTACK_RED=15
- NOSE_OPENSTACK_YELLOW=3
- NOSE_OPENSTACK_SHOW_ELAPSED=1
- NOSE_OPENSTACK_STDOUT=1
+ LANG=en_US.UTF-8
+ LANGUAGE=en_US:en
+ LC_ALL=C
[testenv:all]
sitepackages = True
setenv = VIRTUAL_ENV={envdir}
- NOSE_WITH_OPENSTACK=1
- NOSE_OPENSTACK_COLOR=1
- NOSE_OPENSTACK_RED=15
- NOSE_OPENSTACK_YELLOW=3
- NOSE_OPENSTACK_SHOW_ELAPSED=1
- NOSE_OPENSTACK_STDOUT=1
commands =
- nosetests --logging-format '%(asctime)-15s %(message)s' --with-xunit --xunit-file=nosetests-all.xml -sv tempest
+ python setup.py testr --slowest
+
[testenv:full]
sitepackages = True
@@ -34,6 +26,12 @@
commands =
nosetests --logging-format '%(asctime)-15s %(message)s' --with-xunit --xunit-file=nosetests-full.xml -sv tempest/api tempest/scenario tempest/thirdparty tempest/cli
+[testenv:testr-full]
+sitepackages = True
+setenv = VIRTUAL_ENV={envdir}
+commands =
+ sh tools/pretty_tox.sh 'tempest.api tempest.scenario tempest.thirdparty tempest.cli'
+
[testenv:smoke]
sitepackages = True
setenv = VIRTUAL_ENV={envdir}
@@ -46,7 +44,6 @@
commands =
nosetests --logging-format '%(asctime)-15s %(message)s' --with-xunit -sv --attr=type=smoke --xunit-file=nosetests-smoke.xml tempest
-
[testenv:coverage]
sitepackages = True
setenv = VIRTUAL_ENV={envdir}
@@ -65,9 +62,7 @@
sitepackages = True
setenv = VIRTUAL_ENV={envdir}
commands =
- python -m tempest/stress/run_stress tempest/stress/etc/sample-test.json -d 60
- python -m tempest/stress/run_stress tempest/stress/etc/volume-create-delete-test.json -d 60
-
+ python -m tempest/stress/run_stress tempest/stress/etc/stress-tox-job.json -d 3600
[testenv:venv]
commands = {posargs}
@@ -83,6 +78,7 @@
local-check-factory = tempest.hacking.checks.factory
[flake8]
+# E125 is a won't fix until https://github.com/jcrocholl/pep8/issues/126 is resolved. For further detail see https://review.openstack.org/#/c/36788/
ignore = E125,H302,H404
show-source = True
exclude = .git,.venv,.tox,dist,doc,openstack,*egg