Merge "Basic starter scenario for testing the dashboard"
diff --git a/HACKING.rst b/HACKING.rst
index 2ac766e..1eb2d4f 100644
--- a/HACKING.rst
+++ b/HACKING.rst
@@ -32,7 +32,7 @@
 Exception Handling
 ------------------
 According to the ``The Zen of Python`` the
- ``Errors should never pass silently.``
+``Errors should never pass silently.``
 Tempest usually runs in special environment (jenkins gate jobs), in every
 error or failure situation we should provide as much error related
 information as possible, because we usually do not have the chance to
@@ -57,6 +57,10 @@
 exception at least logged.  When the exception is logged you usually need
 to ``raise`` the same or a different exception anyway.
 
+Use of ``self.addCleanup`` is often a good way to avoid having to catch
+exceptions and still ensure resources are correctly cleaned up if the
+test fails part way through.
+
 Use the ``self.assert*`` methods provided by the unit test framework
  the signal failures early.
 
@@ -74,5 +78,10 @@
 This and the service logs are your only guide to find the root cause of flaky
 issue.
 
-
-
+Guidelines
+----------
+- Do not submit changesets with only testcases which are skipped as
+  they will not be merged.
+- Consistently check the status code of responses in testcases. The
+  earlier a problem is detected the easier it is to debug, especially
+  where there is complicated setup required.
diff --git a/README.rst b/README.rst
index da0f5f3..f18628a 100644
--- a/README.rst
+++ b/README.rst
@@ -1,5 +1,3 @@
-::
-
 Tempest - The OpenStack Integration Test Suite
 ==============================================
 
@@ -37,9 +35,11 @@
 Tempest is not tied to any single test runner, but Nose been the most commonly
 used tool. After setting up your configuration file, you can execute
 the set of Tempest tests by using ``nosetests`` ::
+
     $> nosetests tempest
 
 To run one single test  ::
+
     $> nosetests -sv tempest.api.compute.servers.test_server_actions.py:
        ServerActionsTestJSON.test_rebuild_nonexistent_server
 
diff --git a/etc/tempest.conf.sample b/etc/tempest.conf.sample
index 8b9bb9c..033bc82 100644
--- a/etc/tempest.conf.sample
+++ b/etc/tempest.conf.sample
@@ -91,6 +91,9 @@
 # IP version of the address used for SSH
 ip_version_for_ssh = 4
 
+# Number of seconds to wait to ping to an instance
+ping_timeout = 60
+
 # Number of seconds to wait to authenticate to an instance
 ssh_timeout = 300
 
diff --git a/tempest/README.rst b/tempest/README.rst
index 8f07a07..33021c8 100644
--- a/tempest/README.rst
+++ b/tempest/README.rst
@@ -1,6 +1,6 @@
-============
+============================
 Tempest Field Guide Overview
-============
+============================
 
 Tempest is designed to be useful for a large number of different
 environments. This includes being useful for gating commits to
@@ -26,7 +26,7 @@
 
 
 api
-------------
+---
 
 API tests are validation tests for the OpenStack API. They should not
 use the existing python clients for OpenStack, but should instead use
@@ -41,7 +41,7 @@
 
 
 cli
-------------
+---
 
 CLI tests use the openstack CLI to interact with the OpenStack
 cloud. CLI testing in unit tests is somewhat difficult because unlike
@@ -51,7 +51,7 @@
 
 
 scenario
-------------
+--------
 
 Scenario tests are complex "through path" tests for OpenStack
 functionality. They are typically a series of steps where complicated
@@ -61,7 +61,7 @@
 
 
 stress
------------
+------
 
 Stress tests are designed to stress an OpenStack environment by
 running a high workload against it and seeing what breaks. Tools may
@@ -72,7 +72,7 @@
 
 
 thirdparty
-------------
+----------
 
 Many openstack components include 3rdparty API support. It is
 completely legitimate for Tempest to include tests of 3rdparty APIs,
@@ -81,7 +81,7 @@
 
 
 whitebox
-----------
+--------
 
 Whitebox tests are tests which require access to the database of the
 target OpenStack machine to verify internal state after operations
diff --git a/tempest/api/compute/base.py b/tempest/api/compute/base.py
index 095be7c..8ba074e 100644
--- a/tempest/api/compute/base.py
+++ b/tempest/api/compute/base.py
@@ -22,7 +22,6 @@
 from tempest.common import log as logging
 from tempest.common.utils.data_utils import parse_image_id
 from tempest.common.utils.data_utils import rand_name
-from tempest import exceptions
 import tempest.test
 
 
@@ -82,89 +81,6 @@
         cls.servers_client_v3_auth = os.servers_client_v3_auth
 
     @classmethod
-    def _get_identity_admin_client(cls):
-        """
-        Returns an instance of the Identity Admin API client
-        """
-        os = clients.AdminManager(interface=cls._interface)
-        admin_client = os.identity_client
-        return admin_client
-
-    @classmethod
-    def _get_client_args(cls):
-
-        return (
-            cls.config,
-            cls.config.identity.admin_username,
-            cls.config.identity.admin_password,
-            cls.config.identity.uri
-        )
-
-    @classmethod
-    def _get_isolated_creds(cls):
-        """
-        Creates a new set of user/tenant/password credentials for a
-        **regular** user of the Compute API so that a test case can
-        operate in an isolated tenant container.
-        """
-        admin_client = cls._get_identity_admin_client()
-        password = "pass"
-
-        while True:
-            try:
-                rand_name_root = rand_name(cls.__name__)
-                if cls.isolated_creds:
-                # Main user already created. Create the alt one...
-                    rand_name_root += '-alt'
-                tenant_name = rand_name_root + "-tenant"
-                tenant_desc = tenant_name + "-desc"
-
-                resp, tenant = admin_client.create_tenant(
-                    name=tenant_name, description=tenant_desc)
-                break
-            except exceptions.Duplicate:
-                if cls.config.compute.allow_tenant_reuse:
-                    tenant = admin_client.get_tenant_by_name(tenant_name)
-                    LOG.info('Re-using existing tenant %s', tenant)
-                    break
-
-        while True:
-            try:
-                rand_name_root = rand_name(cls.__name__)
-                if cls.isolated_creds:
-                # Main user already created. Create the alt one...
-                    rand_name_root += '-alt'
-                username = rand_name_root + "-user"
-                email = rand_name_root + "@example.com"
-                resp, user = admin_client.create_user(username,
-                                                      password,
-                                                      tenant['id'],
-                                                      email)
-                break
-            except exceptions.Duplicate:
-                if cls.config.compute.allow_tenant_reuse:
-                    user = admin_client.get_user_by_username(tenant['id'],
-                                                             username)
-                    LOG.info('Re-using existing user %s', user)
-                    break
-        # Store the complete creds (including UUID ids...) for later
-        # but return just the username, tenant_name, password tuple
-        # that the various clients will use.
-        cls.isolated_creds.append((user, tenant))
-
-        return username, tenant_name, password
-
-    @classmethod
-    def clear_isolated_creds(cls):
-        if not cls.isolated_creds:
-            return
-        admin_client = cls._get_identity_admin_client()
-
-        for user, tenant in cls.isolated_creds:
-            admin_client.delete_user(user['id'])
-            admin_client.delete_tenant(tenant['id'])
-
-    @classmethod
     def clear_servers(cls):
         for server in cls.servers:
             try:
@@ -192,7 +108,7 @@
     def tearDownClass(cls):
         cls.clear_images()
         cls.clear_servers()
-        cls.clear_isolated_creds()
+        cls._clear_isolated_creds()
 
     @classmethod
     def create_server(cls, **kwargs):
@@ -266,10 +182,16 @@
         admin_username = cls.config.compute_admin.username
         admin_password = cls.config.compute_admin.password
         admin_tenant = cls.config.compute_admin.tenant_name
-
         if not (admin_username and admin_password and admin_tenant):
             msg = ("Missing Compute Admin API credentials "
                    "in configuration.")
             raise cls.skipException(msg)
-
-        cls.os_adm = clients.ComputeAdminManager(interface=cls._interface)
+        if cls.config.compute.allow_tenant_isolation:
+            creds = cls._get_isolated_creds(admin=True)
+            admin_username, admin_tenant_name, admin_password = creds
+            cls.os_adm = clients.Manager(username=admin_username,
+                                         password=admin_password,
+                                         tenant_name=admin_tenant_name,
+                                         interface=cls._interface)
+        else:
+            cls.os_adm = clients.ComputeAdminManager(interface=cls._interface)
diff --git a/tempest/api/compute/images/test_images.py b/tempest/api/compute/images/test_images.py
index 95ea820..4f9364b 100644
--- a/tempest/api/compute/images/test_images.py
+++ b/tempest/api/compute/images/test_images.py
@@ -1,7 +1,6 @@
 # vim: tabstop=4 shiftwidth=4 softtabstop=4
 
 # Copyright 2012 OpenStack, LLC
-# All Rights Reserved.
 #
 #    Licensed under the Apache License, Version 2.0 (the "License"); you may
 #    not use this file except in compliance with the License. You may obtain
@@ -92,6 +91,31 @@
                           '!@#$%^&*()', name, meta)
 
     @attr(type=['negative', 'gate'])
+    def test_create_image_from_stopped_server(self):
+        resp, server = self.create_server(wait_until='ACTIVE')
+        self.servers_client.stop(server['id'])
+        self.servers_client.wait_for_server_status(server['id'],
+                                                   'SHUTOFF')
+        self.addCleanup(self.servers_client.delete_server, server['id'])
+        snapshot_name = rand_name('test-snap-')
+        resp, image = self.create_image_from_server(server['id'],
+                                                    name=snapshot_name,
+                                                    wait_until='ACTIVE')
+        self.addCleanup(self.client.delete_image, image['id'])
+        self.assertEqual(snapshot_name, image['name'])
+
+    @attr(type='gate')
+    def test_delete_saving_image(self):
+        snapshot_name = rand_name('test-snap-')
+        resp, server = self.create_server(wait_until='ACTIVE')
+        self.addCleanup(self.servers_client.delete_server, server['id'])
+        resp, image = self.create_image_from_server(server['id'],
+                                                    name=snapshot_name,
+                                                    wait_until='SAVING')
+        resp, body = self.client.delete_image(image['id'])
+        self.assertEqual('204', resp['status'])
+
+    @attr(type=['negative', 'gate'])
     def test_create_image_specify_uuid_35_characters_or_less(self):
         # Return an error if Image ID passed is 35 characters or less
         snapshot_name = rand_name('test-snap-')
diff --git a/tempest/api/compute/servers/test_server_rescue.py b/tempest/api/compute/servers/test_server_rescue.py
index 8225a4c..13c2f74 100644
--- a/tempest/api/compute/servers/test_server_rescue.py
+++ b/tempest/api/compute/servers/test_server_rescue.py
@@ -126,6 +126,13 @@
                           self.rescue_id, 'HARD')
 
     @attr(type=['negative', 'gate'])
+    def test_rescue_non_existent_server(self):
+        # Rescue a non-existing server
+        self.assertRaises(exceptions.NotFound,
+                          self.servers_client.rescue_server,
+                          '999erra43')
+
+    @attr(type=['negative', 'gate'])
     def test_rescued_vm_rebuild(self):
         self.assertRaises(exceptions.Duplicate,
                           self.servers_client.rebuild,
diff --git a/tempest/api/compute/servers/test_servers_negative.py b/tempest/api/compute/servers/test_servers_negative.py
index 5cc8dc6..af58b5f 100644
--- a/tempest/api/compute/servers/test_servers_negative.py
+++ b/tempest/api/compute/servers/test_servers_negative.py
@@ -99,6 +99,17 @@
                           self.server_id, 'SOFT')
 
     @attr(type=['negative', 'gate'])
+    def test_pause_paused_server(self):
+        # Pause a paused server.
+        resp, server = self.create_server(wait_until='ACTIVE')
+        self.server_id = server['id']
+        self.client.pause_server(self.server_id)
+        self.client.wait_for_server_status(self.server_id, 'PAUSED')
+        self.assertRaises(exceptions.Duplicate,
+                          self.client.pause_server,
+                          self.server_id)
+
+    @attr(type=['negative', 'gate'])
     def test_rebuild_deleted_server(self):
         # Rebuild a deleted server
 
diff --git a/tempest/api/utils.py b/tempest/api/utils.py
index 0738201..69ab7fb 100644
--- a/tempest/api/utils.py
+++ b/tempest/api/utils.py
@@ -17,7 +17,7 @@
 
 """Common utilities used in testing."""
 
-from testtools import TestCase
+from tempest.test import BaseTestCase
 
 
 class skip_unless_attr(object):
@@ -32,7 +32,7 @@
             """Wrapped skipper function."""
             testobj = args[0]
             if not getattr(testobj, self.attr, False):
-                raise TestCase.skipException(self.message)
+                raise BaseTestCase.skipException(self.message)
             func(*args, **kw)
         _skipper.__name__ = func.__name__
         _skipper.__doc__ = func.__doc__
diff --git a/tempest/api/volume/base.py b/tempest/api/volume/base.py
index 5770d28..a84f9e8 100644
--- a/tempest/api/volume/base.py
+++ b/tempest/api/volume/base.py
@@ -19,7 +19,6 @@
 
 from tempest import clients
 from tempest.common import log as logging
-from tempest.common.utils.data_utils import rand_name
 import tempest.test
 
 LOG = logging.getLogger(__name__)
@@ -65,59 +64,10 @@
                                          cls.os.tenant_name)
 
     @classmethod
-    def _get_identity_admin_client(cls):
-        """
-        Returns an instance of the Identity Admin API client
-        """
-        os = clients.ComputeAdminManager()
-        return os.identity_client
-
-    @classmethod
-    def _get_isolated_creds(cls):
-        """
-        Creates a new set of user/tenant/password credentials for a
-        **regular** user of the Volume API so that a test case can
-        operate in an isolated tenant container.
-        """
-        admin_client = cls._get_identity_admin_client()
-        rand_name_root = rand_name(cls.__name__)
-        if cls.isolated_creds:
-            # Main user already created. Create the alt one...
-            rand_name_root += '-alt'
-        username = rand_name_root + "-user"
-        email = rand_name_root + "@example.com"
-        tenant_name = rand_name_root + "-tenant"
-        tenant_desc = tenant_name + "-desc"
-        password = "pass"
-
-        resp, tenant = admin_client.create_tenant(name=tenant_name,
-                                                  description=tenant_desc)
-        resp, user = admin_client.create_user(username,
-                                              password,
-                                              tenant['id'],
-                                              email)
-        # Store the complete creds (including UUID ids...) for later
-        # but return just the username, tenant_name, password tuple
-        # that the various clients will use.
-        cls.isolated_creds.append((user, tenant))
-
-        return username, tenant_name, password
-
-    @classmethod
-    def clear_isolated_creds(cls):
-        if not cls.isolated_creds:
-            return
-        admin_client = cls._get_identity_admin_client()
-
-        for user, tenant in cls.isolated_creds:
-            admin_client.delete_user(user['id'])
-            admin_client.delete_tenant(tenant['id'])
-
-    @classmethod
     def tearDownClass(cls):
         cls.clear_snapshots()
         cls.clear_volumes()
-        cls.clear_isolated_creds()
+        cls._clear_isolated_creds()
 
     @classmethod
     def create_snapshot(cls, volume_id=1, **kwargs):
@@ -198,6 +148,13 @@
             msg = ("Missing Volume Admin API credentials "
                    "in configuration.")
             raise cls.skipException(msg)
-
-        cls.os_adm = clients.AdminManager(interface=cls._interface)
+        if cls.config.compute.allow_tenant_isolation:
+            creds = cls._get_isolated_creds(admin=True)
+            admin_username, admin_tenant_name, admin_password = creds
+            cls.os_adm = clients.Manager(username=admin_username,
+                                         password=admin_password,
+                                         tenant_name=admin_tenant_name,
+                                         interface=cls._interface)
+        else:
+            cls.os_adm = clients.AdminManager(interface=cls._interface)
         cls.client = cls.os_adm.volume_types_client
diff --git a/tempest/cli/README.rst b/tempest/cli/README.rst
index 3eae492..f86adf3 100644
--- a/tempest/cli/README.rst
+++ b/tempest/cli/README.rst
@@ -12,7 +12,7 @@
 Why are these tests in tempest?
 -------------------------------
 These tests exist here because it is extremely difficult to build a
-functional enough environment in the python-*client unit tests to
+functional enough environment in the python-\*client unit tests to
 provide this kind of testing. Because we already put up a cloud in the
 gate with devstack + tempest it was decided it was better to have
 these as a side tree in tempest instead of another QA effort which
diff --git a/tempest/config.py b/tempest/config.py
index c9b82ca..a918d0b 100644
--- a/tempest/config.py
+++ b/tempest/config.py
@@ -160,6 +160,10 @@
     cfg.StrOpt('ssh_user',
                default='root',
                help="User name used to authenticate to an instance."),
+    cfg.IntOpt('ping_timeout',
+               default=60,
+               help="Timeout in seconds to wait for ping to "
+                    "succeed."),
     cfg.IntOpt('ssh_timeout',
                default=300,
                help="Timeout in seconds to wait for authentication to "
@@ -288,7 +292,7 @@
                default="10.100.0.0/16",
                help="The cidr block to allocate tenant networks from"),
     cfg.IntOpt('tenant_network_mask_bits',
-               default=29,
+               default=28,
                help="The mask bits for tenant networks"),
     cfg.BoolOpt('tenant_networks_reachable',
                 default=False,
diff --git a/tempest/scenario/manager.py b/tempest/scenario/manager.py
index fcd5d0e..8b24b2e 100644
--- a/tempest/scenario/manager.py
+++ b/tempest/scenario/manager.py
@@ -425,24 +425,24 @@
             if proc.returncode == 0:
                 return True
 
-        # TODO(mnewby) Allow configuration of execution and sleep duration.
-        return tempest.test.call_until_true(ping, 20, 1)
+        return tempest.test.call_until_true(
+            ping, self.config.compute.ping_timeout, 1)
 
     def _is_reachable_via_ssh(self, ip_address, username, private_key,
-                              timeout=120):
+                              timeout):
         ssh_client = ssh.Client(ip_address, username,
                                 pkey=private_key,
                                 timeout=timeout)
         return ssh_client.test_connection_auth()
 
-    def _check_vm_connectivity(self, ip_address, username, private_key,
-                               timeout=120):
+    def _check_vm_connectivity(self, ip_address, username, private_key):
         self.assertTrue(self._ping_ip_address(ip_address),
                         "Timed out waiting for %s to become "
                         "reachable" % ip_address)
-        self.assertTrue(self._is_reachable_via_ssh(ip_address,
-                                                   username,
-                                                   private_key,
-                                                   timeout=timeout),
-                        'Auth failure in connecting to %s@%s via ssh' %
-                        (username, ip_address))
+        self.assertTrue(self._is_reachable_via_ssh(
+            ip_address,
+            username,
+            private_key,
+            timeout=self.config.compute.ssh_timeout),
+            'Auth failure in connecting to %s@%s via ssh' %
+            (username, ip_address))
diff --git a/tempest/services/compute/xml/servers_client.py b/tempest/services/compute/xml/servers_client.py
index f2cca72..ea8b0e0 100644
--- a/tempest/services/compute/xml/servers_client.py
+++ b/tempest/services/compute/xml/servers_client.py
@@ -437,6 +437,12 @@
     def revert_resize(self, server_id, **kwargs):
         return self.action(server_id, 'revertResize', None, **kwargs)
 
+    def stop(self, server_id, **kwargs):
+        return self.action(server_id, 'os-stop', None, **kwargs)
+
+    def start(self, server_id, **kwargs):
+        return self.action(server_id, 'os-start', None, **kwargs)
+
     def create_image(self, server_id, name):
         return self.action(server_id, 'createImage', None, name=name)
 
diff --git a/tempest/stress/driver.py b/tempest/stress/driver.py
index b04a93a..d170eb8 100644
--- a/tempest/stress/driver.py
+++ b/tempest/stress/driver.py
@@ -206,3 +206,7 @@
     if not had_errors:
         logger.info("cleaning up")
         cleanup.cleanup(logger)
+    if had_errors:
+        return 1
+    else:
+        return 0
diff --git a/tempest/stress/run_stress.py b/tempest/stress/run_stress.py
index 9ec1527..106049d 100755
--- a/tempest/stress/run_stress.py
+++ b/tempest/stress/run_stress.py
@@ -18,17 +18,25 @@
 
 import argparse
 import json
+import sys
 
 
 def main(ns):
     #NOTE(kodererm): moved import to make "-h" possible without OpenStack
     from tempest.stress import driver
+    result = 0
     tests = json.load(open(ns.tests, 'r'))
     if ns.serial:
         for test in tests:
-            driver.stress_openstack([test], ns.duration, ns.number)
+            step_result = driver.stress_openstack([test],
+                                                  ns.duration,
+                                                  ns.number)
+            #NOTE(kodererm): we just save the last result code
+            if (step_result != 0):
+                result = step_result
     else:
         driver.stress_openstack(tests, ns.duration, ns.number)
+    return result
 
 
 parser = argparse.ArgumentParser(description='Run stress tests. ')
@@ -39,4 +47,6 @@
 parser.add_argument('-n', '--number', type=int,
                     help="How often an action is executed for each process.")
 parser.add_argument('tests', help="Name of the file with test description.")
-main(parser.parse_args())
+
+if __name__ == "__main__":
+    sys.exit(main(parser.parse_args()))
diff --git a/tempest/test.py b/tempest/test.py
index d7008a7..5040f34 100644
--- a/tempest/test.py
+++ b/tempest/test.py
@@ -15,14 +15,18 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
+import os
 import time
 
 import nose.plugins.attrib
 import testresources
 import testtools
 
+from tempest import clients
 from tempest.common import log as logging
+from tempest.common.utils.data_utils import rand_name
 from tempest import config
+from tempest import exceptions
 from tempest import manager
 
 LOG = logging.getLogger(__name__)
@@ -54,6 +58,42 @@
     return decorator
 
 
+# there is a mis-match between nose and testtools for older pythons.
+# testtools will set skipException to be either
+# unittest.case.SkipTest, unittest2.case.SkipTest or an internal skip
+# exception, depending on what it can find. Python <2.7 doesn't have
+# unittest.case.SkipTest; so if unittest2 is not installed it falls
+# back to the internal class.
+#
+# The current nose skip plugin will decide to raise either
+# unittest.case.SkipTest or its own internal exception; it does not
+# look for unittest2 or the internal unittest exception.  Thus we must
+# monkey-patch testtools.TestCase.skipException to be the exception
+# the nose skip plugin expects.
+#
+# However, with the switch to testr nose may not be available, so we
+# require you to opt-in to this fix with an environment variable.
+#
+# This is temporary until upstream nose starts looking for unittest2
+# as testtools does; we can then remove this and ensure unittest2 is
+# available for older pythons; then nose and testtools will agree
+# unittest2.case.SkipTest is the one-true skip test exception.
+#
+#   https://review.openstack.org/#/c/33056
+#   https://github.com/nose-devs/nose/pull/699
+if 'TEMPEST_PY26_NOSE_COMPAT' in os.environ:
+    try:
+        import unittest.case.SkipTest
+        # convince pep8 we're using the import...
+        if unittest.case.SkipTest:
+            pass
+        raise RuntimeError("You have unittest.case.SkipTest; "
+                           "no need to override")
+    except ImportError:
+        LOG.info("Overriding skipException to nose SkipTest")
+        testtools.TestCase.skipException = nose.plugins.skip.SkipTest
+
+
 class BaseTestCase(testtools.TestCase,
                    testtools.testcase.WithAttributes,
                    testresources.ResourcedTestCase):
@@ -65,6 +105,104 @@
         if hasattr(super(BaseTestCase, cls), 'setUpClass'):
             super(BaseTestCase, cls).setUpClass()
 
+    @classmethod
+    def _get_identity_admin_client(cls):
+        """
+        Returns an instance of the Identity Admin API client
+        """
+        os = clients.AdminManager(interface=cls._interface)
+        admin_client = os.identity_client
+        return admin_client
+
+    @classmethod
+    def _get_client_args(cls):
+
+        return (
+            cls.config,
+            cls.config.identity.admin_username,
+            cls.config.identity.admin_password,
+            cls.config.identity.uri
+        )
+
+    @classmethod
+    def _get_isolated_creds(cls, admin=False):
+        """
+        Creates a new set of user/tenant/password credentials for a
+        **regular** user of the Compute API so that a test case can
+        operate in an isolated tenant container.
+        """
+        admin_client = cls._get_identity_admin_client()
+        password = "pass"
+
+        while True:
+            try:
+                rand_name_root = rand_name(cls.__name__)
+                if cls.isolated_creds:
+                # Main user already created. Create the alt or admin one...
+                    if admin:
+                        rand_name_root += '-admin'
+                    else:
+                        rand_name_root += '-alt'
+                tenant_name = rand_name_root + "-tenant"
+                tenant_desc = tenant_name + "-desc"
+
+                resp, tenant = admin_client.create_tenant(
+                    name=tenant_name, description=tenant_desc)
+                break
+            except exceptions.Duplicate:
+                if cls.config.compute.allow_tenant_reuse:
+                    tenant = admin_client.get_tenant_by_name(tenant_name)
+                    LOG.info('Re-using existing tenant %s', tenant)
+                    break
+
+        while True:
+            try:
+                rand_name_root = rand_name(cls.__name__)
+                if cls.isolated_creds:
+                # Main user already created. Create the alt one...
+                    rand_name_root += '-alt'
+                username = rand_name_root + "-user"
+                email = rand_name_root + "@example.com"
+                resp, user = admin_client.create_user(username,
+                                                      password,
+                                                      tenant['id'],
+                                                      email)
+                break
+            except exceptions.Duplicate:
+                if cls.config.compute.allow_tenant_reuse:
+                    user = admin_client.get_user_by_username(tenant['id'],
+                                                             username)
+                    LOG.info('Re-using existing user %s', user)
+                    break
+        # Store the complete creds (including UUID ids...) for later
+        # but return just the username, tenant_name, password tuple
+        # that the various clients will use.
+        cls.isolated_creds.append((user, tenant))
+
+        # Assign admin role if this is for admin creds
+        if admin:
+            _, roles = admin_client.list_roles()
+            role = None
+            try:
+                _, roles = admin_client.list_roles()
+                role = next(r for r in roles if r['name'] == 'admin')
+            except StopIteration:
+                msg = "No admin role found"
+                raise exceptions.NotFound(msg)
+            admin_client.assign_user_role(tenant['id'], user['id'], role['id'])
+
+        return username, tenant_name, password
+
+    @classmethod
+    def _clear_isolated_creds(cls):
+        if not cls.isolated_creds:
+            return
+        admin_client = cls._get_identity_admin_client()
+
+        for user, tenant in cls.isolated_creds:
+            admin_client.delete_user(user['id'])
+            admin_client.delete_tenant(tenant['id'])
+
 
 def call_until_true(func, duration, sleep_for):
     """
diff --git a/tempest/thirdparty/README.rst b/tempest/thirdparty/README.rst
index 41d31f3..b775817 100644
--- a/tempest/thirdparty/README.rst
+++ b/tempest/thirdparty/README.rst
@@ -1,9 +1,9 @@
 Tempest Guide to Third Party API tests
-========
+======================================
 
 
 What are these tests?
---------
+---------------------
 
 Third party tests are tests for non native OpenStack APIs that are
 part of OpenStack projects. If we ship an API, we're really required
@@ -14,14 +14,14 @@
 
 
 Why are these tests in tempest?
---------
+-------------------------------
 
 If we ship an API in an OpenStack component, there should be tests in
 tempest to exercise it in some way.
 
 
 Scope of these tests
---------
+--------------------
 
 Third party API testing should be limited to the functional testing of
 third party API compliance. Complex scenarios should be avoided, and
diff --git a/test-requirements.txt b/test-requirements.txt
index 693daff..236a473 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -4,4 +4,5 @@
 flake8==2.0
 hacking>=0.5.6,<0.7
 # needed for doc build
+docutils==0.9.1
 sphinx>=1.1.2