Merge "remove gate tag (part 3)"
diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst
index e21deea..b6e00ce 100644
--- a/doc/source/configuration.rst
+++ b/doc/source/configuration.rst
@@ -83,8 +83,16 @@
This is also the currently the default credential provider enabled by tempest,
due to it's common use and ease of configuration.
-Locking Test Accounts
-"""""""""""""""""""""
+It is worth pointing out that depending on your cloud configuration you might
+need to assign a role to each of the users created Tempest's tenant isolation.
+This can be set using the *tempest_roles* option. It takes in a list of role
+names each of which will be assigned to each of the users created by tenant
+isolation. This option will not have any effect when set and tempest is not
+configured to use tenant isolation.
+
+
+Locking Test Accounts (aka accounts.yaml or accounts file)
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
For a long time using tenant isolation was the only method available if you
wanted to enable parallel execution of tempest tests. However this was
insufficient for certain use cases because of the admin credentials requirement
@@ -95,11 +103,6 @@
accounts.yaml before executing any of its tests so that each class is isolated
like in tenant isolation.
-Currently, this mechanism has some limitations, mostly around networking. The
-locking test accounts provider will only work with a single flat network as
-the default for each tenant/project. If another network configuration is used
-in your cloud you might face unexpected failures.
-
To enable and use locking test accounts you need do a few things:
#. Create a accounts.yaml file which contains the set of pre-existing
@@ -112,20 +115,20 @@
#. Provide tempest with the location of you accounts.yaml file with the
test_accounts_file option in the auth section
+It is worth pointing out that each set of credentials in the accounts.yaml
+should have a unique tenant. This is required to provide proper isolation
+to the tests using the credentials, and failure to do this will likely cause
+unexpected failures in some tests.
-Non-locking test accounts
-"""""""""""""""""""""""""
-When tempest was refactored to allow for locking test accounts, the original
-non-tenant isolated case was converted to support the new accounts.yaml file.
-This mechanism is the non-locking test accounts provider. It only makes sense
-to use it if parallel execution isn't needed. If the role restrictions were too
-limiting with the locking accounts provider and tenant isolation is not wanted
-then you can use the non-locking test accounts credential provider without the
-accounts.yaml file.
-To use the non-locking test accounts provider you have 2 ways to configure it.
-First you can specify the sets of credentials in the configuration file like
-detailed above with following 9 options in the identity section:
+Non-locking test accounts (aka credentials config options)
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+When Tempest was refactored to allow for locking test accounts, the original
+non-tenant isolated case was converted to internally work similarly to the
+accounts.yaml file. This mechanism was then called the non-locking test accounts
+provider. To use the non-locking test accounts provider you can specify the sets
+of credentials in the configuration file like detailed above with following 9
+options in the identity section:
#. username
#. password
@@ -137,10 +140,14 @@
#. alt_password
#. alt_tenant_name
-The only restriction with using the traditional config options for credentials
-is that if a test requires specific roles on accounts these tests can not be
-run. This is because the config options do not give sufficient flexibility to
-describe the roles assigned to a user for running the tests.
+It only makes sense to use it if parallel execution isn't needed, since tempest
+won't be able to properly isolate tests using this. Additionally, using the
+traditional config options for credentials is not able to provide credentials to
+tests which requires specific roles on accounts. This is because the config
+options do not give sufficient flexibility to describe the roles assigned to a
+user for running the tests. There are additional limitations with regard to
+network configuration when using this credential provider mechanism, see the
+`Networking`_ section below.
Compute
-------
diff --git a/etc/accounts.yaml.sample b/etc/accounts.yaml.sample
index 31ceb33..3f57eb7 100644
--- a/etc/accounts.yaml.sample
+++ b/etc/accounts.yaml.sample
@@ -1,4 +1,7 @@
# The number of accounts required can be estimated as CONCURRENCY x 2
+# It is expected that each user provided here will be in a different tenant.
+# This is required to provide isolation between test for running in parallel
+#
# Valid fields for credentials are defined in the descendants of
# auth.Credentials - see KeystoneV[2|3]Credentials.CONF_ATTRIBUTES
@@ -28,6 +31,9 @@
- 'reseller_admin'
- 'operator'
+# Networks can be specified to tell tempest which network it should use when
+# creating servers with an account
+
- username: 'admin_user_1'
tenant_name: 'admin_tenant_1'
password: 'test_password'
diff --git a/tempest/api/compute/images/test_image_metadata.py b/tempest/api/compute/images/test_image_metadata.py
index 159a814..9664c61 100644
--- a/tempest/api/compute/images/test_image_metadata.py
+++ b/tempest/api/compute/images/test_image_metadata.py
@@ -13,7 +13,7 @@
# License for the specific language governing permissions and limitations
# under the License.
-import StringIO
+import six
from tempest_lib.common.utils import data_utils
@@ -51,7 +51,7 @@
is_public=False)
cls.image_id = body['id']
cls.images.append(cls.image_id)
- image_file = StringIO.StringIO(('*' * 1024))
+ image_file = six.StringIO(('*' * 1024))
cls.glance_client.update_image(cls.image_id, data=image_file)
cls.client.wait_for_image_status(cls.image_id, 'ACTIVE')
diff --git a/tempest/api/compute/images/test_list_image_filters.py b/tempest/api/compute/images/test_list_image_filters.py
index 7afbdc7..abe7be5 100644
--- a/tempest/api/compute/images/test_list_image_filters.py
+++ b/tempest/api/compute/images/test_list_image_filters.py
@@ -13,10 +13,10 @@
# License for the specific language governing permissions and limitations
# under the License.
-import StringIO
import time
from oslo_log import log as logging
+import six
from tempest_lib.common.utils import data_utils
import testtools
@@ -59,7 +59,7 @@
# Wait 1 second between creation and upload to ensure a delta
# between created_at and updated_at.
time.sleep(1)
- image_file = StringIO.StringIO(('*' * 1024))
+ image_file = six.StringIO(('*' * 1024))
cls.glance_client.update_image(image_id, data=image_file)
cls.client.wait_for_image_status(image_id, 'ACTIVE')
body = cls.client.get_image(image_id)
diff --git a/tempest/api/compute/servers/test_server_actions.py b/tempest/api/compute/servers/test_server_actions.py
index 19ecffe..def4c47 100644
--- a/tempest/api/compute/servers/test_server_actions.py
+++ b/tempest/api/compute/servers/test_server_actions.py
@@ -15,8 +15,8 @@
import base64
import logging
-import urlparse
+from six.moves.urllib import parse as urlparse
from tempest_lib.common.utils import data_utils
from tempest_lib import decorators
from tempest_lib import exceptions as lib_exc
diff --git a/tempest/api/compute/test_authorization.py b/tempest/api/compute/test_authorization.py
index 4458357..ba43617 100644
--- a/tempest/api/compute/test_authorization.py
+++ b/tempest/api/compute/test_authorization.py
@@ -13,7 +13,7 @@
# License for the specific language governing permissions and limitations
# under the License.
-import StringIO
+import six
from oslo_log import log as logging
from tempest_lib.common.utils import data_utils
@@ -75,7 +75,7 @@
disk_format='raw',
is_public=False)
image_id = body['id']
- image_file = StringIO.StringIO(('*' * 1024))
+ image_file = six.StringIO(('*' * 1024))
body = cls.glance_client.update_image(image_id, data=image_file)
cls.glance_client.wait_for_image_status(image_id, 'active')
cls.image = cls.images_client.get_image(image_id)
diff --git a/tempest/api/image/base.py b/tempest/api/image/base.py
index d513b0c..74044dc 100644
--- a/tempest/api/image/base.py
+++ b/tempest/api/image/base.py
@@ -12,9 +12,8 @@
# License for the specific language governing permissions and limitations
# under the License.
-import cStringIO as StringIO
-
from oslo_log import log as logging
+from six import moves
from tempest_lib.common.utils import data_utils
from tempest_lib import exceptions as lib_exc
@@ -113,7 +112,7 @@
cls.alt_tenant_id = cls.alt_img_cli.tenant_id
def _create_image(self):
- image_file = StringIO.StringIO(data_utils.random_bytes())
+ image_file = moves.cStringIO(data_utils.random_bytes())
image = self.create_image(container_format='bare',
disk_format='raw',
is_public=False,
diff --git a/tempest/api/image/v1/test_images.py b/tempest/api/image/v1/test_images.py
index 154f43f..ea95059 100644
--- a/tempest/api/image/v1/test_images.py
+++ b/tempest/api/image/v1/test_images.py
@@ -13,8 +13,7 @@
# License for the specific language governing permissions and limitations
# under the License.
-import cStringIO as StringIO
-
+from six import moves
from tempest_lib.common.utils import data_utils
from tempest.api.image import base
@@ -45,7 +44,7 @@
self.assertEqual(val, body.get('properties')[key])
# Now try uploading an image file
- image_file = StringIO.StringIO(data_utils.random_bytes())
+ image_file = moves.cStringIO(data_utils.random_bytes())
body = self.client.update_image(image_id, data=image_file)
self.assertIn('size', body)
self.assertEqual(1024, body.get('size'))
@@ -157,7 +156,7 @@
image. Note that the size of the new image is a random number between
1024 and 4096
"""
- image_file = StringIO.StringIO(data_utils.random_bytes(size))
+ image_file = moves.cStringIO(data_utils.random_bytes(size))
name = 'New Standard Image %s' % name
image = cls.create_image(name=name,
container_format=container_format,
@@ -246,7 +245,7 @@
Create a new standard image and return the ID of the newly-registered
image.
"""
- image_file = StringIO.StringIO(data_utils.random_bytes(size))
+ image_file = moves.cStringIO(data_utils.random_bytes(size))
name = 'New Standard Image %s' % name
image = cls.create_image(name=name,
container_format=container_format,
diff --git a/tempest/api/image/v2/test_images.py b/tempest/api/image/v2/test_images.py
index adb9a73..832ddf0 100644
--- a/tempest/api/image/v2/test_images.py
+++ b/tempest/api/image/v2/test_images.py
@@ -14,9 +14,9 @@
# License for the specific language governing permissions and limitations
# under the License.
-import cStringIO as StringIO
import random
+from six import moves
from tempest_lib.common.utils import data_utils
from tempest.api.image import base
@@ -54,7 +54,7 @@
# Now try uploading an image file
file_content = data_utils.random_bytes()
- image_file = StringIO.StringIO(file_content)
+ image_file = moves.cStringIO(file_content)
self.client.store_image(image_id, image_file)
# Now try to get image details
@@ -105,7 +105,7 @@
image_id = body['id']
# Now try uploading an image file
- image_file = StringIO.StringIO(data_utils.random_bytes())
+ image_file = moves.cStringIO(data_utils.random_bytes())
self.client.store_image(image_id, image_file)
# Update Image
@@ -146,7 +146,7 @@
1024 and 4096
"""
size = random.randint(1024, 4096)
- image_file = StringIO.StringIO(data_utils.random_bytes(size))
+ image_file = moves.cStringIO(data_utils.random_bytes(size))
name = data_utils.rand_name('image')
body = cls.create_image(name=name,
container_format=container_format,
diff --git a/tempest/api/messaging/test_claims.py b/tempest/api/messaging/test_claims.py
index 896de81..34e1fe8 100644
--- a/tempest/api/messaging/test_claims.py
+++ b/tempest/api/messaging/test_claims.py
@@ -14,8 +14,8 @@
# limitations under the License.
import logging
-import urlparse
+from six.moves.urllib import parse as urlparse
from tempest_lib.common.utils import data_utils
from tempest_lib import decorators
diff --git a/tempest/api/object_storage/test_container_sync.py b/tempest/api/object_storage/test_container_sync.py
index ff99c15..53bcfa6 100644
--- a/tempest/api/object_storage/test_container_sync.py
+++ b/tempest/api/object_storage/test_container_sync.py
@@ -14,8 +14,8 @@
# under the License.
import time
-import urlparse
+from six.moves.urllib import parse as urlparse
from tempest_lib.common.utils import data_utils
from tempest_lib import decorators
import testtools
diff --git a/tempest/api/object_storage/test_object_formpost.py b/tempest/api/object_storage/test_object_formpost.py
index 7ab5c6d..ce587d7 100644
--- a/tempest/api/object_storage/test_object_formpost.py
+++ b/tempest/api/object_storage/test_object_formpost.py
@@ -15,8 +15,8 @@
import hashlib
import hmac
import time
-import urlparse
+from six.moves.urllib import parse as urlparse
from tempest_lib.common.utils import data_utils
from tempest.api.object_storage import base
diff --git a/tempest/api/object_storage/test_object_formpost_negative.py b/tempest/api/object_storage/test_object_formpost_negative.py
index e945c7c..89deca2 100644
--- a/tempest/api/object_storage/test_object_formpost_negative.py
+++ b/tempest/api/object_storage/test_object_formpost_negative.py
@@ -15,8 +15,8 @@
import hashlib
import hmac
import time
-import urlparse
+from six.moves.urllib import parse as urlparse
from tempest_lib.common.utils import data_utils
from tempest_lib import exceptions as lib_exc
diff --git a/tempest/api/object_storage/test_object_services.py b/tempest/api/object_storage/test_object_services.py
index f424442..6acd0f2 100644
--- a/tempest/api/object_storage/test_object_services.py
+++ b/tempest/api/object_storage/test_object_services.py
@@ -13,7 +13,6 @@
# License for the specific language governing permissions and limitations
# under the License.
-import cStringIO as StringIO
import hashlib
import random
import re
@@ -21,6 +20,7 @@
import zlib
import six
+from six import moves
from tempest_lib.common.utils import data_utils
from tempest.api.object_storage import base
@@ -210,7 +210,7 @@
status, _, resp_headers = self.object_client.put_object_with_chunk(
container=self.container_name,
name=object_name,
- contents=StringIO.StringIO(data),
+ contents=moves.cStringIO(data),
chunk_size=512)
self.assertHeaders(resp_headers, 'Object', 'PUT')
diff --git a/tempest/api/object_storage/test_object_temp_url.py b/tempest/api/object_storage/test_object_temp_url.py
index b405216..8748269 100644
--- a/tempest/api/object_storage/test_object_temp_url.py
+++ b/tempest/api/object_storage/test_object_temp_url.py
@@ -15,8 +15,8 @@
import hashlib
import hmac
import time
-import urlparse
+from six.moves.urllib import parse as urlparse
from tempest_lib.common.utils import data_utils
from tempest.api.object_storage import base
diff --git a/tempest/api/object_storage/test_object_temp_url_negative.py b/tempest/api/object_storage/test_object_temp_url_negative.py
index ed4dc6d..233cced 100644
--- a/tempest/api/object_storage/test_object_temp_url_negative.py
+++ b/tempest/api/object_storage/test_object_temp_url_negative.py
@@ -15,8 +15,8 @@
import hashlib
import hmac
import time
-import urlparse
+from six.moves.urllib import parse as urlparse
from tempest_lib.common.utils import data_utils
from tempest_lib import exceptions as lib_exc
diff --git a/tempest/cmd/verify_tempest_config.py b/tempest/cmd/verify_tempest_config.py
index b61f286..f1965bc 100755
--- a/tempest/cmd/verify_tempest_config.py
+++ b/tempest/cmd/verify_tempest_config.py
@@ -18,10 +18,10 @@
import json
import os
import sys
-import urlparse
import httplib2
from six import moves
+from six.moves.urllib import parse as urlparse
from tempest import clients
from tempest.common import credentials
diff --git a/tempest/common/glance_http.py b/tempest/common/glance_http.py
index c6b8ba3..fb5958c 100644
--- a/tempest/common/glance_http.py
+++ b/tempest/common/glance_http.py
@@ -22,14 +22,13 @@
import posixpath
import re
import socket
-import StringIO
import struct
-import urlparse
-
import OpenSSL
from oslo_log import log as logging
+import six
from six import moves
+from six.moves.urllib import parse as urlparse
from tempest_lib import exceptions as lib_exc
from tempest import exceptions as exc
@@ -129,7 +128,7 @@
# Read body into string if it isn't obviously image data
if resp.getheader('content-type', None) != 'application/octet-stream':
body_str = ''.join([body_chunk for body_chunk in body_iter])
- body_iter = StringIO.StringIO(body_str)
+ body_iter = six.StringIO(body_str)
self._log_response(resp, None)
else:
self._log_response(resp, body_iter)
diff --git a/tempest/common/ssh.py b/tempest/common/ssh.py
index fe67ff8..d0e484c 100644
--- a/tempest/common/ssh.py
+++ b/tempest/common/ssh.py
@@ -13,8 +13,6 @@
# License for the specific language governing permissions and limitations
# under the License.
-
-import cStringIO
import select
import socket
import time
@@ -22,6 +20,7 @@
from oslo_log import log as logging
import six
+from six import moves
from tempest import exceptions
@@ -43,7 +42,7 @@
self.password = password
if isinstance(pkey, six.string_types):
pkey = paramiko.RSAKey.from_private_key(
- cStringIO.StringIO(str(pkey)))
+ moves.cStringIO(str(pkey)))
self.pkey = pkey
self.look_for_keys = look_for_keys
self.key_filename = key_filename
diff --git a/tempest/services/botoclients.py b/tempest/services/botoclients.py
index 6a1af6c..95c05d7 100644
--- a/tempest/services/botoclients.py
+++ b/tempest/services/botoclients.py
@@ -15,15 +15,15 @@
import ConfigParser
import contextlib
-from tempest_lib import exceptions as lib_exc
import types
-import urlparse
-
-from tempest import config
import boto
import boto.ec2
import boto.s3.connection
+from six.moves.urllib import parse as urlparse
+from tempest_lib import exceptions as lib_exc
+
+from tempest import config
CONF = config.CONF
diff --git a/tempest/services/object_storage/object_client.py b/tempest/services/object_storage/object_client.py
index eaa894d..0e02bbc 100644
--- a/tempest/services/object_storage/object_client.py
+++ b/tempest/services/object_storage/object_client.py
@@ -15,7 +15,8 @@
import httplib
import urllib
-import urlparse
+
+from six.moves.urllib import parse as urlparse
from tempest.common import service_client
diff --git a/tempest/tests/test_ssh.py b/tempest/tests/test_ssh.py
index 27cd6b5..aaacaab 100644
--- a/tempest/tests/test_ssh.py
+++ b/tempest/tests/test_ssh.py
@@ -29,7 +29,7 @@
def test_pkey_calls_paramiko_RSAKey(self):
with contextlib.nested(
mock.patch('paramiko.RSAKey.from_private_key'),
- mock.patch('cStringIO.StringIO')) as (rsa_mock, cs_mock):
+ mock.patch('six.moves.cStringIO')) as (rsa_mock, cs_mock):
cs_mock.return_value = mock.sentinel.csio
pkey = 'mykey'
ssh.Client('localhost', 'root', pkey=pkey)
diff --git a/tempest/tests/test_wrappers.py b/tempest/tests/test_wrappers.py
index ae7860d..a4ef699 100644
--- a/tempest/tests/test_wrappers.py
+++ b/tempest/tests/test_wrappers.py
@@ -14,10 +14,11 @@
import os
import shutil
-import StringIO
import subprocess
import tempfile
+import six
+
from tempest.tests import base
DEVNULL = open(os.devnull, 'wb')
@@ -50,8 +51,8 @@
shutil.copy('tools/pretty_tox_serial.sh',
os.path.join(self.directory, 'pretty_tox_serial.sh'))
- self.stdout = StringIO.StringIO()
- self.stderr = StringIO.StringIO()
+ self.stdout = six.StringIO()
+ self.stderr = six.StringIO()
# Change directory, run wrapper and check result
self.addCleanup(os.chdir, os.path.abspath(os.curdir))
os.chdir(self.directory)
diff --git a/tempest/thirdparty/boto/test.py b/tempest/thirdparty/boto/test.py
index cd35e7f..d3846a8 100644
--- a/tempest/thirdparty/boto/test.py
+++ b/tempest/thirdparty/boto/test.py
@@ -17,7 +17,6 @@
import logging as orig_logging
import os
import re
-import urlparse
import boto
from boto import ec2
@@ -25,7 +24,7 @@
from boto import s3
from oslo_log import log as logging
import six
-
+from six.moves.urllib import parse as urlparse
from tempest_lib import exceptions as lib_exc
import tempest.clients
diff --git a/tempest/thirdparty/boto/test_ec2_instance_run.py b/tempest/thirdparty/boto/test_ec2_instance_run.py
index 8894de0..19a77dc 100644
--- a/tempest/thirdparty/boto/test_ec2_instance_run.py
+++ b/tempest/thirdparty/boto/test_ec2_instance_run.py
@@ -68,12 +68,21 @@
"ari":
{"name": data_utils.rand_name("ari-name"),
"location": cls.bucket_name + "/" + ari_manifest}}
- for image in cls.images.itervalues():
+ for image_type in ("aki", "ari"):
+ image = cls.images[image_type]
image["image_id"] = cls.ec2_client.register_image(
name=image["name"],
image_location=image["location"])
cls.addResourceCleanUp(cls.ec2_client.deregister_image,
image["image_id"])
+ image = cls.images["ami"]
+ image["image_id"] = cls.ec2_client.register_image(
+ name=image["name"],
+ image_location=image["location"],
+ kernel_id=cls.images["aki"]["image_id"],
+ ramdisk_id=cls.images["ari"]["image_id"])
+ cls.addResourceCleanUp(cls.ec2_client.deregister_image,
+ image["image_id"])
for image in cls.images.itervalues():
def _state():