Merge "Refactor skip_because decorator"
diff --git a/REVIEWING.rst b/REVIEWING.rst
index e07e358..4c63aa0 100644
--- a/REVIEWING.rst
+++ b/REVIEWING.rst
@@ -160,13 +160,11 @@
When to approve
---------------
* It's OK to hold off on an approval until a subject matter expert reviews it.
-* Every patch needs two +2's before being approved.
-* However, a single Tempest core reviewer can approve patches without waiting
- for another +2 in the following cases:
+* Every patch needs at least single +2's before being approved. A single
+ Tempest core reviewer can approve patches but can always wait for another
+ +2 in any case. Following cases where single +2 can be used without any
+ issue:
- * If a patch has already been approved but requires a trivial rebase to
- merge, then there is no need to wait for a second +2, since the patch has
- already had two +2's.
* If any trivial patch set fixes one of the items below:
* Documentation or code comment typo
@@ -187,7 +185,4 @@
voting ``tempest-tox-plugin-sanity-check`` job) and unblock the
tempest gate
- Note that such a policy should be used judiciously, as we should strive to
- have two +2's on each patch set, prior to approval.
-
.. _example: https://review.opendev.org/#/c/611032/
diff --git a/doc/source/contributor/contributing.rst b/doc/source/contributor/contributing.rst
index 9c79a1f..62953ff 100644
--- a/doc/source/contributor/contributing.rst
+++ b/doc/source/contributor/contributing.rst
@@ -43,10 +43,9 @@
Getting Your Patch Merged
~~~~~~~~~~~~~~~~~~~~~~~~~
-All changes proposed to the Tempest require two ``Code-Review +2`` votes from
-Tempest core reviewers before one of the core reviewers can approve the patch by
-giving ``Workflow +1`` vote. More detailed guidelines for reviewers are available
-at :doc:`../REVIEWING`.
+All changes proposed to the Tempest require single ``Code-Review +2`` votes from
+Tempest core reviewers by giving ``Workflow +1`` vote. More detailed guidelines
+for reviewers are available at :doc:`../REVIEWING`.
Project Team Lead Duties
~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/source/data/tempest-blacklisted-plugins-registry.header b/doc/source/data/tempest-non-active-plugins-registry.header
similarity index 67%
rename from doc/source/data/tempest-blacklisted-plugins-registry.header
rename to doc/source/data/tempest-non-active-plugins-registry.header
index 6b6af11..06d8eaa 100644
--- a/doc/source/data/tempest-blacklisted-plugins-registry.header
+++ b/doc/source/data/tempest-non-active-plugins-registry.header
@@ -1,7 +1,7 @@
-Blacklisted Plugins
+Non Active Plugins
===================
List of Tempest plugin projects that are stale or unmaintained for a long
-time (6 months or more). They can be moved out of blacklist state once one
+time (6 months or more). They can be moved out of nonactivelist state once one
of the relevant patches gets merged:
https://review.opendev.org/#/q/topic:tempest-sanity-gate+%28status:open%29
diff --git a/doc/source/microversion_testing.rst b/doc/source/microversion_testing.rst
index c1981f9..06062c2 100644
--- a/doc/source/microversion_testing.rst
+++ b/doc/source/microversion_testing.rst
@@ -302,6 +302,10 @@
.. _2.2: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id2
+ * `2.3`_
+
+ .. _2.3: http://docs.openstack.org/developer/nova/api_microversion_history.html#maximum-in-kilo
+
* `2.6`_
.. _2.6: https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id5
diff --git a/doc/source/overview.rst b/doc/source/overview.rst
index e51b90b..2eaf72f 100644
--- a/doc/source/overview.rst
+++ b/doc/source/overview.rst
@@ -113,7 +113,7 @@
There is also the option to use `stestr`_ directly. For example, from
the workspace dir run::
- $ stestr run --black-regex '\[.*\bslow\b.*\]' '^tempest\.(api|scenario)'
+ $ stestr run --exclude-regex '\[.*\bslow\b.*\]' '^tempest\.(api|scenario)'
will run the same set of tests as the default gate jobs. Or you can
use `unittest`_ compatible test runners such as `stestr`_, `pytest`_ etc.
diff --git a/doc/source/stable_branch_support_policy.rst b/doc/source/stable_branch_support_policy.rst
index 87e3ad1..9c2d1ed 100644
--- a/doc/source/stable_branch_support_policy.rst
+++ b/doc/source/stable_branch_support_policy.rst
@@ -20,7 +20,7 @@
testing branches in these phases, it's possible that we'll introduce changes to
Tempest on master which will break support on *Extended Maintenance* phase
branches. When this happens the expectation for those branches is to either
-switch to running Tempest from a tag with support for the branch, or blacklist
+switch to running Tempest from a tag with support for the branch, or exclude
a newly introduced test (if that is the cause of the issue). Tempest will not
be creating stable branches to support *Extended Maintenance* phase branches, as
the burden is on the *Extended Maintenance* phase branche maintainers, not the Tempest
diff --git a/etc/whitelist.yaml b/etc/allow-list.yaml
similarity index 100%
rename from etc/whitelist.yaml
rename to etc/allow-list.yaml
diff --git a/releasenotes/notes/Inclusive-jargon-17621346744f0cf4.yaml b/releasenotes/notes/Inclusive-jargon-17621346744f0cf4.yaml
new file mode 100644
index 0000000..089569e
--- /dev/null
+++ b/releasenotes/notes/Inclusive-jargon-17621346744f0cf4.yaml
@@ -0,0 +1,13 @@
+---
+deprecations:
+ - |
+ In this release the following tempest arguments are deprecated and
+ replaced by new ones which are functionally equivalent:
+
+ * --black-regex is replaced by --exclude-regex
+ * --blacklist-file is replaced by --exclude-list
+ * --whitelist-file is replaced by --include-list
+
+ For now Tempest supports both (new and old ones) in order to make the
+ transition for all consumers smoother. However, that's just a temporary
+ case and the old options will be removed soon.
diff --git a/releasenotes/notes/Remove-manager-2e0b0af48f01294a.yaml b/releasenotes/notes/Remove-manager-2e0b0af48f01294a.yaml
new file mode 100644
index 0000000..822df7d
--- /dev/null
+++ b/releasenotes/notes/Remove-manager-2e0b0af48f01294a.yaml
@@ -0,0 +1,5 @@
+---
+upgrade:
+ - |
+ In this release tempest/manager.py is removed after more than 4 years
+ of deprecation.
diff --git a/releasenotes/notes/create_loginable_secgroup_rule-73722fd4b4eb12d0.yaml b/releasenotes/notes/create_loginable_secgroup_rule-73722fd4b4eb12d0.yaml
new file mode 100644
index 0000000..e53411d
--- /dev/null
+++ b/releasenotes/notes/create_loginable_secgroup_rule-73722fd4b4eb12d0.yaml
@@ -0,0 +1,6 @@
+---
+features:
+ - |
+ Added public interface create_loginable_secgroup_rule().
+ Since this interface is meant to be used by tempest plugins,
+ It doesn't neccessarily require to be private api.
diff --git a/releasenotes/notes/create_security_group_rule-16d58a8f0f0ff262.yaml b/releasenotes/notes/create_security_group_rule-16d58a8f0f0ff262.yaml
new file mode 100644
index 0000000..3354f65
--- /dev/null
+++ b/releasenotes/notes/create_security_group_rule-16d58a8f0f0ff262.yaml
@@ -0,0 +1,6 @@
+---
+features:
+ - |
+ Added public interface create_security_group_rule().
+ Since this interface is meant to be used by tempest plugins,
+ It doesn't neccessarily require to be private api.
diff --git a/roles/run-tempest/README.rst b/roles/run-tempest/README.rst
index 3643edb..f9fcf28 100644
--- a/roles/run-tempest/README.rst
+++ b/roles/run-tempest/README.rst
@@ -32,7 +32,11 @@
.. zuul:rolevar:: tempest_test_blacklist
- Specifies a blacklist file to skip tests that are not needed.
+ DEPRECATED option, please use tempest_test_exclude_list instead.
+
+.. zuul:rolevar:: tempest_test_exclude_list
+
+ Specifies an excludelist file to skip tests that are not needed.
Pass a full path to the file.
@@ -44,6 +48,11 @@
.. zuul:rolevar:: tempest_black_regex
:default: ''
+ DEPRECATED option, please use tempest_exclude_regex instead.
+
+.. zuul:rolevar:: tempest_exclude_regex
+ :default: ''
+
A regular expression used to skip the tests.
It works only when used with some specific tox environments
@@ -51,7 +60,7 @@
::
vars:
- tempest_black_regex: (tempest.api.identity).*$
+ tempest_exclude_regex: (tempest.api.identity).*$
.. zuul:rolevar:: tox_extra_args
:default: ''
diff --git a/roles/run-tempest/defaults/main.yaml b/roles/run-tempest/defaults/main.yaml
index 5867b6c..e7a1cc6 100644
--- a/roles/run-tempest/defaults/main.yaml
+++ b/roles/run-tempest/defaults/main.yaml
@@ -1,7 +1,10 @@
devstack_base_dir: /opt/stack
tempest_test_regex: ''
tox_envlist: smoke
+# TODO(kopecmartin) remove tempest_black_regex once all consumers of this
+# role have switched to the tempest_exclude_regex option.
tempest_black_regex: ''
+tempest_exclude_regex: ''
tox_extra_args: ''
tempest_test_timeout: ''
stable_constraints_file: "{{ devstack_base_dir }}/requirements/upper-constraints.txt"
diff --git a/roles/run-tempest/tasks/main.yaml b/roles/run-tempest/tasks/main.yaml
index 1de3725..e9c8e92 100644
--- a/roles/run-tempest/tasks/main.yaml
+++ b/roles/run-tempest/tasks/main.yaml
@@ -36,6 +36,9 @@
tempest_tox_environment: "{{ tempest_tox_environment | combine({'OS_TEST_TIMEOUT': tempest_test_timeout}) }}"
when: tempest_test_timeout != ''
+# TODO(kopecmartin) remove the following 'when block' after all consumers of
+# the role have switched to tempest_test_exclude_list option, until then it's
+# kept here for backward compatibility
- when:
- tempest_test_blacklist is defined
block:
@@ -50,10 +53,48 @@
blacklist_option: "--blacklist-file={{ tempest_test_blacklist|quote }}"
when: blacklist_stat.stat.exists
+- when:
+ - tempest_test_exclude_list is defined
+ block:
+ - name: Check for test exclude list file
+ stat:
+ path: "{{ tempest_test_exclude_list }}"
+ register:
+ exclude_list_stat
+
+ - name: Build exclude list option
+ set_fact:
+ exclude_list_option: "--exclude-list={{ tempest_test_exclude_list|quote }}"
+ when: exclude_list_stat.stat.exists
+
+# TODO(kopecmartin) remove this after all consumers of the role have switched
+# to tempest_exclude_regex option, until then it's kept here for the backward
+# compatibility
+- name: Set tempest_exclude_regex
+ set_fact:
+ tempest_exclude_regex: "{{ tempest_black_regex }}"
+ when:
+ - tempest_black_regex is defined
+ - tempest_exclude_regex is not defined
+
+- name: Build exclude regex (old param)
+ set_fact:
+ tempest_exclude_regex: "--black-regex={{tempest_black_regex|quote}}"
+ when:
+ - tempest_black_regex is defined
+
+- name: Build exclude regex (new param)
+ set_fact:
+ tempest_exclude_regex: "--exclude-regex={{tempest_exclude_regex|quote}}"
+ when:
+ - tempest_black_regex is not defined
+ - tempest_exclude_regex is defined
+
- name: Run Tempest
- command: tox -e {{tox_envlist}} {{tox_extra_args}} -- {{tempest_test_regex|quote}} {{blacklist_option|default('')}} \
+ command: tox -e {{tox_envlist}} {{tox_extra_args}} -- {{tempest_test_regex|quote}} \
+ {{blacklist_option|default('')}} {{exclude_list_option|default('')}} \
--concurrency={{tempest_concurrency|default(default_concurrency)}} \
- --black-regex={{tempest_black_regex|quote}}
+ {{tempest_exclude_regex|default('')}}
args:
chdir: "{{devstack_base_dir}}/tempest"
register: tempest_run_result
diff --git a/tempest/api/image/v2/test_images.py b/tempest/api/image/v2/test_images.py
index d1f6f98..ca72388 100644
--- a/tempest/api/image/v2/test_images.py
+++ b/tempest/api/image/v2/test_images.py
@@ -90,7 +90,7 @@
self.assertEqual('uploading', body['status'])
# import image from staging to backend
self.client.image_import(image['id'], method='glance-direct')
- self.client.wait_for_resource_activation(image['id'])
+ waiters.wait_for_image_imported_to_stores(self.client, image['id'])
@decorators.idempotent_id('f6feb7a4-b04f-4706-a011-206129f83e62')
def test_image_web_download_import(self):
@@ -111,7 +111,7 @@
image_uri = CONF.image.http_image
self.client.image_import(image['id'], method='web-download',
image_uri=image_uri)
- self.client.wait_for_resource_activation(image['id'])
+ waiters.wait_for_image_imported_to_stores(self.client, image['id'])
class MultiStoresImportImagesTest(base.BaseV2ImageTest):
diff --git a/tempest/cmd/run.py b/tempest/cmd/run.py
index 8bebce2..2669ff7 100644
--- a/tempest/cmd/run.py
+++ b/tempest/cmd/run.py
@@ -22,10 +22,10 @@
* ``--regex/-r``: This is a selection regex like what stestr uses. It will run
any tests that match on re.match() with the regex
* ``--smoke/-s``: Run all the tests tagged as smoke
-* ``--black-regex``: It allows to do simple test exclusion via passing a
- rejection/black regexp
+* ``--exclude-regex``: It allows to do simple test exclusion via passing a
+ rejection/exclude regexp
-There are also the ``--blacklist-file`` and ``--whitelist-file`` options that
+There are also the ``--exclude-list`` and ``--include-list`` options that
let you pass a filepath to tempest run with the file format being a line
separated regex, with '#' used to signify the start of a comment on a line.
For example::
@@ -128,6 +128,7 @@
import sys
from cliff import command
+from oslo_log import log
from oslo_serialization import jsonutils as json
from stestr import commands
@@ -141,6 +142,8 @@
CONF = config.CONF
SAVED_STATE_JSON = "saved_state.json"
+LOG = log.getLogger(__name__)
+
class TempestRun(command.Command):
@@ -201,23 +204,71 @@
self._init_state()
regex = self._build_regex(parsed_args)
+
+ # temporary method for parsing deprecated and new stestr options
+ # and showing warning messages in order to make the transition
+ # smoother for all tempest consumers
+ # TODO(kopecmartin) remove this after stestr>=3.1.0 is used
+ # in all supported OpenStack releases
+ def parse_dep(old_o, old_v, new_o, new_v):
+ ret = ''
+ if old_v:
+ LOG.warning("'%s' option is deprecated, use '%s' instead "
+ "which is functionally equivalent. Right now "
+ "Tempest still supports this option for "
+ "backward compatibility, however, it will be "
+ "removed soon.",
+ old_o, new_o)
+ ret = old_v
+ if old_v and new_v:
+ # both options are specified
+ LOG.warning("'%s' and '%s' are specified at the same time, "
+ "'%s' takes precedence over '%s'",
+ new_o, old_o, new_o, old_o)
+ if new_v:
+ ret = new_v
+ return ret
+ ex_regex = parse_dep('--black-regex', parsed_args.black_regex,
+ '--exclude-regex', parsed_args.exclude_regex)
+ ex_list = parse_dep('--blacklist-file', parsed_args.blacklist_file,
+ '--exclude-list', parsed_args.exclude_list)
+ in_list = parse_dep('--whitelist-file', parsed_args.whitelist_file,
+ '--include-list', parsed_args.include_list)
+
return_code = 0
if parsed_args.list_tests:
- return_code = commands.list_command(
- filters=regex, whitelist_file=parsed_args.whitelist_file,
- blacklist_file=parsed_args.blacklist_file,
- black_regex=parsed_args.black_regex)
+ try:
+ return_code = commands.list_command(
+ filters=regex, include_list=in_list,
+ exclude_list=ex_list, exclude_regex=ex_regex)
+ except TypeError:
+ # exclude_list, include_list and exclude_regex are defined only
+ # in stestr >= 3.1.0, this except block catches the case when
+ # tempest is executed with an older stestr
+ return_code = commands.list_command(
+ filters=regex, whitelist_file=in_list,
+ blacklist_file=ex_list, black_regex=ex_regex)
else:
serial = not parsed_args.parallel
- return_code = commands.run_command(
- filters=regex, subunit_out=parsed_args.subunit,
- serial=serial, concurrency=parsed_args.concurrency,
- blacklist_file=parsed_args.blacklist_file,
- whitelist_file=parsed_args.whitelist_file,
- black_regex=parsed_args.black_regex,
- worker_path=parsed_args.worker_file,
- load_list=parsed_args.load_list, combine=parsed_args.combine)
+ params = {
+ 'filters': regex, 'subunit_out': parsed_args.subunit,
+ 'serial': serial, 'concurrency': parsed_args.concurrency,
+ 'worker_path': parsed_args.worker_file,
+ 'load_list': parsed_args.load_list,
+ 'combine': parsed_args.combine
+ }
+ try:
+ return_code = commands.run_command(
+ **params, exclude_list=ex_list,
+ include_list=in_list, exclude_regex=ex_regex)
+ except TypeError:
+ # exclude_list, include_list and exclude_regex are defined only
+ # in stestr >= 3.1.0, this except block catches the case when
+ # tempest is executed with an older stestr
+ return_code = commands.run_command(
+ **params, blacklist_file=ex_list,
+ whitelist_file=in_list, black_regex=ex_regex)
if return_code > 0:
sys.exit(return_code)
return return_code
@@ -271,15 +322,38 @@
help='A normal stestr selection regex used to '
'specify a subset of tests to run')
parser.add_argument('--black-regex', dest='black_regex',
+ help='DEPRECATED: This option is deprecated and '
+ 'will be removed soon, use --exclude-regex '
+ 'which is functionally equivalent. If this '
+ 'is specified at the same time as '
+ '--exclude-regex, this flag will be ignored '
+ 'and --exclude-regex will be used')
+ parser.add_argument('--exclude-regex', dest='exclude_regex',
help='A regex to exclude tests that match it')
parser.add_argument('--whitelist-file', '--whitelist_file',
- help="Path to a whitelist file, this file "
- "contains a separate regex on each "
- "newline.")
+ help='DEPRECATED: This option is deprecated and '
+ 'will be removed soon, use --include-list '
+ 'which is functionally equivalent. If this '
+ 'is specified at the same time as '
+ '--include-list, this flag will be ignored '
+ 'and --include-list will be used')
+ parser.add_argument('--include-list', '--include_list',
+ help="Path to an include file which contains the "
+ "regex for tests to be included in tempest "
+ "run, this file contains a separate regex on "
+ "each newline.")
parser.add_argument('--blacklist-file', '--blacklist_file',
- help='Path to a blacklist file, this file '
- 'contains a separate regex exclude on '
- 'each newline')
+ help='DEPRECATED: This option is deprecated and '
+ 'will be removed soon, use --exclude-list '
+ 'which is functionally equivalent. If this '
+ 'is specified at the same time as '
+ '--exclude-list, this flag will be ignored '
+ 'and --exclude-list will be used')
+ parser.add_argument('--exclude-list', '--exclude_list',
+ help='Path to an exclude file which contains the '
+ 'regex for tests to be excluded in tempest '
+ 'run, this file contains a separate regex on '
+ 'each newline.')
parser.add_argument('--load-list', '--load_list',
help='Path to a non-regex whitelist file, '
'this file contains a separate test '
diff --git a/tempest/common/waiters.py b/tempest/common/waiters.py
index e3c33c7..eaac05e 100644
--- a/tempest/common/waiters.py
+++ b/tempest/common/waiters.py
@@ -193,26 +193,34 @@
raise lib_exc.TimeoutException(message)
-def wait_for_image_imported_to_stores(client, image_id, stores):
+def wait_for_image_imported_to_stores(client, image_id, stores=None):
"""Waits for an image to be imported to all requested stores.
+ Short circuits to fail if the serer reports failure of any store.
+ If stores is None, just wait for status==active.
+
The client should also have build_interval and build_timeout attributes.
"""
+ exc_cls = lib_exc.TimeoutException
start = int(time.time())
while int(time.time()) - start < client.build_timeout:
image = client.show_image(image_id)
- if image['status'] == 'active' and image['stores'] == stores:
+ if image['status'] == 'active' and (stores is None or
+ image['stores'] == stores):
return
+ if image.get('os_glance_failed_import'):
+ exc_cls = lib_exc.OtherRestClientException
+ break
time.sleep(client.build_interval)
message = ('Image %s failed to import on stores: %s' %
- (image_id, str(image['os_glance_failed_import'])))
+ (image_id, str(image.get('os_glance_failed_import'))))
caller = test_utils.find_test_caller()
if caller:
message = '(%s) %s' % (caller, message)
- raise lib_exc.TimeoutException(message)
+ raise exc_cls(message)
def wait_for_image_copied_to_stores(client, image_id):
diff --git a/tempest/config.py b/tempest/config.py
index 956b593..31d9b1b 100644
--- a/tempest/config.py
+++ b/tempest/config.py
@@ -1207,6 +1207,39 @@
help="Whether or not horizon is expected to be available"),
]
+enforce_scope_group = cfg.OptGroup(name="enforce_scope",
+ title="OpenStack Services with "
+ "enforce scope")
+
+
+EnforceScopeGroup = [
+ cfg.BoolOpt('nova',
+ default=False,
+ help='Does the compute service API policies enforce scope? '
+ 'This configuration value should be same as '
+ 'nova.conf: [oslo_policy].enforce_scope option.'),
+ cfg.BoolOpt('neutron',
+ default=False,
+ help='Does the network service API policies enforce scope? '
+ 'This configuration value should be same as '
+ 'neutron.conf: [oslo_policy].enforce_scope option.'),
+ cfg.BoolOpt('glance',
+ default=False,
+ help='Does the Image service API policies enforce scope? '
+ 'This configuration value should be same as '
+ 'glance.conf: [oslo_policy].enforce_scope option.'),
+ cfg.BoolOpt('cinder',
+ default=False,
+ help='Does the Volume service API policies enforce scope? '
+ 'This configuration value should be same as '
+ 'cinder.conf: [oslo_policy].enforce_scope option.'),
+ cfg.BoolOpt('keystone',
+ default=False,
+ help='Does the Identity service API policies enforce scope? '
+ 'This configuration value should be same as '
+ 'keystone.conf: [oslo_policy].enforce_scope option.'),
+]
+
debug_group = cfg.OptGroup(name="debug",
title="Debug System")
@@ -1276,6 +1309,7 @@
(object_storage_feature_group, ObjectStoreFeaturesGroup),
(scenario_group, ScenarioGroup),
(service_available_group, ServiceAvailableGroup),
+ (enforce_scope_group, EnforceScopeGroup),
(debug_group, DebugGroup),
(placement_group, PlacementGroup),
(profiler_group, ProfilerGroup),
@@ -1345,6 +1379,7 @@
'object-storage-feature-enabled']
self.scenario = _CONF.scenario
self.service_available = _CONF.service_available
+ self.enforce_scope = _CONF.enforce_scope
self.debug = _CONF.debug
logging.tempest_set_log_file('tempest.log')
# Setting attributes for plugins
diff --git a/tempest/lib/common/dynamic_creds.py b/tempest/lib/common/dynamic_creds.py
index 220d96c..ecbbe8f 100644
--- a/tempest/lib/common/dynamic_creds.py
+++ b/tempest/lib/common/dynamic_creds.py
@@ -376,15 +376,18 @@
elif scope and self._creds.get("%s_%s" % (scope, credential_type[0])):
credentials = self._creds["%s_%s" % (scope, credential_type[0])]
else:
- if credential_type in ['primary', 'alt', 'admin']:
+ if scope:
+ if credential_type == 'admin':
+ credentials = self._create_creds(
+ admin=True, scope=scope)
+ else:
+ credentials = self._create_creds(
+ roles=credential_type, scope=scope)
+ elif credential_type in ['primary', 'alt', 'admin']:
is_admin = (credential_type == 'admin')
credentials = self._create_creds(admin=is_admin)
else:
- if scope:
- credentials = self._create_creds(
- roles=credential_type, scope=scope)
- else:
- credentials = self._create_creds(roles=credential_type)
+ credentials = self._create_creds(roles=credential_type)
if scope:
self._creds["%s_%s" %
(scope, credential_type[0])] = credentials
diff --git a/tempest/manager.py b/tempest/manager.py
deleted file mode 100644
index b485ef2..0000000
--- a/tempest/manager.py
+++ /dev/null
@@ -1,62 +0,0 @@
-# Copyright 2012 OpenStack Foundation
-# All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
-# not use this file except in compliance with the License. You may obtain
-# a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-# License for the specific language governing permissions and limitations
-# under the License.
-
-from oslo_log import log as logging
-
-from tempest import clients as tempest_clients
-from tempest import config
-from tempest.lib.services import clients
-
-CONF = config.CONF
-LOG = logging.getLogger(__name__)
-
-
-class Manager(clients.ServiceClients):
- """Service client manager class for backward compatibility
-
- The former manager.Manager is not a stable interface in Tempest,
- nonetheless it is consumed by a number of plugins already. This class
- exists to provide some grace time for the move to tempest.lib.
- """
-
- def __init__(self, credentials, scope='project'):
- msg = ("tempest.manager.Manager is not a stable interface and as such "
- "it should not be imported directly. It will be removed as "
- "soon as the client manager becomes available in tempest.lib.")
- LOG.warning(msg)
- dscv = CONF.identity.disable_ssl_certificate_validation
- _, uri = tempest_clients.get_auth_provider_class(credentials)
- super(Manager, self).__init__(
- credentials=credentials, scope=scope,
- identity_uri=uri,
- disable_ssl_certificate_validation=dscv,
- ca_certs=CONF.identity.ca_certificates_file,
- trace_requests=CONF.debug.trace_requests)
-
-
-def get_auth_provider(credentials, pre_auth=False, scope='project'):
- """Shim to get_auth_provider in clients.py
-
- get_auth_provider used to be hosted in this module, but it has been
- moved to clients.py now as a more permanent location.
- This module will be removed eventually, and this shim is only
- maintained for the benefit of plugins already consuming this interface.
- """
- msg = ("tempest.manager.get_auth_provider is not a stable interface and "
- "as such it should not imported directly. It will be removed as "
- "the client manager becomes available in tempest.lib.")
- LOG.warning(msg)
- return tempest_clients.get_auth_provider(credentials=credentials,
- pre_auth=pre_auth, scope=scope)
diff --git a/tempest/scenario/manager.py b/tempest/scenario/manager.py
index 8866a22..f725b3f 100644
--- a/tempest/scenario/manager.py
+++ b/tempest/scenario/manager.py
@@ -375,21 +375,35 @@
def create_backup(self, volume_id, name=None, description=None,
force=False, snapshot_id=None, incremental=False,
- container=None):
- """Creates backup
+ container=None, **kwargs):
+ """Creates a backup of the given volume_id or snapshot_id
- This wrapper utility creates backup and waits for backup to be
- in 'available' state.
+ This wrapper utility creates a backup and waits until it is in
+ 'available' state.
+
+ :param volume_id: UUID of the volume to back up
+ :param name: backup name, '$classname-backup' by default
+ :param description: Description of the backup, None by default
+ :param force: boolean whether to backup even if the volume is attached
+ False by default
+ :param snapshot_id: UUID of the source snapshot to back up
+ None by default
+ :param incremental: boolean, False by default
+ :param container: a container name, None by default
+ :param **kwargs: additional parameters per the documentation:
+ https://docs.openstack.org/api-ref/block-storage/v3/
+ #create-a-backup
"""
name = name or data_utils.rand_name(
self.__class__.__name__ + "-backup")
- kwargs = {'name': name,
- 'description': description,
- 'force': force,
- 'snapshot_id': snapshot_id,
- 'incremental': incremental,
- 'container': container}
+ args = {'name': name,
+ 'description': description,
+ 'force': force,
+ 'snapshot_id': snapshot_id,
+ 'incremental': incremental,
+ 'container': container}
+ args.update(kwargs)
backup = self.backups_client.create_backup(volume_id=volume_id,
**kwargs)['backup']
self.addCleanup(self.backups_client.delete_backup, backup['id'])
@@ -397,14 +411,20 @@
backup['id'], 'available')
return backup
- def restore_backup(self, backup_id):
- """Restore backup
+ def restore_backup(self, backup_id, **kwargs):
+ """Restores a backup given by the backup_id
- This wrapper utility restores backup and waits for backup to be
- in 'available' state.
+ This wrapper utility restores a backup and waits until it is in
+ 'available' state.
+
+ :param backup_id: UUID of a backup to restore
+ :param **kwargs: additional parameters per the documentation:
+ https://docs.openstack.org/api-ref/block-storage/v3/
+ #restore-a-backup
"""
- restore = self.backups_client.restore_backup(backup_id)['restore']
+ body = self.backups_client.restore_backup(backup_id, **kwargs)
+ restore = body['restore']
self.addCleanup(self.volumes_client.delete_volume,
restore['volume_id'])
waiters.wait_for_volume_resource_status(self.backups_client,
@@ -515,7 +535,7 @@
self.addCleanup(self._cleanup_volume_type, volume_type)
return volume_type
- def _create_loginable_secgroup_rule(self, secgroup_id=None, rulesets=None):
+ def create_loginable_secgroup_rule(self, secgroup_id=None, rulesets=None):
"""Create loginable security group rule by compute clients.
This function will create by default the following rules:
@@ -575,7 +595,7 @@
secgroup['id'])
# Add rules to the security group
- self._create_loginable_secgroup_rule(secgroup['id'])
+ self.create_loginable_secgroup_rule(secgroup['id'])
return secgroup
def get_remote_client(self, ip_address, username=None, private_key=None,
@@ -1346,7 +1366,7 @@
project_id=project_id)
# Add rules to the security group
- rules = self._create_loginable_secgroup_rule(
+ rules = self.create_loginable_secgroup_rule(
security_group_rules_client=security_group_rules_client,
secgroup=secgroup,
security_groups_client=security_groups_client)
@@ -1387,10 +1407,10 @@
client.delete_security_group, secgroup['id'])
return secgroup
- def _create_security_group_rule(self, secgroup=None,
- sec_group_rules_client=None,
- project_id=None,
- security_groups_client=None, **kwargs):
+ def create_security_group_rule(self, secgroup=None,
+ sec_group_rules_client=None,
+ project_id=None,
+ security_groups_client=None, **kwargs):
"""Create a rule from a dictionary of rule parameters.
Create a rule in a secgroup. if secgroup not defined will search for
@@ -1435,9 +1455,9 @@
return sg_rule
- def _create_loginable_secgroup_rule(self, security_group_rules_client=None,
- secgroup=None,
- security_groups_client=None):
+ def create_loginable_secgroup_rule(self, security_group_rules_client=None,
+ secgroup=None,
+ security_groups_client=None):
"""Create loginable security group rule by neutron clients by default.
This function will create:
@@ -1474,7 +1494,7 @@
for r_direction in ['ingress', 'egress']:
ruleset['direction'] = r_direction
try:
- sg_rule = self._create_security_group_rule(
+ sg_rule = self.create_security_group_rule(
sec_group_rules_client=sec_group_rules_client,
secgroup=secgroup,
security_groups_client=security_groups_client,
@@ -1490,7 +1510,7 @@
return rules
- def _get_router(self, client=None, project_id=None):
+ def _get_router(self, client=None, project_id=None, **kwargs):
"""Retrieve a router for the given tenant id.
If a public router has been configured, it will be returned.
@@ -1510,11 +1530,20 @@
body = client.show_router(router_id)
return body['router']
elif network_id:
+ name = kwargs.pop('name', None)
+ if not name:
+ namestart = self.__class__.__name__ + '-router'
+ name = data_utils.rand_name(namestart)
+
+ ext_gw_info = kwargs.pop('external_gateway_info', None)
+ if not ext_gw_info:
+ ext_gw_info = dict(network_id=network_id)
router = client.create_router(
- name=data_utils.rand_name(self.__class__.__name__ + '-router'),
- admin_state_up=True,
+ name=name,
+ admin_state_up=kwargs.get('admin_state_up', True),
project_id=project_id,
- external_gateway_info=dict(network_id=network_id))['router']
+ external_gateway_info=ext_gw_info,
+ **kwargs)['router']
self.addCleanup(test_utils.call_and_ignore_notfound_exc,
client.delete_router, router['id'])
return router
diff --git a/tempest/scenario/test_security_groups_basic_ops.py b/tempest/scenario/test_security_groups_basic_ops.py
index 03a4a39..496a371 100644
--- a/tempest/scenario/test_security_groups_basic_ops.py
+++ b/tempest/scenario/test_security_groups_basic_ops.py
@@ -217,7 +217,7 @@
direction='ingress',
)
sec_group_rules_client = tenant.manager.security_group_rules_client
- self._create_security_group_rule(
+ self.create_security_group_rule(
secgroup=access_sg,
sec_group_rules_client=sec_group_rules_client,
**ssh_rule)
@@ -385,7 +385,7 @@
remote_group_id=tenant.security_groups['default']['id'],
direction='ingress'
)
- self._create_security_group_rule(
+ self.create_security_group_rule(
secgroup=tenant.security_groups['default'],
security_groups_client=tenant.manager.security_groups_client,
**ruleset
@@ -413,7 +413,7 @@
protocol = ruleset['protocol']
sec_group_rules_client = (
dest_tenant.manager.security_group_rules_client)
- self._create_security_group_rule(
+ self.create_security_group_rule(
secgroup=dest_tenant.security_groups['default'],
sec_group_rules_client=sec_group_rules_client,
**ruleset
@@ -429,7 +429,7 @@
# allow reverse traffic and check
sec_group_rules_client = (
source_tenant.manager.security_group_rules_client)
- self._create_security_group_rule(
+ self.create_security_group_rule(
secgroup=source_tenant.security_groups['default'],
sec_group_rules_client=sec_group_rules_client,
**ruleset
@@ -543,7 +543,7 @@
direction='ingress',
)
sec_group_rules_client = new_tenant.manager.security_group_rules_client
- self._create_security_group_rule(
+ self.create_security_group_rule(
secgroup=new_sg,
sec_group_rules_client=sec_group_rules_client,
**icmp_rule)
@@ -596,7 +596,7 @@
protocol='icmp',
direction='ingress'
)
- self._create_security_group_rule(
+ self.create_security_group_rule(
secgroup=tenant.security_groups['default'],
**ruleset
)
diff --git a/tempest/scenario/test_server_advanced_ops.py b/tempest/scenario/test_server_advanced_ops.py
index 8aa729b..990b325 100644
--- a/tempest/scenario/test_server_advanced_ops.py
+++ b/tempest/scenario/test_server_advanced_ops.py
@@ -37,7 +37,7 @@
@classmethod
def setup_credentials(cls):
- cls.set_network_resources()
+ cls.set_network_resources(network=True, subnet=True)
super(TestServerAdvancedOps, cls).setup_credentials()
@decorators.attr(type='slow')
diff --git a/tempest/test.py b/tempest/test.py
index f383bc1..68602d6 100644
--- a/tempest/test.py
+++ b/tempest/test.py
@@ -38,12 +38,6 @@
CONF = config.CONF
-# TODO(oomichi): This test.idempotent_id should be removed after all projects
-# switch to use decorators.idempotent_id.
-idempotent_id = debtcollector.moves.moved_function(
- decorators.idempotent_id, 'idempotent_id', __name__,
- version='Mitaka', removal_version='?')
-
attr = debtcollector.moves.moved_function(
decorators.attr, 'attr', __name__,
diff --git a/tempest/tests/cmd/test_run.py b/tempest/tests/cmd/test_run.py
index 3c99bbe..ec7b760 100644
--- a/tempest/tests/cmd/test_run.py
+++ b/tempest/tests/cmd/test_run.py
@@ -68,6 +68,11 @@
class TestRunReturnCode(base.TestCase):
+
+ exclude_regex = '--exclude-regex'
+ exclude_list = '--exclude-list'
+ include_list = '--include-list'
+
def setUp(self):
super(TestRunReturnCode, self).setUp()
# Setup test dirs
@@ -92,6 +97,14 @@
self.addCleanup(os.chdir, os.path.abspath(os.curdir))
os.chdir(self.directory)
+ def _get_test_list_file(self, content):
+ fd, path = tempfile.mkstemp()
+ self.addCleanup(os.remove, path)
+ test_file = os.fdopen(fd, 'wb', 0)
+ self.addCleanup(test_file.close)
+ test_file.write(content.encode('utf-8'))
+ return path
+
def assertRunExit(self, cmd, expected):
p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
@@ -115,19 +128,23 @@
subprocess.call(['stestr', 'init'])
self.assertRunExit(['tempest', 'run', '--regex', 'failing'], 1)
- def test_tempest_run_blackregex_failing(self):
- self.assertRunExit(['tempest', 'run', '--black-regex', 'failing'], 0)
+ def test_tempest_run_exclude_regex_failing(self):
+ self.assertRunExit(['tempest', 'run',
+ self.exclude_regex, 'failing'], 0)
- def test_tempest_run_blackregex_failing_with_stestr_repository(self):
+ def test_tempest_run_exclude_regex_failing_with_stestr_repository(self):
subprocess.call(['stestr', 'init'])
- self.assertRunExit(['tempest', 'run', '--black-regex', 'failing'], 0)
+ self.assertRunExit(['tempest', 'run',
+ self.exclude_regex, 'failing'], 0)
- def test_tempest_run_blackregex_passing(self):
- self.assertRunExit(['tempest', 'run', '--black-regex', 'passing'], 1)
+ def test_tempest_run_exclude_regex_passing(self):
+ self.assertRunExit(['tempest', 'run',
+ self.exclude_regex, 'passing'], 1)
- def test_tempest_run_blackregex_passing_with_stestr_repository(self):
+ def test_tempest_run_exclude_regex_passing_with_stestr_repository(self):
subprocess.call(['stestr', 'init'])
- self.assertRunExit(['tempest', 'run', '--black-regex', 'passing'], 1)
+ self.assertRunExit(['tempest', 'run',
+ self.exclude_regex, 'passing'], 1)
def test_tempest_run_fails(self):
self.assertRunExit(['tempest', 'run'], 1)
@@ -149,47 +166,31 @@
self.assertEqual(result, tests)
def test_tempest_run_with_worker_file(self):
- fd, path = tempfile.mkstemp()
- self.addCleanup(os.remove, path)
- worker_file = os.fdopen(fd, 'wb', 0)
- self.addCleanup(worker_file.close)
- worker_file.write(
- '- worker:\n - passing\n concurrency: 3'.encode('utf-8'))
+ path = self._get_test_list_file(
+ '- worker:\n - passing\n concurrency: 3')
self.assertRunExit(['tempest', 'run', '--worker-file=%s' % path], 0)
- def test_tempest_run_with_whitelist(self):
- fd, path = tempfile.mkstemp()
- self.addCleanup(os.remove, path)
- whitelist_file = os.fdopen(fd, 'wb', 0)
- self.addCleanup(whitelist_file.close)
- whitelist_file.write('passing'.encode('utf-8'))
- self.assertRunExit(['tempest', 'run', '--whitelist-file=%s' % path], 0)
+ def test_tempest_run_with_include_list(self):
+ path = self._get_test_list_file('passing')
+ self.assertRunExit(['tempest', 'run',
+ '%s=%s' % (self.include_list, path)], 0)
- def test_tempest_run_with_whitelist_regex_include_pass_check_fail(self):
- fd, path = tempfile.mkstemp()
- self.addCleanup(os.remove, path)
- whitelist_file = os.fdopen(fd, 'wb', 0)
- self.addCleanup(whitelist_file.close)
- whitelist_file.write('passing'.encode('utf-8'))
- self.assertRunExit(['tempest', 'run', '--whitelist-file=%s' % path,
+ def test_tempest_run_with_include_regex_include_pass_check_fail(self):
+ path = self._get_test_list_file('passing')
+ self.assertRunExit(['tempest', 'run',
+ '%s=%s' % (self.include_list, path),
'--regex', 'fail'], 1)
- def test_tempest_run_with_whitelist_regex_include_pass_check_pass(self):
- fd, path = tempfile.mkstemp()
- self.addCleanup(os.remove, path)
- whitelist_file = os.fdopen(fd, 'wb', 0)
- self.addCleanup(whitelist_file.close)
- whitelist_file.write('passing'.encode('utf-8'))
- self.assertRunExit(['tempest', 'run', '--whitelist-file=%s' % path,
+ def test_tempest_run_with_include_regex_include_pass_check_pass(self):
+ path = self._get_test_list_file('passing')
+ self.assertRunExit(['tempest', 'run',
+ '%s=%s' % (self.include_list, path),
'--regex', 'passing'], 0)
- def test_tempest_run_with_whitelist_regex_include_fail_check_pass(self):
- fd, path = tempfile.mkstemp()
- self.addCleanup(os.remove, path)
- whitelist_file = os.fdopen(fd, 'wb', 0)
- self.addCleanup(whitelist_file.close)
- whitelist_file.write('failing'.encode('utf-8'))
- self.assertRunExit(['tempest', 'run', '--whitelist-file=%s' % path,
+ def test_tempest_run_with_include_regex_include_fail_check_pass(self):
+ path = self._get_test_list_file('failing')
+ self.assertRunExit(['tempest', 'run',
+ '%s=%s' % (self.include_list, path),
'--regex', 'pass'], 1)
def test_tempest_run_passes_with_config_file(self):
@@ -197,50 +198,75 @@
'--config-file', self.stestr_conf_file,
'--regex', 'passing'], 0)
- def test_tempest_run_with_blacklist_failing(self):
- fd, path = tempfile.mkstemp()
- self.addCleanup(os.remove, path)
- blacklist_file = os.fdopen(fd, 'wb', 0)
- self.addCleanup(blacklist_file.close)
- blacklist_file.write('failing'.encode('utf-8'))
- self.assertRunExit(['tempest', 'run', '--blacklist-file=%s' % path], 0)
+ def test_tempest_run_with_exclude_list_failing(self):
+ path = self._get_test_list_file('failing')
+ self.assertRunExit(['tempest', 'run',
+ '%s=%s' % (self.exclude_list, path)], 0)
- def test_tempest_run_with_blacklist_passing(self):
- fd, path = tempfile.mkstemp()
- self.addCleanup(os.remove, path)
- blacklist_file = os.fdopen(fd, 'wb', 0)
- self.addCleanup(blacklist_file.close)
- blacklist_file.write('passing'.encode('utf-8'))
- self.assertRunExit(['tempest', 'run', '--blacklist-file=%s' % path], 1)
+ def test_tempest_run_with_exclude_list_passing(self):
+ path = self._get_test_list_file('passing')
+ self.assertRunExit(['tempest', 'run',
+ '%s=%s' % (self.exclude_list, path)], 1)
- def test_tempest_run_with_blacklist_regex_exclude_fail_check_pass(self):
- fd, path = tempfile.mkstemp()
- self.addCleanup(os.remove, path)
- blacklist_file = os.fdopen(fd, 'wb', 0)
- self.addCleanup(blacklist_file.close)
- blacklist_file.write('failing'.encode('utf-8'))
- self.assertRunExit(['tempest', 'run', '--blacklist-file=%s' % path,
+ def test_tempest_run_with_exclude_list_regex_exclude_fail_check_pass(self):
+ path = self._get_test_list_file('failing')
+ self.assertRunExit(['tempest', 'run',
+ '%s=%s' % (self.exclude_list, path),
'--regex', 'pass'], 0)
- def test_tempest_run_with_blacklist_regex_exclude_pass_check_pass(self):
- fd, path = tempfile.mkstemp()
- self.addCleanup(os.remove, path)
- blacklist_file = os.fdopen(fd, 'wb', 0)
- self.addCleanup(blacklist_file.close)
- blacklist_file.write('passing'.encode('utf-8'))
- self.assertRunExit(['tempest', 'run', '--blacklist-file=%s' % path,
+ def test_tempest_run_with_exclude_list_regex_exclude_pass_check_pass(self):
+ path = self._get_test_list_file('passing')
+ self.assertRunExit(['tempest', 'run',
+ '%s=%s' % (self.exclude_list, path),
'--regex', 'pass'], 1)
- def test_tempest_run_with_blacklist_regex_exclude_pass_check_fail(self):
- fd, path = tempfile.mkstemp()
- self.addCleanup(os.remove, path)
- blacklist_file = os.fdopen(fd, 'wb', 0)
- self.addCleanup(blacklist_file.close)
- blacklist_file.write('passing'.encode('utf-8'))
- self.assertRunExit(['tempest', 'run', '--blacklist-file=%s' % path,
+ def test_tempest_run_with_exclude_list_regex_exclude_pass_check_fail(self):
+ path = self._get_test_list_file('passing')
+ self.assertRunExit(['tempest', 'run',
+ '%s=%s' % (self.exclude_list, path),
'--regex', 'fail'], 1)
+class TestOldArgRunReturnCode(TestRunReturnCode):
+ """A class for testing deprecated but still supported args.
+
+ This class will be removed once we remove the following arguments:
+ * --black-regex
+ * --blacklist-file
+ * --whitelist-file
+ """
+ exclude_regex = '--black-regex'
+ exclude_list = '--blacklist-file'
+ include_list = '--whitelist-file'
+
+ def _test_args_passing(self, args):
+ self.assertRunExit(['tempest', 'run'] + args, 0)
+
+ def test_tempest_run_new_old_arg_comb(self):
+ path = self._get_test_list_file('failing')
+ self._test_args_passing(['--black-regex', 'failing',
+ '--exclude-regex', 'failing'])
+ self._test_args_passing(['--blacklist-file=' + path,
+ '--exclude-list=' + path])
+ path = self._get_test_list_file('passing')
+ self._test_args_passing(['--whitelist-file=' + path,
+ '--include-list=' + path])
+
+ def _test_args_passing_with_stestr_repository(self, args):
+ subprocess.call(['stestr', 'init'])
+ self.assertRunExit(['tempest', 'run'] + args, 0)
+
+ def test_tempest_run_new_old_arg_comb_with_stestr_repository(self):
+ path = self._get_test_list_file('failing')
+ self._test_args_passing_with_stestr_repository(
+ ['--black-regex', 'failing', '--exclude-regex', 'failing'])
+ self._test_args_passing_with_stestr_repository(
+ ['--blacklist-file=' + path, '--exclude-list=' + path])
+ path = self._get_test_list_file('passing')
+ self._test_args_passing_with_stestr_repository(
+ ['--whitelist-file=' + path, '--include-list=' + path])
+
+
class TestConfigPathCheck(base.TestCase):
def setUp(self):
super(TestConfigPathCheck, self).setUp()
diff --git a/tempest/tests/common/test_waiters.py b/tempest/tests/common/test_waiters.py
index ff74877..d64d7b0 100755
--- a/tempest/tests/common/test_waiters.py
+++ b/tempest/tests/common/test_waiters.py
@@ -66,7 +66,7 @@
# Ensure waiter returns before build_timeout
self.assertLess((end_time - start_time), 10)
- def test_wait_for_image_imported_to_stores_timeout(self):
+ def test_wait_for_image_imported_to_stores_failure(self):
time_mock = self.patch('time.time')
client = mock.MagicMock()
client.build_timeout = 2
@@ -77,6 +77,20 @@
'status': 'saving',
'stores': 'fake_store',
'os_glance_failed_import': 'fake_os_glance_failed_import'})
+ self.assertRaises(lib_exc.OtherRestClientException,
+ waiters.wait_for_image_imported_to_stores,
+ client, 'fake_image_id', 'fake_store')
+
+ def test_wait_for_image_imported_to_stores_timeout(self):
+ time_mock = self.patch('time.time')
+ client = mock.MagicMock()
+ client.build_timeout = 2
+ self.patch('time.time', side_effect=[0., 1., 2.])
+ time_mock.side_effect = utils.generate_timeout_series(1)
+
+ client.show_image.return_value = ({
+ 'status': 'saving',
+ 'stores': 'fake_store'})
self.assertRaises(lib_exc.TimeoutException,
waiters.wait_for_image_imported_to_stores,
client, 'fake_image_id', 'fake_store')
diff --git a/tempest/tests/test_decorators.py b/tempest/tests/test_decorators.py
index 6018441..1889420 100644
--- a/tempest/tests/test_decorators.py
+++ b/tempest/tests/test_decorators.py
@@ -19,7 +19,6 @@
from tempest.common import utils
from tempest import config
from tempest import exceptions
-from tempest.lib.common.utils import data_utils
from tempest import test
from tempest.tests import base
from tempest.tests import fake_config
@@ -33,47 +32,6 @@
fake_config.FakePrivate)
-# NOTE: The test module is for tempest.test.idempotent_id.
-# After all projects switch to use decorators.idempotent_id,
-# we can remove tempest.test.idempotent_id as well as this
-# test module
-class TestIdempotentIdDecorator(BaseDecoratorsTest):
-
- def _test_helper(self, _id, **decorator_args):
- @test.idempotent_id(_id)
- def foo():
- """Docstring"""
- pass
-
- return foo
-
- def _test_helper_without_doc(self, _id, **decorator_args):
- @test.idempotent_id(_id)
- def foo():
- pass
-
- return foo
-
- def test_positive(self):
- _id = data_utils.rand_uuid()
- foo = self._test_helper(_id)
- self.assertIn('id-%s' % _id, getattr(foo, '__testtools_attrs'))
- self.assertTrue(foo.__doc__.startswith('Test idempotent id: %s' % _id))
-
- def test_positive_without_doc(self):
- _id = data_utils.rand_uuid()
- foo = self._test_helper_without_doc(_id)
- self.assertTrue(foo.__doc__.startswith('Test idempotent id: %s' % _id))
-
- def test_idempotent_id_not_str(self):
- _id = 42
- self.assertRaises(TypeError, self._test_helper, _id)
-
- def test_idempotent_id_not_valid_uuid(self):
- _id = '42'
- self.assertRaises(ValueError, self._test_helper, _id)
-
-
class TestServicesDecorator(BaseDecoratorsTest):
def _test_services_helper(self, *decorator_args):
class TestFoo(test.BaseTestCase):
diff --git a/tools/check_logs.py b/tools/check_logs.py
index de7e41d..7e191a0 100755
--- a/tools/check_logs.py
+++ b/tools/check_logs.py
@@ -56,39 +56,39 @@
's-proxy'])
-def process_files(file_specs, url_specs, whitelists):
+def process_files(file_specs, url_specs, allow_lists):
regexp = re.compile(r"^.* (ERROR|CRITICAL|TRACE) .*\[.*\-.*\]")
logs_with_errors = []
for (name, filename) in file_specs:
- whitelist = whitelists.get(name, [])
+ allow_list = allow_lists.get(name, [])
with open(filename) as content:
- if scan_content(content, regexp, whitelist):
+ if scan_content(content, regexp, allow_list):
logs_with_errors.append(name)
for (name, url) in url_specs:
- whitelist = whitelists.get(name, [])
+ allow_list = allow_lists.get(name, [])
req = urlreq.Request(url)
req.add_header('Accept-Encoding', 'gzip')
page = urlreq.urlopen(req)
buf = six.StringIO(page.read())
f = gzip.GzipFile(fileobj=buf)
- if scan_content(f.read().splitlines(), regexp, whitelist):
+ if scan_content(f.read().splitlines(), regexp, allow_list):
logs_with_errors.append(name)
return logs_with_errors
-def scan_content(content, regexp, whitelist):
+def scan_content(content, regexp, allow_list):
had_errors = False
for line in content:
if not line.startswith("Stderr:") and regexp.match(line):
- whitelisted = False
- for w in whitelist:
+ allowed = False
+ for w in allow_list:
pat = ".*%s.*%s.*" % (w['module'].replace('.', '\\.'),
w['message'])
if re.match(pat, line):
- whitelisted = True
+ allowed = True
break
- if not whitelisted or dump_all_errors:
- if not whitelisted:
+ if not allowed or dump_all_errors:
+ if not allowed:
had_errors = True
return had_errors
@@ -105,9 +105,9 @@
print("Must provide exactly one of -d or -u")
return 1
print("Checking logs...")
- WHITELIST_FILE = os.path.join(
+ ALLOW_LIST_FILE = os.path.join(
os.path.abspath(os.path.dirname(os.path.dirname(__file__))),
- "etc", "whitelist.yaml")
+ "etc", "allow-list.yaml")
file_matcher = re.compile(r".*screen-([\w-]+)\.log")
files = []
@@ -132,17 +132,17 @@
if m:
urls_to_process.append((m.group(1), u))
- whitelists = {}
- with open(WHITELIST_FILE) as stream:
+ allow_lists = {}
+ with open(ALLOW_LIST_FILE) as stream:
loaded = yaml.safe_load(stream)
if loaded:
for (name, l) in six.iteritems(loaded):
for w in l:
assert 'module' in w, 'no module in %s' % name
assert 'message' in w, 'no message in %s' % name
- whitelists = loaded
+ allow_lists = loaded
logs_with_errors = process_files(files_to_process, urls_to_process,
- whitelists)
+ allow_lists)
failed = False
if logs_with_errors:
@@ -164,14 +164,14 @@
usage = """
-Find non-white-listed log errors in log files from a devstack-gate run.
+Find non-allow-listed log errors in log files from a devstack-gate run.
Log files will be searched for ERROR or CRITICAL messages. If any
-error messages do not match any of the whitelist entries contained in
-etc/whitelist.yaml, those messages will be printed to the console and
+error messages do not match any of the allow-list entries contained in
+etc/allow-list.yaml, those messages will be printed to the console and
failure will be returned. A file directory containing logs or a url to the
log files of an OpenStack gate job can be provided.
-The whitelist yaml looks like:
+The allow-list yaml looks like:
log-name:
- module: "a.b.c"
@@ -179,7 +179,7 @@
- module: "a.b.c"
message: "regexp"
-repeated for each log file with a whitelist.
+repeated for each log file with an allow-list.
"""
parser = argparse.ArgumentParser(description=usage)
diff --git a/tools/generate-tempest-plugins-list.py b/tools/generate-tempest-plugins-list.py
index 618c388..1b5b369 100644
--- a/tools/generate-tempest-plugins-list.py
+++ b/tools/generate-tempest-plugins-list.py
@@ -32,9 +32,9 @@
# List of projects having tempest plugin stale or unmaintained for a long time
# (6 months or more)
-# TODO(masayukig): Some of these can be removed from BLACKLIST in the future
-# when the patches are merged.
-BLACKLIST = [
+# TODO(masayukig): Some of these can be removed from NON_ACTIVE_LIST in the
+# future when the patches are merged.
+NON_ACTIVE_LIST = [
'x/gce-api', # It looks gce-api doesn't support python3 yet.
'x/glare', # To avoid sanity-job failure
'x/group-based-policy', # It looks this doesn't support python3 yet.
@@ -52,8 +52,11 @@
'x/tap-as-a-service', # To avoid sanity-job failure
'x/valet', # https://review.opendev.org/#/c/638339/
'x/kingbird', # https://bugs.launchpad.net/kingbird/+bug/1869722
- # vmware-nsx is blacklisted since https://review.opendev.org/#/c/736952
+ # vmware-nsx is excluded since https://review.opendev.org/#/c/736952
'x/vmware-nsx-tempest-plugin',
+ # mogan is unmaintained now, remove from the list when this is merged:
+ # https://review.opendev.org/c/x/mogan/+/767718
+ 'x/mogan',
]
url = 'https://review.opendev.org/projects/'
@@ -86,10 +89,10 @@
False
-if len(sys.argv) > 1 and sys.argv[1] == 'blacklist':
- for black_plugin in BLACKLIST:
- print(black_plugin)
- # We just need BLACKLIST when we use this `blacklist` option.
+if len(sys.argv) > 1 and sys.argv[1] == 'nonactivelist':
+ for non_active_plugin in NON_ACTIVE_LIST:
+ print(non_active_plugin)
+ # We just need NON_ACTIVE_LIST when we use this `nonactivelist` option.
# So, this exits here.
sys.exit()
diff --git a/tools/generate-tempest-plugins-list.sh b/tools/generate-tempest-plugins-list.sh
index 33675ed..4430bbf 100755
--- a/tools/generate-tempest-plugins-list.sh
+++ b/tools/generate-tempest-plugins-list.sh
@@ -81,17 +81,17 @@
printf "\n\n"
-# Print BLACKLIST
-if [[ -r doc/source/data/tempest-blacklisted-plugins-registry.header ]]; then
- cat doc/source/data/tempest-blacklisted-plugins-registry.header
+# Print NON_ACTIVE_LIST
+if [[ -r doc/source/data/tempest-non-active-plugins-registry.header ]]; then
+ cat doc/source/data/tempest-non-active-plugins-registry.header
fi
-blacklist=$(python tools/generate-tempest-plugins-list.py blacklist)
-name_col_len=$(echo "${blacklist}" | wc -L)
+nonactivelist=$(python tools/generate-tempest-plugins-list.py nonactivelist)
+name_col_len=$(echo "${nonactivelist}" | wc -L)
name_col_len=$(( name_col_len + 20 ))
printf "\n\n"
-print_plugin_table "${blacklist}"
+print_plugin_table "${nonactivelist}"
printf "\n\n"
diff --git a/tools/tempest-integrated-gate-compute-blacklist.txt b/tools/tempest-integrated-gate-compute-exclude-list.txt
similarity index 100%
rename from tools/tempest-integrated-gate-compute-blacklist.txt
rename to tools/tempest-integrated-gate-compute-exclude-list.txt
diff --git a/tools/tempest-integrated-gate-networking-blacklist.txt b/tools/tempest-integrated-gate-networking-exclude-list.txt
similarity index 100%
rename from tools/tempest-integrated-gate-networking-blacklist.txt
rename to tools/tempest-integrated-gate-networking-exclude-list.txt
diff --git a/tools/tempest-integrated-gate-object-storage-blacklist.txt b/tools/tempest-integrated-gate-object-storage-exclude-list.txt
similarity index 100%
rename from tools/tempest-integrated-gate-object-storage-blacklist.txt
rename to tools/tempest-integrated-gate-object-storage-exclude-list.txt
diff --git a/tools/tempest-integrated-gate-placement-blacklist.txt b/tools/tempest-integrated-gate-placement-exclude-list.txt
similarity index 100%
rename from tools/tempest-integrated-gate-placement-blacklist.txt
rename to tools/tempest-integrated-gate-placement-exclude-list.txt
diff --git a/tools/tempest-integrated-gate-storage-blacklist.txt b/tools/tempest-integrated-gate-storage-blacklist.txt
new file mode 120000
index 0000000..2d691f8
--- /dev/null
+++ b/tools/tempest-integrated-gate-storage-blacklist.txt
@@ -0,0 +1 @@
+tempest-integrated-gate-storage-exclude-list.txt
\ No newline at end of file
diff --git a/tools/tempest-integrated-gate-storage-blacklist.txt b/tools/tempest-integrated-gate-storage-exclude-list.txt
similarity index 100%
rename from tools/tempest-integrated-gate-storage-blacklist.txt
rename to tools/tempest-integrated-gate-storage-exclude-list.txt
diff --git a/tools/tempest-plugin-sanity.sh b/tools/tempest-plugin-sanity.sh
index c983da9..106a9c6 100644
--- a/tools/tempest-plugin-sanity.sh
+++ b/tools/tempest-plugin-sanity.sh
@@ -44,7 +44,7 @@
# retrieve a list of projects having tempest plugins
PROJECT_LIST="$(python tools/generate-tempest-plugins-list.py)"
-BLACKLIST="$(python tools/generate-tempest-plugins-list.py blacklist)"
+NON_ACTIVE_LIST="$(python tools/generate-tempest-plugins-list.py nonactivelist)"
# Function to clone project using zuul-cloner or from git
function clone_project {
@@ -117,8 +117,8 @@
failed_plugin=''
# Perform sanity on all tempest plugin projects
for project in $PROJECT_LIST; do
- # Remove blacklisted tempest plugins
- if ! [[ `echo $BLACKLIST | grep -c $project ` -gt 0 ]]; then
+ # Remove non-active tempest plugins
+ if ! [[ `echo $NON_ACTIVE_LIST | grep -c $project ` -gt 0 ]]; then
plugin_sanity_check $project && passed_plugin+=", $project" || \
failed_plugin+="$project, " > $SANITY_DIR/$project.txt
fi
diff --git a/tox.ini b/tox.ini
index d8e059a..2315163 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,6 +1,6 @@
[tox]
envlist = pep8,py36,py38,bashate,pip-check-reqs
-minversion = 3.1.1
+minversion = 3.18.0
skipsdist = True
ignore_basepython_conflict = True
@@ -26,7 +26,7 @@
passenv = OS_STDOUT_CAPTURE OS_STDERR_CAPTURE OS_TEST_TIMEOUT OS_TEST_LOCK_PATH TEMPEST_CONFIG TEMPEST_CONFIG_DIR http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY ZUUL_CACHE_DIR REQUIREMENTS_PIP_LOCATION GENERATE_TEMPEST_PLUGIN_LIST
usedevelop = True
install_command = pip install {opts} {packages}
-whitelist_externals = *
+allowlist_externals = *
deps =
-c{env:UPPER_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master}
-r{toxinidir}/requirements.txt
@@ -108,7 +108,7 @@
deps = {[tempestenv]deps}
# The regex below is used to select which tests to run and exclude the slow tag:
# See the testrepository bug: https://bugs.launchpad.net/testrepository/+bug/1208610
-# FIXME: We can replace it with the `--black-regex` option to exclude tests now.
+# FIXME: We can replace it with the `--exclude-regex` option to exclude tests now.
commands =
find . -type f -name "*.pyc" -delete
tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' {posargs}
@@ -132,11 +132,11 @@
setenv = {[tempestenv]setenv}
deps = {[tempestenv]deps}
# The regex below is used to select which tests to run and exclude the slow tag and
-# tests listed in blacklist file:
+# tests listed in exclude-list file:
commands =
find . -type f -name "*.pyc" -delete
- tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' --blacklist_file ./tools/tempest-integrated-gate-networking-blacklist.txt {posargs}
- tempest run --combine --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.scenario)' --blacklist_file ./tools/tempest-integrated-gate-networking-blacklist.txt {posargs}
+ tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' --exclude-list ./tools/tempest-integrated-gate-networking-exclude-list.txt {posargs}
+ tempest run --combine --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.scenario)' --exclude-list ./tools/tempest-integrated-gate-networking-exclude-list.txt {posargs}
[testenv:integrated-compute]
envdir = .tox/tempest
@@ -145,11 +145,11 @@
setenv = {[tempestenv]setenv}
deps = {[tempestenv]deps}
# The regex below is used to select which tests to run and exclude the slow tag and
-# tests listed in blacklist file:
+# tests listed in exclude-list file:
commands =
find . -type f -name "*.pyc" -delete
- tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' --blacklist_file ./tools/tempest-integrated-gate-compute-blacklist.txt {posargs}
- tempest run --combine --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.scenario)' --blacklist_file ./tools/tempest-integrated-gate-compute-blacklist.txt {posargs}
+ tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' --exclude-list ./tools/tempest-integrated-gate-compute-exclude-list.txt {posargs}
+ tempest run --combine --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.scenario)' --exclude-list ./tools/tempest-integrated-gate-compute-exclude-list.txt {posargs}
[testenv:integrated-placement]
envdir = .tox/tempest
@@ -158,11 +158,11 @@
setenv = {[tempestenv]setenv}
deps = {[tempestenv]deps}
# The regex below is used to select which tests to run and exclude the slow tag and
-# tests listed in blacklist file:
+# tests listed in exclude-list file:
commands =
find . -type f -name "*.pyc" -delete
- tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' --blacklist_file ./tools/tempest-integrated-gate-placement-blacklist.txt {posargs}
- tempest run --combine --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.scenario)' --blacklist_file ./tools/tempest-integrated-gate-placement-blacklist.txt {posargs}
+ tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' --exclude-list ./tools/tempest-integrated-gate-placement-exclude-list.txt {posargs}
+ tempest run --combine --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.scenario)' --exclude-list ./tools/tempest-integrated-gate-placement-exclude-list.txt {posargs}
[testenv:integrated-storage]
envdir = .tox/tempest
@@ -171,11 +171,11 @@
setenv = {[tempestenv]setenv}
deps = {[tempestenv]deps}
# The regex below is used to select which tests to run and exclude the slow tag and
-# tests listed in blacklist file:
+# tests listed in exclude-list file:
commands =
find . -type f -name "*.pyc" -delete
- tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' --blacklist_file ./tools/tempest-integrated-gate-storage-blacklist.txt {posargs}
- tempest run --combine --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.scenario)' --blacklist_file ./tools/tempest-integrated-gate-storage-blacklist.txt {posargs}
+ tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' --exclude-list ./tools/tempest-integrated-gate-storage-exclude-list.txt {posargs}
+ tempest run --combine --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.scenario)' --exclude-list ./tools/tempest-integrated-gate-storage-exclude-list.txt {posargs}
[testenv:integrated-object-storage]
envdir = .tox/tempest
@@ -184,11 +184,11 @@
setenv = {[tempestenv]setenv}
deps = {[tempestenv]deps}
# The regex below is used to select which tests to run and exclude the slow tag and
-# tests listed in blacklist file:
+# tests listed in exclude-list file:
commands =
find . -type f -name "*.pyc" -delete
- tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' --blacklist_file ./tools/tempest-integrated-gate-object-storage-blacklist.txt {posargs}
- tempest run --combine --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.scenario)' --blacklist_file ./tools/tempest-integrated-gate-object-storage-blacklist.txt {posargs}
+ tempest run --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.api)' --exclude-list ./tools/tempest-integrated-gate-object-storage-exclude-list.txt {posargs}
+ tempest run --combine --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.scenario)' --exclude-list ./tools/tempest-integrated-gate-object-storage-exclude-list.txt {posargs}
[testenv:full-serial]
envdir = .tox/tempest
@@ -198,7 +198,7 @@
deps = {[tempestenv]deps}
# The regex below is used to select which tests to run and exclude the slow tag:
# See the testrepository bug: https://bugs.launchpad.net/testrepository/+bug/1208610
-# FIXME: We can replace it with the `--black-regex` option to exclude tests now.
+# FIXME: We can replace it with the `--exclude-regex` option to exclude tests now.
commands =
find . -type f -name "*.pyc" -delete
tempest run --serial --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.(api|scenario))' {posargs}
@@ -290,12 +290,12 @@
sphinx-apidoc -f -o doc/source/tests/volume tempest/api/volume
rm -rf doc/build
sphinx-build -W -b html doc/source doc/build/html
-whitelist_externals =
+allowlist_externals =
rm
[testenv:pdf-docs]
deps = {[testenv:docs]deps}
-whitelist_externals =
+allowlist_externals =
rm
make
commands =
@@ -369,7 +369,7 @@
rm -rf releasenotes/build
sphinx-build -a -E -W -d releasenotes/build/doctrees \
-b html releasenotes/source releasenotes/build/html
-whitelist_externals = rm
+allowlist_externals = rm
[testenv:bashate]
# if you want to test out some changes you have made to bashate
@@ -377,7 +377,7 @@
# modified bashate tree
deps =
{env:BASHATE_INSTALL_PATH:bashate}
-whitelist_externals = bash
+allowlist_externals = bash
commands = bash -c "find {toxinidir}/tools \
-not \( -type d -name .?\* -prune \) \
-type f \
@@ -406,6 +406,6 @@
[testenv:plugin-sanity-check]
# perform tempest plugin sanity
-whitelist_externals = bash
+allowlist_externals = bash
commands =
bash tools/tempest-plugin-sanity.sh