Merge "Allow tempest cleanup delete resources based on prefix"
diff --git a/HACKING.rst b/HACKING.rst
index caf954b..23bc61b 100644
--- a/HACKING.rst
+++ b/HACKING.rst
@@ -184,7 +184,7 @@
the ``slow`` attribute is leveraged to run slow tests on a selective basis,
to keep total `Zuul`_ job runtime down to a reasonable time frame.
-.. _Zuul: https://docs.openstack.org/infra/zuul/
+.. _Zuul: https://zuul-ci.org/docs/zuul/latest/
Smoke Attribute
^^^^^^^^^^^^^^^
@@ -488,7 +488,7 @@
Otherwise the bug fix won't be able to land in the project.
Handily, `Zuul's cross-repository dependencies
-<https://docs.openstack.org/infra/zuul/user/gating.html#cross-project-dependencies>`_.
+<https://zuul-ci.org/docs/zuul/latest/gating.html#cross-project-dependencies>`_.
can be leveraged to do without step 2 and to have steps 3 and 4 happen
"atomically". To do that, make the patch written in step 1 to depend (refer to
Zuul's documentation above) on the patch written in step 4. The commit message
diff --git a/README.rst b/README.rst
index 3cde2bf..7880357 100644
--- a/README.rst
+++ b/README.rst
@@ -3,7 +3,6 @@
========================
.. image:: https://governance.openstack.org/tc/badges/tempest.svg
- :target: https://governance.openstack.org/tc/reference/tags/index.html
.. Change things from this point on
diff --git a/doc/source/contributor/contributing.rst b/doc/source/contributor/contributing.rst
index 139f0b7..81a1874 100644
--- a/doc/source/contributor/contributing.rst
+++ b/doc/source/contributor/contributing.rst
@@ -14,8 +14,9 @@
Communication
~~~~~~~~~~~~~
* IRC channel ``#openstack-qa`` at OFTC
-* Mailing list (prefix subjects with ``[qa]`` for faster responses)
- http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss
+* `Mailing list <https://lists.openstack.org/mailman3/lists/openstack-discuss.lists.openstack.org/>`_
+ (prefix subjects with ``[qa]`` for faster responses)
+
Contacting the Core Team
~~~~~~~~~~~~~~~~~~~~~~~~
@@ -26,25 +27,29 @@
~~~~~~~~~~~~~~~~~~~~
If you want to propose a new feature please read `Feature Proposal Process`_
Tempest features are tracked on `Launchpad BP <https://blueprints.launchpad.net/tempest>`_.
+It also helps to bring the feature up during the next PTG and contact
+`the current PTL of QA project <https://governance.openstack.org/tc/reference/projects/>`_.
+Information about PTG is always posted on `the Mailing list
+<https://lists.openstack.org/mailman3/lists/openstack-discuss.lists.openstack.org/>`_.
Task Tracking
~~~~~~~~~~~~~
We track our tasks in `Launchpad <https://bugs.launchpad.net/tempest>`_.
-If you're looking for some smaller, easier work item to pick up and get started
+If you're looking for some smaller, easier work items to pick up and get started
on, search for the 'low-hanging-fruit' tag.
Reporting a Bug
~~~~~~~~~~~~~~~
-You found an issue and want to make sure we are aware of it? You can do so on
-`Launchpad <https://bugs.launchpad.net/tempest/+filebug>`__.
+Have you found an issue and want to make sure we are aware of it? You can do so
+on `Launchpad <https://bugs.launchpad.net/tempest/+filebug>`__.
More info about Launchpad usage can be found on `OpenStack docs page
<https://docs.openstack.org/contributors/common/task-tracking.html#launchpad>`_
Getting Your Patch Merged
~~~~~~~~~~~~~~~~~~~~~~~~~
-All changes proposed to the Tempest require single ``Code-Review +2`` votes from
-Tempest core reviewers by giving ``Workflow +1`` vote. More detailed guidelines
+All changes proposed to the Tempest require a single ``Code-Review +2`` vote
+from a Tempest core followed by a ``Workflow +1`` vote. More detailed guidelines
for reviewers are available at :doc:`../REVIEWING`.
Project Team Lead Duties
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 2f29cf2..0340f8d 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -83,6 +83,7 @@
:maxdepth: 2
HACKING
+ serial_tests
REVIEWING
microversion_testing
test_removal
diff --git a/doc/source/library/api_microversion_testing.rst b/doc/source/library/api_microversion_testing.rst
index 8be924d..e979683 100644
--- a/doc/source/library/api_microversion_testing.rst
+++ b/doc/source/library/api_microversion_testing.rst
@@ -9,9 +9,9 @@
Many of the OpenStack components have implemented API microversions.
It is important to test those microversions in Tempest or external plugins.
-Tempest now provides stable interfaces to support to test the API microversions.
+Tempest now provides stable interfaces to support testing the API microversions.
Based on the microversion range coming from the combination of both configuration
-and each test case, APIs request will be made with selected microversion.
+and each test case, APIs requests will be made with the selected microversion.
This document explains the interfaces needed for microversion testing.
diff --git a/doc/source/library/clients.rst b/doc/source/library/clients.rst
index 0f4ba4c..fe9f4ca 100644
--- a/doc/source/library/clients.rst
+++ b/doc/source/library/clients.rst
@@ -6,11 +6,11 @@
Tests make requests against APIs using service clients. Service clients are
specializations of the ``RestClient`` class. The service clients that cover the
APIs exposed by a service should be grouped in a service clients module.
-A service clients module is python module where all service clients are
+A service clients module is a Python module where all service clients are
defined. If major API versions are available, submodules should be defined,
one for each version.
-The ``ClientsFactory`` class helps initializing all clients of a specific
+The ``ClientsFactory`` class helps to initialize all clients of a specific
service client module from a set of shared parameters.
The ``ServiceClients`` class provides a convenient way to get access to all
diff --git a/doc/source/library/credential_providers.rst b/doc/source/library/credential_providers.rst
index d25f85c..8c9a16a 100644
--- a/doc/source/library/credential_providers.rst
+++ b/doc/source/library/credential_providers.rst
@@ -4,12 +4,12 @@
====================
These library interfaces are used to deal with allocating credentials on demand
-either dynamically by calling keystone to allocate new credentials, or from
+either dynamically by calling Keystone to allocate new credentials, or from
a list of preprovisioned credentials. These 2 modules are implementations of
the same abstract credential providers class and can be used interchangeably.
However, each implementation has some additional parameters that are used to
influence the behavior of the modules. The API reference at the bottom of this
-doc shows the interface definitions for both modules, however that may be a bit
+doc shows the interface definitions for both modules, however, that may be a bit
opaque. You can see some examples of how to leverage this interface below.
Initialization Example
@@ -30,7 +30,7 @@
# If a test requires a new account to work, it can have it via forcing
# dynamic credentials. A new account will be produced only for that test.
# In case admin credentials are not available for the account creation,
- # the test should be skipped else it would fail.
+ # the test should be skipped else it will fail.
identity_version = identity_version or CONF.identity.auth_version
if CONF.auth.use_dynamic_credentials or force_tenant_isolation:
admin_creds = get_configured_admin_credentials(
@@ -81,12 +81,12 @@
Once you have a credential provider object created the access patterns for
allocating and removing credentials are the same across both the dynamic
and preprovisioned credentials. These are defined in the abstract
-CredentialProvider class. At a high level the credentials provider enables
-you to get 3 basic types of credentials at once (per object): a primary, alt,
+CredentialProvider class. At a high level, the credentials provider enables
+you to get 3 basic types of credentials at once (per object): primary, alt,
and admin. You're also able to allocate a credential by role. These credentials
-are tracked by the provider object and delete must manually be called otherwise
-the created resources will not be deleted (or returned to the pool in the case
-of preprovisioned creds)
+are tracked by the provider object and delete must be called manually,
+otherwise, the created resources will not be deleted (or returned to the pool
+in the case of preprovisioned creds).
Examples
''''''''
diff --git a/doc/source/plugins/plugin.rst b/doc/source/plugins/plugin.rst
index 0771318..31aa134 100644
--- a/doc/source/plugins/plugin.rst
+++ b/doc/source/plugins/plugin.rst
@@ -80,30 +80,30 @@
Since all that's required for a plugin to be detected by Tempest is a valid
setuptools entry point in the proper namespace there is no difference from the
-Tempest perspective on either creating a separate python package to
-house the plugin or adding the code to an existing python project. However,
+Tempest perspective on either creating a separate Python package to
+house the plugin or adding the code to an existing Python project. However,
there are tradeoffs to consider when deciding which approach to take when
creating a new plugin.
-If you create a separate python project for your plugin this makes a lot of
+If you create a separate Python project for your plugin this makes a lot of
things much easier. Firstly it makes packaging and versioning much simpler, you
can easily decouple the requirements for the plugin from the requirements for
the other project. It lets you version the plugin independently and maintain a
single version of the test code across project release boundaries (see the
`Branchless Tempest Spec`_ for more details on this). It also greatly
simplifies the install time story for external users. Instead of having to
-install the right version of a project in the same python namespace as Tempest
+install the right version of a project in the same Python namespace as Tempest
they simply need to pip install the plugin in that namespace. It also means
that users don't have to worry about inadvertently installing a Tempest plugin
when they install another package.
.. _Branchless Tempest Spec: https://specs.openstack.org/openstack/qa-specs/specs/tempest/implemented/branchless-tempest.html
-The sole advantage to integrating a plugin into an existing python project is
+The sole advantage of integrating a plugin into an existing Python project is
that it enables you to land code changes at the same time you land test changes
in the plugin. This reduces some of the burden on contributors by not having
-to land 2 changes to add a new API feature and then test it and doing it as a
-single combined commit.
+to land 2 changes to add a new API feature and then test it, and do it as a
+single combined commit instead.
Plugin Class
@@ -122,7 +122,7 @@
class MyPlugin(plugins.TempestPlugin):
Then you need to ensure you locally define all of the mandatory methods in the
-abstract class, you can refer to the api doc below for a reference of what that
+abstract class, you can refer to the API doc below for a reference of what that
entails.
Abstract Plugin Class
@@ -135,7 +135,7 @@
================
While there are no hard and fast rules for the structure of a plugin, there are
basically no constraints on what the plugin looks like as long as the 2 steps
-above are done. However, there are some recommended patterns to follow to make
+above are done. However, there are some recommended patterns to follow to make
it easy for people to contribute and work with your plugin. For example, if you
create a directory structure with something like::
@@ -214,7 +214,7 @@
Parameters:
* **name**: Name of the attribute used to access the ``ClientsFactory`` from
- the ``ServiceClients`` instance. See example below.
+ the ``ServiceClients`` instance. See the example below.
* **service_version**: Tempest enforces a single implementation for each
service client. Available service clients are held in a ``ClientsRegistry``
singleton, and registered with ``service_version``, which means that
@@ -229,7 +229,7 @@
.. code-block:: python
- # my_creds is instance of tempest.lib.auth.Credentials
+ # my_creds is an instance of tempest.lib.auth.Credentials
# identity_uri is v2 or v3 depending on the configuration
from tempest.lib.services import clients
@@ -241,13 +241,13 @@
constraints on the structure of the configuration options exposed by the
plugin.
-First ``service_version`` should be in the format `service_config[.version]`.
+Firstly, ``service_version`` should be in the format `service_config[.version]`.
The `.version` part is optional, and should only be used if there are multiple
versions of the same API available. The `service_config` must match the name of
a configuration options group defined by the plugin. Different versions of one
API must share the same configuration group.
-Second the configuration options group `service_config` must contain the
+Secondly, the configuration options group `service_config` must contain the
following options:
* `catalog_type`: corresponds to `service` in the catalog
@@ -257,10 +257,10 @@
as they do not necessarily apply to all service clients.
* `region`: default to identity.region
-* `build_timeout` : default to compute.build_timeout
+* `build_timeout`: default to compute.build_timeout
* `build_interval`: default to compute.build_interval
-Third the service client classes should inherit from ``RestClient``, should
+Thirdly, the service client classes should inherit from ``RestClient``, should
accept generic keyword arguments, and should pass those arguments to the
``__init__`` method of ``RestClient``. Extra arguments can be added. For
instance:
@@ -276,7 +276,7 @@
self.my_arg = my_arg
self.my_args2 = my_arg
-Finally the service client should be structured in a python module, so that all
+Finally, the service client should be structured in a Python module, so that all
service client classes are importable from it. Each major API version should
have its own module.
@@ -299,7 +299,7 @@
__all__ = ['API1Client', 'API2Client']
The following folder and module structure is recommended for multiple major
-API version::
+API versions::
plugin_dir/
services/
@@ -325,14 +325,14 @@
=============
Tempest will automatically discover any installed plugins when it is run. So by
-just installing the python packages which contain your plugin you'll be using
+just installing the Python packages, which contain your plugin, you'll be using
them with Tempest, nothing else is really required.
-However, you should take care when installing plugins. By their very nature
+However, you should take care when installing plugins. By their very nature,
there are no guarantees when running Tempest with plugins enabled about the
quality of the plugin. Additionally, while there is no limitation on running
with multiple plugins, it's worth noting that poorly written plugins might not
-properly isolate their tests which could cause unexpected cross interactions
+properly isolate their tests which could cause unexpected cross-interactions
between plugins.
Notes for using plugins with virtualenvs
diff --git a/doc/source/serial_tests.rst b/doc/source/serial_tests.rst
new file mode 120000
index 0000000..6709115
--- /dev/null
+++ b/doc/source/serial_tests.rst
@@ -0,0 +1 @@
+../../tempest/serial_tests/README.rst
\ No newline at end of file
diff --git a/doc/source/supported_version.rst b/doc/source/supported_version.rst
index 89f0f90..968c821 100644
--- a/doc/source/supported_version.rst
+++ b/doc/source/supported_version.rst
@@ -9,10 +9,10 @@
Tempest master supports the below OpenStack Releases:
+* 2024.1
* 2023.2
* 2023.1
* Zed
-* Yoga
For older OpenStack Release:
diff --git a/doc/source/tests/modules.rst b/doc/source/tests/modules.rst
index 026a7a5..697b011 100644
--- a/doc/source/tests/modules.rst
+++ b/doc/source/tests/modules.rst
@@ -19,3 +19,10 @@
network/modules
object_storage/modules
volume/modules
+
+Serial Tests
+------------
+.. toctree::
+ :maxdepth: 2
+
+ serial_tests/modules
diff --git a/playbooks/devstack-tempest.yaml b/playbooks/devstack-tempest.yaml
index f9b1db0..5fb1afc 100644
--- a/playbooks/devstack-tempest.yaml
+++ b/playbooks/devstack-tempest.yaml
@@ -20,6 +20,11 @@
include_role:
name: acl-devstack-files
+ - name: Set source and destination host
+ include_role:
+ name: set-src-dest-host
+ when: tempest_set_src_dest_host is defined and tempest_set_src_dest_host | bool
+
- name: Run tempest cleanup init-saved-state
include_role:
name: tempest-cleanup
diff --git a/releasenotes/notes/add-option-to-specify-source-host.yaml b/releasenotes/notes/add-option-to-specify-source-host.yaml
new file mode 100644
index 0000000..f8df40a
--- /dev/null
+++ b/releasenotes/notes/add-option-to-specify-source-host.yaml
@@ -0,0 +1,5 @@
+---
+features:
+ - Add a new config options migration_source_host and migration_dest_host
+ in the compute section, which if is set takes source or destination
+ host from options, otherwise a host is chosen automatically.
diff --git a/releasenotes/notes/cleanup-decorator-aliases-e940b6e114e6f481.yaml b/releasenotes/notes/cleanup-decorator-aliases-e940b6e114e6f481.yaml
new file mode 100644
index 0000000..fd4a546
--- /dev/null
+++ b/releasenotes/notes/cleanup-decorator-aliases-e940b6e114e6f481.yaml
@@ -0,0 +1,9 @@
+---
+upgrade:
+ - |
+ The following decorators are no longer available in the ``tempest.test``
+ module. Use the ``tempest.common.utils`` module instead.
+
+ - ``services``
+ - ``requires_ext``
+ - ``is_extension_enabled``
diff --git a/releasenotes/notes/end-of-support-of-yoga-4ad45e91fe893024.yaml b/releasenotes/notes/end-of-support-of-yoga-4ad45e91fe893024.yaml
new file mode 100644
index 0000000..ceeb2b2
--- /dev/null
+++ b/releasenotes/notes/end-of-support-of-yoga-4ad45e91fe893024.yaml
@@ -0,0 +1,12 @@
+---
+prelude: >
+ This is an intermediate release during the 2024.1 development cycle to
+ mark the end of support for Yoga release in Tempest.
+ After this release, Tempest will support below OpenStack Releases:
+
+ * 2023.2
+ * 2023.1
+ * Zed
+
+ Current development of Tempest is for OpenStack 2024.1 development
+ cycle.
diff --git a/releasenotes/notes/identity-feature-opt-cleanup-caracal-7afd283855a07025.yaml b/releasenotes/notes/identity-feature-opt-cleanup-caracal-7afd283855a07025.yaml
new file mode 100644
index 0000000..67f6ede
--- /dev/null
+++ b/releasenotes/notes/identity-feature-opt-cleanup-caracal-7afd283855a07025.yaml
@@ -0,0 +1,21 @@
+---
+upgrade:
+ - |
+ The following deprecated options in the ``[identity-feature-enabled]``
+ section have been removed. Project tags API and application credentials
+ API are now always tested if identity v3 API is available.
+
+ - ``project_tag``
+ - ``application_credentials``
+
+ - |
+ Default value of the ``[identity-feature-enabled] access_rule`` option has
+ been changed from ``False`` to ``True`` and now the access rule API is
+ always tested when identity API is available.
+
+deprecations:
+ - |
+ The Keystone access_rule is enabled by default since Train release and we
+ no longer need a separate config in Tempest to enable it. Therefore
+ the ``[identity-feature-enabled] access_rule`` option has been deprecated
+ and will be removed in a future release.
diff --git a/releasenotes/notes/remove-compute-feature-enabled-block-migrate-cinder-iscsi-882da88096019f3c.yaml b/releasenotes/notes/remove-compute-feature-enabled-block-migrate-cinder-iscsi-882da88096019f3c.yaml
new file mode 100644
index 0000000..2808677
--- /dev/null
+++ b/releasenotes/notes/remove-compute-feature-enabled-block-migrate-cinder-iscsi-882da88096019f3c.yaml
@@ -0,0 +1,8 @@
+---
+upgrade:
+ - |
+ The deprecated ``[compute-feature-enabled] block_migrate_cinder_iscsi``
+ option has been removed.
+ Now the ``[compute-feature-enabled] block_migration_for_live_migration``
+ option is solely used to determine when to run block migration based tests
+ during live migration.
diff --git a/releasenotes/notes/remove-dns_servers_option-f49fdb2b4eb50f8f.yaml b/releasenotes/notes/remove-dns_servers_option-f49fdb2b4eb50f8f.yaml
new file mode 100644
index 0000000..6be1db9
--- /dev/null
+++ b/releasenotes/notes/remove-dns_servers_option-f49fdb2b4eb50f8f.yaml
@@ -0,0 +1,4 @@
+---
+upgrade:
+ - |
+ The deprecated ``[network] dns_servers`` option has been removed.
diff --git a/releasenotes/notes/remove-nova_cert-e2ee70a40e117e8a.yaml b/releasenotes/notes/remove-nova_cert-e2ee70a40e117e8a.yaml
new file mode 100644
index 0000000..1a292f0
--- /dev/null
+++ b/releasenotes/notes/remove-nova_cert-e2ee70a40e117e8a.yaml
@@ -0,0 +1,6 @@
+---
+upgrade:
+ - |
+ The deprecated ``[compute-feature-enabled] nova_cert`` option has been
+ removed. The nova-cert service was removed from nova in 16.0.0 release.
+ Tests of compute root certificates API have also been removed.
diff --git a/releasenotes/notes/remove-rdp_console-34e11f58d525905a.yaml b/releasenotes/notes/remove-rdp_console-34e11f58d525905a.yaml
new file mode 100644
index 0000000..4f03150
--- /dev/null
+++ b/releasenotes/notes/remove-rdp_console-34e11f58d525905a.yaml
@@ -0,0 +1,5 @@
+---
+upgrade:
+ - |
+ The deprecated ``[compute-feature-enabled] rdp_console`` config option has
+ been removed.
diff --git a/releasenotes/notes/remove-vnc-server-header-1a9731ba10242603.yaml b/releasenotes/notes/remove-vnc-server-header-1a9731ba10242603.yaml
new file mode 100644
index 0000000..cf14513
--- /dev/null
+++ b/releasenotes/notes/remove-vnc-server-header-1a9731ba10242603.yaml
@@ -0,0 +1,5 @@
+---
+upgrade:
+ - |
+ The deprecated ``[compute-feature-enabled] vnc_server_header`` option has
+ been removed.
diff --git a/releasenotes/notes/tempest-2024-1-release-d51f15c6bfe60b35.yaml b/releasenotes/notes/tempest-2024-1-release-d51f15c6bfe60b35.yaml
new file mode 100644
index 0000000..81d6a05
--- /dev/null
+++ b/releasenotes/notes/tempest-2024-1-release-d51f15c6bfe60b35.yaml
@@ -0,0 +1,17 @@
+---
+prelude: >
+ This release is to tag Tempest for OpenStack 2024.1 release.
+ This release marks the start of 2024.1 release support in Tempest.
+ After this release, Tempest will support below OpenStack Releases:
+
+ * 2024.1
+ * 2023.2
+ * 2023.1
+ * Zed
+
+ Current development of Tempest is for OpenStack 2024.2 development
+ cycle. Every Tempest commit is also tested against master during
+ the 2024.2 cycle. However, this does not necessarily mean that using
+ Tempest as of this tag will work against a 2024.2 (or future release)
+ cloud.
+ To be on safe side, use this tag to test the OpenStack 2024.1 release.
diff --git a/releasenotes/source/index.rst b/releasenotes/source/index.rst
index 989d3b5..e3018b4 100644
--- a/releasenotes/source/index.rst
+++ b/releasenotes/source/index.rst
@@ -6,6 +6,8 @@
:maxdepth: 1
unreleased
+ v38.0.0
+ v37.0.0
v36.0.0
v35.0.0
v34.2.0
diff --git a/releasenotes/source/v37.0.0.rst b/releasenotes/source/v37.0.0.rst
new file mode 100644
index 0000000..72b8bc6
--- /dev/null
+++ b/releasenotes/source/v37.0.0.rst
@@ -0,0 +1,6 @@
+=====================
+v37.0.0 Release Notes
+=====================
+
+.. release-notes:: 37.0.0 Release Notes
+ :version: 37.0.0
diff --git a/releasenotes/source/v38.0.0.rst b/releasenotes/source/v38.0.0.rst
new file mode 100644
index 0000000..2664374
--- /dev/null
+++ b/releasenotes/source/v38.0.0.rst
@@ -0,0 +1,6 @@
+=====================
+v38.0.0 Release Notes
+=====================
+
+.. release-notes:: 38.0.0 Release Notes
+ :version: 38.0.0
diff --git a/roles/set-src-dest-host/defaults/main.yaml b/roles/set-src-dest-host/defaults/main.yaml
new file mode 100644
index 0000000..fea05c8
--- /dev/null
+++ b/roles/set-src-dest-host/defaults/main.yaml
@@ -0,0 +1 @@
+devstack_base_dir: /opt/stack
diff --git a/roles/set-src-dest-host/tasks/main.yaml b/roles/set-src-dest-host/tasks/main.yaml
new file mode 100644
index 0000000..78b7a2c
--- /dev/null
+++ b/roles/set-src-dest-host/tasks/main.yaml
@@ -0,0 +1,29 @@
+- name: Find out hostnames
+ set_fact:
+ devstack_hostnames: "{{ devstack_hostnames|default([]) + [hostvars[zj_item]['ansible_hostname'] | default('unknown')] }}"
+ loop: "{{ query('inventory_hostnames', 'all,!localhost') }}"
+ loop_control:
+ loop_var: zj_item
+ ignore_errors: yes # noqa ignore-errors
+
+- name: Found hostnames
+ debug:
+ msg: |
+ # Available hosts
+ {{ devstack_hostnames }}
+
+- name: Set migration_source_host in tempest.conf
+ become: true
+ community.general.ini_file:
+ path: "{{ devstack_base_dir }}/tempest/etc/tempest.conf"
+ section: compute
+ option: migration_source_host
+ value: "{{ devstack_hostnames[0] }}"
+
+- name: Set migration_dest_host in tempest.conf
+ become: true
+ community.general.ini_file:
+ path: "{{ devstack_base_dir }}/tempest/etc/tempest.conf"
+ section: compute
+ option: migration_dest_host
+ value: "{{ devstack_hostnames[1] }}"
diff --git a/tempest/README.rst b/tempest/README.rst
index b345032..b300dcb 100644
--- a/tempest/README.rst
+++ b/tempest/README.rst
@@ -8,7 +8,7 @@
implementations for both correctness, as well as a burn in tool for
OpenStack clouds.
-As such Tempest tests come in many flavors, each with their own rules
+As such Tempest tests come in many flavors, each with its own rules
and guidelines. Below is the overview of the Tempest repository structure
to make this clear.
@@ -17,6 +17,7 @@
tempest/
api/ - API tests
scenario/ - complex scenario tests
+ serial_tests/ - tests that run always in the serial mode
tests/ - unit tests for Tempest internals
Each of these directories contains different types of tests. What
@@ -41,13 +42,20 @@
---------------------------
Scenario tests are complex "through path" tests for OpenStack
-functionality. They are typically a series of steps where complicated
+functionality. They are typically a series of steps where a complicated
state requiring multiple services is set up exercised, and torn down.
Scenario tests should not use the existing Python clients for OpenStack,
but should instead use the Tempest implementations of clients.
+:ref:`serial_tests_guide`
+--------------------------------
+
+Tests within this category will always be executed serially from the rest of
+the test cases.
+
+
:ref:`unit_tests_field_guide`
-----------------------------
diff --git a/tempest/api/README.rst b/tempest/api/README.rst
index a796922..7051230 100644
--- a/tempest/api/README.rst
+++ b/tempest/api/README.rst
@@ -7,20 +7,20 @@
What are these tests?
---------------------
-One of Tempest's prime function is to ensure that your OpenStack cloud
+One of Tempest's prime functions is to ensure that your OpenStack cloud
works with the OpenStack API as documented. The current largest
portion of Tempest code is devoted to test cases that do exactly this.
It's also important to test not only the expected positive path on
APIs, but also to provide them with invalid data to ensure they fail
in expected and documented ways. The latter type of tests is called
-``negative tests`` in Tempest source code. Over the course of the OpenStack
-project Tempest has discovered many fundamental bugs by doing just
+``negative tests`` in Tempest source code. Throughout the OpenStack
+project, Tempest has discovered many fundamental bugs by doing just
this.
In order for some APIs to return meaningful results, there must be
enough data in the system. This means these tests might start by
-spinning up a server, image, etc, then operating on it.
+spinning up a server, image, etc., and then operating on it.
Why are these tests in Tempest?
@@ -32,7 +32,7 @@
It could be argued that some of the negative testing could be done
back in the projects themselves, and we might evolve there over time,
-but currently in the OpenStack gate this is a fundamentally important
+but currently, in the OpenStack gate, this is a fundamentally important
place to keep things.
@@ -43,7 +43,7 @@
OpenStack API, as we want to ensure that bugs aren't hidden by the
official clients.
-They should test specific API calls, and can build up complex state if
+They should test specific API calls and can build up complex states if
it's needed for the API call to be meaningful.
They should send not only good data, but bad data at the API and look
diff --git a/tempest/api/compute/admin/test_live_migration.py b/tempest/api/compute/admin/test_live_migration.py
index 429755a..f6a1ae9 100644
--- a/tempest/api/compute/admin/test_live_migration.py
+++ b/tempest/api/compute/admin/test_live_migration.py
@@ -175,9 +175,6 @@
@testtools.skipIf(not CONF.compute_feature_enabled.
block_migration_for_live_migration,
'Block Live migration not available')
- @testtools.skipIf(not CONF.compute_feature_enabled.
- block_migrate_cinder_iscsi,
- 'Block Live migration not configured for iSCSI')
@utils.services('volume')
def test_live_block_migration_with_attached_volume(self):
"""Test the live-migration of an instance with an attached volume.
diff --git a/tempest/api/compute/certificates/test_certificates.py b/tempest/api/compute/certificates/test_certificates.py
deleted file mode 100644
index 5917931..0000000
--- a/tempest/api/compute/certificates/test_certificates.py
+++ /dev/null
@@ -1,40 +0,0 @@
-# Copyright 2012 OpenStack Foundation
-# All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
-# not use this file except in compliance with the License. You may obtain
-# a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-# License for the specific language governing permissions and limitations
-# under the License.
-
-from tempest.api.compute import base
-from tempest import config
-from tempest.lib import decorators
-
-CONF = config.CONF
-
-
-class CertificatesV2TestJSON(base.BaseV2ComputeTest):
- """Test Certificates API"""
-
- @classmethod
- def skip_checks(cls):
- super(CertificatesV2TestJSON, cls).skip_checks()
- if not CONF.compute_feature_enabled.nova_cert:
- raise cls.skipException("Nova cert is not available")
-
- @decorators.idempotent_id('c070a441-b08e-447e-a733-905909535b1b')
- def test_create_root_certificate(self):
- """Test creating root certificate"""
- self.certificates_client.create_certificate()
-
- @decorators.idempotent_id('3ac273d0-92d2-4632-bdfc-afbc21d4606c')
- def test_get_root_certificate(self):
- """Test getting root certificate details"""
- self.certificates_client.show_certificate('root')
diff --git a/tempest/api/identity/admin/v3/test_project_tags.py b/tempest/api/identity/admin/v3/test_project_tags.py
index 2cc7257..2004cbc 100644
--- a/tempest/api/identity/admin/v3/test_project_tags.py
+++ b/tempest/api/identity/admin/v3/test_project_tags.py
@@ -13,8 +13,6 @@
# License for the specific language governing permissions and limitations
# under the License.
-import testtools
-
from tempest.api.identity import base
from tempest import config
from tempest.lib.common.utils import data_utils
@@ -33,8 +31,6 @@
force_tenant_isolation = False
@decorators.idempotent_id('7c123aac-999d-416a-a0fb-84b915ab10de')
- @testtools.skipUnless(CONF.identity_feature_enabled.project_tags,
- 'Project tags not available.')
def test_list_update_delete_project_tags(self):
"""Test listing, updating and deleting of project tags"""
project = self.setup_test_project()
diff --git a/tempest/api/identity/base.py b/tempest/api/identity/base.py
index c9e0e1c..9cdd917 100644
--- a/tempest/api/identity/base.py
+++ b/tempest/api/identity/base.py
@@ -322,13 +322,6 @@
class BaseApplicationCredentialsV3Test(BaseIdentityV3Test):
@classmethod
- def skip_checks(cls):
- super(BaseApplicationCredentialsV3Test, cls).skip_checks()
- if not CONF.identity_feature_enabled.application_credentials:
- raise cls.skipException("Application credentials are not available"
- " in this environment")
-
- @classmethod
def resource_setup(cls):
super(BaseApplicationCredentialsV3Test, cls).resource_setup()
cls.user_id = cls.os_primary.credentials.user_id
diff --git a/tempest/api/image/v2/admin/test_image_caching.py b/tempest/api/image/v2/admin/test_image_caching.py
index 75369c9..333f946 100644
--- a/tempest/api/image/v2/admin/test_image_caching.py
+++ b/tempest/api/image/v2/admin/test_image_caching.py
@@ -37,13 +37,17 @@
# NOTE(abhishekk): As caching is enabled instance boot or volume
# boot or image download can also cache image, so we are going to
# maintain our caching information to avoid disturbing other tests
- self.cached_info = {}
+ self.cached_info = []
+ self.cached_info_remote = []
def tearDown(self):
# Delete all from cache/queue if we exit abruptly
for image_id in self.cached_info:
- self.os_admin.image_cache_client.cache_delete(
- image_id)
+ self.os_admin.image_cache_client.cache_delete(image_id)
+
+ for image_id in self.cached_info_remote:
+ self.os_admin.image_cache_client.cache_delete(image_id)
+
super(ImageCachingTest, self).tearDown()
@classmethod
@@ -75,19 +79,13 @@
image = self.client.show_image(image['id'])
return image
- def _assertCheckQueues(self, queued_images):
- for image in self.cached_info:
- if self.cached_info[image] == 'queued':
- self.assertIn(image, queued_images)
-
- def _assertCheckCache(self, cached_images):
+ def _assertCheckCache(self, cached_images, cached):
cached_list = []
for image in cached_images:
cached_list.append(image['image_id'])
- for image in self.cached_info:
- if self.cached_info[image] == 'cached':
- self.assertIn(image, cached_list)
+ for image in cached:
+ self.assertIn(image, cached_list)
@decorators.idempotent_id('4bf6adba-2f9f-47e9-a6d5-37f21ad4387c')
def test_image_caching_cycle(self):
@@ -97,10 +95,9 @@
self.assertRaises(lib_exc.Forbidden,
self.os_primary.image_cache_client.list_cache)
- # Check there is nothing is queued for cached by us
+ # Check there is nothing cached by us
output = self.os_admin.image_cache_client.list_cache()
- self._assertCheckQueues(output['queued_images'])
- self._assertCheckCache(output['cached_images'])
+ self._assertCheckCache(output['cached_images'], self.cached_info)
# Non-existing image should raise NotFound exception
self.assertRaises(lib_exc.NotFound,
@@ -122,12 +119,6 @@
# Queue image for caching
self.os_admin.image_cache_client.cache_queue(image['id'])
- self.cached_info[image['id']] = 'queued'
- # Verify that we have 1 image for queueing and 0 for caching
- output = self.os_admin.image_cache_client.list_cache()
- self._assertCheckQueues(output['queued_images'])
- self._assertCheckCache(output['cached_images'])
-
# Wait for image caching
LOG.info("Waiting for image %s to get cached", image['id'])
caching = waiters.wait_for_caching(
@@ -135,10 +126,9 @@
self.os_admin.image_cache_client,
image['id'])
- self.cached_info[image['id']] = 'cached'
- # verify that we have image in cache and not in queued
- self._assertCheckQueues(caching['queued_images'])
- self._assertCheckCache(caching['cached_images'])
+ self.cached_info.append(image['id'])
+ # verify that we have image cached
+ self._assertCheckCache(caching['cached_images'], self.cached_info)
# Verify that we can delete images from caching and queueing with
# api call.
@@ -152,4 +142,78 @@
self.os_admin.image_cache_client.cache_clear,
target="invalid")
# Remove all data from local information
- self.cached_info = {}
+ self.cached_info = []
+
+ @decorators.idempotent_id('0a6b7e10-bc30-4a41-91ff-69fb4f5e65f2')
+ def test_remote_and_self_cache(self):
+ """Test image cache works with self and remote glance service"""
+ if not CONF.image.alternate_image_endpoint:
+ raise self.skipException('No image_remote service to test '
+ 'against')
+
+ # Check there is nothing is cached by us on current and
+ # remote node
+ output = self.os_admin.image_cache_client.list_cache()
+ self._assertCheckCache(output['cached_images'], self.cached_info)
+
+ output = self.os_admin.cache_client_remote.list_cache()
+ self._assertCheckCache(output['cached_images'],
+ self.cached_info_remote)
+
+ # Create one image
+ image = self.image_create_and_upload(name='first',
+ container_format='bare',
+ disk_format='raw',
+ visibility='private')
+ self.assertEqual('active', image['status'])
+
+ # Queue image for caching on local node
+ self.os_admin.image_cache_client.cache_queue(image['id'])
+ # Wait for image caching
+ LOG.info("Waiting for image %s to get cached", image['id'])
+ caching = waiters.wait_for_caching(
+ self.client,
+ self.os_admin.image_cache_client,
+ image['id'])
+ self.cached_info.append(image['id'])
+ # verify that we have image in cache on local node
+ self._assertCheckCache(caching['cached_images'], self.cached_info)
+ # verify that we don't have anything cached on remote node
+ output = self.os_admin.cache_client_remote.list_cache()
+ self._assertCheckCache(output['cached_images'],
+ self.cached_info_remote)
+
+ # cache same image on remote node
+ self.os_admin.cache_client_remote.cache_queue(image['id'])
+ # Wait for image caching
+ LOG.info("Waiting for image %s to get cached", image['id'])
+ caching = waiters.wait_for_caching(
+ self.client,
+ self.os_admin.cache_client_remote,
+ image['id'])
+ self.cached_info_remote.append(image['id'])
+
+ # verify that we have image cached on remote node
+ output = self.os_admin.cache_client_remote.list_cache()
+ self._assertCheckCache(output['cached_images'],
+ self.cached_info_remote)
+
+ # Verify that we can delete image from remote cache and it
+ # still present in local cache
+ self.os_admin.cache_client_remote.cache_clear()
+ output = self.os_admin.cache_client_remote.list_cache()
+ self.assertEqual(0, len(output['queued_images']))
+ self.assertEqual(0, len(output['cached_images']))
+
+ output = self.os_admin.image_cache_client.list_cache()
+ self._assertCheckCache(output['cached_images'], self.cached_info)
+
+ # Delete image from local cache as well
+ self.os_admin.image_cache_client.cache_clear()
+ output = self.os_admin.image_cache_client.list_cache()
+ self.assertEqual(0, len(output['queued_images']))
+ self.assertEqual(0, len(output['cached_images']))
+
+ # Remove all data from local and remote information
+ self.cached_info = []
+ self.cached_info_remote = []
diff --git a/tempest/api/image/v2/test_images_negative.py b/tempest/api/image/v2/test_images_negative.py
index 80c01a5..f0b891f 100644
--- a/tempest/api/image/v2/test_images_negative.py
+++ b/tempest/api/image/v2/test_images_negative.py
@@ -58,7 +58,10 @@
def test_get_delete_deleted_image(self):
"""Get and delete the deleted image"""
# create and delete image
- image = self.client.create_image(name='test',
+ image_name = data_utils.rand_name(
+ prefix=CONF.resource_name_prefix,
+ name="test")
+ image = self.client.create_image(name=image_name,
container_format='bare',
disk_format='raw')
self.client.delete_image(image['id'])
@@ -111,7 +114,10 @@
@decorators.idempotent_id('ab980a34-8410-40eb-872b-f264752f46e5')
def test_delete_protected_image(self):
"""Create a protected image"""
- image = self.create_image(protected=True)
+ image_name = data_utils.rand_name(
+ prefix=CONF.resource_name_prefix,
+ name="test")
+ image = self.create_image(name=image_name, protected=True)
self.addCleanup(self.client.update_image, image['id'],
[dict(replace="/protected", value=False)])
@@ -132,7 +138,10 @@
if not CONF.image_feature_enabled.os_glance_reserved:
raise self.skipException('os_glance_reserved is not enabled')
- image = self.create_image(name='test',
+ image_name = data_utils.rand_name(
+ prefix=CONF.resource_name_prefix,
+ name="test")
+ image = self.create_image(name=image_name,
container_format='bare',
disk_format='raw')
self.assertRaises(lib_exc.Forbidden,
@@ -152,9 +161,12 @@
if not CONF.image_feature_enabled.os_glance_reserved:
raise self.skipException('os_glance_reserved is not enabled')
+ image_name = data_utils.rand_name(
+ prefix=CONF.resource_name_prefix,
+ name="test")
self.assertRaises(lib_exc.Forbidden,
self.create_image,
- name='test',
+ name=image_name,
container_format='bare',
disk_format='raw',
os_glance_foo='bar')
@@ -195,7 +207,10 @@
if 'web-download' not in self.available_import_methods:
raise self.skipException('Server does not support '
'web-download import method')
- image = self.client.create_image(name='test',
+ image_name = data_utils.rand_name(
+ prefix=CONF.resource_name_prefix,
+ name="test")
+ image = self.client.create_image(name=image_name,
container_format='bare',
disk_format='raw')
# Now try to get image details
diff --git a/tempest/api/volume/test_volumes_backup.py b/tempest/api/volume/test_volumes_backup.py
index bfe962a..a3ba974 100644
--- a/tempest/api/volume/test_volumes_backup.py
+++ b/tempest/api/volume/test_volumes_backup.py
@@ -172,6 +172,52 @@
self.assertTrue(restored_volume_info['bootable'])
+ @decorators.idempotent_id('f86eff09-2a6d-43c1-905e-8079e5754f1e')
+ @utils.services('compute')
+ @decorators.related_bug('1703011')
+ def test_volume_backup_incremental(self):
+ """Test create a backup when latest incremental backup is deleted"""
+ # Create a volume
+ volume = self.create_volume()
+
+ # Create a server
+ server = self.create_server(wait_until='SSHABLE')
+
+ # Attach volume to the server
+ self.attach_volume(server['id'], volume['id'])
+
+ # Create a backup to the attached volume
+ backup1 = self.create_backup(volume['id'], force=True)
+
+ # Validate backup details
+ backup_info = self.backups_client.show_backup(backup1['id'])['backup']
+ self.assertEqual(False, backup_info['has_dependent_backups'])
+ self.assertEqual(False, backup_info['is_incremental'])
+
+ # Create another incremental backup
+ backup2 = self.backups_client.create_backup(
+ volume_id=volume['id'], incremental=True, force=True)['backup']
+ waiters.wait_for_volume_resource_status(self.backups_client,
+ backup2['id'], 'available')
+
+ # Validate incremental backup details
+ backup2_info = self.backups_client.show_backup(backup2['id'])['backup']
+ self.assertEqual(True, backup2_info['is_incremental'])
+ self.assertEqual(False, backup2_info['has_dependent_backups'])
+
+ # Delete the last incremental backup that was created
+ self.backups_client.delete_backup(backup2['id'])
+ self.backups_client.wait_for_resource_deletion(backup2['id'])
+
+ # Create another incremental backup
+ backup3 = self.create_backup(
+ volume_id=volume['id'], incremental=True, force=True)
+
+ # Validate incremental backup details
+ backup3_info = self.backups_client.show_backup(backup3['id'])['backup']
+ self.assertEqual(True, backup3_info['is_incremental'])
+ self.assertEqual(False, backup3_info['has_dependent_backups'])
+
class VolumesBackupsV39Test(base.BaseVolumeTest):
"""Test volumes backup with volume microversion greater than 3.8"""
diff --git a/tempest/clients.py b/tempest/clients.py
index 5b31cf8..5338ed4 100644
--- a/tempest/clients.py
+++ b/tempest/clients.py
@@ -104,6 +104,15 @@
service=CONF.image.alternate_image_endpoint,
endpoint_type=CONF.image.alternate_image_endpoint_type,
region=CONF.image.region)
+ # NOTE(abhishekk): If no alternate endpoint is configured,
+ # this client will work the same as the base
+ # self.image_cache_client. If your test needs to know if
+ # these are different, check the config option to see if
+ # the alternate_image_endpoint is set.
+ self.cache_client_remote = self.image_v2.ImageCacheClient(
+ service=CONF.image.alternate_image_endpoint,
+ endpoint_type=CONF.image.alternate_image_endpoint_type,
+ region=CONF.image.region)
def _set_compute_clients(self):
self.agents_client = self.compute.AgentsClient()
diff --git a/tempest/cmd/run.py b/tempest/cmd/run.py
index 2669ff7..e305646 100644
--- a/tempest/cmd/run.py
+++ b/tempest/cmd/run.py
@@ -269,6 +269,8 @@
return_code = commands.run_command(
**params, blacklist_file=ex_list,
whitelist_file=in_list, black_regex=ex_regex)
+ if parsed_args.slowest:
+ commands.slowest_command()
if return_code > 0:
sys.exit(return_code)
return return_code
@@ -392,6 +394,9 @@
help='Combine the output of this run with the '
"previous run's as a combined stream in the "
"stestr repository after it finish")
+ parser.add_argument('--slowest', action='store_true',
+ help='Show the longest running tests in the '
+ 'stestr repository after it finishes')
parser.set_defaults(parallel=True)
return parser
diff --git a/tempest/config.py b/tempest/config.py
index 699e271..0a084ea 100644
--- a/tempest/config.py
+++ b/tempest/config.py
@@ -126,9 +126,11 @@
default=None,
help='Specify a CA bundle file to use in verifying a '
'TLS (https) server certificate.'),
- cfg.StrOpt('uri',
+ cfg.URIOpt('uri',
+ schemes=['http', 'https'],
help="Full URI of the OpenStack Identity API (Keystone), v2"),
- cfg.StrOpt('uri_v3',
+ cfg.URIOpt('uri_v3',
+ schemes=['http', 'https'],
help='Full URI of the OpenStack Identity API (Keystone), v3'),
cfg.StrOpt('auth_version',
default='v3',
@@ -263,23 +265,11 @@
default=False,
help='Does the environment have the security compliance '
'settings enabled?'),
- cfg.BoolOpt('project_tags',
- default=True,
- help='Is the project tags identity v3 API available?',
- deprecated_for_removal=True,
- deprecated_reason='Project tags API is a default feature '
- 'since Queens'),
- cfg.BoolOpt('application_credentials',
- default=True,
- help='Does the environment have application credentials '
- 'enabled?',
- deprecated_for_removal=True,
- deprecated_reason='Application credentials is a default '
- 'feature since Queens'),
- # Access rules for application credentials is a default feature in Train.
- # This config option can removed once Stein is EOL.
cfg.BoolOpt('access_rules',
- default=False,
+ default=True,
+ deprecated_for_removal=True,
+ deprecated_reason='Access rules for application credentials '
+ 'is a default feature since Train',
help='Does the environment have access rules enabled?'),
cfg.BoolOpt('immutable_user_source',
default=False,
@@ -405,6 +395,17 @@
'allow_availability_zone_fallback=False in cinder.conf), '
'the volume create request will fail and the instance '
'will fail the build request.'),
+ cfg.StrOpt('migration_source_host',
+ default=None,
+ help="Specify source host for live-migration, cold-migration"
+ " and resize tests. If option is not set tests will use"
+ " host automatically."),
+ cfg.StrOpt('migration_dest_host',
+ default=None,
+ help="Specify destination host for live-migration and cold"
+ " migration. If option is not set tests will use host"
+ " automatically."),
+
]
placement_group = cfg.OptGroup(name='placement',
@@ -501,18 +502,6 @@
default=False,
help="Does the test environment use block devices for live "
"migration"),
- cfg.BoolOpt('block_migrate_cinder_iscsi',
- default=False,
- help="Does the test environment support block migration with "
- "Cinder iSCSI volumes. Note: libvirt >= 1.2.17 is required "
- "to support this if using the libvirt compute driver.",
- deprecated_for_removal=True,
- deprecated_reason='This option duplicates the more generic '
- '[compute-feature-enabled]/block_migration '
- '_for_live_migration now that '
- 'MIN_LIBVIRT_VERSION is >= 1.2.17 on all '
- 'branches from stable/rocky and will be '
- 'removed in a future release.'),
cfg.BoolOpt('can_migrate_between_any_hosts',
default=True,
help="Does the test environment support migrating between "
@@ -523,15 +512,6 @@
default=False,
help='Enable VNC console. This configuration value should '
'be same as nova.conf: vnc.enabled'),
- cfg.StrOpt('vnc_server_header',
- default='WebSockify',
- help='Expected VNC server name (WebSockify, nginx, etc) '
- 'in response header.',
- deprecated_for_removal=True,
- deprecated_reason='This option will be ignored because the '
- 'usage of different response header fields '
- 'to accomplish the same goal (in accordance '
- 'with RFC7231 S6.2.2) makes it obsolete.'),
cfg.BoolOpt('spice_console',
default=False,
help='Enable Spice console. This configuration value should '
@@ -540,14 +520,6 @@
deprecated_reason="This config option is not being used "
"in Tempest, we can add it back when "
"adding the test cases."),
- cfg.BoolOpt('rdp_console',
- default=False,
- help='Enable RDP console. This configuration value should '
- 'be same as nova.conf: rdp.enabled',
- deprecated_for_removal=True,
- deprecated_reason="This config option is not being used "
- "in Tempest, we can add it back when "
- "adding the test cases."),
cfg.BoolOpt('serial_console',
default=False,
help='Enable serial console. This configuration value '
@@ -575,13 +547,6 @@
default=True,
help='Does the test environment support creating snapshot '
'images of running instances?'),
- cfg.BoolOpt('nova_cert',
- default=False,
- help='Does the test environment have the nova cert running?',
- deprecated_for_removal=True,
- deprecated_reason="On Nova side, the nova-cert service is "
- "deprecated and the service will be removed "
- "as early as Ocata."),
cfg.BoolOpt('personality',
default=False,
help='Does the test environment support server personality'),
@@ -801,13 +766,6 @@
default=1,
help="Time in seconds between network operation status "
"checks."),
- cfg.ListOpt('dns_servers',
- default=["8.8.8.8", "8.8.4.4"],
- help="List of dns servers which should be used"
- " for subnet creation",
- deprecated_for_removal=True,
- deprecated_reason="This config option is no longer "
- "used anywhere, so it can be removed."),
cfg.StrOpt('port_vnic_type',
choices=[None, 'normal', 'direct', 'macvtap', 'direct-physical',
'baremetal', 'virtio-forwarder'],
@@ -883,8 +841,9 @@
title="Dashboard options")
DashboardGroup = [
- cfg.StrOpt('dashboard_url',
+ cfg.URIOpt('dashboard_url',
default='http://localhost/',
+ schemes=['http', 'https'],
help="Where the dashboard can be found"),
cfg.BoolOpt('disable_ssl_certificate_validation',
default=False,
@@ -909,10 +868,11 @@
help='Enable/disable security group rules.'),
cfg.StrOpt('connect_method',
default='floating',
- choices=['fixed', 'floating'],
- help='Default IP type used for validation: '
- '-fixed: uses the first IP belonging to the fixed network '
- '-floating: creates and uses a floating IP'),
+ choices=[('fixed',
+ 'uses the first IP belonging to the fixed network'),
+ ('floating',
+ 'creates and uses a floating IP')],
+ help='Default IP type used for validation'),
cfg.StrOpt('auth_method',
default='keypair',
choices=['keypair'],
diff --git a/tempest/lib/cli/base.py b/tempest/lib/cli/base.py
index c9cffd2..c308c30 100644
--- a/tempest/lib/cli/base.py
+++ b/tempest/lib/cli/base.py
@@ -374,12 +374,13 @@
:param merge_stderr: if True the stderr buffer is merged into stdout
:type merge_stderr: boolean
"""
- creds = ('--os-username %s --os-project-name %s --os-password %s '
+ creds = ('--os-username %s --os-password %s '
'--os-auth-url %s' %
(self.username,
- self.tenant_name,
self.password,
self.uri))
+ if self.tenant_name is not None:
+ creds += ' --os-project-name %s' % self.tenant_name
if self.identity_api_version:
if cmd not in self.CLIENTS_WITHOUT_IDENTITY_VERSION:
creds += ' --os-identity-api-version %s' % (
diff --git a/tempest/lib/common/utils/test_utils.py b/tempest/lib/common/utils/test_utils.py
index c79db15..7b85dec 100644
--- a/tempest/lib/common/utils/test_utils.py
+++ b/tempest/lib/common/utils/test_utils.py
@@ -93,7 +93,7 @@
if attempt >= 3:
raise
LOG.warning('Got ServerFault while running %s, retrying...', func)
- time.sleep(1)
+ time.sleep(5)
def call_until_true(func, duration, sleep_for, *args, **kwargs):
diff --git a/tempest/scenario/README.rst b/tempest/scenario/README.rst
index efcd139..6c51f22 100644
--- a/tempest/scenario/README.rst
+++ b/tempest/scenario/README.rst
@@ -7,14 +7,14 @@
What are these tests?
---------------------
-Scenario tests are "through path" tests of OpenStack
-function. Complicated setups where one part might depend on completion
+Scenario tests are "through path" tests of OpenStack function.
+Complicated setups where one part might depend on the completion
of a previous part. They ideally involve the integration between
multiple OpenStack services to exercise the touch points between them.
Any scenario test should have a real-life use case. An example would be:
-- "As operator I want to start with a blank environment":
+- "As an operator, I want to start with a blank environment":
1. upload a glance image
2. deploy a vm from it
@@ -24,12 +24,14 @@
Why are these tests in Tempest?
-------------------------------
+
This is one of Tempest's core purposes, testing the integration between
projects.
Scope of these tests
--------------------
+
Scenario tests should always use the Tempest implementation of the
OpenStack API, as we want to ensure that bugs aren't hidden by the
official clients.
@@ -40,6 +42,7 @@
Example of a good test
----------------------
+
While we are looking for interaction of 2 or more services, be
specific in your interactions. A giant "this is my data center" smoke
test is hard to debug when it goes wrong.
diff --git a/tempest/scenario/manager.py b/tempest/scenario/manager.py
index 7c986cc..5f30909 100644
--- a/tempest/scenario/manager.py
+++ b/tempest/scenario/manager.py
@@ -441,7 +441,7 @@
'container': container}
args.update(kwargs)
backup = self.backups_client.create_backup(volume_id=volume_id,
- **kwargs)['backup']
+ **args)['backup']
self.addCleanup(self.backups_client.delete_backup, backup['id'])
waiters.wait_for_volume_resource_status(self.backups_client,
backup['id'], 'available')
diff --git a/tempest/scenario/test_network_advanced_server_ops.py b/tempest/scenario/test_network_advanced_server_ops.py
index 882afff..3a93f74 100644
--- a/tempest/scenario/test_network_advanced_server_ops.py
+++ b/tempest/scenario/test_network_advanced_server_ops.py
@@ -28,25 +28,12 @@
LOG = log.getLogger(__name__)
-class TestNetworkAdvancedServerOps(manager.NetworkScenarioTest):
- """Check VM connectivity after some advanced instance operations executed:
-
- * Stop/Start an instance
- * Reboot an instance
- * Rebuild an instance
- * Pause/Unpause an instance
- * Suspend/Resume an instance
- * Resize an instance
- """
-
- @classmethod
- def setup_clients(cls):
- super(TestNetworkAdvancedServerOps, cls).setup_clients()
- cls.admin_servers_client = cls.os_admin.servers_client
+class BaseTestNetworkAdvancedServerOps(manager.NetworkScenarioTest):
+ """Base class for defining methods used in tests."""
@classmethod
def skip_checks(cls):
- super(TestNetworkAdvancedServerOps, cls).skip_checks()
+ super(BaseTestNetworkAdvancedServerOps, cls).skip_checks()
if not (CONF.network.project_networks_reachable or
CONF.network.public_network_id):
msg = ('Either project_networks_reachable must be "true", or '
@@ -56,26 +43,52 @@
raise cls.skipException("Floating ips are not available")
@classmethod
+ def setup_clients(cls):
+ super(BaseTestNetworkAdvancedServerOps, cls).setup_clients()
+ cls.admin_servers_client = cls.os_admin.servers_client
+ cls.sec_group_rules_client = \
+ cls.os_primary.security_group_rules_client
+ cls.sec_groups_client = cls.os_primary.security_groups_client
+ cls.keypairs_client = cls.os_primary.keypairs_client
+ cls.floating_ips_client = cls.os_primary.floating_ips_client
+ cls.servers_client = cls.os_primary.servers_client
+
+ @classmethod
def setup_credentials(cls):
# Create no network resources for these tests.
cls.set_network_resources()
- super(TestNetworkAdvancedServerOps, cls).setup_credentials()
+ super(BaseTestNetworkAdvancedServerOps, cls).setup_credentials()
- def _setup_server(self, keypair):
+ def _setup_server(self, keypair, host_spec=None):
security_groups = []
if utils.is_extension_enabled('security-group', 'network'):
- security_group = self.create_security_group()
+ sec_args = {
+ 'security_group_rules_client':
+ self.sec_group_rules_client,
+ 'security_groups_client':
+ self.sec_groups_client
+ }
+ security_group = self.create_security_group(**sec_args)
security_groups = [{'name': security_group['name']}]
network, _, _ = self.setup_network_subnet_with_router()
- server = self.create_server(
- networks=[{'uuid': network['id']}],
- key_name=keypair['name'],
- security_groups=security_groups)
+ server_args = {
+ 'networks': [{'uuid': network['id']}],
+ 'key_name': keypair['name'],
+ 'security_groups': security_groups,
+ }
+
+ if host_spec is not None:
+ server_args['host'] = host_spec
+ # by default, host can be specified by administrators only
+ server_args['clients'] = self.os_admin
+
+ server = self.create_server(**server_args)
return server
def _setup_network(self, server, keypair):
public_network_id = CONF.network.public_network_id
- floating_ip = self.create_floating_ip(server, public_network_id)
+ floating_ip = self.create_floating_ip(
+ server, public_network_id, client=self.floating_ips_client)
# Verify that we can indeed connect to the server before we mess with
# it's state
self._wait_server_status_and_check_network_connectivity(
@@ -107,6 +120,148 @@
self._check_network_connectivity(server, keypair, floating_ip,
username=username)
+ def _test_server_connectivity_resize(self, src_host=None):
+ resize_flavor = CONF.compute.flavor_ref_alt
+ keypair = self.create_keypair()
+ server = self._setup_server(keypair, src_host)
+ if src_host:
+ server_host = self.get_host_for_server(server['id'])
+ self.assertEqual(server_host, src_host)
+ floating_ip = self._setup_network(server, keypair)
+ self.servers_client.resize_server(server['id'],
+ flavor_ref=resize_flavor)
+ waiters.wait_for_server_status(self.servers_client, server['id'],
+ 'VERIFY_RESIZE')
+ self.servers_client.confirm_resize_server(server['id'])
+ server = self.servers_client.show_server(server['id'])['server']
+ # Nova API > 2.46 no longer includes flavor.id, and schema check
+ # will cover whether 'id' should be in flavor
+ if server['flavor'].get('id'):
+ self.assertEqual(resize_flavor, server['flavor']['id'])
+ else:
+ flavor = self.flavors_client.show_flavor(resize_flavor)['flavor']
+ self.assertEqual(flavor['name'], server['flavor']['original_name'])
+ for key in ['ram', 'vcpus', 'disk']:
+ self.assertEqual(flavor[key], server['flavor'][key])
+ self._wait_server_status_and_check_network_connectivity(
+ server, keypair, floating_ip)
+
+ def _test_server_connectivity_cold_migration(self, source_host=None,
+ dest_host=None):
+ keypair = self.create_keypair(client=self.keypairs_client)
+ server = self._setup_server(keypair, source_host)
+ floating_ip = self._setup_network(server, keypair)
+ src_host = self.get_host_for_server(server['id'])
+ if source_host:
+ self.assertEqual(src_host, source_host)
+ self._wait_server_status_and_check_network_connectivity(
+ server, keypair, floating_ip)
+
+ self.admin_servers_client.migrate_server(
+ server['id'], host=dest_host)
+ waiters.wait_for_server_status(self.servers_client, server['id'],
+ 'VERIFY_RESIZE')
+ self.servers_client.confirm_resize_server(server['id'])
+ self._wait_server_status_and_check_network_connectivity(
+ server, keypair, floating_ip)
+ dst_host = self.get_host_for_server(server['id'])
+ if dest_host:
+ self.assertEqual(dst_host, dest_host)
+ self.assertNotEqual(src_host, dst_host)
+
+ def _test_server_connectivity_live_migration(self, source_host=None,
+ dest_host=None,
+ migration=False):
+ keypair = self.create_keypair(client=self.keypairs_client)
+ server = self._setup_server(keypair, source_host)
+ floating_ip = self._setup_network(server, keypair)
+ self._wait_server_status_and_check_network_connectivity(
+ server, keypair, floating_ip)
+
+ block_migration = (CONF.compute_feature_enabled.
+ block_migration_for_live_migration)
+ src_host = self.get_host_for_server(server['id'])
+ if source_host:
+ self.assertEqual(src_host, source_host)
+
+ downtime_meter = net_downtime.NetDowntimeMeter(
+ floating_ip['floating_ip_address'])
+ self.useFixture(downtime_meter)
+
+ migration_kwargs = {'host': None, 'block_migration': block_migration}
+
+ # check if microversion is less than 2.25 because of
+ # disk_over_commit is depracted since compute api version 2.25
+ # if min_microversion is None, it runs on version < 2.25
+ if not migration and (CONF.compute.min_microversion is None or
+ CONF.compute.min_microversion < '2.25'):
+ migration_kwargs['disk_over_commit'] = False
+
+ if dest_host:
+ migration_kwargs['host'] = dest_host
+
+ self.admin_servers_client.live_migrate_server(
+ server['id'], **migration_kwargs)
+ waiters.wait_for_server_status(self.servers_client,
+ server['id'], 'ACTIVE')
+
+ dst_host = self.get_host_for_server(server['id'])
+ if dest_host:
+ self.assertEqual(dst_host, dest_host)
+
+ self.assertNotEqual(src_host, dst_host, 'Server did not migrate')
+
+ # we first wait until the VM replies pings again, then check the
+ # network downtime
+ self._wait_server_status_and_check_network_connectivity(
+ server, keypair, floating_ip)
+
+ downtime = downtime_meter.get_downtime()
+ self.assertIsNotNone(downtime)
+ LOG.debug("Downtime seconds measured with downtime_meter = %r",
+ downtime)
+ allowed_downtime = CONF.validation.allowed_network_downtime
+ self.assertLessEqual(
+ downtime, allowed_downtime,
+ "Downtime of {} seconds is higher than expected '{}'".format(
+ downtime, allowed_downtime))
+
+ def _test_server_connectivity_cold_migration_revert(self, source_host=None,
+ dest_host=None):
+ keypair = self.create_keypair(client=self.keypairs_client)
+ server = self._setup_server(keypair, source_host)
+ floating_ip = self._setup_network(server, keypair)
+ src_host = self.get_host_for_server(server['id'])
+ if source_host:
+ self.assertEqual(src_host, source_host)
+ self._wait_server_status_and_check_network_connectivity(
+ server, keypair, floating_ip)
+
+ self.admin_servers_client.migrate_server(
+ server['id'], host=dest_host)
+ waiters.wait_for_server_status(self.servers_client, server['id'],
+ 'VERIFY_RESIZE')
+ if dest_host:
+ self.assertEqual(dest_host,
+ self.get_host_for_server(server['id']))
+ self.servers_client.revert_resize_server(server['id'])
+ self._wait_server_status_and_check_network_connectivity(
+ server, keypair, floating_ip)
+ dst_host = self.get_host_for_server(server['id'])
+
+ self.assertEqual(src_host, dst_host)
+
+
+class TestNetworkAdvancedServerOps(BaseTestNetworkAdvancedServerOps):
+ """Check VM connectivity after some advanced instance operations executed:
+
+ * Stop/Start an instance
+ * Reboot an instance
+ * Rebuild an instance
+ * Pause/Unpause an instance
+ * Suspend/Resume an instance
+ """
+
@decorators.idempotent_id('61f1aa9a-1573-410e-9054-afa557cab021')
@decorators.attr(type='slow')
@utils.services('compute', 'network')
@@ -190,27 +345,7 @@
@decorators.attr(type='slow')
@utils.services('compute', 'network')
def test_server_connectivity_resize(self):
- resize_flavor = CONF.compute.flavor_ref_alt
- keypair = self.create_keypair()
- server = self._setup_server(keypair)
- floating_ip = self._setup_network(server, keypair)
- self.servers_client.resize_server(server['id'],
- flavor_ref=resize_flavor)
- waiters.wait_for_server_status(self.servers_client, server['id'],
- 'VERIFY_RESIZE')
- self.servers_client.confirm_resize_server(server['id'])
- server = self.servers_client.show_server(server['id'])['server']
- # Nova API > 2.46 no longer includes flavor.id, and schema check
- # will cover whether 'id' should be in flavor
- if server['flavor'].get('id'):
- self.assertEqual(resize_flavor, server['flavor']['id'])
- else:
- flavor = self.flavors_client.show_flavor(resize_flavor)['flavor']
- self.assertEqual(flavor['name'], server['flavor']['original_name'])
- for key in ['ram', 'vcpus', 'disk']:
- self.assertEqual(flavor[key], server['flavor'][key])
- self._wait_server_status_and_check_network_connectivity(
- server, keypair, floating_ip)
+ self._test_server_connectivity_resize()
@decorators.idempotent_id('a4858f6c-401e-4155-9a49-d5cd053d1a2f')
@testtools.skipUnless(CONF.compute_feature_enabled.cold_migration,
@@ -221,22 +356,7 @@
@decorators.attr(type=['slow', 'multinode'])
@utils.services('compute', 'network')
def test_server_connectivity_cold_migration(self):
- keypair = self.create_keypair()
- server = self._setup_server(keypair)
- floating_ip = self._setup_network(server, keypair)
- src_host = self.get_host_for_server(server['id'])
- self._wait_server_status_and_check_network_connectivity(
- server, keypair, floating_ip)
-
- self.admin_servers_client.migrate_server(server['id'])
- waiters.wait_for_server_status(self.servers_client, server['id'],
- 'VERIFY_RESIZE')
- self.servers_client.confirm_resize_server(server['id'])
- self._wait_server_status_and_check_network_connectivity(
- server, keypair, floating_ip)
- dst_host = self.get_host_for_server(server['id'])
-
- self.assertNotEqual(src_host, dst_host)
+ self._test_server_connectivity_cold_migration()
@decorators.idempotent_id('03fd1562-faad-11e7-9ea0-fa163e65f5ce')
@testtools.skipUnless(CONF.compute_feature_enabled.live_migration,
@@ -247,52 +367,7 @@
@decorators.attr(type=['slow', 'multinode'])
@utils.services('compute', 'network')
def test_server_connectivity_live_migration(self):
- keypair = self.create_keypair()
- server = self._setup_server(keypair)
- floating_ip = self._setup_network(server, keypair)
- self._wait_server_status_and_check_network_connectivity(
- server, keypair, floating_ip)
-
- block_migration = (CONF.compute_feature_enabled.
- block_migration_for_live_migration)
- old_host = self.get_host_for_server(server['id'])
-
- downtime_meter = net_downtime.NetDowntimeMeter(
- floating_ip['floating_ip_address'])
- self.useFixture(downtime_meter)
-
- migration_kwargs = {'host': None, 'block_migration': block_migration}
-
- # check if microversion is less than 2.25 because of
- # disk_over_commit is depracted since compute api version 2.25
- # if min_microversion is None, it runs on version < 2.25
- if (CONF.compute.min_microversion is None or
- CONF.compute.min_microversion < 2.25):
- migration_kwargs['disk_over_commit'] = False
-
- self.admin_servers_client.live_migrate_server(
- server['id'], **migration_kwargs)
-
- waiters.wait_for_server_status(self.servers_client,
- server['id'], 'ACTIVE')
-
- new_host = self.get_host_for_server(server['id'])
- self.assertNotEqual(old_host, new_host, 'Server did not migrate')
-
- # we first wait until the VM replies pings again, then check the
- # network downtime
- self._wait_server_status_and_check_network_connectivity(
- server, keypair, floating_ip)
-
- downtime = downtime_meter.get_downtime()
- self.assertIsNotNone(downtime)
- LOG.debug("Downtime seconds measured with downtime_meter = %r",
- downtime)
- allowed_downtime = CONF.validation.allowed_network_downtime
- self.assertLessEqual(
- downtime, allowed_downtime,
- "Downtime of {} seconds is higher than expected '{}'".format(
- downtime, allowed_downtime))
+ self._test_server_connectivity_live_migration()
@decorators.idempotent_id('25b188d7-0183-4b1e-a11d-15840c8e2fd6')
@testtools.skipUnless(CONF.compute_feature_enabled.cold_migration,
@@ -303,19 +378,95 @@
@decorators.attr(type=['slow', 'multinode'])
@utils.services('compute', 'network')
def test_server_connectivity_cold_migration_revert(self):
- keypair = self.create_keypair()
- server = self._setup_server(keypair)
- floating_ip = self._setup_network(server, keypair)
- src_host = self.get_host_for_server(server['id'])
- self._wait_server_status_and_check_network_connectivity(
- server, keypair, floating_ip)
+ self._test_server_connectivity_cold_migration_revert()
- self.admin_servers_client.migrate_server(server['id'])
- waiters.wait_for_server_status(self.servers_client, server['id'],
- 'VERIFY_RESIZE')
- self.servers_client.revert_resize_server(server['id'])
- self._wait_server_status_and_check_network_connectivity(
- server, keypair, floating_ip)
- dst_host = self.get_host_for_server(server['id'])
- self.assertEqual(src_host, dst_host)
+class TestNetworkAdvancedServerMigrationWithHost(
+ BaseTestNetworkAdvancedServerOps):
+
+ """Check VM connectivity with specifying source and destination hosts:
+
+ * Resize an instance by creating server on configured source host
+ * Migrate server by creating it on configured source host and migrate it
+ - Cold Migration
+ - Cold Migration with revert
+ - Live Migration
+ """
+ credentials = ['primary', 'admin']
+ compute_min_microversion = "2.74"
+
+ @classmethod
+ def skip_checks(cls):
+ super(TestNetworkAdvancedServerMigrationWithHost, cls).skip_checks()
+ if not (CONF.compute.migration_source_host or
+ CONF.compute.migration_dest_host):
+ raise cls.skipException("migration_source_host or "
+ "migration_dest_host is required")
+ if (CONF.compute.migration_source_host and
+ CONF.compute.migration_dest_host and
+ CONF.compute.migration_source_host ==
+ CONF.compute.migration_dest_host):
+ raise cls.skipException("migration_source_host and "
+ "migration_dest_host must be different")
+
+ @classmethod
+ def setup_clients(cls):
+ super(BaseTestNetworkAdvancedServerOps, cls).setup_clients()
+ cls.sec_group_rules_client = \
+ cls.os_admin.security_group_rules_client
+ cls.sec_groups_client = cls.os_admin.security_groups_client
+ cls.keypairs_client = cls.os_admin.keypairs_client
+ cls.floating_ips_client = cls.os_admin.floating_ips_client
+ cls.servers_client = cls.os_admin.servers_client
+ cls.admin_servers_client = cls.os_admin.servers_client
+
+ @decorators.idempotent_id('06e23934-79ae-11ee-b962-0242ac120002')
+ @testtools.skipUnless(CONF.compute_feature_enabled.resize,
+ 'Resize is not available.')
+ @decorators.attr(type='slow')
+ @utils.services('compute', 'network')
+ def test_server_connectivity_resize(self):
+ source_host = CONF.compute.migration_source_host
+ self._test_server_connectivity_resize(src_host=source_host)
+
+ @decorators.idempotent_id('14f0c9e6-79ae-11ee-b962-0242ac120002')
+ @testtools.skipUnless(CONF.compute_feature_enabled.cold_migration,
+ 'Cold migration is not available.')
+ @testtools.skipUnless(CONF.compute.min_compute_nodes > 1,
+ 'Less than 2 compute nodes, skipping multinode '
+ 'tests.')
+ @decorators.attr(type=['slow', 'multinode'])
+ @utils.services('compute', 'network')
+ def test_server_connectivity_cold_migration(self):
+ source_host = CONF.compute.migration_source_host
+ dest_host = CONF.compute.migration_dest_host
+ self._test_server_connectivity_cold_migration(
+ source_host=source_host, dest_host=dest_host)
+
+ @decorators.idempotent_id('1c13933e-79ae-11ee-b962-0242ac120002')
+ @testtools.skipUnless(CONF.compute_feature_enabled.live_migration,
+ 'Live migration is not available.')
+ @testtools.skipUnless(CONF.compute.min_compute_nodes > 1,
+ 'Less than 2 compute nodes, skipping multinode '
+ 'tests.')
+ @decorators.attr(type=['slow', 'multinode'])
+ @utils.services('compute', 'network')
+ def test_server_connectivity_live_migration(self):
+ source_host = CONF.compute.migration_source_host
+ dest_host = CONF.compute.migration_dest_host
+ self._test_server_connectivity_live_migration(
+ source_host=source_host, dest_host=dest_host, migration=True)
+
+ @decorators.idempotent_id('2627789a-79ae-11ee-b962-0242ac120002')
+ @testtools.skipUnless(CONF.compute_feature_enabled.cold_migration,
+ 'Cold migration is not available.')
+ @testtools.skipUnless(CONF.compute.min_compute_nodes > 1,
+ 'Less than 2 compute nodes, skipping multinode '
+ 'tests.')
+ @decorators.attr(type=['slow', 'multinode'])
+ @utils.services('compute', 'network')
+ def test_server_connectivity_cold_migration_revert(self):
+ source_host = CONF.compute.migration_source_host
+ dest_host = CONF.compute.migration_dest_host
+ self._test_server_connectivity_cold_migration_revert(
+ source_host=source_host, dest_host=dest_host)
diff --git a/tempest/scenario/test_network_qos_placement.py b/tempest/scenario/test_network_qos_placement.py
index dbbc314..055dcb6 100644
--- a/tempest/scenario/test_network_qos_placement.py
+++ b/tempest/scenario/test_network_qos_placement.py
@@ -67,10 +67,10 @@
cls.networks_client = cls.os_admin.networks_client
cls.subnets_client = cls.os_admin.subnets_client
cls.ports_client = cls.os_primary.ports_client
- cls.routers_client = cls.os_adm.routers_client
+ cls.routers_client = cls.os_admin.routers_client
cls.qos_client = cls.os_admin.qos_client
cls.qos_min_bw_client = cls.os_admin.qos_min_bw_client
- cls.flavors_client = cls.os_adm.flavors_client
+ cls.flavors_client = cls.os_admin.flavors_client
cls.servers_client = cls.os_primary.servers_client
def _create_flavor_to_resize_to(self):
diff --git a/tempest/serial_tests/README.rst b/tempest/serial_tests/README.rst
new file mode 100644
index 0000000..40a50f4
--- /dev/null
+++ b/tempest/serial_tests/README.rst
@@ -0,0 +1,61 @@
+.. _serial_tests_guide:
+
+Tempest Field Guide to Serial tests
+===================================
+
+
+What are these tests?
+---------------------
+
+Tempest can run tests serially as well as in parallel, depending on the
+configuration that is fully up to the user. However, sometimes you need to
+make sure that tests are not interfering with each other via OpenStack
+resources with the other tests running in parallel. Tempest creates separate
+projects for each test class to separate project based resources between test
+cases.
+
+If your tests use resources outside of projects, e.g. host aggregates then
+you might need to explicitly separate interfering test cases. If you only need
+to separate a small set of test cases from each other then you can use the
+``LockFixture``.
+
+However, in some cases, a small set of tests needs to be run serially. For
+example, some of the host aggregate and availability zone testing needs
+compute nodes without any running nova server to be able to move compute hosts
+between availability zones. But many tempest tests start one or more nova
+servers.
+
+
+Why are these tests in Tempest?
+-------------------------------
+
+This is one of Tempest's core purposes, testing the integration between
+projects.
+
+
+Scope of these tests
+--------------------
+
+The tests should always use the Tempest implementation of the OpenStack API,
+as we want to ensure that bugs aren't hidden by the official clients.
+
+Tests should be tagged with which services they exercise, as
+determined by which client libraries are used directly by the test.
+
+
+Example of a good test
+----------------------
+
+While we are looking for interaction of 2 or more services, be specific in
+your interactions. A giant "this is my data center" smoke test is hard to
+debug when it goes wrong.
+
+The tests that need to be run serially need to be marked with the
+``@serial`` class decorator. This will make sure that even if tempest is
+configured to run the tests in parallel, these tests will always be executed
+separately from the rest of the test cases.
+
+Please note that due to test ordering optimization reasons test cases marked
+for ``@serial`` execution need to be put under ``tempest/serial_tests``
+directory. This will ensure that the serial tests will block the parallel tests
+in the least amount of time.
diff --git a/tempest/test.py b/tempest/test.py
index 3360221..a766367 100644
--- a/tempest/test.py
+++ b/tempest/test.py
@@ -26,7 +26,6 @@
from tempest import clients
from tempest.common import credentials_factory as credentials
-from tempest.common import utils
from tempest import config
from tempest.lib.common import api_microversion_fixture
from tempest.lib.common import fixed_network
@@ -45,20 +44,6 @@
version='Pike', removal_version='?')
-services = debtcollector.moves.moved_function(
- utils.services, 'services', __name__,
- version='Pike', removal_version='?')
-
-
-requires_ext = debtcollector.moves.moved_function(
- utils.requires_ext, 'requires_ext', __name__,
- version='Pike', removal_version='?')
-
-
-is_extension_enabled = debtcollector.moves.moved_function(
- utils.is_extension_enabled, 'is_extension_enabled', __name__,
- version='Pike', removal_version='?')
-
at_exit_set = set()
diff --git a/tempest/tests/README.rst b/tempest/tests/README.rst
index 0587e7b..081dd07 100644
--- a/tempest/tests/README.rst
+++ b/tempest/tests/README.rst
@@ -14,6 +14,7 @@
Why are these tests in Tempest?
-------------------------------
+
These tests exist to make sure that the mechanisms that we use inside of
Tempest are valid and remain functional. They are only here for self
validation of Tempest.
@@ -21,6 +22,7 @@
Scope of these tests
--------------------
+
Unit tests should not require an external service to be running or any extra
configuration to run. Any state that is required for a test should either be
mocked out or created in a temporary test directory. (see test_wrappers.py for
diff --git a/tempest/tests/cmd/test_run.py b/tempest/tests/cmd/test_run.py
index 3b5e901..b487c3f 100644
--- a/tempest/tests/cmd/test_run.py
+++ b/tempest/tests/cmd/test_run.py
@@ -225,6 +225,11 @@
'%s=%s' % (self.exclude_list, path),
'--regex', 'fail'], 1)
+ def test_tempest_run_with_slowest(self):
+ out, err = self.assertRunExit(['tempest', 'run', '--regex', 'passing',
+ '--slowest'], 0)
+ self.assertRegex(str(out), r'Test id\s+Runtime \(s\)')
+
class TestOldArgRunReturnCode(TestRunReturnCode):
"""A class for testing deprecated but still supported args.
@@ -363,6 +368,7 @@
parsed_args.state = None
parsed_args.list_tests = False
parsed_args.config_file = path
+ parsed_args.slowest = False
with mock.patch('stestr.commands.run_command') as m:
m.return_value = 0
@@ -393,6 +399,7 @@
parsed_args.state = None
parsed_args.list_tests = False
parsed_args.config_file = path
+ parsed_args.slowest = False
with mock.patch('stestr.commands.run_command') as m:
m.return_value = 0
@@ -409,6 +416,7 @@
parsed_args.state = None
parsed_args.list_tests = False
parsed_args.config_file = ''
+ parsed_args.slowest = False
with mock.patch('stestr.commands.run_command') as m:
m.return_value = 0
@@ -441,6 +449,7 @@
parsed_args.state = True
parsed_args.list_tests = False
parsed_args.config_file = ''
+ parsed_args.slowest = False
with mock.patch('stestr.commands.run_command') as m:
m.return_value = 0
@@ -460,6 +469,7 @@
parsed_args.state = True
parsed_args.list_tests = False
parsed_args.config_file = path
+ parsed_args.slowest = False
with mock.patch('stestr.commands.run_command') as m:
m.return_value = 0
diff --git a/tempest/tests/common/test_credentials_factory.py b/tempest/tests/common/test_credentials_factory.py
index 374474d..8a1158d 100644
--- a/tempest/tests/common/test_credentials_factory.py
+++ b/tempest/tests/common/test_credentials_factory.py
@@ -37,7 +37,7 @@
fake_config.FakePrivate)
def test_get_dynamic_provider_params_creds_v2(self):
- expected_uri = 'EXPECTED_V2_URI'
+ expected_uri = 'http://v2.identy.example.com'
cfg.CONF.set_default('uri', expected_uri, group='identity')
admin_creds = fake_credentials.FakeCredentials()
params = cf.get_dynamic_provider_params('v2', admin_creds=admin_creds)
@@ -48,7 +48,7 @@
self.assertEqual(expected_params[key], params[key])
def test_get_dynamic_provider_params_creds_v3(self):
- expected_uri = 'EXPECTED_V3_URI'
+ expected_uri = 'http://v3.identy.example.com'
cfg.CONF.set_default('uri_v3', expected_uri, group='identity')
admin_creds = fake_credentials.FakeCredentials()
params = cf.get_dynamic_provider_params('v3', admin_creds=admin_creds)
@@ -76,14 +76,14 @@
fill_in=True, identity_version=expected_identity_version)
def test_get_preprov_provider_params_creds_v2(self):
- expected_uri = 'EXPECTED_V2_URI'
+ expected_uri = 'http://v2.identy.example.com'
cfg.CONF.set_default('uri', expected_uri, group='identity')
params = cf.get_preprov_provider_params('v2')
self.assertIn('identity_uri', params)
self.assertEqual(expected_uri, params['identity_uri'])
def test_get_preprov_provider_params_creds_v3(self):
- expected_uri = 'EXPECTED_V3_URI'
+ expected_uri = 'http://v3.identy.example.com'
cfg.CONF.set_default('uri_v3', expected_uri, group='identity')
params = cf.get_preprov_provider_params('v3')
self.assertIn('identity_uri', params)
@@ -237,7 +237,7 @@
@mock.patch('tempest.lib.auth.get_credentials')
def test_get_credentials_v2(self, mock_auth_get_credentials):
- expected_uri = 'V2_URI'
+ expected_uri = 'http://v2.identity.example.com'
expected_result = 'my_creds'
mock_auth_get_credentials.return_value = expected_result
cfg.CONF.set_default('uri', expected_uri, 'identity')
@@ -252,7 +252,7 @@
@mock.patch('tempest.lib.auth.get_credentials')
def test_get_credentials_v3_no_domain(self, mock_auth_get_credentials):
- expected_uri = 'V3_URI'
+ expected_uri = 'https://v3.identity.exmaple.com'
expected_result = 'my_creds'
expected_domain = 'my_domain'
mock_auth_get_credentials.return_value = expected_result
@@ -272,7 +272,7 @@
@mock.patch('tempest.lib.auth.get_credentials')
def test_get_credentials_v3_domain(self, mock_auth_get_credentials):
- expected_uri = 'V3_URI'
+ expected_uri = 'https://v3.identity.exmaple.com'
expected_result = 'my_creds'
expected_domain = 'my_domain'
mock_auth_get_credentials.return_value = expected_result
@@ -291,7 +291,7 @@
@mock.patch('tempest.lib.auth.get_credentials')
def test_get_credentials_v3_system(self, mock_auth_get_credentials):
- expected_uri = 'V3_URI'
+ expected_uri = 'https://v3.identity.exmaple.com'
expected_result = 'my_creds'
mock_auth_get_credentials.return_value = expected_result
cfg.CONF.set_default('uri_v3', expected_uri, 'identity')
diff --git a/tox.ini b/tox.ini
index de81707..51c38f2 100644
--- a/tox.ini
+++ b/tox.ini
@@ -198,7 +198,7 @@
commands =
find . -type f -name "*.pyc" -delete
tempest run --regex {[testenv:integrated-compute]regex1} --exclude-list ./tools/tempest-integrated-gate-compute-exclude-list.txt {posargs}
- tempest run --combine --serial --regex {[testenv:integrated-compute]regex2} --exclude-list ./tools/tempest-integrated-gate-compute-exclude-list.txt {posargs}
+ tempest run --combine --serial --slowest --regex {[testenv:integrated-compute]regex2} --exclude-list ./tools/tempest-integrated-gate-compute-exclude-list.txt {posargs}
[testenv:integrated-placement]
envdir = .tox/tempest
@@ -359,8 +359,9 @@
sphinx-apidoc -f -o doc/source/tests/image tempest/api/image
sphinx-apidoc -f -o doc/source/tests/network tempest/api/network
sphinx-apidoc -f -o doc/source/tests/object_storage tempest/api/object_storage
- sphinx-apidoc -f -o doc/source/tests/scenario tempest/scenario
sphinx-apidoc -f -o doc/source/tests/volume tempest/api/volume
+ sphinx-apidoc -f -o doc/source/tests/scenario tempest/scenario
+ sphinx-apidoc -f -o doc/source/tests/serial_tests tempest/serial_tests
rm -rf doc/build
sphinx-build -W -b html doc/source doc/build/html
allowlist_externals =
@@ -377,8 +378,9 @@
sphinx-apidoc -f -o doc/source/tests/image tempest/api/image
sphinx-apidoc -f -o doc/source/tests/network tempest/api/network
sphinx-apidoc -f -o doc/source/tests/object_storage tempest/api/object_storage
- sphinx-apidoc -f -o doc/source/tests/scenario tempest/scenario
sphinx-apidoc -f -o doc/source/tests/volume tempest/api/volume
+ sphinx-apidoc -f -o doc/source/tests/scenario tempest/scenario
+ sphinx-apidoc -f -o doc/source/tests/serial_tests tempest/serial_tests
sphinx-build -W -b latex doc/source doc/build/pdf
make -C doc/build/pdf
diff --git a/zuul.d/base.yaml b/zuul.d/base.yaml
index 0ac893a..3b402c8 100644
--- a/zuul.d/base.yaml
+++ b/zuul.d/base.yaml
@@ -24,6 +24,8 @@
min_compute_nodes: "{{ groups['compute'] | default(['controller']) | length }}"
test_results_stage_name: test_results
zuul_copy_output:
+ '/var/log/openvswitch': logs
+ '/var/log/ovn': logs
'{{ devstack_base_dir }}/tempest/etc/tempest.conf': logs
'{{ devstack_base_dir }}/tempest/etc/accounts.yaml': logs
'{{ devstack_base_dir }}/tempest/tempest.log': logs
diff --git a/zuul.d/integrated-gate.yaml b/zuul.d/integrated-gate.yaml
index 4b4306c..596acb1 100644
--- a/zuul.d/integrated-gate.yaml
+++ b/zuul.d/integrated-gate.yaml
@@ -8,6 +8,7 @@
Integration test that runs all tests.
Former name for this job was:
* legacy-periodic-tempest-dsvm-all-master
+ timeout: 10800
vars:
tox_envlist: all
tempest_test_regex: tempest
@@ -29,32 +30,6 @@
tox_envlist: ipv6-only
- job:
- name: tempest-full
- parent: devstack-tempest
- description: |
- Base integration test with Neutron networking and py27.
- This job is supposed to run until stable/train setup only.
- If you are running it on stable/ussuri gate onwards for python2.7
- coverage then you need to do override-checkout with any stable
- branch less than or equal to stable/train.
- Former names for this job where:
- * legacy-tempest-dsvm-neutron-full
- * gate-tempest-dsvm-neutron-full-ubuntu-xenial
- vars:
- tox_envlist: full
- devstack_localrc:
- ENABLE_FILE_INJECTION: true
- ENABLE_VOLUME_MULTIATTACH: true
- USE_PYTHON3: False
- devstack_services:
- # NOTE(mriedem): Disable the cinder-backup service from tempest-full
- # since tempest-full is in the integrated-gate project template but
- # the backup tests do not really involve other services so they should
- # be run in some more cinder-specific job, especially because the
- # tests fail at a high rate (see bugs 1483434, 1813217, 1745168)
- c-bak: false
-
-- job:
name: tempest-extra-tests
parent: tempest-full-py3
description: |
@@ -73,7 +48,7 @@
# this job definition is only for stable/xena onwards
# and separate job definition until stable/wallaby
branches:
- regex: ^stable/(stein|train|ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
description: |
Base integration test with Neutron networking, horizon, swift enable,
@@ -107,7 +82,7 @@
nodeset: devstack-single-node-centos-9-stream
# centos-9-stream is supported from yoga release onwards
branches:
- regex: ^stable/(stein|train|ussuri|victoria|wallaby|xena)$
+ regex: ^.*/(victoria|wallaby|xena)$
negate: true
description: |
Base integration test on CentOS 9 stream
@@ -168,7 +143,7 @@
nodeset: devstack-single-node-centos-9-stream
# centos-9-stream is supported from yoga release onwards
branches:
- regex: ^stable/(stein|train|ussuri|victoria|wallaby|xena)$
+ regex: ^.*/(victoria|wallaby|xena)$
negate: true
description: |
This job runs integration tests for compute. This is
@@ -255,7 +230,7 @@
nodeset: openstack-two-node-jammy
# This job runs on ubuntu Jammy and after stable/zed.
branches:
- regex: ^stable/(stein|train|ussuri|victoria|wallaby|xena|yoga|zed)$
+ regex: ^.*/(victoria|wallaby|xena|yoga|zed)$
negate: true
vars:
# NOTE(gmann): Default concurrency is higher (number of cpu -2) which
@@ -263,6 +238,7 @@
# requests to services and can cause more oom issues. To avoid the
# oom issue, setting the concurrency to 4 in this job.
tempest_concurrency: 4
+ tempest_set_src_dest_host: true
devstack_localrc:
USE_PYTHON3: true
devstack_plugins:
@@ -319,7 +295,7 @@
# till stable/wallaby, this job definition is only for stable/xena
# onwards and separate job definition until stable/wallaby
branches:
- regex: ^stable/(stein|train|ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
vars:
tox_envlist: slow
@@ -440,11 +416,12 @@
voting: false
branches:
- stable/2023.1
- # on master (SLURP 2024.1) grenade-skip-level which test stable/2023.1
- # to stable/2024.1 upgrade is voting.
+ # on stable/2024.1(SLURP) grenade-skip-level is voting which test
+ # stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
+ # SLURP release only.
- grenade-skip-level:
branches:
- - master
+ - ^.*/2024.1
- tempest-integrated-networking
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
@@ -452,24 +429,25 @@
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
- regex: ^stable/(ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
gate:
jobs:
- grenade
- tempest-integrated-networking
- # on master (SLURP 2024.1) grenade-skip-level which test stable/2023.1
- # to stable/2024.1 upgrade is voting.
+ # on stable/2024.1(SLURP) grenade-skip-level is voting which test
+ # stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
+ # SLURP release only.
- grenade-skip-level:
branches:
- - master
+ - ^.*/2024.1
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
- regex: ^stable/(ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
- project-template:
@@ -501,35 +479,39 @@
# to make sure we do not run this job on older than 2023.2 gate.
- grenade-skip-level-always:
branches:
+ - ^.*/2023.2
+ - ^.*/2024.1
- master
- tempest-integrated-compute
# centos-8-stream is tested from wallaby -> yoga branches
- tempest-integrated-compute-centos-8-stream:
- branches: ^stable/(wallaby|xena|yoga).*$
+ branches: ^.*/(wallaby|xena|yoga)$
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
- regex: ^stable/(ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
gate:
jobs:
- grenade-skip-level-always:
branches:
+ - ^.*/2023.2
+ - ^.*/2024.1
- master
- tempest-integrated-compute
- openstacksdk-functional-devstack:
branches:
- regex: ^stable/(ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
periodic-weekly:
jobs:
# centos-9-stream is tested from zed release onwards
- tempest-integrated-compute-centos-9-stream:
branches:
- regex: ^stable/(stein|train|ussuri|victoria|wallaby|xena|yoga)$
+ regex: ^.*/(victoria|wallaby|xena|yoga)$
negate: true
- project-template:
@@ -549,11 +531,12 @@
voting: false
branches:
- stable/2023.1
- # on master (SLURP 2024.1) grenade-skip-level which test stable/2023.1
- # to stable/2024.1 upgrade is voting.
+ # on stable/2024.1(SLURP) grenade-skip-level is voting which test
+ # stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
+ # SLURP release only.
- grenade-skip-level:
branches:
- - master
+ - ^.*/2024.1
- tempest-integrated-placement
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
@@ -561,24 +544,25 @@
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
- regex: ^stable/(ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
gate:
jobs:
- grenade
- tempest-integrated-placement
- # on master (SLURP 2024.1) grenade-skip-level which test stable/2023.1
- # to stable/2024.1 upgrade is voting.
+ # on stable/2024.1(SLURP) grenade-skip-level is voting which test
+ # stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
+ # SLURP release only.
- grenade-skip-level:
branches:
- - master
+ - ^.*/2024.1
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
- regex: ^stable/(ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
- project-template:
@@ -598,11 +582,12 @@
voting: false
branches:
- stable/2023.1
- # on master (SLURP 2024.1) grenade-skip-level which test stable/2023.1
- # to stable/2024.1 upgrade is voting.
+ # on stable/2024.1(SLURP) grenade-skip-level is voting which test
+ # stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
+ # SLURP release only.
- grenade-skip-level:
branches:
- - master
+ - ^.*/2024.1
- tempest-integrated-storage
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
@@ -610,16 +595,17 @@
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
- regex: ^stable/(ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
gate:
jobs:
- grenade
- # on master (SLURP 2024.1) grenade-skip-level which test stable/2023.1
- # to stable/2024.1 upgrade is voting.
+ # on stable/2024.1(SLURP) grenade-skip-level is voting which test
+ # stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
+ # SLURP release only.
- grenade-skip-level:
branches:
- - master
+ - ^.*/2024.1
- tempest-integrated-storage
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
@@ -627,7 +613,7 @@
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
- regex: ^stable/(ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
- project-template:
@@ -640,11 +626,12 @@
check:
jobs:
- grenade
- # on master (SLURP 2024.1) grenade-skip-level which test stable/2023.1
- # to stable/2024.1 upgrade is voting.
+ # on stable/2024.1(SLURP) grenade-skip-level is voting which test
+ # stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
+ # SLURP release only.
- grenade-skip-level:
branches:
- - master
+ - ^.*/2024.1
- tempest-integrated-object-storage
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
@@ -652,16 +639,17 @@
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
- regex: ^stable/(ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
gate:
jobs:
- grenade
- # on master (SLURP 2024.1) grenade-skip-level which test stable/2023.1
- # to stable/2024.1 upgrade is voting.
+ # on stable/2024.1(SLURP) grenade-skip-level is voting which test
+ # stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
+ # SLURP release only.
- grenade-skip-level:
branches:
- - master
+ - ^.*/2024.1
- tempest-integrated-object-storage
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
@@ -669,5 +657,5 @@
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
- regex: ^stable/(ussuri|victoria|wallaby)$
+ regex: ^.*/(victoria|wallaby)$
negate: true
diff --git a/zuul.d/project.yaml b/zuul.d/project.yaml
index 3f32f9f..e2505cb 100644
--- a/zuul.d/project.yaml
+++ b/zuul.d/project.yaml
@@ -37,9 +37,9 @@
# if things are working in latest and oldest it will work in between
# stable branches also. If anything is breaking we will be catching
# those in respective stable branch gate.
- - tempest-full-2023-2:
+ - tempest-full-2024-1:
irrelevant-files: *tempest-irrelevant-files
- - tempest-full-yoga:
+ - tempest-full-zed:
irrelevant-files: *tempest-irrelevant-files
- tempest-multinode-full-py3:
irrelevant-files: *tempest-irrelevant-files
@@ -111,7 +111,7 @@
irrelevant-files: *tempest-irrelevant-files
- grenade:
irrelevant-files: *tempest-irrelevant-files
- - grenade-skip-level:
+ - grenade-skip-level-always:
irrelevant-files: *tempest-irrelevant-files
- neutron-ovs-tempest-dvr:
voting: false
@@ -142,7 +142,7 @@
irrelevant-files: *tempest-irrelevant-files
- grenade:
irrelevant-files: *tempest-irrelevant-files
- - grenade-skip-level:
+ - grenade-skip-level-always:
irrelevant-files: *tempest-irrelevant-files
- tempest-ipv6-only:
irrelevant-files: *tempest-irrelevant-files-3
@@ -155,7 +155,7 @@
- nova-live-migration:
irrelevant-files: *tempest-irrelevant-files
- ironic-tempest-bios-ipmi-direct-tinyipa:
- irrelevant-files: *tempest-irrelevant-files
+ irrelevant-files: *tempest-irrelevant-files
experimental:
jobs:
- nova-multi-cell
@@ -170,7 +170,6 @@
- tempest-all-rbac-old-defaults
- tempest-full-parallel
- tempest-full-zed-extra-tests
- - tempest-full-yoga-extra-tests
- tempest-full-enforce-scope-new-defaults-zed
- neutron-ovs-tempest-dvr-ha-multinode-full:
irrelevant-files: *tempest-irrelevant-files
@@ -190,18 +189,18 @@
irrelevant-files: *tempest-irrelevant-files
periodic-stable:
jobs:
+ - tempest-full-2024-1
- tempest-full-2023-2
- tempest-full-2023-1
- tempest-full-zed
- - tempest-full-yoga
+ - tempest-slow-2024-1
- tempest-slow-2023-2
- tempest-slow-2023-1
- tempest-slow-zed
- - tempest-slow-yoga
+ - tempest-full-2024-1-extra-tests
- tempest-full-2023-2-extra-tests
- tempest-full-2023-1-extra-tests
- tempest-full-zed-extra-tests
- - tempest-full-yoga-extra-tests
periodic:
jobs:
- tempest-all
diff --git a/zuul.d/stable-jobs.yaml b/zuul.d/stable-jobs.yaml
index 2fdc2af..a662685 100644
--- a/zuul.d/stable-jobs.yaml
+++ b/zuul.d/stable-jobs.yaml
@@ -1,5 +1,11 @@
# NOTE(gmann): This file includes all stable release jobs definition.
- job:
+ name: tempest-full-2024-1
+ parent: tempest-full-py3
+ nodeset: openstack-single-node-jammy
+ override-checkout: stable/2024.1
+
+- job:
name: tempest-full-2023-2
parent: tempest-full-py3
nodeset: openstack-single-node-jammy
@@ -18,10 +24,10 @@
override-checkout: stable/zed
- job:
- name: tempest-full-yoga
- parent: tempest-full-py3
- nodeset: openstack-single-node-focal
- override-checkout: stable/yoga
+ name: tempest-full-2024-1-extra-tests
+ parent: tempest-extra-tests
+ nodeset: openstack-single-node-jammy
+ override-checkout: stable/2024.1
- job:
name: tempest-full-2023-2-extra-tests
@@ -42,10 +48,10 @@
override-checkout: stable/zed
- job:
- name: tempest-full-yoga-extra-tests
- parent: tempest-extra-tests
- nodeset: openstack-single-node-focal
- override-checkout: stable/yoga
+ name: tempest-slow-2024-1
+ parent: tempest-slow-py3
+ nodeset: openstack-two-node-jammy
+ override-checkout: stable/2024.1
- job:
name: tempest-slow-2023-2
@@ -72,20 +78,13 @@
override-checkout: stable/zed
- job:
- name: tempest-slow-yoga
- parent: tempest-slow-py3
- nodeset: openstack-two-node-focal
- override-checkout: stable/yoga
-
-- job:
name: tempest-full-py3
parent: devstack-tempest
# This job version is to use the 'full' tox env which
# is available for stable/ussuri to stable/wallaby also.
branches:
- - stable/ussuri
- - stable/victoria
- - stable/wallaby
+ - ^.*/victoria
+ - ^.*/wallaby
description: |
Base integration test with Neutron networking, horizon, swift enable,
and py3.
@@ -108,89 +107,16 @@
horizon: true
- job:
- name: tempest-full-py3
- parent: devstack-tempest
- # This job version is with swift disabled on py3
- # as swift was not ready on py3 until stable/train.
- branches:
- - stable/stein
- - stable/train
- description: |
- Base integration test with Neutron networking, swift disabled, and py3.
- Former names for this job where:
- * legacy-tempest-dsvm-py35
- * gate-tempest-dsvm-py35
- required-projects:
- - openstack/horizon
- vars:
- tox_envlist: full
- devstack_localrc:
- USE_PYTHON3: true
- FORCE_CONFIG_DRIVE: true
- ENABLE_VOLUME_MULTIATTACH: true
- GLANCE_USE_IMPORT_WORKFLOW: True
- devstack_plugins:
- neutron: https://opendev.org/openstack/neutron
- devstack_local_conf:
- post-config:
- "/$NEUTRON_CORE_PLUGIN_CONF":
- ovs:
- bridge_mappings: public:br-ex
- resource_provider_bandwidths: br-ex:1000000:1000000
- test-config:
- $TEMPEST_CONFIG:
- network-feature-enabled:
- qos_placement_physnet: public
- devstack_services:
- # Enbale horizon so that we can run horizon test.
- horizon: true
- s-account: false
- s-container: false
- s-object: false
- s-proxy: false
- # without Swift, c-bak cannot run (in the Gate at least)
- # NOTE(mriedem): Disable the cinder-backup service from
- # tempest-full-py3 since tempest-full-py3 is in the integrated-gate-py3
- # project template but the backup tests do not really involve other
- # services so they should be run in some more cinder-specific job,
- # especially because the tests fail at a high rate (see bugs 1483434,
- # 1813217, 1745168)
- c-bak: false
- neutron-placement: true
- neutron-qos: true
-
-- job:
- name: tempest-multinode-full-py3
- parent: tempest-multinode-full
- nodeset: openstack-two-node-bionic
- # This job runs on Bionic.
- branches:
- - stable/stein
- - stable/train
- - stable/ussuri
- vars:
- devstack_localrc:
- USE_PYTHON3: true
- devstack_plugins:
- neutron: https://opendev.org/openstack/neutron
- devstack_services:
- neutron-trunk: true
- group-vars:
- subnode:
- devstack_localrc:
- USE_PYTHON3: true
-
-- job:
name: tempest-multinode-full-py3
parent: tempest-multinode-full
nodeset: openstack-two-node-focal
# This job runs on Focal and supposed to run until stable/zed.
branches:
- - stable/victoria
- - stable/wallaby
- - stable/xena
- - stable/yoga
- - stable/zed
+ - ^.*/victoria
+ - ^.*/wallaby
+ - ^.*/xena
+ - ^.*/yoga
+ - ^.*/zed
vars:
devstack_localrc:
USE_PYTHON3: true
@@ -209,11 +135,11 @@
nodeset: openstack-two-node-focal
# This job runs on Focal and on python2. This is for stable/victoria to stable/zed.
branches:
- - stable/victoria
- - stable/wallaby
- - stable/xena
- - stable/yoga
- - stable/zed
+ - ^.*/victoria
+ - ^.*/wallaby
+ - ^.*/xena
+ - ^.*/yoga
+ - ^.*/zed
vars:
devstack_localrc:
USE_PYTHON3: False
@@ -223,56 +149,13 @@
USE_PYTHON3: False
- job:
- name: tempest-multinode-full
- parent: tempest-multinode-full-base
- nodeset: openstack-two-node-bionic
- # This job runs on Bionic and on python2. This is for stable/stein and stable/train.
- # This job is prepared to make sure all stable branches from stable/stein till stable/train
- # will keep running on bionic. This can be removed once stable/train is EOL.
- branches:
- - stable/stein
- - stable/train
- - stable/ussuri
- vars:
- devstack_localrc:
- USE_PYTHON3: False
- group-vars:
- subnode:
- devstack_localrc:
- USE_PYTHON3: False
-
-- job:
- name: tempest-slow-py3
- parent: tempest-slow
- # This job version is with swift disabled on py3
- # as swift was not ready on py3 until stable/train.
- branches:
- - stable/stein
- - stable/train
- vars:
- devstack_localrc:
- USE_PYTHON3: true
- devstack_services:
- s-account: false
- s-container: false
- s-object: false
- s-proxy: false
- # without Swift, c-bak cannot run (in the Gate at least)
- c-bak: false
- group-vars:
- subnode:
- devstack_localrc:
- USE_PYTHON3: true
-
-- job:
name: tempest-slow-py3
parent: tempest-slow
# This job version is to use the 'slow-serial' tox env for
# the stable/ussuri to stable/wallaby testing.
branches:
- - stable/ussuri
- - stable/victoria
- - stable/wallaby
+ - ^.*/victoria
+ - ^.*/wallaby
vars:
tox_envlist: slow-serial
@@ -287,12 +170,9 @@
# This job is not used after stable/xena and can be
# removed once stable/xena is EOL.
branches:
- - stable/stein
- - stable/train
- - stable/ussuri
- - stable/victoria
- - stable/wallaby
- - stable/xena
+ - ^.*/victoria
+ - ^.*/wallaby
+ - ^.*/xena
- job:
name: tempest-integrated-compute-centos-8-stream
@@ -301,9 +181,9 @@
voting: false
nodeset: devstack-single-node-centos-8-stream
branches:
- - stable/wallaby
- - stable/xena
- - stable/yoga
+ - ^.*/wallaby
+ - ^.*/xena
+ - ^.*/yoga
description: |
This job runs integration tests for compute. This is
subset of 'tempest-full-py3' job and run Nova, Neutron, Cinder (except backup tests)
@@ -320,9 +200,9 @@
# TODO(gmann): Make this job non voting until bug#1957941 if fixed.
voting: false
branches:
- - stable/wallaby
- - stable/xena
- - stable/yoga
+ - ^.*/wallaby
+ - ^.*/xena
+ - ^.*/yoga
nodeset: devstack-single-node-centos-8-stream
description: |
Base integration test with Neutron networking and py36 running