Leverage the remote microversion without requiring human config
Tempest plugins are... weird.
The challenge is to have a branchless utility which can be loaded
and help provide feedback if the remote service is correctly
responding as we would expect.
This works great in theory, until you have to either do some
sort of negative test, or plan in advance, or until you have
some sort of mixed state environment. This also weirdly
restraints testing against older versions on older branches,
requiring further care and feeding to keep things passing.
And the way issues like these are resolved, originally, was to
leverage manual human configuration. The problem is, that doesn't
always work and operationally becomes an increased burden.
So the logical path forward is for the plugin to automatically
skip specific tests *based upon* the remote offered API
microversion, much like many of the tests do if a driver or
running configuration does not exist.
This can be done because when we compose tests, we have a minimum
and maximum API version where we know the test is valid, and if
the remote endpoint is outside of that bound.
The result is now the plugin will query the remote endpoint and
collect the minimum and maximum API versions as part of skip
version testing, so if either are defined on a test class, then
we make a decision automatically removing the need to configure
aspects specifically.
Change-Id: I197e6c30c8514e1f72cb1ce3ebad851802632203
diff --git a/ironic_tempest_plugin/tests/api/base.py b/ironic_tempest_plugin/tests/api/base.py
index 6ebb162..59f2f94 100644
--- a/ironic_tempest_plugin/tests/api/base.py
+++ b/ironic_tempest_plugin/tests/api/base.py
@@ -76,6 +76,8 @@
cfg_min_version = CONF.baremetal.min_microversion
cfg_max_version = CONF.baremetal.max_microversion
+
+ # Check versions and skip based upon *configuration*
api_version_utils.check_skip_with_microversion(cls.min_microversion,
cls.max_microversion,
cfg_min_version,
@@ -100,6 +102,15 @@
else:
cls.client = cls.os_admin.baremetal.BaremetalClient()
+ # Skip the test if the class version doesn't match.
+ if cls.min_microversion or cls.max_microversion:
+ api_min, api_max = cls.client.get_min_max_api_microversions()
+ api_version_utils.check_skip_with_microversion(
+ cls.min_microversion,
+ cls.max_microversion,
+ api_min,
+ api_max)
+
@classmethod
def resource_setup(cls):
super(BaseBaremetalTest, cls).resource_setup()