Merge "Add microversion testing doc in tempest"
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 17def1c..5f357b2 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -12,6 +12,7 @@
REVIEWING
plugin
library
+ microversion_testing
------------
Field Guides
diff --git a/doc/source/microversion_testing.rst b/doc/source/microversion_testing.rst
new file mode 100644
index 0000000..bb48b3f
--- /dev/null
+++ b/doc/source/microversion_testing.rst
@@ -0,0 +1,206 @@
+===================================
+How To Implement Microversion Tests
+===================================
+
+Tempest provides stable interfaces to test API microversion.
+For Details, see: `API Microversion testing Framework`_
+This document explains how to implement microversion tests using those
+interfaces.
+
+.. _API Microversion testing Framework: http://docs.openstack.org/developer/tempest/library/api_microversion_testing.html
+
+
+Configuration options for Microversion
+""""""""""""""""""""""""""""""""""""""
+
+* Add configuration options for specifying test target Microversions.
+ We need to specify test target Microversions because the supported
+ microversions may be different between OpenStack clouds. For operating
+ multiple Microversion tests in a single Tempest operation, configuration
+ options should represent the range of test target Microversions.
+ New configuration options are:
+ - min_microversion
+ - max_microversion
+ Those should be defined under respective section of each service.
+ For Example::
+ [compute]
+ min_microversion = None
+ max_microversion = latest
+
+
+How To Implement Microversion Tests
+"""""""""""""""""""""""""""""""""""
+
+Step1: Add skip logic based on configured microversion range
+''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+Add logic to skip the tests based on Tests class and configured Microversion
+range.
+api_version_utils.check_skip_with_microversion function can be used
+to automatically skip the tests which does not fall under configured
+Microversion range.
+For example::
+
+ class BaseTestCase1(api_version_utils.BaseMicroversionTest):
+
+ [..]
+ @classmethod
+ def skip_checks(cls):
+ super(BaseTestCase1, cls).skip_checks()
+ api_version_utils.check_skip_with_microversion(cls.min_microversion,
+ cls.max_microversion,
+ CONF.compute.min_microversion,
+ CONF.compute.max_microversion)
+
+Skip logic can be added in tests base class or any specific test class depends on
+tests class structure.
+
+Step2: Selected API request microversion
+''''''''''''''''''''''''''''''''''''''''
+
+Select appropriate Microversion which needs to be used
+to send with API request.
+api_version_utils.select_request_microversion function can be used
+to select the appropriate microversion which will be used for API request.
+For example::
+
+ @classmethod
+ def resource_setup(cls):
+ super(BaseTestCase1, cls).resource_setup()
+ cls.request_microversion = (
+ api_version_utils.select_request_microversion(
+ cls.min_microversion,
+ CONF.compute.min_microversion))
+
+
+Step3: Set Microversion on Service Clients
+''''''''''''''''''''''''''''''''''''''''''
+
+Microversion selected by Test Class in previous step needs to be set on
+service clients so that APIs can be requested with selected microversion.
+
+Microversion can be defined as global variable on service clients which
+can be set using fixture.
+Also microversion header name needs to be defined on service clients which
+should be constant because it is not supposed to be changed by project
+as per API contract.
+For example::
+
+ COMPUTE_MICROVERSION = None
+
+ class BaseClient1(rest_client.RestClient):
+ api_microversion_header_name = 'X-OpenStack-Nova-API-Version'
+
+Now test class can set the selected microversion on required service clients
+using fixture which can take care of reseting the same once tests is completed.
+For example::
+
+ def setUp(self):
+ super(BaseTestCase1,, self).setUp()
+ self.useFixture(api_microversion_fixture.APIMicroversionFixture(
+ self.request_microversion))
+
+Service clients needs to add set microversion in API request header which
+can be done by overriding the get_headers() method of rest_client.
+For example::
+
+ COMPUTE_MICROVERSION = None
+
+ class BaseClient1(rest_client.RestClient):
+ api_microversion_header_name = 'X-OpenStack-Nova-API-Version'
+
+ def get_headers(self):
+ headers = super(BaseClient1, self).get_headers()
+ if COMPUTE_MICROVERSION:
+ headers[self.api_microversion_header_name] = COMPUTE_MICROVERSION
+ return headers
+
+
+Step4: Separate Test classes for each Microversion
+''''''''''''''''''''''''''''''''''''''''''''''''''
+
+This is last step to implement microversion test class.
+
+For any microversion tests, basically we need to implement a
+separate test class. In addition, each test class defines its
+microversion range with class variable like min_microversion
+and max_microversion. Tests will be valid for that defined range.
+If that range is out of configured microversion range then, test
+will be skipped.
+
+*NOTE: Microversion testing is supported at test class level not at individual
+test case level.*
+For example:
+
+Below test is applicable for microversion from 2.2 till 2.9::
+
+ class BaseTestCase1(api_version_utils.BaseMicroversionTest,
+ tempest.test.BaseTestCase):
+
+ [..]
+
+
+ class Test1(BaseTestCase1):
+ min_microversion = '2.2'
+ max_microversion = '2.9'
+
+ [..]
+
+Below test is applicable for microversion from 2.10 till latest::
+
+ class Test2(BaseTestCase1):
+ min_microversion = '2.10'
+ max_microversion = 'latest'
+
+ [..]
+
+
+
+
+Notes about Compute Microversion Tests
+"""""""""""""""""""""""""""""""""""
+Some of the compute Microversion tests have been already implemented
+with the microversion testing framework. So for further tests only
+step 4 is needed.
+
+Along with that JSON response schema might need versioning if needed.
+
+Compute service clients strictly validate the response against defined JSON
+schema and does not allow additional elements in response.
+So if that microversion changed the API response then schema needs to be version.
+New JSON schema file needs be defined with new response attributes and service
+client methods will select the schema based on requested microversion.
+
+If microversion tests are implemented randomly means not
+in sequence order(v2.20 tests added and previous microversion tests are not yet added)
+then, still schema might needs to be version for older microversion if they changed
+the response.
+This is because Nova microversion includes all the previous microversions behvaior.
+
+For Example:
+ Implementing the v2.20 microversion tests before v2.9 and 2.19-
+ v2.20 API request will respond as latest behavior of Nova till v2.20,
+ and in v2.9 and 2.19, server response has been changed so response schema needs
+ to be version accordingly.
+
+That can be done by using the get_schema method in below module:
+
+The base_compute_client module
+''''''''''''''''''''''''''''''
+
+.. automodule:: tempest.lib.services.compute.base_compute_client
+ :members:
+
+
+Microversion tests implemented in Tempest
+"""""""""""""""""""""""""""""""""""""""""
+
+* Compute
+
+ * `2.1`_
+
+ .. _2.1: http://docs.openstack.org/developer/nova/api_microversion_history.html#id1
+
+ * `2.2`_
+
+ .. _2.2: http://docs.openstack.org/developer/nova/api_microversion_history.html#id2