Merge "test_l3_agent_scheduler: remove workaround code for Liberty"
diff --git a/doc/source/microversion_testing.rst b/doc/source/microversion_testing.rst
index adbd2dc..b516055 100644
--- a/doc/source/microversion_testing.rst
+++ b/doc/source/microversion_testing.rst
@@ -1,13 +1,83 @@
-===================================
-How To Implement Microversion Tests
-===================================
+=================================
+Microversion Testing With Tempest
+=================================
 
-Tempest provides stable interfaces to test API Microversion.
-For Details, see: `API Microversion testing Framework`_
-This document explains how to implement Microversion tests using those
-interfaces.
+Many OpenStack Services provide their APIs with `microversion`_
+support and want to test them in Tempest.
 
-.. _API Microversion testing Framework: http://docs.openstack.org/developer/tempest/library/api_microversion_testing.html
+.. _microversion: http://specs.openstack.org/openstack/api-wg/guidelines/microversion_specification.html
+
+This document covers how to test microversions for each project and
+whether tests should live in Tempest or on project side.
+
+Tempest Scope For Microversion Testing
+""""""""""""""""""""""""""""""""""""""
+APIs microversions for any OpenStack service grow rapidly and
+testing each and every microversion in Tempest is not feasible and
+efficient way.
+Also not every API microversion changes the complete system behavior
+and many of them only change the API or DB layer to accept and return more
+data on API.
+
+Tempest is an integration test suite, but not all API microversion testing fall under this category.
+As a result, Tempest mainly covers integration test cases for microversions, Other testing coverage
+for microversion should be hosted on project side as functional tests or via Tempest plugin as per
+project guidelines.
+
+.. note:: Integration tests are those tests which involve more than one service to
+     verify the expected behavior by single or combination of API requests.
+     If a test is just to verify the API behavior as success and failure cases
+     or verify its expected response object, then it does not fall under integration
+     tests.
+
+Tempest will cover only integration testing of applicable microversions with
+below exceptions:
+
+  #. Test covers a feature which is important for interoperability. This covers tests requirement
+     from Defcore.
+  #. Test needed to fill Schema gaps.
+     Tempest validates API responses with defined JSON schema. API responses can be different on
+     each microversion and the JSON schemas need to be defined separately for the microversion.
+     While implementing new integration tests for a specific microversion, there
+     may be a gap in the JSON schemas (caused by previous microversions) implemented
+     in Tempest.
+     Filling that gap while implementing the new integration test cases is not efficient due to
+     many reasons:
+
+     * Hard to review
+     * Sync between multiple integration tests patches which try to fill the same schema gap at same
+       time
+     * Might delay the microversion change on project side where project team wants Tempest
+       tests to verify the results.
+
+     Tempest will allow to fill the schema gaps at the end of each cycle, or more
+     often if required.
+     Schema gap can be filled with testing those with a minimal set of tests. Those
+     tests might not be integration tests and might be already covered on project
+     side also.
+     This exception is needed because:
+
+     * Allow to create microversion response schema in Tempest at the same time that projects are
+       implementing their API microversions. This will make implementation easier for adding
+       required tests before a new microversion change can be merged in the corresponding project
+       and hence accelerate the development of microversions.
+     * New schema must be verified by at least one test case which exercises such schema.
+
+     For example:
+        If any projects implemented 4 API microversion say- v2.3, v2.4, v2.5, v2.6
+        Assume microversion v2.3, v2.4, v2.6 change the API Response which means Tempest
+        needs to add JSON schema for v2.3, v2.4, v2.6.
+        In that case if only 1 or 2 tests can verify all new schemas then we do not need
+        separate tests for each new schemas. In worst case, we have to add 3 separate tests.
+  #. Test covers service behavior at large scale with involvement of more deep layer like hypervisor
+     etc not just API/DB layer. This type of tests will be added case by case basis and
+     with project team consultation about why it cannot be covered on project side and worth to test
+     in Tempest.
+
+Project Scope For Microversion Testing
+""""""""""""""""""""""""""""""""""""""
+All microversions testing which are not covered under Tempest as per above section, should be
+tested on project side as functional tests or as Tempest plugin as per project decision.
 
 
 Configuration options for Microversion
@@ -36,6 +106,14 @@
 How To Implement Microversion Tests
 """""""""""""""""""""""""""""""""""
 
+Tempest provides stable interfaces to test API Microversion.
+For Details, see: `API Microversion testing Framework`_
+This document explains how to implement Microversion tests using those
+interfaces.
+
+.. _API Microversion testing Framework: http://docs.openstack.org/developer/tempest/library/api_microversion_testing.html
+
+
 Step1: Add skip logic based on configured Microversion range
 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
 
diff --git a/doc/source/write_tests.rst b/doc/source/write_tests.rst
index 63ff61c..8488fb1 100644
--- a/doc/source/write_tests.rst
+++ b/doc/source/write_tests.rst
@@ -188,7 +188,7 @@
 +-------------------+---------------------+
 
 By default cls.os is available since it is allocated in the base tempest test
-class. (located in tempest/test.py) If your TestCase inherits from a different
+class (located in tempest/test.py). If your TestCase inherits from a different
 direct parent class (it'll still inherit from the BaseTestCase, just not
 directly) be sure to check if that class overrides allocated credentials.
 
@@ -199,10 +199,10 @@
 automatically setup when a tenant is created. Since tempest needs isolated
 tenants to function properly it also needs to handle network allocation. By
 default the base test class will allocate a network, subnet, and router
-automatically. (this depends on the configured credential provider, for more
-details see: :ref:`tempest_conf_network_allocation`) However, there are
-situations where you do no need all of these resources allocated. (or your
-TestCase inherits from a class that overrides the default in tempest/test.py)
+automatically (this depends on the configured credential provider, for more
+details see: :ref:`tempest_conf_network_allocation`). However, there are
+situations where you do no need all of these resources allocated (or your
+TestCase inherits from a class that overrides the default in tempest/test.py).
 There is a class level mechanism to override this allocation and specify which
 resources you need. To do this you need to call `cls.set_network_resources()`
 in the `setup_credentials()` method before the `super()`. For example::
@@ -256,8 +256,8 @@
 
 The primary interface with which you interact with both credentials and
 API clients is the client manager object. These objects are created
-automatically by the base test class as part of credential setup. (for more
-details see the previous :ref:`credentials` section) Each manager object is
+automatically by the base test class as part of credential setup (for more
+details see the previous :ref:`credentials` section). Each manager object is
 initialized with a set of credentials and has each client object already setup
 to use that set of credentials for making all the API requests. Each client is
 accessible as a top level attribute on the manager object. So to start making
@@ -281,9 +281,9 @@
 Credentials Objects
 -------------------
 
-In certain cases you need direct access to the credentials. (the most common
+In certain cases you need direct access to the credentials (the most common
 use case would be an API request that takes a user or project id in the request
-body) If you're in a situation where you need to access this you'll need to
+body). If you're in a situation where you need to access this you'll need to
 access the ``credentials`` object which is allocated from the configured
 credential provider in the base test class. This is accessible from the manager
 object via the manager's ``credentials`` attribute. For example::
diff --git a/releasenotes/notes/add-list-volume-transfers-with-detail-to-transfers-client-80169bf78cf4fa66.yaml b/releasenotes/notes/add-list-volume-transfers-with-detail-to-transfers-client-80169bf78cf4fa66.yaml
new file mode 100644
index 0000000..8e85d3a
--- /dev/null
+++ b/releasenotes/notes/add-list-volume-transfers-with-detail-to-transfers-client-80169bf78cf4fa66.yaml
@@ -0,0 +1,5 @@
+---
+features:
+  - |
+    Add list volume transfers with details API to v2 transfers_client library.
+    This feature enables the possibility to list volume transfers with details.
diff --git a/tempest/api/image/admin/__init__.py b/tempest/api/image/admin/__init__.py
deleted file mode 100644
index e69de29..0000000
--- a/tempest/api/image/admin/__init__.py
+++ /dev/null
diff --git a/tempest/api/image/admin/v2/__init__.py b/tempest/api/image/admin/v2/__init__.py
deleted file mode 100644
index e69de29..0000000
--- a/tempest/api/image/admin/v2/__init__.py
+++ /dev/null
diff --git a/tempest/api/image/admin/v2/test_images.py b/tempest/api/image/admin/v2/test_images.py
deleted file mode 100644
index fc5ed79..0000000
--- a/tempest/api/image/admin/v2/test_images.py
+++ /dev/null
@@ -1,58 +0,0 @@
-# Copyright 2015 Red Hat, Inc.
-# All Rights Reserved.
-#
-#    Licensed under the Apache License, Version 2.0 (the "License"); you may
-#    not use this file except in compliance with the License. You may obtain
-#    a copy of the License at
-#
-#         http://www.apache.org/licenses/LICENSE-2.0
-#
-#    Unless required by applicable law or agreed to in writing, software
-#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-#    License for the specific language governing permissions and limitations
-#    under the License.
-
-import six
-import testtools
-
-from tempest.api.image import base
-from tempest import config
-from tempest.lib.common.utils import data_utils
-from tempest.lib import decorators
-from tempest.lib import exceptions as lib_exc
-
-CONF = config.CONF
-
-
-class BasicAdminOperationsImagesTest(base.BaseV2ImageAdminTest):
-    """Here we test admin operations of images"""
-
-    @testtools.skipUnless(CONF.image_feature_enabled.deactivate_image,
-                          'deactivate-image is not available.')
-    @decorators.idempotent_id('951ebe01-969f-4ea9-9898-8a3f1f442ab0')
-    def test_admin_deactivate_reactivate_image(self):
-        # Create image by non-admin tenant
-        image_name = data_utils.rand_name('image')
-        image = self.create_image(name=image_name,
-                                  container_format='bare',
-                                  disk_format='raw',
-                                  visibility='private')
-        # upload an image file
-        content = data_utils.random_bytes()
-        image_file = six.BytesIO(content)
-        self.client.store_image_file(image['id'], image_file)
-        # deactivate image
-        self.admin_client.deactivate_image(image['id'])
-        body = self.client.show_image(image['id'])
-        self.assertEqual("deactivated", body['status'])
-        # non-admin user unable to download deactivated image
-        self.assertRaises(lib_exc.Forbidden, self.client.show_image_file,
-                          image['id'])
-        # reactivate image
-        self.admin_client.reactivate_image(image['id'])
-        body = self.client.show_image(image['id'])
-        self.assertEqual("active", body['status'])
-        # non-admin user able to download image after reactivation by admin
-        body = self.client.show_image_file(image['id'])
-        self.assertEqual(content, body.data)
diff --git a/tempest/api/image/v2/test_images.py b/tempest/api/image/v2/test_images.py
index 8310001..a5d9773 100644
--- a/tempest/api/image/v2/test_images.py
+++ b/tempest/api/image/v2/test_images.py
@@ -18,11 +18,14 @@
 
 import six
 
+import testtools
+
 from oslo_log import log as logging
 from tempest.api.image import base
 from tempest import config
 from tempest.lib.common.utils import data_utils
 from tempest.lib import decorators
+from tempest.lib import exceptions as lib_exc
 
 CONF = config.CONF
 LOG = logging.getLogger(__name__)
@@ -125,6 +128,40 @@
         self.assertEqual(image['id'], body['id'])
         self.assertEqual(new_image_name, body['name'])
 
+    @testtools.skipUnless(CONF.image_feature_enabled.deactivate_image,
+                          'deactivate-image is not available.')
+    @decorators.idempotent_id('951ebe01-969f-4ea9-9898-8a3f1f442ab0')
+    def test_deactivate_reactivate_image(self):
+        # Create image
+        image_name = data_utils.rand_name('image')
+        image = self.create_image(name=image_name,
+                                  container_format='bare',
+                                  disk_format='raw',
+                                  visibility='private')
+
+        # Upload an image file
+        content = data_utils.random_bytes()
+        image_file = six.BytesIO(content)
+        self.client.store_image_file(image['id'], image_file)
+
+        # Deactivate image
+        self.client.deactivate_image(image['id'])
+        body = self.client.show_image(image['id'])
+        self.assertEqual("deactivated", body['status'])
+
+        # User unable to download deactivated image
+        self.assertRaises(lib_exc.Forbidden, self.client.show_image_file,
+                          image['id'])
+
+        # Reactivate image
+        self.client.reactivate_image(image['id'])
+        body = self.client.show_image(image['id'])
+        self.assertEqual("active", body['status'])
+
+        # User able to download image after reactivation
+        body = self.client.show_image_file(image['id'])
+        self.assertEqual(content, body.data)
+
 
 class ListUserImagesTest(base.BaseV2ImageTest):
     """Here we test the listing of image information"""
diff --git a/tempest/api/volume/base.py b/tempest/api/volume/base.py
index a19af5d..767b3c7 100644
--- a/tempest/api/volume/base.py
+++ b/tempest/api/volume/base.py
@@ -65,7 +65,6 @@
     def setup_clients(cls):
         super(BaseVolumeTest, cls).setup_clients()
         cls.servers_client = cls.os.servers_client
-        cls.compute_networks_client = cls.os.compute_networks_client
         cls.compute_images_client = cls.os.compute_images_client
 
         cls.snapshots_client = cls.os.snapshots_v2_client
diff --git a/tempest/api/volume/test_volume_transfers.py b/tempest/api/volume/test_volume_transfers.py
index afcffc2..75f2a73 100644
--- a/tempest/api/volume/test_volume_transfers.py
+++ b/tempest/api/volume/test_volume_transfers.py
@@ -73,19 +73,20 @@
                         volume['id'])
 
         # Create a volume transfer
-        body = self.client.create_volume_transfer(
-            volume_id=volume['id'])['transfer']
-        transfer_id = body['id']
+        transfer_id = self.client.create_volume_transfer(
+            volume_id=volume['id'])['transfer']['id']
         waiters.wait_for_volume_resource_status(
             self.volumes_client, volume['id'], 'awaiting-transfer')
 
-        # List all volume transfers (looking for the one we created)
-        body = self.client.list_volume_transfers()['transfers']
-        for transfer in body:
-            if volume['id'] == transfer['volume_id']:
-                break
-        else:
-            self.fail('Transfer not found for volume %s' % volume['id'])
+        # List all volume transfers with details, check the detail-specific
+        # elements, and look for the created transfer.
+        transfers = self.client.list_volume_transfers(detail=True)['transfers']
+        self.assertNotEmpty(transfers)
+        for transfer in transfers:
+            self.assertIn('created_at', transfer)
+        volume_list = [transfer['volume_id'] for transfer in transfers]
+        self.assertIn(volume['id'], volume_list,
+                      'Transfer not found for volume %s' % volume['id'])
 
         # Delete a volume transfer
         self.client.delete_volume_transfer(transfer_id)
diff --git a/tempest/lib/services/volume/v2/transfers_client.py b/tempest/lib/services/volume/v2/transfers_client.py
index 853948e..2dfbe7b 100644
--- a/tempest/lib/services/volume/v2/transfers_client.py
+++ b/tempest/lib/services/volume/v2/transfers_client.py
@@ -44,14 +44,17 @@
         self.expected_success(200, resp.status)
         return rest_client.ResponseBody(resp, body)
 
-    def list_volume_transfers(self, **params):
+    def list_volume_transfers(self, detail=False, **params):
         """List all the volume transfers created.
 
         For a full list of available parameters, please refer to the official
         API reference:
         https://developer.openstack.org/api-ref/block-storage/v2/#list-volume-transfers
+        https://developer.openstack.org/api-ref/block-storage/v2/#list-volume-transfers-with-details
         """
         url = 'os-volume-transfer'
+        if detail:
+            url += '/detail'
         if params:
             url += '?%s' % urllib.urlencode(params)
         resp, body = self.get(url)
diff --git a/tempest/tests/lib/services/volume/v2/test_transfers_client.py b/tempest/tests/lib/services/volume/v2/test_transfers_client.py
new file mode 100644
index 0000000..0c59bf2
--- /dev/null
+++ b/tempest/tests/lib/services/volume/v2/test_transfers_client.py
@@ -0,0 +1,61 @@
+# Copyright 2017 FiberHome Telecommunication Technologies CO.,LTD
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from tempest.lib.services.volume.v2 import transfers_client
+from tempest.tests.lib import fake_auth_provider
+from tempest.tests.lib.services import base
+
+
+class TestTransfersClient(base.BaseServiceTest):
+
+    FAKE_LIST_VOLUME_TRANSFERS_WITH_DETAIL = {
+        "transfers": [{
+            "created_at": "2017-04-18T09:10:03.000000",
+            "volume_id": "47bf04ef-1ea5-4c5f-a375-430a086d6747",
+            "id": "0e89cdd1-6249-421b-96d8-25fac0623d42",
+            "links": [
+                {
+                    "href": "fake-url-1",
+                    "rel": "self"
+                },
+                {
+                    "href": "fake-url-2",
+                    "rel": "bookmark"
+                }
+            ],
+            "name": "fake-volume-transfer"
+        }]
+    }
+
+    def setUp(self):
+        super(TestTransfersClient, self).setUp()
+        fake_auth = fake_auth_provider.FakeAuthProvider()
+        self.client = transfers_client.TransfersClient(fake_auth,
+                                                       'volume',
+                                                       'regionOne')
+
+    def _test_list_volume_transfers_with_detail(self, bytes_body=False):
+        self.check_service_client_function(
+            self.client.list_volume_transfers,
+            'tempest.lib.common.rest_client.RestClient.get',
+            self.FAKE_LIST_VOLUME_TRANSFERS_WITH_DETAIL,
+            bytes_body,
+            detail=True)
+
+    def test_list_volume_transfers_with_detail_with_str_body(self):
+        self._test_list_volume_transfers_with_detail()
+
+    def test_list_volume_transfers_with_detail_with_bytes_body(self):
+        self._test_list_volume_transfers_with_detail(bytes_body=True)