Fix doc for usage of python clients in scenario tests
Scenario tests has been migrated from official python clients to
Tempest clients.
Documents for the same needs to be fixed.
This patch fix the README & HACKING file for above changes.
This patch also extends hacking rule of not import python clients
for scenario tests too.
Change-Id: Ieb19a2c0b09f00fb3d4f6c7c73541275a4cf24ae
diff --git a/HACKING.rst b/HACKING.rst
index e57b670..e920634 100644
--- a/HACKING.rst
+++ b/HACKING.rst
@@ -8,7 +8,8 @@
Tempest Specific Commandments
------------------------------
-- [T102] Cannot import OpenStack python clients in tempest/api tests
+- [T102] Cannot import OpenStack python clients in tempest/api &
+ tempest/scenario tests
- [T104] Scenario tests require a services decorator
- [T105] Unit tests cannot use setUpClass
- [T106] vim configuration should not be kept in source files.
diff --git a/tempest/README.rst b/tempest/README.rst
index fb25151..d28c3f9 100644
--- a/tempest/README.rst
+++ b/tempest/README.rst
@@ -55,7 +55,8 @@
functionality. They are typically a series of steps where complicated
state requiring multiple services is set up exercised, and torn down.
-Scenario tests can and should use the OpenStack python clients.
+Scenario tests should not use the existing python clients for OpenStack,
+but should instead use the tempest implementations of clients.
:ref:`stress_field_guide`
diff --git a/tempest/hacking/checks.py b/tempest/hacking/checks.py
index 6014cff..29898a9 100644
--- a/tempest/hacking/checks.py
+++ b/tempest/hacking/checks.py
@@ -30,18 +30,18 @@
mutable_default_args = re.compile(r"^\s*def .+\((.+=\{\}|.+=\[\])")
-def import_no_clients_in_api(physical_line, filename):
- """Check for client imports from tempest/api tests
+def import_no_clients_in_api_and_scenario_tests(physical_line, filename):
+ """Check for client imports from tempest/api & tempest/scenario tests
T102: Cannot import OpenStack python clients
"""
- if "tempest/api" in filename:
+ if "tempest/api" in filename or "tempest/scenario" in filename:
res = PYTHON_CLIENT_RE.match(physical_line)
if res:
return (physical_line.find(res.group(1)),
("T102: python clients import not allowed"
- " in tempest/api/* tests"))
+ " in tempest/api/* or tempest/scenario/* tests"))
def scenario_tests_need_service_tags(physical_line, filename,
@@ -117,7 +117,7 @@
def factory(register):
- register(import_no_clients_in_api)
+ register(import_no_clients_in_api_and_scenario_tests)
register(scenario_tests_need_service_tags)
register(no_setup_teardown_class_for_tests)
register(no_vi_headers)
diff --git a/tempest/scenario/README.rst b/tempest/scenario/README.rst
index 5a287d6..38e0de9 100644
--- a/tempest/scenario/README.rst
+++ b/tempest/scenario/README.rst
@@ -29,9 +29,9 @@
Scope of these tests
--------------------
-Scenario tests should use the official python client libraries for
-OpenStack, as they provide a more realistic approach in how people
-will interact with the services.
+Scenario tests should always use the Tempest implementation of the
+OpenStack API, as we want to ensure that bugs aren't hidden by the
+official clients.
Tests should be tagged with which services they exercise, as
determined by which client libraries are used directly by the test.
diff --git a/tempest/tests/test_hacking.py b/tempest/tests/test_hacking.py
index 6857461..fd01887 100644
--- a/tempest/tests/test_hacking.py
+++ b/tempest/tests/test_hacking.py
@@ -69,13 +69,18 @@
self.assertFalse(checks.no_setup_teardown_class_for_tests(
" def tearDownClass(cls):", './tempest/test.py'))
- def test_import_no_clients_in_api(self):
+ def test_import_no_clients_in_api_and_scenario_tests(self):
for client in checks.PYTHON_CLIENTS:
string = "import " + client + "client"
- self.assertTrue(checks.import_no_clients_in_api(
- string, './tempest/api/fake_test.py'))
- self.assertFalse(checks.import_no_clients_in_api(
- string, './tempest/scenario/fake_test.py'))
+ self.assertTrue(
+ checks.import_no_clients_in_api_and_scenario_tests(
+ string, './tempest/api/fake_test.py'))
+ self.assertTrue(
+ checks.import_no_clients_in_api_and_scenario_tests(
+ string, './tempest/scenario/fake_test.py'))
+ self.assertFalse(
+ checks.import_no_clients_in_api_and_scenario_tests(
+ string, './tempest/test.py'))
def test_scenario_tests_need_service_tags(self):
self.assertFalse(checks.scenario_tests_need_service_tags(