Merge "trivial: Fix HACKING.rst nits causing rendering issues"
diff --git a/HACKING.rst b/HACKING.rst
index 3171a10..28a977d 100644
--- a/HACKING.rst
+++ b/HACKING.rst
@@ -1,5 +1,5 @@
-Patrole Style Commandments
-==========================
+Patrole Coding Guide
+====================
- Step 1: Read the OpenStack Style Commandments: `<https://docs.openstack.org/hacking/latest/>`__
- Step 2: Review Tempest's Style Commandments: `<https://docs.openstack.org/tempest/latest/HACKING.html>`__
diff --git a/README.rst b/README.rst
index fb8976f..e9c03c8 100644
--- a/README.rst
+++ b/README.rst
@@ -8,10 +8,15 @@
Patrole - RBAC Integration Tempest Plugin
=========================================
-Patrole is a security validation tool for verifying that Role-Based Access
-Control is correctly configured and enforced in a system. It runs
-`Tempest`_-based API tests using specified RBAC roles, thus allowing
-deployments to verify that only intended roles have access to those APIs.
+Patrole is a set of integration tests to be run against a live OpenStack
+cluster. It has a battery of tests dedicated to validating the correctness and
+integrity of the cloud's RBAC implementation.
+
+More importantly, Patrole is a security validation tool for verifying that
+Role-Based Access Control is correctly configured and enforced in an OpenStack
+cloud. It runs `Tempest`_-based API tests using specified RBAC roles, thus
+allowing deployments to verify that only intended roles have access to those
+APIs.
Patrole currently offers testing for the following OpenStack services: Nova,
Neutron, Glance, Cinder and Keystone.
@@ -20,19 +25,25 @@
toward policy in code, Patrole will align its testing with the appropriate
documentation.
-.. _Tempest: https://docs.openstack.org/tempest/latest/
+* Free software: Apache license
+* Documentation: https://docs.openstack.org/patrole/latest
+* Source: https://git.openstack.org/cgit/openstack/patrole
+* Bugs: https://bugs.launchpad.net/patrole
+* Release notes: https://docs.openstack.org/releasenotes/patrole/
Design Principles
-----------------
-Patrole borrows some design principles from Tempest, but not all, as its
-testing scope is confined to policies.
+As a `Tempest plugin`_, Patrole borrows some `design principles`_ from Tempest,
+but not all, as its testing scope is confined to policies.
* *Stability*. Patrole uses OpenStack public interfaces. Tests in Patrole
should only touch public OpenStack APIs.
* *Atomicity*. Patrole tests should be atomic: they should test policies in
isolation. Unlike Tempest, a Patrole test strives to only call a single
- endpoint at a time.
+ endpoint at a time. This is because it is important to validate each policy
+ is authorized correctly and the best way to do that is to validate each
+ policy alone, to avoid test contamination.
* *Complete coverage*. Patrole should validate all policy in code defaults. For
testing, Patrole uses the API-to-policy mapping contained in each project's
`policy in code`_ documentation where applicable.
@@ -59,6 +70,8 @@
* *Self-testing*. Patrole should be self-testing.
+.. _Tempest plugin: https://docs.openstack.org/tempest/latest/plugin.html
+.. _design principles: https://docs.openstack.org/tempest/latest/overview.html#design-principles
.. _policy in code: https://specs.openstack.org/openstack/oslo-specs/specs/newton/policy-in-code.html
.. _Nova repository: https://github.com/openstack/nova/tree/master/nova/policies
.. _Keystone repository: https://github.com/openstack/keystone/tree/master/keystone/common/policies
@@ -86,45 +99,135 @@
expected result) and the actual result from test execution are compared to
each other: if both results match, then the test passes; else it fails.
-* Documentation: https://docs.openstack.org/patrole/latest/
-* Bugs: https://bugs.launchpad.net/patrole
+Terminology
+^^^^^^^^^^^
+* Expected Result - The expected result of a given test.
+* Actual Result - The actual result of a given test.
+* Final Result - A match between both expected and actual results. A mismatch
+ in the expected result and the actual result will result in a test failure.
+
+ * Expected: Pass | Actual: Pass - Test Case Success
+ * Expected: Pass | Actual: Fail - Test Case Under-Permission Failure
+ * Expected: Fail | Actual: Pass - Test Case Over-Permission Failure
+ * Expected: Fail | Actual: Fail (Expected exception) - Test Case Success
+ * Expected: Fail | Actual: Fail (Unexpected exception) - Test Case Failure
Quickstart
----------
-Tempest is a prerequisite for running Patrole. If you do not have Tempest
-installed, please reference the official Tempest documentation for guidance.
+To run Patrole, you must first have `Tempest`_ installed and configured
+properly. Please reference Tempest's `Quickstart`_ guide to do so. Follow all
+the steps outlined therein. Afterward, proceed with the steps below.
-Assuming Tempest is installed, the simplest way to configure Patrole is:
+#. You first need to install Patrole. This is done with pip after you check out
+ the Patrole repo::
-1. Open up the ``tempest.conf`` configuration file and include the following
-settings:
+ $ git clone https://git.openstack.org/openstack/patrole
+ $ pip install patrole/
-.. code-block:: ini
+ This can be done within a venv.
- [rbac]
- enable_rbac = True
- rbac_test_role = admin
+ .. note::
-These settings tell Patrole to run RBAC tests using the "admin" role (which
-is the default admin role in OpenStack) to verify the default policy
-definitions used by OpenStack services. Specifying a different role
-for ``rbac_test_role`` will run Patrole tests against that role. For additional
-information about Patrole's configuration settings, please refer to
-:ref:`patrole-configuration` and :ref:`patrole-sampleconf` for a sample
-configuration file.
+ You may also install Patrole from source code by running::
-2. You are now ready to run Patrole. To do so, you can use any testr-based test
-runner::
+ pip install -e patrole/
- $ testr run patrole_tempest_plugin.tests.api
+#. Next you must properly configure Patrole, which is relatively
+ straightforward. For details on configuring Patrole refer to the
+ :ref:`patrole-configuration`.
-or::
+#. Once the configuration is done you're now ready to run Patrole. This can
+ be done using the `tempest_run`_ command. This can be done by running::
- $ ostestr --regex '(?!.*\[.*\bslow\b.*\])(^patrole_tempest_plugin\.tests\.api)'
+ $ tempest run --regex '^patrole_tempest_plugin\.tests\.api'
-It is also possible to run Patrole using tox::
+ There is also the option to use testr directly, or any `testr`_ based test
+ runner, like `ostestr`_. For example, from the workspace dir run::
- tox -eall-plugin -- patrole_tempest_plugin.tests.api
+ $ stestr --regex '(?!.*\[.*\bslow\b.*\])(^patrole_tempest_plugin\.tests\.api))'
+
+ will run the same set of tests as the default gate jobs.
+
+ You can also run Patrole tests using `tox`_. To do so, ``cd`` into the
+ **Tempest** directory and run::
+
+ $ tox -eall-plugin -- patrole_tempest_plugin.tests.api
+
+ .. note::
+
+ It is possible to run Patrole via ``tox -eall`` in order to run Patrole
+ isolated from other plugins. This can be accomplished by including the
+ installation of services that currently use policy in code -- for example,
+ Nova and Keystone. For example::
+
+ $ tox -evenv-tempest -- pip install /opt/stack/patrole /opt/stack/keystone /opt/stack/nova
+ $ tox -eall -- patrole_tempest_plugin.tests.api
+
+#. Log information from tests is captured in ``tempest.log`` under the Tempest
+ repository. Some Patrole debugging information is captured in that log
+ related to expected test results and :ref:`role-overriding`.
+
+ More detailed RBAC testing log output is emitted to ``patrole.log`` under
+ the Patrole repository. To configure Patrole's logging, see the
+ :ref:`patrole-configuration` guide.
+
+.. _Tempest: https://github.com/openstack/tempest
+.. _Quickstart: https://docs.openstack.org/tempest/latest/overview.html#quickstart
+.. _tempest_run: https://docs.openstack.org/tempest/latest/run.html
+.. _testr: https://testrepository.readthedocs.org/en/latest/MANUAL.html
+.. _ostestr: https://docs.openstack.org/os-testr/latest/
+.. _tox: https://tox.readthedocs.io/en/latest/
+
+RBAC Tests
+----------
+
+To change the role that the patrole tests are being run as, edit
+``rbac_test_role`` in the ``patrole`` section of tempest.conf: ::
+
+ [patrole]
+ rbac_test_role = member
+ ...
+
+.. note::
+
+ The ``rbac_test_role`` is service-specific. member, for example,
+ is an arbitrary role, but by convention is used to designate the default
+ non-admin role in the system. Most Patrole tests should be run with
+ **admin** and **member** roles. However, other services may use entirely
+ different roles.
+
+For more information about the member role and its nomenclature,
+please see: `<https://ask.openstack.org/en/question/4759/member-vs-_member_/>`__.
+
+Unit Tests
+----------
+
+Patrole also has a set of unit tests which test the Patrole code itself. These
+tests can be run by specifying the test discovery path::
+
+ $ stestr --test-path ./patrole_tempest_plugin/tests/unit run
+
+By setting ``--test-path`` option to ``./patrole_tempest_plugin/tests/unit``
+it specifies that test discovery should only be run on the unit test directory.
+
+Alternatively, there are the py27 and py35 tox jobs which will run the unit
+tests with the corresponding version of Python.
+
+One common activity is to just run a single test; you can do this with tox
+simply by specifying to just run py27 or py35 tests against a single test::
+
+ $ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils.RBACUtilsTest.test_override_role_with_missing_admin_role
+
+Or all tests in the test_rbac_utils.py file::
+
+ $ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils
+
+You may also use regular expressions to run any matching tests::
+
+ $ tox -e py27 -- test_rbac_utils
+
+For more information on these options and details about stestr, please see the
+`stestr documentation <http://stestr.readthedocs.io/en/latest/MANUAL.html>`_.
Release Versioning
------------------
diff --git a/REVIEWING.rst b/REVIEWING.rst
new file mode 100644
index 0000000..7e3c71f
--- /dev/null
+++ b/REVIEWING.rst
@@ -0,0 +1,136 @@
+Reviewing Patrole Code
+======================
+To start read the `OpenStack Common Review Checklist
+<https://docs.openstack.org/infra/manual/developers.html#peer-review>`_
+
+
+Ensuring code is executed
+-------------------------
+Any new test or change to an existing test has to be verified in the gate. This
+means that the first thing to check with any change is that a gate job actually
+runs it. Tests which aren't executed either because of configuration or skips
+should not be accepted.
+
+
+Execution time
+--------------
+Along with checking that the jobs log that a new test is actually executed,
+also pay attention to the execution time of that test. Patrole already runs
+hundreds of tests per job in its check and gate pipelines and it is important
+that the overall runtime of the jobs be constrained as much as possible.
+Consider applying the ``@decorators.attr(type='slow')``
+`test attribute decorator`_ to a test if its runtime is longer than 30 seconds.
+
+.. _test attribute decorator: https://docs.openstack.org/tempest/latest/HACKING.html#test-attributes
+
+
+Unit Tests
+----------
+For any change that adds new functionality to common functionality unit tests
+are required. This is to ensure we don't introduce future regressions and to
+test conditions which we may not hit in the gate runs.
+
+
+API Stability
+-------------
+Tests should only be added for published stable APIs. If a patch contains
+tests for an API which hasn't been marked as stable or for an API which
+doesn't conform to the `API stability guidelines
+<https://wiki.openstack.org/wiki/Governance/Approved/APIStability>`_ then it
+should not be approved.
+
+Similarly, tests should only be added for policies that are covered by
+`policy in code documentation
+<https://specs.openstack.org/openstack/keystone-specs/specs/keystone/pike/policy-in-code.html>`_.
+Any existing tests that test policies not covered by such documentation
+are either:
+
+* part of a service that has not yet migrated to policy in code; or
+* legacy in the sense that they were created prior to policy in code
+
+For the first bullet, the tests should not be considered stable, but should be
+kept around to maintain coverage. These tests are a best-effort attempt at
+offering RBAC test coverage for the service that has not yet migrated to
+policy in code.
+
+For the second bullet, the tests should be updated to conform to policy in
+code documentation, if applicable.
+
+
+Reject Copy and Paste Test Code
+-------------------------------
+When creating new tests that are similar to existing tests it is tempting to
+simply copy the code and make a few modifications. This increases code size and
+the maintenance burden. Such changes should not be approved if it is easy to
+abstract the duplicated code into a function or method.
+
+
+Tests overlap
+-------------
+When a new test is being proposed, question whether this feature is not already
+tested with Patrole. Patrole has more than 600 tests, spread amongst many
+directories, so it's easy to introduce test duplication.
+
+Test Duplication
+^^^^^^^^^^^^^^^^
+
+Test duplication means:
+
+* testing an API endpoint in more than one test
+* testing the same policy in more than one test
+
+For the first bullet, try to avoid calling the same API inside the
+``self.rbac_utils.override_role`` call.
+
+.. note::
+
+ If the same API is tested against different policies, consider combining
+ the different tests into only 1 test, that tests the API against all
+ the policies it enforces.
+
+For the second bullet, try to avoid testing the same policies across multiple
+tests.
+
+.. note::
+
+ This is not always possible since policy granularity doesn't exist for all
+ APIs. In cases where policy granularity doesn't exist, make sure that the
+ policy overlap only exists for the non-granular APIs that enforce the same
+ policy.
+
+
+Being explicit
+--------------
+When tests are being added that depend on a configurable feature or extension,
+polling the API to discover that it is enabled should not be done. This will
+just result in bugs being masked because the test can be skipped automatically.
+Instead the config file should be used to determine whether a test should be
+skipped or not. Do not approve changes that depend on an API call to determine
+whether to skip or not.
+
+
+Release Notes
+-------------
+Release notes are how we indicate to users and other consumers of Patrole what
+has changed in a given release. There are certain types of changes that
+require release notes and we should not approve them without including a release
+note. These include but aren't limited to, any addition, deprecation or removal
+from the framework code, any change to configuration options (including
+deprecation), major feature additions, and anything backwards incompatible or
+would require a user to take note or do something extra.
+
+
+Deprecated Code
+---------------
+Sometimes we have some bugs in deprecated code. Basically, we leave it. Because
+we don't need to maintain it. However, if the bug is critical, we might need to
+fix it. When it will happen, we will deal with it on a case-by-case basis.
+
+
+When to approve
+---------------
+* Every patch needs two +2's before being approved.
+* It's OK to hold off on an approval until a subject matter expert reviews it.
+* If a patch has already been approved but requires a trivial rebase to merge,
+ you do not have to wait for a second +2, since the patch has already had
+ two +2's.
diff --git a/devstack/README.rst b/devstack/README.rst
new file mode 100644
index 0000000..053afd4
--- /dev/null
+++ b/devstack/README.rst
@@ -0,0 +1,25 @@
+====================
+Enabling in Devstack
+====================
+
+.. warning::
+
+ The ``stack.sh`` script must be run in a disposable VM that is not
+ being created automatically. See the `README file`_ in the DevStack
+ repository for more information.
+
+1. Download DevStack::
+
+ git clone https://git.openstack.org/openstack-dev/devstack.git
+ cd devstack
+
+2. Patrole can be installed like any other DevStack plugin by including the
+ ``enable_plugin`` directive inside local.conf::
+
+ > cat local.conf
+ [[local|localrc]]
+ enable_plugin patrole https://git.openstack.org/openstack/patrole
+
+3. Run ``stack.sh`` found in the DevStack repo.
+
+.. _README file: https://github.com/openstack-dev/devstack/blob/master/README.rst
diff --git a/doc/source/REVIEWING.rst b/doc/source/REVIEWING.rst
new file mode 100644
index 0000000..e8e3c1a
--- /dev/null
+++ b/doc/source/REVIEWING.rst
@@ -0,0 +1,5 @@
+======================
+Reviewing Patrole Code
+======================
+
+.. include:: ../../REVIEWING.rst
diff --git a/doc/source/field_guide/rbac.rst b/doc/source/field_guide/rbac.rst
index d2f4af0..a383099 100644
--- a/doc/source/field_guide/rbac.rst
+++ b/doc/source/field_guide/rbac.rst
@@ -62,7 +62,7 @@
* Tempest tests aren't factored the right way: They're not granular enough.
They call too many APIs and too many policies are enforced by each test.
* Tempest tests assume default policy rules: Tempest uses ``os_admin``
- credentials for admin APIs and ``os_primary`` for non-admin APIs.
+ `credentials`_ for admin APIs and ``os_primary`` for non-admin APIs.
This breaks for custom policy overrides.
* Tempest doesn't have tests that enforce all the policy actions, regardless.
Some RBAC tests require that tests be written a very precise way for the
@@ -71,7 +71,9 @@
Why are these tests not in Tempest?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Patrole should be a separate project that specializes in RBAC tests.
+Patrole should be a separate project that specializes in RBAC tests. This
+was agreed upon during `discussion`_ that led to the approval of the RBAC
+testing framework `spec`_, which was the genesis for Patrole.
Philosophically speaking:
@@ -90,6 +92,9 @@
in Tempest, then adding more Zuul checks/gates for Patrole would only make it
harder to get changes merged in Tempest.
+.. _credentials: https://docs.openstack.org/tempest/latest/write_tests.html#allocating-credentials
+.. _discussion: https://review.openstack.org/#/c/382672/
+.. _spec: https://specs.openstack.org/openstack/qa-specs/specs/tempest/rbac-policy-testing.html
.. _API and scenario testing: https://docs.openstack.org/tempest/latest/overview.html#tempest-the-openstack-integration-test-suite
.. _OpenStack project structure reform: https://governance.openstack.org/tc/resolutions/20141202-project-structure-reform-spec.html#impact-for-horizontal-teams
.. _Tempest external plugin interface: https://specs.openstack.org/openstack/qa-specs/specs/tempest/implemented/tempest-external-plugin-interface.html
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 255fd9a..560857e 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -45,6 +45,7 @@
:maxdepth: 1
HACKING
+ REVIEWING
Framework
---------
diff --git a/doc/source/installation.rst b/doc/source/installation.rst
index b9cc924..827239f 100644
--- a/doc/source/installation.rst
+++ b/doc/source/installation.rst
@@ -26,10 +26,4 @@
DevStack Installation
=====================
-Patrole can be installed like any other DevStack plugin by including the
-``install_plugin`` directive inside local.conf::
-
- [[local|localrc]]
- ...
-
- enable_plugin patrole git://git.openstack.org/openstack/patrole
+.. include:: ../../devstack/README.rst
diff --git a/doc/source/overview.rst b/doc/source/overview.rst
deleted file mode 100644
index 795359e..0000000
--- a/doc/source/overview.rst
+++ /dev/null
@@ -1,178 +0,0 @@
-========================
-Team and repository tags
-========================
-
-.. image:: https://governance.openstack.org/tc/badges/patrole.svg
- :target: https://governance.openstack.org/tc/reference/tags/index.html
-
-Patrole - The OpenStack RBAC Validation Test Suite
-==================================================
-
-The documentation for Patrole is officially hosted at:
-https://docs.openstack.org/patrole/latest/
-
-This is a set of integration tests to be run against a live OpenStack
-cluster. Patrole has a battery of tests dedicated to validating the correctness
-and security of the cloud's RBAC implementation.
-
-Design Principles
------------------
-
-As a `Tempest plugin`_, Patrole borrows some `design principles`_ from Tempest,
-but not all, as its testing scope is confined to policies.
-
-* Patrole uses OpenStack public interfaces. Tests in Patrole should only touch
- public OpenStack APIs.
-* Patrole tests should be atomic: they should test policies in isolation.
- Unlike Tempest, a Patrole test strives to only call a single endpoint at a
- time. This is because it is important to validate each policy is authorized
- correctly and the best way to do that is to validate the policy alone.
-* Patrole should validate all policy in code defaults. For testing, Patrole
- uses the API-to-policy mapping contained in each project's `policy in code`_
- documentation where applicable.
-
- For example, Nova's policy in code documentation is located in the
- `Nova repository`_ under ``nova/policies``. Likewise, Keystone's policy in
- code documentation is located in the `Keystone repository`_ under
- ``keystone/common/policies``. The other OpenStack services follow the same
- directory layout pattern with respect to policy in code.
-
- .. note::
-
- Realistically this is not always possible because some services have
- not yet moved to policy in code.
-
-* Patrole should attempt to clean up after itself; whenever possible it should
- tear down resources when done.
-
- .. note::
-
- Patrole modifies roles dynamically in the background, which affects
- pre-provisioned credentials. Work is currently underway to clean up
- modifications made to pre-provisioned credentials.
-
-* Patrole should be self-testing.
-
-.. _Tempest plugin: https://docs.openstack.org/tempest/latest/plugin.html
-.. _design principles: https://docs.openstack.org/tempest/latest/overview.html#design-principles
-.. _policy in code: https://specs.openstack.org/openstack/oslo-specs/specs/newton/policy-in-code.html
-.. _Nova repository: https://github.com/openstack/nova/tree/master/nova/policies
-.. _Keystone repository: https://github.com/openstack/keystone/tree/master/keystone/common/policies
-
-Quickstart
-----------
-
-To run Patrole, you must first have `Tempest`_ installed and configured
-properly. Please reference Tempest's `Quickstart`_ guide to do so. Follow all
-the steps outlined therein. Afterward, proceed with the steps below.
-
-#. You first need to install Patrole. This is done with pip after you check out
- the Patrole repo::
-
- $ git clone https://git.openstack.org/openstack/patrole
- $ pip install patrole/
-
- This can be done within a venv.
-
- .. note::
-
- You may also install Patrole from source code by running::
-
- pip install -e patrole/
-
-#. Next you must properly configure Patrole, which is relatively
- straightforward. For details on configuring Patrole refer to the
- :ref:`patrole-configuration`.
-
-#. Once the configuration is done you're now ready to run Patrole. This can
- be done using the `tempest_run`_ command. This can be done by running::
-
- $ tempest run --regex '^patrole_tempest_plugin\.tests\.api'
-
- There is also the option to use testr directly, or any `testr`_ based test
- runner, like `ostestr`_. For example, from the workspace dir run::
-
- $ stestr --regex '(?!.*\[.*\bslow\b.*\])(^patrole_tempest_plugin\.tests\.api))'
-
- will run the same set of tests as the default gate jobs.
-
- You can also run Patrole tests using `tox`_. To do so, ``cd`` into the
- **Tempest** directory and run::
-
- $ tox -eall-plugin -- patrole_tempest_plugin.tests.api
-
- .. note::
-
- It is possible to run Patrole via ``tox -eall`` in order to run Patrole
- isolated from other plugins. This can be accomplished by including the
- installation of services that currently use policy in code -- for example,
- Nova and Keystone. For example::
-
- $ tox -evenv-tempest -- pip install /opt/stack/patrole /opt/stack/keystone /opt/stack/nova
- $ tox -eall -- patrole_tempest_plugin.tests.api
-
-#. Log information from tests is captured in ``tempest.log`` under the Tempest
- repository. Some Patrole debugging information is captured in that log
- related to expected test results and :ref:`role-overriding`.
-
- More detailed RBAC testing log output is emitted to ``patrole.log`` under
- the Patrole repository. To configure Patrole's logging, see the
- :ref:`patrole-configuration` guide.
-
-.. _Tempest: https://github.com/openstack/tempest
-.. _Quickstart: https://docs.openstack.org/tempest/latest/overview.html#quickstart
-.. _tempest_run: https://docs.openstack.org/tempest/latest/run.html
-.. _testr: https://testrepository.readthedocs.org/en/latest/MANUAL.html
-.. _ostestr: https://docs.openstack.org/os-testr/latest/
-.. _tox: https://tox.readthedocs.io/en/latest/
-
-RBAC Tests
-----------
-
-To change the role that the patrole tests are being run as, edit
-``rbac_test_role`` in the ``patrole`` section of tempest.conf: ::
-
- [patrole]
- rbac_test_role = member
- ...
-
-.. note::
-
- The ``rbac_test_role`` is service-specific. member, for example,
- is an arbitrary role, but by convention is used to designate the default
- non-admin role in the system. Most Patrole tests should be run with
- **admin** and **member** roles. However, other services may use entirely
- different roles.
-
-For more information about the member role and its nomenclature,
-please see: `<https://ask.openstack.org/en/question/4759/member-vs-_member_/>`__.
-
-Unit Tests
-----------
-
-Patrole also has a set of unit tests which test the Patrole code itself. These
-tests can be run by specifying the test discovery path::
-
- $ stestr --test-path ./patrole_tempest_plugin/tests/unit run
-
-By setting ``--test-path`` option to ``./patrole_tempest_plugin/tests/unit``
-it specifies that test discovery should only be run on the unit test directory.
-
-Alternatively, there are the py27 and py35 tox jobs which will run the unit
-tests with the corresponding version of Python.
-
-One common activity is to just run a single test; you can do this with tox
-simply by specifying to just run py27 or py35 tests against a single test::
-
- $ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils.RBACUtilsTest.test_override_role_with_missing_admin_role
-
-Or all tests in the test_rbac_utils.py file::
-
- $ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils
-
-You may also use regular expressions to run any matching tests::
-
- $ tox -e py27 -- test_rbac_utils
-
-For more information on these options and details about stestr, please see the
-`stestr documentation <http://stestr.readthedocs.io/en/latest/MANUAL.html>`_.
diff --git a/doc/source/overview.rst b/doc/source/overview.rst
new file mode 120000
index 0000000..c768ff7
--- /dev/null
+++ b/doc/source/overview.rst
@@ -0,0 +1 @@
+../../README.rst
\ No newline at end of file
diff --git a/lower-constraints.txt b/lower-constraints.txt
index 9422676..2b77dff 100644
--- a/lower-constraints.txt
+++ b/lower-constraints.txt
@@ -16,7 +16,7 @@
extras==1.0.0
fasteners==0.14.1
fixtures==3.0.0
-flake8==2.5.5
+flake8==2.6.2
future==0.16.0
hacking==1.0.0
idna==2.6
diff --git a/patrole_tempest_plugin/hacking/checks.py b/patrole_tempest_plugin/hacking/checks.py
index eb73ef1..d106da8 100644
--- a/patrole_tempest_plugin/hacking/checks.py
+++ b/patrole_tempest_plugin/hacking/checks.py
@@ -17,7 +17,7 @@
import os
import re
-import pep8
+import pycodestyle
PYTHON_CLIENTS = ['cinder', 'glance', 'keystone', 'nova', 'swift', 'neutron',
@@ -59,7 +59,7 @@
T105: Tests cannot use setUpClass/tearDownClass
"""
- if pep8.noqa(physical_line):
+ if pycodestyle.noqa(physical_line):
return
if SETUP_TEARDOWN_CLASS_DEFINITION.match(physical_line):
diff --git a/patrole_tempest_plugin/policy_authority.py b/patrole_tempest_plugin/policy_authority.py
index b813f88..3339a5d 100644
--- a/patrole_tempest_plugin/policy_authority.py
+++ b/patrole_tempest_plugin/policy_authority.py
@@ -136,7 +136,7 @@
if not service or service not in cls.available_services:
LOG.debug("%s is NOT a valid service.", service)
- raise rbac_exceptions.RbacInvalidService(
+ raise rbac_exceptions.RbacInvalidServiceException(
"%s is NOT a valid service." % service)
@classmethod
diff --git a/patrole_tempest_plugin/rbac_exceptions.py b/patrole_tempest_plugin/rbac_exceptions.py
index 980672a..809a7ed 100644
--- a/patrole_tempest_plugin/rbac_exceptions.py
+++ b/patrole_tempest_plugin/rbac_exceptions.py
@@ -64,7 +64,10 @@
"instead. Actual exception: %(exception)s")
-class RbacInvalidService(exceptions.TempestException):
+class RbacInvalidServiceException(exceptions.TempestException):
+ """Raised when an invalid service is passed to ``rbac_rule_validation``
+ decorator.
+ """
message = "Attempted to test an invalid service"
diff --git a/patrole_tempest_plugin/rbac_rule_validation.py b/patrole_tempest_plugin/rbac_rule_validation.py
index 2f6759d..e6d1e80 100644
--- a/patrole_tempest_plugin/rbac_rule_validation.py
+++ b/patrole_tempest_plugin/rbac_rule_validation.py
@@ -22,7 +22,7 @@
import six
from tempest import config
-from tempest.lib import exceptions
+from tempest.lib import exceptions as lib_exc
from tempest import test
from patrole_tempest_plugin import policy_authority
@@ -123,7 +123,7 @@
"os_alt.auth_provider.credentials.user_id"
})
- :raises NotFound: If ``service`` is invalid.
+ :raises RbacInvalidServiceException: If ``service`` is invalid.
:raises RbacUnderPermissionException: For item (2) above.
:raises RbacOverPermissionException: For item (3) above.
:raises RbacExpectedWrongException: When a 403 is expected but a 404
@@ -184,12 +184,13 @@
try:
test_func(*args, **kwargs)
- except rbac_exceptions.RbacInvalidService as e:
- msg = ("%s is not a valid service." % service)
- test_status = ('Error, %s' % (msg))
- LOG.error(msg)
- raise exceptions.NotFound(
- "%s RbacInvalidService was: %s" % (msg, e))
+ except rbac_exceptions.RbacInvalidServiceException:
+ with excutils.save_and_reraise_exception():
+ msg = ("%s is not a valid service." % service)
+ # FIXME(felipemonteiro): This test_status is logged too
+ # late. Need a function to log it before re-raising.
+ test_status = ('Error, %s' % (msg))
+ LOG.error(msg)
except (expected_exception,
rbac_exceptions.RbacConflictingPolicies,
rbac_exceptions.RbacMalformedResponse) as e:
@@ -382,14 +383,13 @@
raise rbac_exceptions.RbacInvalidErrorCode(msg)
if expected_error_code == 403:
- expected_exception = exceptions.Forbidden
+ expected_exception = lib_exc.Forbidden
elif expected_error_code == 404:
- expected_exception = exceptions.NotFound
+ expected_exception = lib_exc.NotFound
irregular_msg = ("NotFound exception was caught for test %s. Expected "
"policies which may have caused the error: %s. The "
"service %s throws a 404 instead of a 403, which is "
"irregular.")
-
return expected_exception, irregular_msg
@@ -431,7 +431,7 @@
def _check_for_expected_mismatch_exception(expected_exception,
actual_exception):
- permission_exceptions = (exceptions.Forbidden, exceptions.NotFound)
+ permission_exceptions = (lib_exc.Forbidden, lib_exc.NotFound)
if isinstance(actual_exception, permission_exceptions):
if not isinstance(actual_exception, expected_exception.__class__):
return True
diff --git a/patrole_tempest_plugin/tests/api/compute/test_floating_ips_bulk_rbac.py b/patrole_tempest_plugin/tests/api/compute/test_floating_ips_bulk_rbac.py
index b83cc46..3ccef73 100644
--- a/patrole_tempest_plugin/tests/api/compute/test_floating_ips_bulk_rbac.py
+++ b/patrole_tempest_plugin/tests/api/compute/test_floating_ips_bulk_rbac.py
@@ -27,12 +27,15 @@
CONF = config.CONF
+# TODO(gmann): Remove this test class once the nova queens branch goes
+# into extended maintenance mode.
class FloatingIpsBulkRbacTest(rbac_base.BaseV2ComputeRbacTest):
# Tests will fail with a 404 starting from microversion 2.36:
# See the following link for details:
# https://developer.openstack.org/api-ref/compute/#floating-ips-bulk-os-floating-ips-bulk-deprecated
max_microversion = '2.35'
+ depends_on_nova_network = True
@classmethod
def skip_checks(cls):
diff --git a/patrole_tempest_plugin/tests/unit/test_policy_authority.py b/patrole_tempest_plugin/tests/unit/test_policy_authority.py
index 3e2cc4c..d396a29 100644
--- a/patrole_tempest_plugin/tests/unit/test_policy_authority.py
+++ b/patrole_tempest_plugin/tests/unit/test_policy_authority.py
@@ -238,7 +238,7 @@
test_user_id = mock.sentinel.user_id
service = 'invalid_service'
- self.assertRaises(rbac_exceptions.RbacInvalidService,
+ self.assertRaises(rbac_exceptions.RbacInvalidServiceException,
policy_authority.PolicyAuthority,
test_tenant_id,
test_user_id,
@@ -253,7 +253,7 @@
test_user_id = mock.sentinel.user_id
service = None
- self.assertRaises(rbac_exceptions.RbacInvalidService,
+ self.assertRaises(rbac_exceptions.RbacInvalidServiceException,
policy_authority.PolicyAuthority,
test_tenant_id,
test_user_id,
@@ -528,8 +528,9 @@
policy_parser = None
expected_exception = 'invalid_service is NOT a valid service'
- with self.assertRaisesRegex(rbac_exceptions.RbacInvalidService,
- expected_exception):
+ with self.assertRaisesRegex(
+ rbac_exceptions.RbacInvalidServiceException,
+ expected_exception):
policy_authority.PolicyAuthority(
test_tenant_id, test_user_id, "INVALID_SERVICE")
else:
diff --git a/patrole_tempest_plugin/tests/unit/test_rbac_rule_validation.py b/patrole_tempest_plugin/tests/unit/test_rbac_rule_validation.py
index 2e275dc..1bf5510 100644
--- a/patrole_tempest_plugin/tests/unit/test_rbac_rule_validation.py
+++ b/patrole_tempest_plugin/tests/unit/test_rbac_rule_validation.py
@@ -430,6 +430,22 @@
"Allowed")
+class RBACRuleValidationNegativeTest(BaseRBACRuleValidationTest):
+
+ @mock.patch.object(rbac_rv, 'policy_authority', autospec=True)
+ def test_rule_validation_invalid_service_raises_exc(self, mock_authority):
+ """Test that invalid service raises the appropriate exception."""
+ mock_authority.PolicyAuthority.return_value.allowed.side_effect = (
+ rbac_exceptions.RbacInvalidServiceException)
+
+ @rbac_rv.action(mock.sentinel.service, mock.sentinel.action)
+ def test_policy(*args):
+ pass
+
+ self.assertRaises(rbac_exceptions.RbacInvalidServiceException,
+ test_policy, self.mock_test_args)
+
+
class RBACRuleValidationTestMultiPolicy(BaseRBACRuleValidationTest):
"""Test suite for validating multi-policy support for the
``rbac_rule_validation`` decorator.
diff --git a/test-requirements.txt b/test-requirements.txt
index 35e4e57..9085c07 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -1,7 +1,7 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
-hacking>=1.0.0,<1.1.0 # Apache-2.0
+hacking>=1.1.0,<1.2.0 # Apache-2.0
fixtures>=3.0.0 # Apache-2.0/BSD
mock>=2.0.0 # BSD
coverage!=4.4,>=4.0 # Apache-2.0