Merge "Adds create_network tests for missing policy actions"
diff --git a/HACKING.rst b/HACKING.rst
index f020063..28a977d 100644
--- a/HACKING.rst
+++ b/HACKING.rst
@@ -1,5 +1,5 @@
-Patrole Style Commandments
-==========================
+Patrole Coding Guide
+====================
- Step 1: Read the OpenStack Style Commandments: `<https://docs.openstack.org/hacking/latest/>`__
- Step 2: Review Tempest's Style Commandments: `<https://docs.openstack.org/tempest/latest/HACKING.html>`__
@@ -17,27 +17,28 @@
The original Tempest Commandments do not include Patrole-specific paths.
Patrole-specific paths replace the Tempest-specific paths within Patrole's
hacking checks.
-..
-- [T102] Cannot import OpenStack python clients in patrole_tempest_plugin/tests/api
+- [T102] Cannot import OpenStack python clients in
+ ``patrole_tempest_plugin/tests/api``
- [T105] Tests cannot use setUpClass/tearDownClass
- [T106] vim configuration should not be kept in source files.
- [T107] Check that a service tag isn't in the module path
- [T108] Check no hyphen at the end of rand_name() argument
- [T109] Cannot use testtools.skip decorator; instead use
- decorators.skip_because from tempest.lib
-- [T113] Check that tests use data_utils.rand_uuid() instead of uuid.uuid4()
+ ``decorators.skip_because`` from ``tempest.lib``
+- [T113] Check that tests use ``data_utils.rand_uuid()`` instead of
+ ``uuid.uuid4()``
- [N322] Method's default argument shouldn't be mutable
The following are Patrole's specific Commandments:
- [P100] The ``rbac_rule_validation.action`` decorator must be applied to
- an RBAC test
+ an RBAC test
- [P101] RBAC test filenames must end with "_rbac.py"; for example,
- test_servers_rbac.py, not test_servers.py
+ test_servers_rbac.py, not test_servers.py
- [P102] RBAC test class names must end in 'RbacTest'
- [P103] ``self.client`` must not be used as a client alias; this allows for
- code that is more maintainable and easier to read
+ code that is more maintainable and easier to read
Role Overriding
---------------
diff --git a/README.rst b/README.rst
index fb8976f..2028536 100644
--- a/README.rst
+++ b/README.rst
@@ -8,10 +8,15 @@
Patrole - RBAC Integration Tempest Plugin
=========================================
-Patrole is a security validation tool for verifying that Role-Based Access
-Control is correctly configured and enforced in a system. It runs
-`Tempest`_-based API tests using specified RBAC roles, thus allowing
-deployments to verify that only intended roles have access to those APIs.
+Patrole is a set of integration tests to be run against a live OpenStack
+cluster. It has a battery of tests dedicated to validating the correctness and
+integrity of the cloud's RBAC implementation.
+
+More importantly, Patrole is a security validation tool for verifying that
+Role-Based Access Control is correctly configured and enforced in an OpenStack
+cloud. It runs `Tempest`_-based API tests using specified RBAC roles, thus
+allowing deployments to verify that only intended roles have access to those
+APIs.
Patrole currently offers testing for the following OpenStack services: Nova,
Neutron, Glance, Cinder and Keystone.
@@ -20,19 +25,27 @@
toward policy in code, Patrole will align its testing with the appropriate
documentation.
-.. _Tempest: https://docs.openstack.org/tempest/latest/
+* Free software: Apache license
+* Documentation: https://docs.openstack.org/patrole/latest
+* Source: https://git.openstack.org/cgit/openstack/patrole
+* Bugs: https://bugs.launchpad.net/patrole
+* Release notes: https://docs.openstack.org/releasenotes/patrole/
+
+.. _design-principles:
Design Principles
-----------------
-Patrole borrows some design principles from Tempest, but not all, as its
-testing scope is confined to policies.
+As a `Tempest plugin`_, Patrole borrows some `design principles`_ from Tempest,
+but not all, as its testing scope is confined to policies.
* *Stability*. Patrole uses OpenStack public interfaces. Tests in Patrole
should only touch public OpenStack APIs.
* *Atomicity*. Patrole tests should be atomic: they should test policies in
isolation. Unlike Tempest, a Patrole test strives to only call a single
- endpoint at a time.
+ endpoint at a time. This is because it is important to validate each policy
+ is authorized correctly and the best way to do that is to validate each
+ policy alone, to avoid test contamination.
* *Complete coverage*. Patrole should validate all policy in code defaults. For
testing, Patrole uses the API-to-policy mapping contained in each project's
`policy in code`_ documentation where applicable.
@@ -48,6 +61,9 @@
Realistically this is not always possible because some services have
not yet moved to policy in code.
+* *Customizable*. Patrole should be able to validate custom policy overrides to
+ ensure that those overrides enhance rather than undermine the cloud's RBAC
+ configuration. In addition, Patrole should be able to validate any role.
* *Self-cleaning*. Patrole should attempt to clean up after itself; whenever
possible we should tear down resources when done.
@@ -59,6 +75,8 @@
* *Self-testing*. Patrole should be self-testing.
+.. _Tempest plugin: https://docs.openstack.org/tempest/latest/plugin.html
+.. _design principles: https://docs.openstack.org/tempest/latest/overview.html#design-principles
.. _policy in code: https://specs.openstack.org/openstack/oslo-specs/specs/newton/policy-in-code.html
.. _Nova repository: https://github.com/openstack/nova/tree/master/nova/policies
.. _Keystone repository: https://github.com/openstack/keystone/tree/master/keystone/common/policies
@@ -86,45 +104,135 @@
expected result) and the actual result from test execution are compared to
each other: if both results match, then the test passes; else it fails.
-* Documentation: https://docs.openstack.org/patrole/latest/
-* Bugs: https://bugs.launchpad.net/patrole
+Terminology
+^^^^^^^^^^^
+* Expected Result - The expected result of a given test.
+* Actual Result - The actual result of a given test.
+* Final Result - A match between both expected and actual results. A mismatch
+ in the expected result and the actual result will result in a test failure.
+
+ * Expected: Pass | Actual: Pass - Test Case Success
+ * Expected: Pass | Actual: Fail - Test Case Under-Permission Failure
+ * Expected: Fail | Actual: Pass - Test Case Over-Permission Failure
+ * Expected: Fail | Actual: Fail (Expected exception) - Test Case Success
+ * Expected: Fail | Actual: Fail (Unexpected exception) - Test Case Failure
Quickstart
----------
-Tempest is a prerequisite for running Patrole. If you do not have Tempest
-installed, please reference the official Tempest documentation for guidance.
+To run Patrole, you must first have `Tempest`_ installed and configured
+properly. Please reference Tempest's `Quickstart`_ guide to do so. Follow all
+the steps outlined therein. Afterward, proceed with the steps below.
-Assuming Tempest is installed, the simplest way to configure Patrole is:
+#. You first need to install Patrole. This is done with pip after you check out
+ the Patrole repo::
-1. Open up the ``tempest.conf`` configuration file and include the following
-settings:
+ $ git clone https://git.openstack.org/openstack/patrole
+ $ pip install patrole/
-.. code-block:: ini
+ This can be done within a venv.
- [rbac]
- enable_rbac = True
- rbac_test_role = admin
+ .. note::
-These settings tell Patrole to run RBAC tests using the "admin" role (which
-is the default admin role in OpenStack) to verify the default policy
-definitions used by OpenStack services. Specifying a different role
-for ``rbac_test_role`` will run Patrole tests against that role. For additional
-information about Patrole's configuration settings, please refer to
-:ref:`patrole-configuration` and :ref:`patrole-sampleconf` for a sample
-configuration file.
+ You may also install Patrole from source code by running::
-2. You are now ready to run Patrole. To do so, you can use any testr-based test
-runner::
+ pip install -e patrole/
- $ testr run patrole_tempest_plugin.tests.api
+#. Next you must properly configure Patrole, which is relatively
+ straightforward. For details on configuring Patrole refer to the
+ :ref:`patrole-configuration`.
-or::
+#. Once the configuration is done you're now ready to run Patrole. This can
+ be done using the `tempest_run`_ command. This can be done by running::
- $ ostestr --regex '(?!.*\[.*\bslow\b.*\])(^patrole_tempest_plugin\.tests\.api)'
+ $ tempest run --regex '^patrole_tempest_plugin\.tests\.api'
-It is also possible to run Patrole using tox::
+ There is also the option to use testr directly, or any `testr`_ based test
+ runner, like `ostestr`_. For example, from the workspace dir run::
- tox -eall-plugin -- patrole_tempest_plugin.tests.api
+ $ stestr --regex '(?!.*\[.*\bslow\b.*\])(^patrole_tempest_plugin\.tests\.api))'
+
+ will run the same set of tests as the default gate jobs.
+
+ You can also run Patrole tests using `tox`_. To do so, ``cd`` into the
+ **Tempest** directory and run::
+
+ $ tox -eall-plugin -- patrole_tempest_plugin.tests.api
+
+ .. note::
+
+ It is possible to run Patrole via ``tox -eall`` in order to run Patrole
+ isolated from other plugins. This can be accomplished by including the
+ installation of services that currently use policy in code -- for example,
+ Nova and Keystone. For example::
+
+ $ tox -evenv-tempest -- pip install /opt/stack/patrole /opt/stack/keystone /opt/stack/nova
+ $ tox -eall -- patrole_tempest_plugin.tests.api
+
+#. Log information from tests is captured in ``tempest.log`` under the Tempest
+ repository. Some Patrole debugging information is captured in that log
+ related to expected test results and :ref:`role-overriding`.
+
+ More detailed RBAC testing log output is emitted to ``patrole.log`` under
+ the Patrole repository. To configure Patrole's logging, see the
+ :ref:`patrole-configuration` guide.
+
+.. _Tempest: https://github.com/openstack/tempest
+.. _Quickstart: https://docs.openstack.org/tempest/latest/overview.html#quickstart
+.. _tempest_run: https://docs.openstack.org/tempest/latest/run.html
+.. _testr: https://testrepository.readthedocs.org/en/latest/MANUAL.html
+.. _ostestr: https://docs.openstack.org/os-testr/latest/
+.. _tox: https://tox.readthedocs.io/en/latest/
+
+RBAC Tests
+----------
+
+To change the role that the patrole tests are being run as, edit
+``rbac_test_role`` in the ``patrole`` section of tempest.conf: ::
+
+ [patrole]
+ rbac_test_role = member
+ ...
+
+.. note::
+
+ The ``rbac_test_role`` is service-specific. member, for example,
+ is an arbitrary role, but by convention is used to designate the default
+ non-admin role in the system. Most Patrole tests should be run with
+ **admin** and **member** roles. However, other services may use entirely
+ different roles.
+
+For more information about the member role and its nomenclature,
+please see: `<https://ask.openstack.org/en/question/4759/member-vs-_member_/>`__.
+
+Unit Tests
+----------
+
+Patrole also has a set of unit tests which test the Patrole code itself. These
+tests can be run by specifying the test discovery path::
+
+ $ stestr --test-path ./patrole_tempest_plugin/tests/unit run
+
+By setting ``--test-path`` option to ``./patrole_tempest_plugin/tests/unit``
+it specifies that test discovery should only be run on the unit test directory.
+
+Alternatively, there are the py27 and py35 tox jobs which will run the unit
+tests with the corresponding version of Python.
+
+One common activity is to just run a single test; you can do this with tox
+simply by specifying to just run py27 or py35 tests against a single test::
+
+ $ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils.RBACUtilsTest.test_override_role_with_missing_admin_role
+
+Or all tests in the test_rbac_utils.py file::
+
+ $ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils
+
+You may also use regular expressions to run any matching tests::
+
+ $ tox -e py27 -- test_rbac_utils
+
+For more information on these options and details about stestr, please see the
+`stestr documentation <http://stestr.readthedocs.io/en/latest/MANUAL.html>`_.
Release Versioning
------------------
diff --git a/REVIEWING.rst b/REVIEWING.rst
new file mode 100644
index 0000000..7e3c71f
--- /dev/null
+++ b/REVIEWING.rst
@@ -0,0 +1,136 @@
+Reviewing Patrole Code
+======================
+To start read the `OpenStack Common Review Checklist
+<https://docs.openstack.org/infra/manual/developers.html#peer-review>`_
+
+
+Ensuring code is executed
+-------------------------
+Any new test or change to an existing test has to be verified in the gate. This
+means that the first thing to check with any change is that a gate job actually
+runs it. Tests which aren't executed either because of configuration or skips
+should not be accepted.
+
+
+Execution time
+--------------
+Along with checking that the jobs log that a new test is actually executed,
+also pay attention to the execution time of that test. Patrole already runs
+hundreds of tests per job in its check and gate pipelines and it is important
+that the overall runtime of the jobs be constrained as much as possible.
+Consider applying the ``@decorators.attr(type='slow')``
+`test attribute decorator`_ to a test if its runtime is longer than 30 seconds.
+
+.. _test attribute decorator: https://docs.openstack.org/tempest/latest/HACKING.html#test-attributes
+
+
+Unit Tests
+----------
+For any change that adds new functionality to common functionality unit tests
+are required. This is to ensure we don't introduce future regressions and to
+test conditions which we may not hit in the gate runs.
+
+
+API Stability
+-------------
+Tests should only be added for published stable APIs. If a patch contains
+tests for an API which hasn't been marked as stable or for an API which
+doesn't conform to the `API stability guidelines
+<https://wiki.openstack.org/wiki/Governance/Approved/APIStability>`_ then it
+should not be approved.
+
+Similarly, tests should only be added for policies that are covered by
+`policy in code documentation
+<https://specs.openstack.org/openstack/keystone-specs/specs/keystone/pike/policy-in-code.html>`_.
+Any existing tests that test policies not covered by such documentation
+are either:
+
+* part of a service that has not yet migrated to policy in code; or
+* legacy in the sense that they were created prior to policy in code
+
+For the first bullet, the tests should not be considered stable, but should be
+kept around to maintain coverage. These tests are a best-effort attempt at
+offering RBAC test coverage for the service that has not yet migrated to
+policy in code.
+
+For the second bullet, the tests should be updated to conform to policy in
+code documentation, if applicable.
+
+
+Reject Copy and Paste Test Code
+-------------------------------
+When creating new tests that are similar to existing tests it is tempting to
+simply copy the code and make a few modifications. This increases code size and
+the maintenance burden. Such changes should not be approved if it is easy to
+abstract the duplicated code into a function or method.
+
+
+Tests overlap
+-------------
+When a new test is being proposed, question whether this feature is not already
+tested with Patrole. Patrole has more than 600 tests, spread amongst many
+directories, so it's easy to introduce test duplication.
+
+Test Duplication
+^^^^^^^^^^^^^^^^
+
+Test duplication means:
+
+* testing an API endpoint in more than one test
+* testing the same policy in more than one test
+
+For the first bullet, try to avoid calling the same API inside the
+``self.rbac_utils.override_role`` call.
+
+.. note::
+
+ If the same API is tested against different policies, consider combining
+ the different tests into only 1 test, that tests the API against all
+ the policies it enforces.
+
+For the second bullet, try to avoid testing the same policies across multiple
+tests.
+
+.. note::
+
+ This is not always possible since policy granularity doesn't exist for all
+ APIs. In cases where policy granularity doesn't exist, make sure that the
+ policy overlap only exists for the non-granular APIs that enforce the same
+ policy.
+
+
+Being explicit
+--------------
+When tests are being added that depend on a configurable feature or extension,
+polling the API to discover that it is enabled should not be done. This will
+just result in bugs being masked because the test can be skipped automatically.
+Instead the config file should be used to determine whether a test should be
+skipped or not. Do not approve changes that depend on an API call to determine
+whether to skip or not.
+
+
+Release Notes
+-------------
+Release notes are how we indicate to users and other consumers of Patrole what
+has changed in a given release. There are certain types of changes that
+require release notes and we should not approve them without including a release
+note. These include but aren't limited to, any addition, deprecation or removal
+from the framework code, any change to configuration options (including
+deprecation), major feature additions, and anything backwards incompatible or
+would require a user to take note or do something extra.
+
+
+Deprecated Code
+---------------
+Sometimes we have some bugs in deprecated code. Basically, we leave it. Because
+we don't need to maintain it. However, if the bug is critical, we might need to
+fix it. When it will happen, we will deal with it on a case-by-case basis.
+
+
+When to approve
+---------------
+* Every patch needs two +2's before being approved.
+* It's OK to hold off on an approval until a subject matter expert reviews it.
+* If a patch has already been approved but requires a trivial rebase to merge,
+ you do not have to wait for a second +2, since the patch has already had
+ two +2's.
diff --git a/devstack/README.rst b/devstack/README.rst
new file mode 100644
index 0000000..053afd4
--- /dev/null
+++ b/devstack/README.rst
@@ -0,0 +1,25 @@
+====================
+Enabling in Devstack
+====================
+
+.. warning::
+
+ The ``stack.sh`` script must be run in a disposable VM that is not
+ being created automatically. See the `README file`_ in the DevStack
+ repository for more information.
+
+1. Download DevStack::
+
+ git clone https://git.openstack.org/openstack-dev/devstack.git
+ cd devstack
+
+2. Patrole can be installed like any other DevStack plugin by including the
+ ``enable_plugin`` directive inside local.conf::
+
+ > cat local.conf
+ [[local|localrc]]
+ enable_plugin patrole https://git.openstack.org/openstack/patrole
+
+3. Run ``stack.sh`` found in the DevStack repo.
+
+.. _README file: https://github.com/openstack-dev/devstack/blob/master/README.rst
diff --git a/doc/source/REVIEWING.rst b/doc/source/REVIEWING.rst
new file mode 100644
index 0000000..e8e3c1a
--- /dev/null
+++ b/doc/source/REVIEWING.rst
@@ -0,0 +1,5 @@
+======================
+Reviewing Patrole Code
+======================
+
+.. include:: ../../REVIEWING.rst
diff --git a/doc/source/field_guide/rbac.rst b/doc/source/field_guide/rbac.rst
index d2f4af0..a383099 100644
--- a/doc/source/field_guide/rbac.rst
+++ b/doc/source/field_guide/rbac.rst
@@ -62,7 +62,7 @@
* Tempest tests aren't factored the right way: They're not granular enough.
They call too many APIs and too many policies are enforced by each test.
* Tempest tests assume default policy rules: Tempest uses ``os_admin``
- credentials for admin APIs and ``os_primary`` for non-admin APIs.
+ `credentials`_ for admin APIs and ``os_primary`` for non-admin APIs.
This breaks for custom policy overrides.
* Tempest doesn't have tests that enforce all the policy actions, regardless.
Some RBAC tests require that tests be written a very precise way for the
@@ -71,7 +71,9 @@
Why are these tests not in Tempest?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Patrole should be a separate project that specializes in RBAC tests.
+Patrole should be a separate project that specializes in RBAC tests. This
+was agreed upon during `discussion`_ that led to the approval of the RBAC
+testing framework `spec`_, which was the genesis for Patrole.
Philosophically speaking:
@@ -90,6 +92,9 @@
in Tempest, then adding more Zuul checks/gates for Patrole would only make it
harder to get changes merged in Tempest.
+.. _credentials: https://docs.openstack.org/tempest/latest/write_tests.html#allocating-credentials
+.. _discussion: https://review.openstack.org/#/c/382672/
+.. _spec: https://specs.openstack.org/openstack/qa-specs/specs/tempest/rbac-policy-testing.html
.. _API and scenario testing: https://docs.openstack.org/tempest/latest/overview.html#tempest-the-openstack-integration-test-suite
.. _OpenStack project structure reform: https://governance.openstack.org/tc/resolutions/20141202-project-structure-reform-spec.html#impact-for-horizontal-teams
.. _Tempest external plugin interface: https://specs.openstack.org/openstack/qa-specs/specs/tempest/implemented/tempest-external-plugin-interface.html
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 255fd9a..2dbf63b 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -10,6 +10,14 @@
overview
+RBAC Overview
+-------------
+
+.. toctree::
+ :maxdepth: 2
+
+ rbac-overview
+
User's Guide
============
@@ -45,6 +53,7 @@
:maxdepth: 1
HACKING
+ REVIEWING
Framework
---------
diff --git a/doc/source/installation.rst b/doc/source/installation.rst
index b9cc924..827239f 100644
--- a/doc/source/installation.rst
+++ b/doc/source/installation.rst
@@ -26,10 +26,4 @@
DevStack Installation
=====================
-Patrole can be installed like any other DevStack plugin by including the
-``install_plugin`` directive inside local.conf::
-
- [[local|localrc]]
- ...
-
- enable_plugin patrole git://git.openstack.org/openstack/patrole
+.. include:: ../../devstack/README.rst
diff --git a/doc/source/overview.rst b/doc/source/overview.rst
deleted file mode 100644
index 795359e..0000000
--- a/doc/source/overview.rst
+++ /dev/null
@@ -1,178 +0,0 @@
-========================
-Team and repository tags
-========================
-
-.. image:: https://governance.openstack.org/tc/badges/patrole.svg
- :target: https://governance.openstack.org/tc/reference/tags/index.html
-
-Patrole - The OpenStack RBAC Validation Test Suite
-==================================================
-
-The documentation for Patrole is officially hosted at:
-https://docs.openstack.org/patrole/latest/
-
-This is a set of integration tests to be run against a live OpenStack
-cluster. Patrole has a battery of tests dedicated to validating the correctness
-and security of the cloud's RBAC implementation.
-
-Design Principles
------------------
-
-As a `Tempest plugin`_, Patrole borrows some `design principles`_ from Tempest,
-but not all, as its testing scope is confined to policies.
-
-* Patrole uses OpenStack public interfaces. Tests in Patrole should only touch
- public OpenStack APIs.
-* Patrole tests should be atomic: they should test policies in isolation.
- Unlike Tempest, a Patrole test strives to only call a single endpoint at a
- time. This is because it is important to validate each policy is authorized
- correctly and the best way to do that is to validate the policy alone.
-* Patrole should validate all policy in code defaults. For testing, Patrole
- uses the API-to-policy mapping contained in each project's `policy in code`_
- documentation where applicable.
-
- For example, Nova's policy in code documentation is located in the
- `Nova repository`_ under ``nova/policies``. Likewise, Keystone's policy in
- code documentation is located in the `Keystone repository`_ under
- ``keystone/common/policies``. The other OpenStack services follow the same
- directory layout pattern with respect to policy in code.
-
- .. note::
-
- Realistically this is not always possible because some services have
- not yet moved to policy in code.
-
-* Patrole should attempt to clean up after itself; whenever possible it should
- tear down resources when done.
-
- .. note::
-
- Patrole modifies roles dynamically in the background, which affects
- pre-provisioned credentials. Work is currently underway to clean up
- modifications made to pre-provisioned credentials.
-
-* Patrole should be self-testing.
-
-.. _Tempest plugin: https://docs.openstack.org/tempest/latest/plugin.html
-.. _design principles: https://docs.openstack.org/tempest/latest/overview.html#design-principles
-.. _policy in code: https://specs.openstack.org/openstack/oslo-specs/specs/newton/policy-in-code.html
-.. _Nova repository: https://github.com/openstack/nova/tree/master/nova/policies
-.. _Keystone repository: https://github.com/openstack/keystone/tree/master/keystone/common/policies
-
-Quickstart
-----------
-
-To run Patrole, you must first have `Tempest`_ installed and configured
-properly. Please reference Tempest's `Quickstart`_ guide to do so. Follow all
-the steps outlined therein. Afterward, proceed with the steps below.
-
-#. You first need to install Patrole. This is done with pip after you check out
- the Patrole repo::
-
- $ git clone https://git.openstack.org/openstack/patrole
- $ pip install patrole/
-
- This can be done within a venv.
-
- .. note::
-
- You may also install Patrole from source code by running::
-
- pip install -e patrole/
-
-#. Next you must properly configure Patrole, which is relatively
- straightforward. For details on configuring Patrole refer to the
- :ref:`patrole-configuration`.
-
-#. Once the configuration is done you're now ready to run Patrole. This can
- be done using the `tempest_run`_ command. This can be done by running::
-
- $ tempest run --regex '^patrole_tempest_plugin\.tests\.api'
-
- There is also the option to use testr directly, or any `testr`_ based test
- runner, like `ostestr`_. For example, from the workspace dir run::
-
- $ stestr --regex '(?!.*\[.*\bslow\b.*\])(^patrole_tempest_plugin\.tests\.api))'
-
- will run the same set of tests as the default gate jobs.
-
- You can also run Patrole tests using `tox`_. To do so, ``cd`` into the
- **Tempest** directory and run::
-
- $ tox -eall-plugin -- patrole_tempest_plugin.tests.api
-
- .. note::
-
- It is possible to run Patrole via ``tox -eall`` in order to run Patrole
- isolated from other plugins. This can be accomplished by including the
- installation of services that currently use policy in code -- for example,
- Nova and Keystone. For example::
-
- $ tox -evenv-tempest -- pip install /opt/stack/patrole /opt/stack/keystone /opt/stack/nova
- $ tox -eall -- patrole_tempest_plugin.tests.api
-
-#. Log information from tests is captured in ``tempest.log`` under the Tempest
- repository. Some Patrole debugging information is captured in that log
- related to expected test results and :ref:`role-overriding`.
-
- More detailed RBAC testing log output is emitted to ``patrole.log`` under
- the Patrole repository. To configure Patrole's logging, see the
- :ref:`patrole-configuration` guide.
-
-.. _Tempest: https://github.com/openstack/tempest
-.. _Quickstart: https://docs.openstack.org/tempest/latest/overview.html#quickstart
-.. _tempest_run: https://docs.openstack.org/tempest/latest/run.html
-.. _testr: https://testrepository.readthedocs.org/en/latest/MANUAL.html
-.. _ostestr: https://docs.openstack.org/os-testr/latest/
-.. _tox: https://tox.readthedocs.io/en/latest/
-
-RBAC Tests
-----------
-
-To change the role that the patrole tests are being run as, edit
-``rbac_test_role`` in the ``patrole`` section of tempest.conf: ::
-
- [patrole]
- rbac_test_role = member
- ...
-
-.. note::
-
- The ``rbac_test_role`` is service-specific. member, for example,
- is an arbitrary role, but by convention is used to designate the default
- non-admin role in the system. Most Patrole tests should be run with
- **admin** and **member** roles. However, other services may use entirely
- different roles.
-
-For more information about the member role and its nomenclature,
-please see: `<https://ask.openstack.org/en/question/4759/member-vs-_member_/>`__.
-
-Unit Tests
-----------
-
-Patrole also has a set of unit tests which test the Patrole code itself. These
-tests can be run by specifying the test discovery path::
-
- $ stestr --test-path ./patrole_tempest_plugin/tests/unit run
-
-By setting ``--test-path`` option to ``./patrole_tempest_plugin/tests/unit``
-it specifies that test discovery should only be run on the unit test directory.
-
-Alternatively, there are the py27 and py35 tox jobs which will run the unit
-tests with the corresponding version of Python.
-
-One common activity is to just run a single test; you can do this with tox
-simply by specifying to just run py27 or py35 tests against a single test::
-
- $ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils.RBACUtilsTest.test_override_role_with_missing_admin_role
-
-Or all tests in the test_rbac_utils.py file::
-
- $ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils
-
-You may also use regular expressions to run any matching tests::
-
- $ tox -e py27 -- test_rbac_utils
-
-For more information on these options and details about stestr, please see the
-`stestr documentation <http://stestr.readthedocs.io/en/latest/MANUAL.html>`_.
diff --git a/doc/source/overview.rst b/doc/source/overview.rst
new file mode 120000
index 0000000..c768ff7
--- /dev/null
+++ b/doc/source/overview.rst
@@ -0,0 +1 @@
+../../README.rst
\ No newline at end of file
diff --git a/doc/source/rbac-overview.rst b/doc/source/rbac-overview.rst
new file mode 100644
index 0000000..5eefa5c
--- /dev/null
+++ b/doc/source/rbac-overview.rst
@@ -0,0 +1,254 @@
+==================================
+Role-Based Access Control Overview
+==================================
+
+Introduction
+------------
+
+Role-Based Access Control (RBAC) is used by most OpenStack services to control
+user access to resources. Authorization is granted if a user has the necessary
+role to perform an action. Patrole is concerned with validating that each of
+these resources *can* be accessed by authorized users and *cannot* be accessed
+by unauthorized users.
+
+OpenStack services use `oslo.policy`_ as the library for RBAC authorization.
+Patrole relies on the same library for deriving expected test results.
+
+.. _policy-in-code:
+
+Policy in Code
+--------------
+
+Services publish their policy-to-API mapping via policy in code documentation.
+This mapping includes the list of APIs that authorize a policy, for each
+policy declared within a service.
+
+For example, Nova's policy in code documentation is located in the
+`Nova repository`_ under ``nova/policies``. Likewise, Keystone's policy in
+code documentation is located in the `Keystone repository`_ under
+``keystone/common/policies``. The other OpenStack services follow the same
+directory layout pattern with respect to policy in code.
+
+The policy in code `governance goal`_ enumerates many advantages with following
+this RBAC design approach. A so-called library of in-code policies offers the
+following advantages, with respect to facilitating validation:
+
+* includes every policy enforced by an OpenStack service, enabling the
+ possibility of complete Patrole test coverage for that service (otherwise
+ one has to read the source code to discover all the policies)
+* provides the policy-to-API mapping for each policy which can be used
+ to write correct Patrole tests (otherwise reading source code and
+ experimentation are required to derive this mapping)
+* by extension, the policy-to-API mapping facilitates writing multi-policy
+ Patrole tests (otherwise even more experimentation and code reading is
+ required to arrive at all the policies enforced by an API)
+* policy in code documentation includes additional information, like
+ descriptions and (in the case of some services, like Keystone)
+ `scope types`_, which help with understanding how to correctly write
+ Patrole tests
+* by extension, such information helps to determine whether a Patrole test
+ should assume :term:`hard authorization` or :term:`soft authorization`
+
+Policy in Code (Default) Validation
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+By default, Patrole validates default OpenStack policies. This is so that
+the out-of-the-box defaults are sanity-checked, to ensure that OpenStack
+services are secure, from an RBAC perspective, for each release.
+
+Patrole strives to validate RBAC by using the policy in code documentation,
+wherever possible.
+
+.. _custom-policies:
+
+Custom Policies
+---------------
+
+Operators can override policy in code defaults using `policy.yaml`_. While
+this allows operators to offer more fine-grained RBAC control to their tenants,
+it opens the door to misconfiguration and bugs. Patrole can be used to validate
+that custom policy overrides don't break anything and work as expected.
+
+Custom Policy Validation
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+While testing default policy behavior is a valid use case, oftentimes default
+policies are modified with custom overrides in production. OpenStack's
+`policy.yaml`_ documentation claims that "modifying policy can have unexpected
+side effects", which is why Patrole was created: to ensure that custom
+overrides allow the principle of least privilege to be tailor-made to exact
+specifications via policy overrides, without:
+
+* causing unintended side effects (breaking API endpoints, breaking
+ cross-service workflows, breaking the policy file itself); or
+* resulting in poor RBAC configuration, promoting security vulnerabilities
+
+This has implications on Patrole's :ref:`design-principles`: validating custom
+overrides requires the ability to handle arbitrary roles, which requires logic
+capable of dynamically determining expected test behavior. See
+:ref:`rbac-validation` for more details.
+
+Note that support for custom policies is limited. This is because custom
+policies can be arbitrarily complex, requiring that tests be very robust
+in order to handle all edge cases.
+
+.. _multiple-policies:
+
+Multiple Policies
+-----------------
+
+Behind the scenes, many APIs enforce multiple policies, for many reasons,
+including:
+
+* to control complex cross-service workflows;
+* to control whether a server is booted from an image or booted from a volume
+ (for example);
+* to control whether a response body should contain additional information
+ conditioned upon successful policy authorization.
+
+This makes `policy in code`_ especially important for policy validation: it
+is difficult to keep track of all the policies being enforced across all the
+individual APIs, without policy in code documentation.
+
+Multi-Policy Validation
+^^^^^^^^^^^^^^^^^^^^^^^
+
+Patrole offers support for validating APIs that enforce multiple policies.
+Perhaps in an ideal world each API endpoint would enforce only one policy,
+but in reality some API endpoints enforce multiple policies. Thus, to offer
+accurate validation, Patrole handles multiple policies:
+
+* for services *with* policy in code documentation: this documentation
+ indicates that a single API endpoint enforces multiple policy actions.
+* for services *without* policy in code documentation: the API code clearly
+ shows multiple policy actions being validated. Note that in this case some
+ degree of log tracing is required by developers to confirm that the expected
+ policies are getting enforced, prior to the tests getting merged.
+
+.. todo::
+
+ Link to multi-policy validation documentation section once it has been
+ written.
+
+.. _error-codes:
+
+Error Codes
+-----------
+
+Most OpenStack services raise a ``403 Forbidden`` following failed
+:term:`hard authorization`. Neutron, however, can raise a ``404 NotFound``
+as well. See Neutron's `authorization policy enforcement`_ documentation
+for more details.
+
+Glossary
+--------
+
+The following nomenclature is used throughout Patrole documentation so it is
+important to understand what each term means in order to understand concepts
+related to RBAC in Patrole.
+
+.. glossary::
+
+ authorize
+
+ The act of ``oslo.policy`` determining whether a user can perform a
+ :term:`policy` given his or her :term:`role`.
+
+ enforce
+
+ See :term:`authorize`.
+
+ hard authorization
+
+ The `do_raise`_ flag controls whether policy authorization should result
+ in an exception getting raised or a boolean value getting returned.
+ Hard authorization results in an exception getting raised. Usually, this
+ results in a ``403 Forbidden`` getting returned for unauthorized requests.
+ (See :ref:`error-codes` for further details.)
+
+ Related term: :term:`soft authorization`.
+
+ oslo.policy
+
+ The OpenStack library providing support for RBAC policy enforcement across
+ all OpenStack services. See the `official documentation`_ for more
+ information.
+
+ policy
+
+ Defines an RBAC rule. Each policy is defined by a one-line statement in
+ the form "<target>" : "<rule>". For more information, reference OpenStack's
+ `policy documentation`_.
+
+ policy action
+
+ See :term:`policy target`.
+
+ policy file
+
+ Prior to `governance goal`_ used by all OpenStack services to define
+ policy defaults. Still used by some services, which is why Patrole
+ needs to read the policy files to derive policy information for testing.
+
+ policy in code
+
+ Registers default OpenStack policies for a service in the service's code
+ base.
+
+ Beginning with the Queens release, policy in code became a
+ `governance goal`_.
+
+ policy rule
+
+ The policy rule determines under which circumstances the API call is
+ permitted.
+
+ policy target
+
+ The name of a policy.
+
+ requirements file
+
+ Requirements-driven approach to declaring the expected RBAC test results
+ referenced by Patrole. Uses a high-level YAML syntax to crystallize policy
+ requirements concisely and unambiguously. See :ref:`requirements-authority`
+ for more information.
+
+ role
+
+ A designation for the set of actions that describe what a user can do in
+ the system. Roles are managed through the `Keystone Roles API`_.
+
+ Role-Based Access Control (RBAC)
+
+ May be formally defined as "an approach to restricting system access to
+ authorized users."
+
+ rule
+
+ See :term:`policy rule`. Note that currently the Patrole code base
+ conflates "rule" with :term:`policy target` in some places.
+
+ soft authorization
+
+ The `do_raise`_ flag controls whether policy authorization should result
+ in an exception getting raised or a boolean value getting returned.
+ Soft authorization results in a boolean value getting returned. When policy
+ authorization evaluates to true, additional operations are performed as a
+ part of the API request or additional information is included in the
+ response body (see `response filtering`_ for an example).
+
+ Related term: :term:`hard authorization`.
+
+.. _Nova repository: https://github.com/openstack/nova/tree/master/nova/policies
+.. _Keystone repository: https://github.com/openstack/keystone/tree/master/keystone/common/policies
+.. _governance goal: https://governance.openstack.org/tc/goals/queens/policy-in-code.html
+.. _scope types: https://docs.openstack.org/keystone/latest/admin/identity-tokens.html#authorization-scopes
+.. _policy.yaml: https://docs.openstack.org/ocata/config-reference/policy-yaml-file.html
+.. _oslo.policy: https://docs.openstack.org/oslo.policy/latest/
+.. _policy documentation: https://docs.openstack.org/kilo/config-reference/content/policy-json-file.html
+.. _do_raise: https://docs.openstack.org/oslo.policy/latest/reference/api/oslo_policy.policy.html#oslo_policy.policy.Enforcer.enforce
+.. _authorization policy enforcement: https://docs.openstack.org/neutron/latest/contributor/internals/policy.html
+.. _official documentation: https://docs.openstack.org/oslo.policy/latest/
+.. _Keystone Roles API: https://developer.openstack.org/api-ref/identity/v3/#roles
+.. _response filtering: https://docs.openstack.org/neutron/latest/contributor/internals/policy.html#response-filtering
diff --git a/lower-constraints.txt b/lower-constraints.txt
index 9422676..2b77dff 100644
--- a/lower-constraints.txt
+++ b/lower-constraints.txt
@@ -16,7 +16,7 @@
extras==1.0.0
fasteners==0.14.1
fixtures==3.0.0
-flake8==2.5.5
+flake8==2.6.2
future==0.16.0
hacking==1.0.0
idna==2.6
diff --git a/patrole_tempest_plugin/hacking/checks.py b/patrole_tempest_plugin/hacking/checks.py
index eb73ef1..d106da8 100644
--- a/patrole_tempest_plugin/hacking/checks.py
+++ b/patrole_tempest_plugin/hacking/checks.py
@@ -17,7 +17,7 @@
import os
import re
-import pep8
+import pycodestyle
PYTHON_CLIENTS = ['cinder', 'glance', 'keystone', 'nova', 'swift', 'neutron',
@@ -59,7 +59,7 @@
T105: Tests cannot use setUpClass/tearDownClass
"""
- if pep8.noqa(physical_line):
+ if pycodestyle.noqa(physical_line):
return
if SETUP_TEARDOWN_CLASS_DEFINITION.match(physical_line):
diff --git a/patrole_tempest_plugin/policy_authority.py b/patrole_tempest_plugin/policy_authority.py
index b813f88..3339a5d 100644
--- a/patrole_tempest_plugin/policy_authority.py
+++ b/patrole_tempest_plugin/policy_authority.py
@@ -136,7 +136,7 @@
if not service or service not in cls.available_services:
LOG.debug("%s is NOT a valid service.", service)
- raise rbac_exceptions.RbacInvalidService(
+ raise rbac_exceptions.RbacInvalidServiceException(
"%s is NOT a valid service." % service)
@classmethod
diff --git a/patrole_tempest_plugin/rbac_exceptions.py b/patrole_tempest_plugin/rbac_exceptions.py
index 980672a..809a7ed 100644
--- a/patrole_tempest_plugin/rbac_exceptions.py
+++ b/patrole_tempest_plugin/rbac_exceptions.py
@@ -64,7 +64,10 @@
"instead. Actual exception: %(exception)s")
-class RbacInvalidService(exceptions.TempestException):
+class RbacInvalidServiceException(exceptions.TempestException):
+ """Raised when an invalid service is passed to ``rbac_rule_validation``
+ decorator.
+ """
message = "Attempted to test an invalid service"
diff --git a/patrole_tempest_plugin/rbac_rule_validation.py b/patrole_tempest_plugin/rbac_rule_validation.py
index 2f6759d..e6d1e80 100644
--- a/patrole_tempest_plugin/rbac_rule_validation.py
+++ b/patrole_tempest_plugin/rbac_rule_validation.py
@@ -22,7 +22,7 @@
import six
from tempest import config
-from tempest.lib import exceptions
+from tempest.lib import exceptions as lib_exc
from tempest import test
from patrole_tempest_plugin import policy_authority
@@ -123,7 +123,7 @@
"os_alt.auth_provider.credentials.user_id"
})
- :raises NotFound: If ``service`` is invalid.
+ :raises RbacInvalidServiceException: If ``service`` is invalid.
:raises RbacUnderPermissionException: For item (2) above.
:raises RbacOverPermissionException: For item (3) above.
:raises RbacExpectedWrongException: When a 403 is expected but a 404
@@ -184,12 +184,13 @@
try:
test_func(*args, **kwargs)
- except rbac_exceptions.RbacInvalidService as e:
- msg = ("%s is not a valid service." % service)
- test_status = ('Error, %s' % (msg))
- LOG.error(msg)
- raise exceptions.NotFound(
- "%s RbacInvalidService was: %s" % (msg, e))
+ except rbac_exceptions.RbacInvalidServiceException:
+ with excutils.save_and_reraise_exception():
+ msg = ("%s is not a valid service." % service)
+ # FIXME(felipemonteiro): This test_status is logged too
+ # late. Need a function to log it before re-raising.
+ test_status = ('Error, %s' % (msg))
+ LOG.error(msg)
except (expected_exception,
rbac_exceptions.RbacConflictingPolicies,
rbac_exceptions.RbacMalformedResponse) as e:
@@ -382,14 +383,13 @@
raise rbac_exceptions.RbacInvalidErrorCode(msg)
if expected_error_code == 403:
- expected_exception = exceptions.Forbidden
+ expected_exception = lib_exc.Forbidden
elif expected_error_code == 404:
- expected_exception = exceptions.NotFound
+ expected_exception = lib_exc.NotFound
irregular_msg = ("NotFound exception was caught for test %s. Expected "
"policies which may have caused the error: %s. The "
"service %s throws a 404 instead of a 403, which is "
"irregular.")
-
return expected_exception, irregular_msg
@@ -431,7 +431,7 @@
def _check_for_expected_mismatch_exception(expected_exception,
actual_exception):
- permission_exceptions = (exceptions.Forbidden, exceptions.NotFound)
+ permission_exceptions = (lib_exc.Forbidden, lib_exc.NotFound)
if isinstance(actual_exception, permission_exceptions):
if not isinstance(actual_exception, expected_exception.__class__):
return True
diff --git a/patrole_tempest_plugin/tests/api/compute/test_floating_ips_bulk_rbac.py b/patrole_tempest_plugin/tests/api/compute/test_floating_ips_bulk_rbac.py
index b83cc46..3ccef73 100644
--- a/patrole_tempest_plugin/tests/api/compute/test_floating_ips_bulk_rbac.py
+++ b/patrole_tempest_plugin/tests/api/compute/test_floating_ips_bulk_rbac.py
@@ -27,12 +27,15 @@
CONF = config.CONF
+# TODO(gmann): Remove this test class once the nova queens branch goes
+# into extended maintenance mode.
class FloatingIpsBulkRbacTest(rbac_base.BaseV2ComputeRbacTest):
# Tests will fail with a 404 starting from microversion 2.36:
# See the following link for details:
# https://developer.openstack.org/api-ref/compute/#floating-ips-bulk-os-floating-ips-bulk-deprecated
max_microversion = '2.35'
+ depends_on_nova_network = True
@classmethod
def skip_checks(cls):
diff --git a/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py b/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py
index 2f630a2..72674f6 100644
--- a/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py
+++ b/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py
@@ -67,6 +67,9 @@
shared_network=None,
router_external=None,
router_private=None,
+ provider_network_type=None,
+ provider_physical_network=None,
+ provider_segmentation_id=None,
segments=None,
**kwargs):
if not net_id:
@@ -74,13 +77,19 @@
if admin is not None:
kwargs['admin_state_up'] = admin
- elif shared_network is not None:
+ if shared_network is not None:
kwargs['shared'] = shared_network
- elif router_external is not None:
+ if router_external is not None:
kwargs['router:external'] = router_external
- elif router_private is not None:
+ if router_private is not None:
kwargs['router:private'] = router_private
- elif segments is not None:
+ if provider_network_type is not None:
+ kwargs['provider:network_type'] = provider_network_type
+ if provider_physical_network is not None:
+ kwargs['provider:physical_network'] = provider_physical_network
+ if provider_segmentation_id is not None:
+ kwargs['provider:segmentation_id'] = provider_segmentation_id
+ if segments is not None:
kwargs['segments'] = segments
updated_network = self.networks_client.update_network(
@@ -253,6 +262,81 @@
with self.rbac_utils.override_role(self):
self._update_network(net_id=network['id'], router_external=True)
+ @utils.requires_ext(extension='provider', service='network')
+ @rbac_rule_validation.action(
+ service="neutron",
+ rules=["get_network",
+ "update_network",
+ "update_network:provider:network_type"],
+ expected_error_codes=[404, 403, 403])
+ @decorators.idempotent_id('d064ef96-662b-47b6-94b7-9106dcd7ba8c')
+ def test_update_network_provider_network_type(self):
+
+ """Update Network Provider Network Type Test
+
+ RBAC test for neutron update_network:provider:network_type policy
+ """
+ try:
+ with self.rbac_utils.override_role(self):
+ self._update_network(self.network['id'],
+ provider_network_type='vxlan')
+ except lib_exc.BadRequest as exc:
+ # Per the api documentation, most plugins don't support updating
+ # provider attributes
+ self.assertIn(
+ "Plugin does not support updating provider attributes",
+ str(exc))
+
+ @utils.requires_ext(extension='provider', service='network')
+ @rbac_rule_validation.action(
+ service="neutron",
+ rules=["get_network",
+ "update_network",
+ "update_network:provider:physical_network"],
+ expected_error_codes=[404, 403, 403])
+ @decorators.idempotent_id('e3a55660-f75c-494e-a1b1-a8b36cc789ef')
+ def test_update_network_provider_physical_network(self):
+
+ """Update Network Provider Physical Network Test
+
+ RBAC test for neutron update_network:provider:physical_network policy
+ """
+ try:
+ with self.rbac_utils.override_role(self):
+ self._update_network(self.network['id'],
+ provider_physical_network='provider')
+ except lib_exc.BadRequest as exc:
+ # Per the api documenation, most plugins don't support updating
+ # provider attributes
+ self.assertIn(
+ "Plugin does not support updating provider attributes",
+ str(exc))
+
+ @utils.requires_ext(extension='provider', service='network')
+ @rbac_rule_validation.action(
+ service="neutron",
+ rules=["get_network",
+ "update_network",
+ "update_network:provider:segmentation_id"],
+ expected_error_codes=[404, 403, 403])
+ @decorators.idempotent_id('f6164228-b670-45fd-9ff9-b101930318c7')
+ def test_update_network_provider_segmentation_id(self):
+
+ """Update Network Provider Segmentation Id Test
+
+ RBAC test for neutron update_network:provider:segmentation_id policy
+ """
+ try:
+ with self.rbac_utils.override_role(self):
+ self._update_network(self.network['id'],
+ provider_segmentation_id=400)
+ except lib_exc.BadRequest as exc:
+ # Per the api documenation, most plugins don't support updating
+ # provider attributes
+ self.assertIn(
+ "Plugin does not support updating provider attributes",
+ str(exc))
+
@rbac_rule_validation.action(service="neutron",
rule="get_network",
expected_error_code=404)
diff --git a/patrole_tempest_plugin/tests/api/volume/rbac_base.py b/patrole_tempest_plugin/tests/api/volume/rbac_base.py
index 798f311..bac173e 100644
--- a/patrole_tempest_plugin/tests/api/volume/rbac_base.py
+++ b/patrole_tempest_plugin/tests/api/volume/rbac_base.py
@@ -12,6 +12,7 @@
# under the License.
from tempest.api.volume import base as vol_base
+from tempest.common import waiters
from tempest import config
from tempest.lib.common.utils import data_utils
from tempest.lib.common.utils import test_utils
@@ -58,6 +59,29 @@
cls.volume_types_client.delete_volume_type, volume_type['id'])
return volume_type
+ @classmethod
+ def _create_backup(cls, volume_id, backup_client=None, **kwargs):
+ """Wrapper utility that returns a test backup.
+
+ Tempest has an instance-level version. This is a class-level version.
+ """
+ if backup_client is None:
+ backup_client = cls.backups_client
+ if 'name' not in kwargs:
+ name = data_utils.rand_name(cls.__name__ + '-Backup')
+ kwargs['name'] = name
+
+ backup = backup_client.create_backup(
+ volume_id=volume_id, **kwargs)['backup']
+ cls.addClassResourceCleanup(
+ test_utils.call_and_ignore_notfound_exc,
+ backup_client.delete_backup, backup['id'])
+ waiters.wait_for_volume_resource_status(backup_client, backup['id'],
+ 'available')
+ waiters.wait_for_volume_resource_status(cls.volumes_client, volume_id,
+ 'available')
+ return backup
+
def create_group_type(self, name=None, ignore_notfound=False, **kwargs):
"""Create a test group-type"""
name = name or data_utils.rand_name(
diff --git a/patrole_tempest_plugin/tests/api/volume/test_volumes_backup_rbac.py b/patrole_tempest_plugin/tests/api/volume/test_volumes_backup_rbac.py
index 7f5f566..5aff7a9 100644
--- a/patrole_tempest_plugin/tests/api/volume/test_volumes_backup_rbac.py
+++ b/patrole_tempest_plugin/tests/api/volume/test_volumes_backup_rbac.py
@@ -41,6 +41,7 @@
def resource_setup(cls):
super(VolumesBackupsV3RbacTest, cls).resource_setup()
cls.volume = cls.create_volume()
+ cls.backup = cls._create_backup(volume_id=cls.volume['id'])
def _decode_url(self, backup_url):
return json.loads(base64.decode_as_text(backup_url))
@@ -59,18 +60,24 @@
rule="backup:create")
@decorators.idempotent_id('6887ec94-0bcf-4ab7-b30f-3808a4b5a2a5')
def test_create_backup(self):
+ backup_name = data_utils.rand_name(self.__class__.__name__ + '-Backup')
+
with self.rbac_utils.override_role(self):
- self.create_backup(volume_id=self.volume['id'])
+ backup = self.backups_client.create_backup(
+ volume_id=self.volume['id'], name=backup_name)['backup']
+ self.addCleanup(self.backups_client.delete_backup, backup['id'])
+ waiters.wait_for_volume_resource_status(
+ self.backups_client, backup['id'], 'available')
+ waiters.wait_for_volume_resource_status(self.volumes_client,
+ self.volume['id'], 'available')
@decorators.attr(type='slow')
@rbac_rule_validation.action(service="cinder",
rule="backup:get")
@decorators.idempotent_id('abd92bdd-b0fb-4dc4-9cfc-de9e968f8c8a')
def test_show_backup(self):
- backup = self.create_backup(volume_id=self.volume['id'])
-
with self.rbac_utils.override_role(self):
- self.backups_client.show_backup(backup['id'])
+ self.backups_client.show_backup(self.backup['id'])
@rbac_rule_validation.action(service="cinder",
rule="backup:get_all")
@@ -92,7 +99,10 @@
service="cinder",
rule="volume_extension:backup_admin_actions:reset_status")
def test_reset_backup_status(self):
+ # Use instance-level create_backup for easier debugging.
backup = self.create_backup(volume_id=self.volume['id'])
+ waiters.wait_for_volume_resource_status(self.volumes_client,
+ self.volume['id'], 'available')
with self.rbac_utils.override_role(self):
self.backups_client.reset_backup_status(backup_id=backup['id'],
@@ -105,11 +115,11 @@
rule="backup:restore")
@decorators.idempotent_id('9c794bf9-2446-4f41-8fe0-80b71e757f9d')
def test_restore_backup(self):
- backup = self.create_backup(volume_id=self.volume['id'])
-
with self.rbac_utils.override_role(self):
restore = self.backups_client.restore_backup(
- backup['id'])['restore']
+ self.backup['id'])['restore']
+ self.addCleanup(self.volumes_client.delete_volume,
+ restore['volume_id'])
waiters.wait_for_volume_resource_status(
self.backups_client, restore['backup_id'], 'available')
@@ -127,6 +137,8 @@
self.backups_client.delete_backup, backup['id'])
waiters.wait_for_volume_resource_status(self.backups_client,
backup['id'], 'available')
+ waiters.wait_for_volume_resource_status(self.volumes_client,
+ self.volume['id'], 'available')
with self.rbac_utils.override_role(self):
self.backups_client.delete_backup(backup['id'])
@@ -138,19 +150,17 @@
rule="backup:export-import")
@decorators.idempotent_id('e984ec8d-e8eb-485c-98bc-f1856020303c')
def test_export_backup(self):
- backup = self.create_backup(volume_id=self.volume['id'])
-
with self.rbac_utils.override_role(self):
- self.backups_client.export_backup(backup['id'])['backup-record']
+ self.backups_client.export_backup(self.backup['id'])[
+ 'backup-record']
@decorators.attr(type='slow')
@rbac_rule_validation.action(service="cinder",
rule="backup:backup-import")
@decorators.idempotent_id('1e70f039-4556-44cc-9cc1-edf2b7ed648b')
def test_import_backup(self):
- backup = self.create_backup(volume_id=self.volume['id'])
export_backup = self.backups_client.export_backup(
- backup['id'])['backup-record']
+ self.backup['id'])['backup-record']
new_id = data_utils.rand_uuid()
new_url = self._modify_backup_url(
export_backup['backup_url'], {'id': new_id})
@@ -160,6 +170,9 @@
backup_service=export_backup['backup_service'],
backup_url=new_url)['backup']
self.addCleanup(self.backups_client.delete_backup, import_backup['id'])
+ waiters.wait_for_volume_resource_status(self.backups_client,
+ import_backup['id'],
+ 'available')
class VolumesBackupsV318RbacTest(rbac_base.BaseVolumeRbacTest):
@@ -175,22 +188,25 @@
if not CONF.volume_feature_enabled.backup:
raise cls.skipException("Cinder backup feature disabled")
+ @classmethod
+ def resource_setup(cls):
+ super(VolumesBackupsV318RbacTest, cls).resource_setup()
+ cls.volume = cls.create_volume()
+ cls.backup = cls._create_backup(volume_id=cls.volume['id'])
+ cls.expected_attr = 'os-backup-project-attr:project_id'
+
@decorators.idempotent_id('69801485-d5be-4e75-bbb4-168d50b5a8c2')
@rbac_rule_validation.action(service="cinder",
rule="backup:backup_project_attribute")
def test_show_backup_project_attribute(self):
- volume = self.create_volume()
- backup = self.create_backup(volume_id=volume['id'])
- expected_attr = 'os-backup-project-attr:project_id'
-
with self.rbac_utils.override_role(self):
- body = self.backups_client.show_backup(backup['id'])['backup']
+ body = self.backups_client.show_backup(self.backup['id'])['backup']
# Show backup API attempts to inject the attribute below into the
# response body but only if policy enforcement succeeds.
- if expected_attr not in body:
+ if self.expected_attr not in body:
raise rbac_exceptions.RbacMalformedResponse(
- attribute=expected_attr)
+ attribute=self.expected_attr)
class VolumesBackupsV39RbacTest(rbac_base.BaseVolumeRbacTest):
@@ -204,18 +220,22 @@
if not CONF.volume_feature_enabled.backup:
raise cls.skipException("Cinder backup feature disabled")
+ @classmethod
+ def resource_setup(cls):
+ super(VolumesBackupsV39RbacTest, cls).resource_setup()
+ cls.volume = cls.create_volume()
+ cls.backup = cls._create_backup(volume_id=cls.volume['id'])
+
@decorators.attr(type='slow')
@decorators.idempotent_id('b45b0e98-6eb8-4c62-aa53-0f8c7c09faa6')
@rbac_rule_validation.action(
service="cinder",
rule="backup:update")
def test_backup_update(self):
- volume = self.create_volume()
- backup = self.create_backup(volume_id=volume['id'])
update_kwargs = {
'name': data_utils.rand_name(self.__class__.__name__ + '-Backup'),
'description': data_utils.rand_name("volume-backup-description")
}
with self.rbac_utils.override_role(self):
- self.backups_client.update_backup(backup['id'],
+ self.backups_client.update_backup(self.backup['id'],
**update_kwargs)
diff --git a/patrole_tempest_plugin/tests/unit/test_policy_authority.py b/patrole_tempest_plugin/tests/unit/test_policy_authority.py
index 3e2cc4c..d396a29 100644
--- a/patrole_tempest_plugin/tests/unit/test_policy_authority.py
+++ b/patrole_tempest_plugin/tests/unit/test_policy_authority.py
@@ -238,7 +238,7 @@
test_user_id = mock.sentinel.user_id
service = 'invalid_service'
- self.assertRaises(rbac_exceptions.RbacInvalidService,
+ self.assertRaises(rbac_exceptions.RbacInvalidServiceException,
policy_authority.PolicyAuthority,
test_tenant_id,
test_user_id,
@@ -253,7 +253,7 @@
test_user_id = mock.sentinel.user_id
service = None
- self.assertRaises(rbac_exceptions.RbacInvalidService,
+ self.assertRaises(rbac_exceptions.RbacInvalidServiceException,
policy_authority.PolicyAuthority,
test_tenant_id,
test_user_id,
@@ -528,8 +528,9 @@
policy_parser = None
expected_exception = 'invalid_service is NOT a valid service'
- with self.assertRaisesRegex(rbac_exceptions.RbacInvalidService,
- expected_exception):
+ with self.assertRaisesRegex(
+ rbac_exceptions.RbacInvalidServiceException,
+ expected_exception):
policy_authority.PolicyAuthority(
test_tenant_id, test_user_id, "INVALID_SERVICE")
else:
diff --git a/patrole_tempest_plugin/tests/unit/test_rbac_rule_validation.py b/patrole_tempest_plugin/tests/unit/test_rbac_rule_validation.py
index 2e275dc..1bf5510 100644
--- a/patrole_tempest_plugin/tests/unit/test_rbac_rule_validation.py
+++ b/patrole_tempest_plugin/tests/unit/test_rbac_rule_validation.py
@@ -430,6 +430,22 @@
"Allowed")
+class RBACRuleValidationNegativeTest(BaseRBACRuleValidationTest):
+
+ @mock.patch.object(rbac_rv, 'policy_authority', autospec=True)
+ def test_rule_validation_invalid_service_raises_exc(self, mock_authority):
+ """Test that invalid service raises the appropriate exception."""
+ mock_authority.PolicyAuthority.return_value.allowed.side_effect = (
+ rbac_exceptions.RbacInvalidServiceException)
+
+ @rbac_rv.action(mock.sentinel.service, mock.sentinel.action)
+ def test_policy(*args):
+ pass
+
+ self.assertRaises(rbac_exceptions.RbacInvalidServiceException,
+ test_policy, self.mock_test_args)
+
+
class RBACRuleValidationTestMultiPolicy(BaseRBACRuleValidationTest):
"""Test suite for validating multi-policy support for the
``rbac_rule_validation`` decorator.
diff --git a/test-requirements.txt b/test-requirements.txt
index 35e4e57..9085c07 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -1,7 +1,7 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
-hacking>=1.0.0,<1.1.0 # Apache-2.0
+hacking>=1.1.0,<1.2.0 # Apache-2.0
fixtures>=3.0.0 # Apache-2.0/BSD
mock>=2.0.0 # BSD
coverage!=4.4,>=4.0 # Apache-2.0