README.rst - packaging/sources/patrole - Gitiles

 Patrole - RBAC Integration Tempest Plugin
 =========================================

 Patrole is a set of integration tests to be run against a live OpenStack
 cluster. It has a battery of tests dedicated to validating the correctness and
 integrity of the cloud's RBAC implementation.

 More importantly, Patrole is a security validation tool for verifying that
 Role-Based Access Control is correctly configured and enforced in an OpenStack
 cloud. It runs `Tempest`_-based API tests using specified RBAC roles, thus
 allowing deployments to verify that only intended roles have access to those
 APIs.

 Patrole is currently undergoing heavy development. As more projects move
 toward policy in code, Patrole will align its testing with the appropriate
 documentation.

 * Free software: Apache license
 * Documentation: https://docs.openstack.org/patrole/latest
 * Source: https://opendev.org/openstack/patrole
 * Bugs: https://storyboard.openstack.org/#!/project/openstack/patrole
 * Release notes: https://docs.openstack.org/releasenotes/patrole/

 Team and repository tags
 ------------------------

 .. image:: https://governance.openstack.org/tc/badges/patrole.svg
     :target: https://governance.openstack.org/tc/reference/tags/index.html

 .. _design-principles:

 Design Principles
 -----------------

 As a `Tempest plugin`_, Patrole borrows some design principles from `Tempest design principles`_,
 but not all, as its testing scope is confined to policies.

 * *Stability*. Patrole uses OpenStack public interfaces. Tests in Patrole
   should only touch public OpenStack APIs.
 * *Atomicity*. Patrole tests should be atomic: they should test policies in
   isolation. Unlike Tempest, a Patrole test strives to only call a single
   endpoint at a time. This is because it is important to validate each policy
   is authorized correctly and the best way to do that is to validate each
   policy alone, to avoid test contamination.
 * *Complete coverage*. Patrole should validate all policy in code defaults. For
   testing, Patrole uses the API-to-policy mapping contained in each project's
   `policy in code`_ documentation where applicable.

   For example, Nova's policy in code documentation is located in the
   `Nova repository`_ under ``nova/policies``. Likewise, Keystone's policy in
   code documentation is located in the `Keystone repository`_ under
   ``keystone/common/policies``. The other OpenStack services follow the same
   directory layout pattern with respect to policy in code.

   .. note::

     Realistically this is not always possible because some services have
     not yet moved to policy in code.

 * *Customizable*. Patrole should be able to validate custom policy overrides to
   ensure that those overrides enhance rather than undermine the cloud's RBAC
   configuration. In addition, Patrole should be able to validate any role.
 * *Self-cleaning*. Patrole should attempt to clean up after itself; whenever
   possible we should tear down resources when done.

   .. note::

       Patrole modifies roles dynamically in the background, which affects
       pre-provisioned credentials. Work is currently underway to clean up
       modifications made to pre-provisioned credentials.

 * *Self-testing*. Patrole should be self-testing.

 .. _Tempest plugin: https://docs.openstack.org/tempest/latest/plugin.html
 .. _Tempest design principles: https://docs.openstack.org/tempest/latest/overview.html#design-principles
 .. _policy in code: https://specs.openstack.org/openstack/oslo-specs/specs/newton/policy-in-code.html
 .. _Nova repository: https://opendev.org/openstack/nova/src/branch/master/nova/policies
 .. _Keystone repository: https://opendev.org/openstack/keystone/src/branch/master/keystone/common/policies

 Features
 --------
 * Validation of default policy definitions located in policy.json files.
 * Validation of in-code policy definitions.
 * Validation of custom policy file definitions that override default policy
   definitions.
 * Built-in positive and negative testing. Positive and negative testing
   are performed using the same tests and role-switching.
 * Valdation of custom roles as well as default OpenStack roles.

 .. note::

     Patrole does not yet support policy.yaml files, the new file format for
     policy files in OpenStack.

 How It Works
 ------------
 Patrole leverages ``oslo.policy`` (OpenStack's policy enforcement engine) to
 determine whether a given role is allowed to perform a policy action, given a
 specific role and OpenStack service. The output from ``oslo.policy`` (the
 expected result) and the actual result from test execution are compared to
 each other: if both results match, then the test passes; else it fails.

 Terminology
 ^^^^^^^^^^^
 * Expected Result - The expected result of a given test.
 * Actual Result - The actual result of a given test.
 * Final Result - A match between both expected and actual results. A mismatch
   in the expected result and the actual result will result in a test failure.

   * Expected: Pass | Actual: Pass - Test Case Success
   * Expected: Pass | Actual: Fail - Test Case Under-Permission Failure
   * Expected: Fail | Actual: Pass - Test Case Over-Permission Failure
   * Expected: Fail | Actual: Fail (Expected exception) - Test Case Success
   * Expected: Fail | Actual: Fail (Unexpected exception) - Test Case Failure

 Quickstart
 ----------
 To run Patrole, you must first have `Tempest`_ installed and configured
 properly. Please reference `Tempest_quickstart`_ guide to do so. Follow all
 the steps outlined therein. Afterward, proceed with the steps below.

 #. You first need to install Patrole. This is done with pip after you check out
    the Patrole repo::

     $ git clone https://opendev.org/openstack/patrole
     $ pip install patrole/

    This can be done within a venv.

    .. note::

      You may also install Patrole from source code by running::

        pip install -e patrole/

 #. Next you must properly configure Patrole, which is relatively
    straightforward. For details on configuring Patrole refer to the
    `Patrole Configuration <https://docs.openstack.org/patrole/latest/configuration.html#patrole-configuration>`_.

 #. Once the configuration is done you're now ready to run Patrole. This can
    be done using the `tempest_run`_ command. This can be done by running::

      $ tempest run --regex '^patrole_tempest_plugin\.tests\.api'

    There is also the option to use testr directly, or any `testr`_ based test
    runner, like `ostestr`_. For example, from the workspace dir run::

      $ stestr --regex '(?!.*\[.*\bslow\b.*\])(^patrole_tempest_plugin\.tests\.api))'

    will run the same set of tests as the default gate jobs.

    You can also run Patrole tests using `tox`_, but as Patrole needs access to
    global packages use ``--sitepackages`` argument. To do so, ``cd`` into the
    **Tempest** directory and run::

      $ tox -eall --sitepackages -- patrole_tempest_plugin.tests.api

    .. note::

      It is possible to run Patrole via ``tox -eall`` in order to run Patrole
      isolated from other plugins. This can be accomplished by including the
      installation of services that currently use policy in code -- for example,
      Nova and Keystone. For example::

        $ tox -evenv-tempest -- pip install /opt/stack/patrole /opt/stack/keystone /opt/stack/nova
        $ tox -eall -- patrole_tempest_plugin.tests.api

 #. Log information from tests is captured in ``tempest.log`` under the Tempest
    repository. Some Patrole debugging information is captured in that log
    related to expected test results and `Role Overriding <https://docs.openstack.org/patrole/latest/test_writing_guide.html#role-overriding>`_.

    More detailed RBAC testing log output is emitted to ``patrole.log`` under
    the Patrole repository. To configure Patrole's logging, see the
    `Patrole Configuration Guide <https://docs.openstack.org/patrole/latest/configuration.html#patrole-configuration>`_.

 .. _Tempest: https://opendev.org/openstack/tempest/
 .. _Tempest_quickstart: https://docs.openstack.org/tempest/latest/overview.html#quickstart
 .. _tempest_run: https://docs.openstack.org/tempest/latest/run.html
 .. _testr: https://testrepository.readthedocs.org/en/latest/MANUAL.html
 .. _ostestr: https://docs.openstack.org/os-testr/latest/
 .. _tox: https://tox.readthedocs.io/en/latest/

 RBAC Tests
 ----------

 To change the roles that the patrole tests are being run as, edit
 ``rbac_test_roles`` in the ``patrole`` section of tempest.conf: ::

     [patrole]
     rbac_test_roles = member,reader
     ...

 .. note::

   The ``rbac_test_roles`` is service-specific. member, for example,
   is an arbitrary role, but by convention is used to designate the default
   non-admin role in the system. Most Patrole tests should be run with
   **admin** and **member** roles. However, other services may use entirely
   different roles or role combinations.

 For more information about RBAC, reference the `rbac-overview`_
 documentation page.

 For information regarding which projects Patrole offers RBAC testing for,
 reference the `HACKING`_ documentation page.

 .. _rbac-overview: https://docs.openstack.org/patrole/latest/rbac-overview.html
 .. _HACKING: https://docs.openstack.org/patrole/latest/HACKING.html#supported-openstack-components

 Unit Tests
 ----------

 Patrole also has a set of unit tests which test the Patrole code itself. These
 tests can be run by specifying the test discovery path::

   $ stestr --test-path ./patrole_tempest_plugin/tests/unit run

 By setting ``--test-path`` option to ``./patrole_tempest_plugin/tests/unit``
 it specifies that test discovery should only be run on the unit test directory.

 Alternatively, there are the py27 and py35 tox jobs which will run the unit
 tests with the corresponding version of Python.

 One common activity is to just run a single test; you can do this with tox
 simply by specifying to just run py27 or py35 tests against a single test::

   $ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils.RBACUtilsTest.test_override_role_with_missing_admin_role

 Or all tests in the test_rbac_utils.py file::

   $ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils

 You may also use regular expressions to run any matching tests::

   $ tox -e py27 -- test_rbac_utils

 For more information on these options and details about stestr, please see the
 `stestr documentation <http://stestr.readthedocs.io/en/latest/MANUAL.html>`_.

 Release Versioning
 ------------------
 `Patrole Release Notes <https://docs.openstack.org/releasenotes/patrole/>`_
 shows which changes have been released for each version.

 Patrole's release versioning follows Tempest's conventions. Like Tempest,
 Patrole is branchless and uses versioning instead.

 Storyboard
 ----------
 Bugs and enhancements are tracked via Patrole's
 `Storyboard Page <https://storyboard.openstack.org/#!/project/openstack/patrole>`_.
	Patrole - RBAC Integration Tempest Plugin
	=========================================

	Patrole is a set of integration tests to be run against a live OpenStack
	cluster. It has a battery of tests dedicated to validating the correctness and
	integrity of the cloud's RBAC implementation.

	More importantly, Patrole is a security validation tool for verifying that
	Role-Based Access Control is correctly configured and enforced in an OpenStack
	cloud. It runs `Tempest`_-based API tests using specified RBAC roles, thus
	allowing deployments to verify that only intended roles have access to those
	APIs.

	Patrole is currently undergoing heavy development. As more projects move
	toward policy in code, Patrole will align its testing with the appropriate
	documentation.

	* Free software: Apache license
	* Documentation: https://docs.openstack.org/patrole/latest
	* Source: https://opendev.org/openstack/patrole
	* Bugs: https://storyboard.openstack.org/#!/project/openstack/patrole
	* Release notes: https://docs.openstack.org/releasenotes/patrole/

	Team and repository tags
	------------------------

	.. image:: https://governance.openstack.org/tc/badges/patrole.svg
	:target: https://governance.openstack.org/tc/reference/tags/index.html

	.. _design-principles:

	Design Principles
	-----------------

	As a `Tempest plugin`_, Patrole borrows some design principles from `Tempest design principles`_,
	but not all, as its testing scope is confined to policies.

	* Stability. Patrole uses OpenStack public interfaces. Tests in Patrole
	should only touch public OpenStack APIs.
	* Atomicity. Patrole tests should be atomic: they should test policies in
	isolation. Unlike Tempest, a Patrole test strives to only call a single
	endpoint at a time. This is because it is important to validate each policy
	is authorized correctly and the best way to do that is to validate each
	policy alone, to avoid test contamination.
	* Complete coverage. Patrole should validate all policy in code defaults. For
	testing, Patrole uses the API-to-policy mapping contained in each project's
	`policy in code`_ documentation where applicable.

	For example, Nova's policy in code documentation is located in the
	`Nova repository`_ under ``nova/policies``. Likewise, Keystone's policy in
	code documentation is located in the `Keystone repository`_ under
	``keystone/common/policies``. The other OpenStack services follow the same
	directory layout pattern with respect to policy in code.

	.. note::

	Realistically this is not always possible because some services have
	not yet moved to policy in code.

	* Customizable. Patrole should be able to validate custom policy overrides to
	ensure that those overrides enhance rather than undermine the cloud's RBAC
	configuration. In addition, Patrole should be able to validate any role.
	* Self-cleaning. Patrole should attempt to clean up after itself; whenever
	possible we should tear down resources when done.

	.. note::

	Patrole modifies roles dynamically in the background, which affects
	pre-provisioned credentials. Work is currently underway to clean up
	modifications made to pre-provisioned credentials.

	* Self-testing. Patrole should be self-testing.

	.. _Tempest plugin: https://docs.openstack.org/tempest/latest/plugin.html
	.. _Tempest design principles: https://docs.openstack.org/tempest/latest/overview.html#design-principles
	.. _policy in code: https://specs.openstack.org/openstack/oslo-specs/specs/newton/policy-in-code.html
	.. _Nova repository: https://opendev.org/openstack/nova/src/branch/master/nova/policies
	.. _Keystone repository: https://opendev.org/openstack/keystone/src/branch/master/keystone/common/policies

	Features
	--------
	* Validation of default policy definitions located in policy.json files.
	* Validation of in-code policy definitions.
	* Validation of custom policy file definitions that override default policy
	definitions.
	* Built-in positive and negative testing. Positive and negative testing
	are performed using the same tests and role-switching.
	* Valdation of custom roles as well as default OpenStack roles.

	.. note::

	Patrole does not yet support policy.yaml files, the new file format for
	policy files in OpenStack.

	How It Works
	------------
	Patrole leverages ``oslo.policy`` (OpenStack's policy enforcement engine) to
	determine whether a given role is allowed to perform a policy action, given a
	specific role and OpenStack service. The output from ``oslo.policy`` (the
	expected result) and the actual result from test execution are compared to
	each other: if both results match, then the test passes; else it fails.

	Terminology
	^^^^^^^^^^^
	* Expected Result - The expected result of a given test.
	* Actual Result - The actual result of a given test.
	* Final Result - A match between both expected and actual results. A mismatch
	in the expected result and the actual result will result in a test failure.

	* Expected: Pass \| Actual: Pass - Test Case Success
	* Expected: Pass \| Actual: Fail - Test Case Under-Permission Failure
	* Expected: Fail \| Actual: Pass - Test Case Over-Permission Failure
	* Expected: Fail \| Actual: Fail (Expected exception) - Test Case Success
	* Expected: Fail \| Actual: Fail (Unexpected exception) - Test Case Failure

	Quickstart
	----------
	To run Patrole, you must first have `Tempest`_ installed and configured
	properly. Please reference `Tempest_quickstart`_ guide to do so. Follow all
	the steps outlined therein. Afterward, proceed with the steps below.

	#. You first need to install Patrole. This is done with pip after you check out
	the Patrole repo::

	$ git clone https://opendev.org/openstack/patrole
	$ pip install patrole/

	This can be done within a venv.

	.. note::

	You may also install Patrole from source code by running::

	pip install -e patrole/

	#. Next you must properly configure Patrole, which is relatively
	straightforward. For details on configuring Patrole refer to the
	`Patrole Configuration <https://docs.openstack.org/patrole/latest/configuration.html#patrole-configuration>`_.

	#. Once the configuration is done you're now ready to run Patrole. This can
	be done using the `tempest_run`_ command. This can be done by running::

	$ tempest run --regex '^patrole_tempest_plugin\.tests\.api'

	There is also the option to use testr directly, or any `testr`_ based test
	runner, like `ostestr`_. For example, from the workspace dir run::

	$ stestr --regex '(?!.\[.\bslow\b.*\])(^patrole_tempest_plugin\.tests\.api))'

	will run the same set of tests as the default gate jobs.

	You can also run Patrole tests using `tox`_, but as Patrole needs access to
	global packages use ``--sitepackages`` argument. To do so, ``cd`` into the
	Tempest directory and run::

	$ tox -eall --sitepackages -- patrole_tempest_plugin.tests.api

	.. note::

	It is possible to run Patrole via ``tox -eall`` in order to run Patrole
	isolated from other plugins. This can be accomplished by including the
	installation of services that currently use policy in code -- for example,
	Nova and Keystone. For example::

	$ tox -evenv-tempest -- pip install /opt/stack/patrole /opt/stack/keystone /opt/stack/nova
	$ tox -eall -- patrole_tempest_plugin.tests.api

	#. Log information from tests is captured in ``tempest.log`` under the Tempest
	repository. Some Patrole debugging information is captured in that log
	related to expected test results and `Role Overriding <https://docs.openstack.org/patrole/latest/test_writing_guide.html#role-overriding>`_.

	More detailed RBAC testing log output is emitted to ``patrole.log`` under
	the Patrole repository. To configure Patrole's logging, see the
	`Patrole Configuration Guide <https://docs.openstack.org/patrole/latest/configuration.html#patrole-configuration>`_.

	.. _Tempest: https://opendev.org/openstack/tempest/
	.. _Tempest_quickstart: https://docs.openstack.org/tempest/latest/overview.html#quickstart
	.. _tempest_run: https://docs.openstack.org/tempest/latest/run.html
	.. _testr: https://testrepository.readthedocs.org/en/latest/MANUAL.html
	.. _ostestr: https://docs.openstack.org/os-testr/latest/
	.. _tox: https://tox.readthedocs.io/en/latest/

	RBAC Tests
	----------

	To change the roles that the patrole tests are being run as, edit
	``rbac_test_roles`` in the ``patrole`` section of tempest.conf: ::

	[patrole]
	rbac_test_roles = member,reader
	...

	.. note::

	The ``rbac_test_roles`` is service-specific. member, for example,
	is an arbitrary role, but by convention is used to designate the default
	non-admin role in the system. Most Patrole tests should be run with
	admin and member roles. However, other services may use entirely
	different roles or role combinations.

	For more information about RBAC, reference the `rbac-overview`_
	documentation page.

	For information regarding which projects Patrole offers RBAC testing for,
	reference the `HACKING`_ documentation page.

	.. _rbac-overview: https://docs.openstack.org/patrole/latest/rbac-overview.html
	.. _HACKING: https://docs.openstack.org/patrole/latest/HACKING.html#supported-openstack-components

	Unit Tests
	----------

	Patrole also has a set of unit tests which test the Patrole code itself. These
	tests can be run by specifying the test discovery path::

	$ stestr --test-path ./patrole_tempest_plugin/tests/unit run

	By setting ``--test-path`` option to ``./patrole_tempest_plugin/tests/unit``
	it specifies that test discovery should only be run on the unit test directory.

	Alternatively, there are the py27 and py35 tox jobs which will run the unit
	tests with the corresponding version of Python.

	One common activity is to just run a single test; you can do this with tox
	simply by specifying to just run py27 or py35 tests against a single test::

	$ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils.RBACUtilsTest.test_override_role_with_missing_admin_role

	Or all tests in the test_rbac_utils.py file::

	$ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils

	You may also use regular expressions to run any matching tests::

	$ tox -e py27 -- test_rbac_utils

	For more information on these options and details about stestr, please see the
	`stestr documentation <http://stestr.readthedocs.io/en/latest/MANUAL.html>`_.

	Release Versioning
	------------------
	`Patrole Release Notes <https://docs.openstack.org/releasenotes/patrole/>`_
	shows which changes have been released for each version.

	Patrole's release versioning follows Tempest's conventions. Like Tempest,
	Patrole is branchless and uses versioning instead.

	Storyboard
	----------
	Bugs and enhancements are tracked via Patrole's
	`Storyboard Page <https://storyboard.openstack.org/#!/project/openstack/patrole>`_.