Merge "Skip CLI tests is identity v2 is not available"
diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst
new file mode 100644
index 0000000..08f37cc
--- /dev/null
+++ b/doc/source/configuration.rst
@@ -0,0 +1,115 @@
+Tempest Configuration Guide
+===========================
+
+Auth/Credentials
+----------------
+
+Tempest currently has 2 different ways in configuration to provide credentials
+to use when running tempest. One is a traditional set of configuration options
+in the tempest.conf file. These options are in the identity section and let you
+specify a regular user, a global admin user, and a alternate user set of
+credentials. (which consist of a username, password, and project/tenant name)
+These options should be clearly labelled in the sample config file in the
+identity section.
+
+The other method to provide credentials is using the accounts.yaml file. This
+file is used to specify an arbitrary number of users available to run tests
+with. You can specify the location of the file in the
+auth section in the tempest.conf file. To see the specific format used in
+the file please refer to the accounts.yaml.sample file included in tempest.
+Currently users that are specified in the accounts.yaml file are assumed to
+have the same set of roles which can be used for executing all the tests you
+are running. This will be addressed in the future, but is a current limitation.
+Eventually the config options for providing credentials to tempest will be
+deprecated and removed in favor of the accounts.yaml file.
+
+Credential Provider Mechanisms
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Tempest currently also has 3 different internal methods for providing
+authentication to tests. Tenant isolation, locking test accounts, and
+non-locking test accounts. Depending on which one is in use the configuration
+of tempest is slightly different.
+
+Tenant Isolation
+""""""""""""""""
+Tenant isolation was originally create to enable running tempest in parallel.
+For each test class it creates a unique set of user credentials to use for the
+tests in the class. It can create up to 3 sets of username, password, and
+tenant/project names for a primary user, an admin user, and an alternate user.
+To enable and use tenant isolation you only need to configure 2 things:
+
+ #. A set of admin credentials with permissions to create users and
+ tenants/projects. This is specified in the identity section with the
+ admin_username, admin_tenant_name, and admin_password options
+ #. To enable tenant_isolation in the auth section with the
+ allow_tenant_isolation option.
+
+
+Locking Test Accounts
+"""""""""""""""""""""
+For a long time using tenant isolation was the only method available if you
+wanted to enable parallel execution of tempest tests. However this was
+insufficient for certain use cases because of the admin credentials requirement
+to create the credential sets on demand. To get around that the accounts.yaml
+file was introduced and with that a new internal credential provider to enable
+using the list of credentials instead of creating them on demand. With locking
+test accounts each test class will reserve a set of credentials from the
+accounts.yaml before executing any of its tests so that each class is isolated
+like in tenant isolation.
+
+Currently, this mechanism has some limitations, first only non-admin users with
+the same role set can be used at one time. The second limitation is around
+networking, locking test accounts will only work with a single flat network as
+the default for each tenant/project. If another network configuration is used
+in your cloud you might face unexpected failures.
+
+To enable and use locking test accounts you need do a few things:
+
+ #. Enable the locking test account provider with the
+ locking_credentials_provider option in the auth section
+ #. Create a accounts.yaml file which contains the set of pre-existing
+ credentials to use for testing. To make sure you don't have a credentials
+ starvation issue when running in parallel make sure you have at least 2
+ times the number of parallel workers you are using to execute tempest
+ available in the file.
+ #. Provide tempest with the location of you accounts.yaml file with the
+ test_accounts_file option in the auth section
+
+
+Non-locking test accounts
+"""""""""""""""""""""""""
+When tempest was refactored to allow for locking test accounts, the original
+non-tenant isolated case was converted to support the new accounts.yaml file.
+This mechanism is the non-locking test accounts provider. It only makes sense
+to use it if parallel execution isn't needed. If the role restrictions were too
+limiting with the locking accounts provider and tenant isolation is not wanted
+then you can use the non-locking test accounts credential provider without the
+accounts.yaml file. This is also the currently the default configuration for
+tempest, since it doesn't require elevated permissions or the extra file.
+
+To use the non-locking test accounts provider you have 2 ways to configure it.
+First you can specify the sets of credentials in the configuration file like
+detailed above with following 9 options in the identity section:
+
+ #. username
+ #. password
+ #. tenant_name
+ #. admin_username
+ #. admin_password
+ #. admin_tenant_name
+ #. alt_username
+ #. alt_password
+ #. alt_tenant_name
+
+You should use this if you need to specify credentials with different roles.
+(i.e. you want to use admin credentials) However, this isn't a requirement for
+its usage.
+
+You also can use the accounts.yaml file to specify the credentials used for
+testing. This will just allocate them serially so you only need to provide
+a pair of credentials. Do note that all the restrictions associated with
+locking test accounts applies to using the accounts.yaml file this way too.
+The procedure for doing this is very similar to with the locking accounts
+provider just don't set the locking_credentials_provider to true and you
+only should need a single pair of credentials.
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 1f06bc5..cb1c504 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -29,6 +29,15 @@
field_guide/thirdparty
field_guide/unit_tests
+---------------------------
+Tempest Configuration Guide
+---------------------------
+
+.. toctree::
+ :maxdepth: 2
+
+ configuration
+
---------------------
Command Documentation
---------------------
diff --git a/etc/tempest.conf.sample b/etc/tempest.conf.sample
index bc4198f..d81d3bb 100644
--- a/etc/tempest.conf.sample
+++ b/etc/tempest.conf.sample
@@ -125,7 +125,7 @@
# achieved configuring a list of test accounts (boolean value)
# Deprecated group/name - [compute]/allow_tenant_isolation
# Deprecated group/name - [orchestration]/allow_tenant_isolation
-#allow_tenant_isolation = false
+#allow_tenant_isolation = true
# If set to True it enables the Accounts provider, which locks
# credentials to allow for parallel execution with pre-provisioned
diff --git a/tempest/api/identity/admin/v3/test_trusts.py b/tempest/api/identity/admin/v3/test_trusts.py
index 4b5075c..db64a5b 100644
--- a/tempest/api/identity/admin/v3/test_trusts.py
+++ b/tempest/api/identity/admin/v3/test_trusts.py
@@ -120,7 +120,7 @@
self.assertIsNotNone(trust['id'])
self.assertEqual(impersonate, trust['impersonation'])
if expires is not None:
- # Omit microseconds of the expiry time
+ # Omit microseconds component of the expiry time
trust_expires_at = re.sub(r'\.([0-9]){6}', '', trust['expires_at'])
self.assertEqual(expires, trust_expires_at)
else:
@@ -220,11 +220,14 @@
# with an expiry specified
expires_at = timeutils.utcnow() + datetime.timedelta(hours=1)
# NOTE(ylobankov) In some cases the expiry time may be rounded up
- # because of microseconds. For example, we have the following expiry
- # time for a trust: 2015-02-17T17:34:01.907051Z. However, if we make
- # a GET request on the trust, the response may contain the time
- # rounded up to 2015-02-17T17:34:02.000000Z. That is why we should
- # omit microseconds when creating a trust.
+ # because of microseconds. In fact, it depends on database and its
+ # version. At least MySQL 5.6.16 does this.
+ # For example, when creating a trust, we will set the expiry time of
+ # the trust to 2015-02-17T17:34:01.907051Z. However, if we make a GET
+ # request on the trust, the response will contain the time rounded up
+ # to 2015-02-17T17:34:02.000000Z. That is why we shouldn't set flag
+ # "subsecond" to True when we invoke timeutils.isotime(...) to avoid
+ # problems with rounding.
expires_str = timeutils.isotime(at=expires_at)
trust = self.create_trust(expires=expires_str)
diff --git a/tempest/api_schema/response/compute/flavors.py b/tempest/api_schema/response/compute/flavors.py
index 44020d2..65f2c28 100644
--- a/tempest/api_schema/response/compute/flavors.py
+++ b/tempest/api_schema/response/compute/flavors.py
@@ -30,8 +30,11 @@
},
'required': ['name', 'links', 'id']
}
- }
+ },
+ 'flavors_links': parameter_types.links
},
+ # NOTE(gmann): flavors_links attribute is not necessary
+ # to be present always So it is not 'required'.
'required': ['flavors']
}
}
diff --git a/tempest/api_schema/response/compute/v2/flavors.py b/tempest/api_schema/response/compute/v2/flavors.py
index 811ea84..76c4cee 100644
--- a/tempest/api_schema/response/compute/v2/flavors.py
+++ b/tempest/api_schema/response/compute/v2/flavors.py
@@ -15,6 +15,7 @@
import copy
from tempest.api_schema.response.compute import flavors
+from tempest.api_schema.response.compute import parameter_types
list_flavors_details = copy.deepcopy(flavors.common_flavor_list_details)
@@ -23,6 +24,12 @@
list_flavors_details['response_body']['properties']['flavors']['items'][
'properties']['swap'] = {'type': ['string', 'integer']}
+# Defining 'flavors_links' attributes for V2 flavor schema
+list_flavors_details['response_body'][
+ 'properties'].update({'flavors_links': parameter_types.links})
+# NOTE(gmann): flavors_links attribute is not necessary to be
+# present always So it is not 'required'.
+
# Defining extra attributes for V2 flavor schema
list_flavors_details['response_body']['properties']['flavors']['items'][
'properties'].update({'OS-FLV-DISABLED:disabled': {'type': 'boolean'},
diff --git a/tempest/api_schema/response/compute/v2/floating_ips.py b/tempest/api_schema/response/compute/v2/floating_ips.py
index def0a78..7250773 100644
--- a/tempest/api_schema/response/compute/v2/floating_ips.py
+++ b/tempest/api_schema/response/compute/v2/floating_ips.py
@@ -146,8 +146,14 @@
'instance_uuid': {'type': ['string', 'null']},
'interface': {'type': ['string', 'null']},
'pool': {'type': ['string', 'null']},
- 'project_id': {'type': ['string', 'null']}
+ 'project_id': {'type': ['string', 'null']},
+ 'fixed_ip': {
+ 'type': ['string', 'null'],
+ 'format': 'ip-address'
+ }
},
+ # NOTE: fixed_ip is introduced after JUNO release,
+ # So it is not defined as 'required'.
'required': ['address', 'instance_uuid', 'interface',
'pool', 'project_id']
}
diff --git a/tempest/api_schema/response/compute/v2/limits.py b/tempest/api_schema/response/compute/v2/limits.py
index b9857f1..a7decb7 100644
--- a/tempest/api_schema/response/compute/v2/limits.py
+++ b/tempest/api_schema/response/compute/v2/limits.py
@@ -38,8 +38,15 @@
'maxSecurityGroupRules': {'type': 'integer'},
'maxTotalKeypairs': {'type': 'integer'},
'totalRAMUsed': {'type': 'integer'},
- 'totalInstancesUsed': {'type': 'integer'}
+ 'totalInstancesUsed': {'type': 'integer'},
+ 'maxServerGroupMembers': {'type': 'integer'},
+ 'maxServerGroups': {'type': 'integer'},
+ 'totalServerGroupsUsed': {'type': 'integer'}
},
+ # NOTE(gmann): maxServerGroupMembers, maxServerGroups
+ # and totalServerGroupsUsed are API extension,
+ # and some environments return a response without these
+ # attributes.So they are not 'required'.
'required': ['maxImageMeta',
'maxPersonality',
'maxPersonalitySize',
diff --git a/tempest/api_schema/response/compute/v2/servers.py b/tempest/api_schema/response/compute/v2/servers.py
index 09abaed..0132350 100644
--- a/tempest/api_schema/response/compute/v2/servers.py
+++ b/tempest/api_schema/response/compute/v2/servers.py
@@ -30,10 +30,10 @@
'links': parameter_types.links,
'OS-DCF:diskConfig': {'type': 'string'}
},
- # NOTE: OS-DCF:diskConfig is API extension, and some
- # environments return a response without the attribute.
- # So it is not 'required'.
- 'required': ['id', 'security_groups', 'links']
+ # NOTE: OS-DCF:diskConfig & security_groups are API extension,
+ # and some environments return a response without these
+ # attributes.So they are not 'required'.
+ 'required': ['id', 'links']
}
},
'required': ['server']
@@ -64,6 +64,7 @@
get_server['response_body']['properties']['server']['properties'].update({
'key_name': {'type': ['string', 'null']},
'hostId': {'type': 'string'},
+ 'security_groups': {'type': 'array'},
# NOTE: Non-admin users also can see "OS-SRV-USG" and "OS-EXT-AZ"
# attributes.
@@ -288,10 +289,11 @@
'properties'].update({
'hostId': {'type': 'string'},
'OS-DCF:diskConfig': {'type': 'string'},
+ 'security_groups': {'type': 'array'},
'accessIPv4': parameter_types.access_ip_v4,
'accessIPv6': parameter_types.access_ip_v6
})
-# NOTE(GMann): OS-DCF:diskConfig and accessIPv4/v6 are API
+# NOTE(GMann): OS-DCF:diskConfig, security_groups and accessIPv4/v6 are API
# extensions, and some environments return a response
# without these attributes. So they are not 'required'.
list_servers_detail['response_body']['properties']['servers']['items'][
diff --git a/tempest/config.py b/tempest/config.py
index 4588b20..a66aa9b 100644
--- a/tempest/config.py
+++ b/tempest/config.py
@@ -40,7 +40,7 @@
help="Path to the yaml file that contains the list of "
"credentials to use for running tests"),
cfg.BoolOpt('allow_tenant_isolation',
- default=False,
+ default=True,
help="Allows test cases to create/destroy tenants and "
"users. This option requires that OpenStack Identity "
"API admin credentials are known. If false, isolated "
diff --git a/tempest/scenario/test_network_basic_ops.py b/tempest/scenario/test_network_basic_ops.py
index f29fecf..4199b2c 100644
--- a/tempest/scenario/test_network_basic_ops.py
+++ b/tempest/scenario/test_network_basic_ops.py
@@ -435,13 +435,15 @@
"admin_state_up of router to True")
@test.idempotent_id('d8bb918e-e2df-48b2-97cd-b73c95450980')
+ @testtools.skipIf(CONF.baremetal.driver_enabled,
+ 'network isolation not available for baremetal nodes')
@testtools.skipUnless(CONF.scenario.dhcp_client,
"DHCP client is not available.")
@test.attr(type='smoke')
@test.services('compute', 'network')
def test_subnet_details(self):
"""Tests that subnet's extra configuration details are affecting
- the VMs
+ the VMs. This test relies on non-shared, isolated tenant networks.
NOTE: Neutron subnets push data to servers via dhcp-agent, so any
update in subnet requires server to actively renew its DHCP lease.