Merge "Rename an internal method to _helper_list()"
diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst
index 4fe2e9f..e21deea 100644
--- a/doc/source/configuration.rst
+++ b/doc/source/configuration.rst
@@ -40,6 +40,24 @@
Eventually the config options for providing credentials to tempest will be
deprecated and removed in favor of the accounts.yaml file.
+Keystone Connection Info
+^^^^^^^^^^^^^^^^^^^^^^^^
+In order for tempest to be able to talk to your OpenStack deployment you need
+to provide it with information about how it communicates with keystone.
+This involves configuring the following options in the identity section:
+
+ #. auth_version
+ #. uri
+ #. uri_v3
+
+The *auth_version* option is used to tell tempest whether it should be using
+keystone's v2 or v3 api for communicating with keystone. (except for the
+identity api tests which will test a specific version) The 2 uri options are
+used to tell tempest the url of the keystone endpoint. The *uri* option is used
+for keystone v2 request and *uri_v3* is used for keystone v3. You want to ensure
+that which ever version you set for *auth_version* has its uri option defined.
+
+
Credential Provider Mechanisms
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -123,3 +141,233 @@
is that if a test requires specific roles on accounts these tests can not be
run. This is because the config options do not give sufficient flexibility to
describe the roles assigned to a user for running the tests.
+
+Compute
+-------
+
+Flavors
+^^^^^^^
+For tempest to be able to create servers you need to specify flavors that it
+can use to boot the servers with. There are 2 options in the tempest config
+for doing this:
+
+ #. flavor_ref
+ #. flavor_ref_alt
+
+Both of these options are in the compute section of the config file and take
+in the flavor id (not the name) from nova. The *flavor_ref* option is what will
+be used for booting almost all of the guests, *flavor_ref_alt* is only used in
+tests where 2 different sized servers are required. (for example a resize test)
+
+Using a smaller flavor is generally recommended, when larger flavors are used
+the extra time required to bring up servers will likely affect total run time
+and probably require tweaking timeout values to ensure tests have ample time to
+finish.
+
+Images
+^^^^^^
+Just like with flavors, tempest needs to know which images to use for booting
+servers. There are 2 options in the compute section just like with flavors:
+
+ #. image_ref
+ #. image_ref_alt
+
+Both options are expecting an image id (not name) from nova. The *image_ref*
+option is what what will be used for booting the majority of servers in tempest.
+*image_ref_alt* is used for tests that require 2 images such as rebuild. If 2
+images are not available you can set both options to the same image_ref and
+those tests will be skipped.
+
+There are also options in the scenario section for images:
+
+ #. img_file
+ #. img_dir
+ #. aki_img_file
+ #. ari_img_file
+ #. ami_img_file
+ #. img_container_format
+ #. img_disk_format
+
+however unlike the other image options these are used for a very small subset
+of scenario tests which are uploading an image. These options are used to tell
+tempest where an image file is located and describe it's metadata for when it's
+uploaded.
+
+The behavior of these options is a bit convoluted (which will likely be fixed
+in future versions). You first need to specify *img_dir*, which is the directory
+tempest will look for the image files in. First it will check if the filename
+set for *img_file* could be found in *img_dir*. If it is found then the
+*img_container_format* and *img_disk_format* options are used to upload that
+image to glance. However if it's not found tempest will look for the 3 uec image
+file name options as a fallback. If neither is found the tests requiring an
+image to upload will fail.
+
+It is worth pointing out that using `cirros`_ is a very good choice for running
+tempest. It's what is used for upstream testing, they boot quickly and have a
+small footprint.
+
+.. _cirros: https://launchpad.net/cirros
+
+Networking
+----------
+OpenStack has a myriad of different networking configurations possible and
+depending on which of the 2 network backends, nova-network or neutron, you are
+using things can vary drastically. Due to this complexity Tempest has to provide
+a certain level of flexibility in it's configuration to ensure it will work
+against any cloud. This ends up causing a large number of permutations in
+Tempest's config around network configuration.
+
+
+Enabling Remote Access to Created Servers
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+When Tempest creates servers for testing, some tests require being able to
+connect those servers. Depending on the configuration of the cloud, the methods
+for doing this can be different. In certain configurations it is required to
+specify a single network with server create calls. Accordingly, Tempest provides
+a few different methods for providing this information in configuration to try
+and ensure that regardless of the clouds configuration it'll still be able to
+run. This section covers the different methods of configuring Tempest to provide
+a network when creating servers.
+
+Fixed Network Name
+""""""""""""""""""
+This is the simplest method of specifying how networks should be used. You can
+just specify a single network name/label to use for all server creations. The
+limitation with this is that all tenants/projects and users must be able to see
+that network name/label if they were to perform a network list and be able to
+use it.
+
+If no network name is assigned in the config file and none of the below
+alternatives are used, then Tempest will not specify a network on server
+creations, which depending on the cloud configuration might prevent them from
+booting.
+
+To set a fixed network name simply do:
+
+ #. Set the fixed_network_name option in the compute group
+
+In the case that the configured fixed network name can not be found by a user
+network list call, it will be treated like one was not provided except that a
+warning will be logged stating that it couldn't be found.
+
+
+Accounts File
+"""""""""""""
+If you are using an accounts file to provide credentials for running Tempest
+then you can leverage it to also specify which network should be used with
+server creations on a per tenant/project and user pair basis. This provides
+the necessary flexibility to work with more intricate networking configurations
+by enabling the user to specify exactly which network to use for which
+tenants/projects. You can refer to the accounts.yaml sample file included in
+the tempest repo for the syntax around specifying networks in the file.
+
+However, specifying a network is not required when using an accounts file. If
+one is not specified you can use a fixed network name to specify the network to
+use when creating servers just as without an accounts file. However, any network
+specified in the accounts file will take precedence over the fixed network name
+provided. If no network is provided in the accounts file and a fixed network
+name is not set then no network will be included in create server requests.
+
+If a fixed network is provided and the accounts.yaml file also contains networks
+this has the benefit of enabling a couple more tests which require a static
+network to perform operations like server lists with a network filter. If a
+fixed network name is not provided these tests are skipped. Additionally, if a
+fixed network name is provided it will serve as a fallback in case of a
+misconfiguration or a missing network in the accounts file.
+
+
+With Tenant Isolation
+"""""""""""""""""""""
+With tenant isolation enabled and using nova-network then nothing changes. Your
+only option for configuration is to either set a fixed network name or not.
+However, in most cases it shouldn't matter because nova-network should have no
+problem booting a server with multiple networks. If this is not the case for
+your cloud then using an accounts file is recommended because it provides the
+necessary flexibility to describe your configuration. Tenant isolation is not
+able to dynamically allocate things as necessary if neutron is not enabled.
+
+With neutron and tenant isolation enabled there should not be any additional
+configuration necessary to enable Tempest to create servers with working
+networking, assuming you have properly configured the network section to work
+for your cloud. Tempest will dynamically create the neutron resources necessary
+to enable using servers with that network. Also, just as with the accounts
+file, if you specify a fixed network name while using neutron and tenant
+isolation it will enable running tests which require a static network and it
+will additionally be used as a fallback for server creation. However, unlike
+accounts.yaml this should never be triggered.
+
+Configuring Available Services
+------------------------------
+OpenStack is really a constellation of several different projects which
+are running together to create a cloud. However which projects you're running
+is not set in stone, and which services are running is up to the deployer.
+Tempest however needs to know which services are available so it can figure
+out which tests it is able to run and certain setup steps which differ based
+on the available services.
+
+The *service_available* section of the config file is used to set which
+services are available. It contains a boolean option for each service (except
+for keystone which is a hard requirement) set it to True if the service is
+available or False if it is not.
+
+Service Catalog
+^^^^^^^^^^^^^^^
+Each project which has its own REST API contains an entry in the service
+catalog. Like most things in OpenStack this is also completely configurable.
+However, for tempest to be able to figure out the endpoints to send REST API
+calls for each service to it needs to know how that project is defined in the
+service catalog. There are 3 options for each service section to accomplish
+this:
+
+ #. catalog_type
+ #. endpoint_type
+ #. region
+
+Setting *catalog_type* and *endpoint_type* should normally give Tempest enough
+information to determine which endpoint it should pull from the service
+catalog to use for talking to that particular service. However, if you're cloud
+has multiple regions available and you need to specify a particular one to use
+a service you can set the *region* option in that service's section.
+
+It should also be noted that the default values for these options are set
+to what devstack uses. (which is a de facto standard for service catalog
+entries) So often nothing actually needs to be set on these options to enable
+communication to a particular service. It is only if you are either not using
+the same *catalog_type* as devstack or you want Tempest to talk to a different
+endpoint type instead of publicURL for a service that these need to be changed.
+
+
+Service feature configuration
+-----------------------------
+
+OpenStack provides its deployers a myriad of different configuration options
+to enable anyone deploying it to create a cloud tailor-made for any individual
+use case. It provides options for several different backend type, databases,
+message queues, etc. However, the downside to this configurability is that
+certain operations and features aren't supported depending on the configuration.
+These features may or may not be discoverable from the API so the burden is
+often on the user to figure out what the cloud they're talking to supports.
+Besides the obvious interoperability issues with this it also leaves Tempest
+in an interesting situation trying to figure out which tests are expected to
+work. However, Tempest tests do not rely on dynamic api discovery for a feature
+(assuming one exists). Instead Tempest has to be explicitly configured as to
+which optional features are enabled. This is in order to prevent bugs in the
+discovery mechanisms from masking failures.
+
+The service feature-enabled config sections are how Tempest addresses the
+optional feature question. Each service that has tests for optional features
+contains one of these sections. The only options in it are boolean options
+with the name of a feature which is used. If it is set to false any test which
+depends on that functionality will be skipped. For a complete list of all these
+options refer to the sample config file.
+
+
+API Extensions
+^^^^^^^^^^^^^^
+The service feature-enabled sections often contain an *api-extensions* option
+(or in the case of swift a *discoverable_apis* option) this is used to tell
+tempest which api extensions (or configurable middleware) is used in your
+deployment. It has 2 valid config states, either it contains a single value
+"all" (which is the default) which means that every api extension is assumed
+to be enabled, or it is set to a list of each individual extension that is
+enabled for that service.
diff --git a/etc/accounts.yaml.sample b/etc/accounts.yaml.sample
index 64ff8a7..31ceb33 100644
--- a/etc/accounts.yaml.sample
+++ b/etc/accounts.yaml.sample
@@ -33,3 +33,5 @@
password: 'test_password'
types:
- 'admin'
+ resources:
+ network: 'public'
diff --git a/etc/tempest.conf.sample b/etc/tempest.conf.sample
index 175f0d9..2a72635 100644
--- a/etc/tempest.conf.sample
+++ b/etc/tempest.conf.sample
@@ -253,10 +253,6 @@
# image. (string value)
#image_alt_ssh_user = root
-# Password used to authenticate to an instance using the alternate
-# image. (string value)
-#image_alt_ssh_password = password
-
# Time in seconds between build status checks. (integer value)
#build_interval = 1
@@ -269,16 +265,16 @@
#run_ssh = false
# Auth method used for authenticate to the instance. Valid choices
-# are: keypair, configured, adminpass. keypair: start the servers with
-# an ssh keypair. configured: use the configured user and password.
-# adminpass: use the injected adminPass. disabled: avoid using ssh
-# when it is an option. (string value)
+# are: keypair, configured, adminpass and disabled. Keypair: start the
+# servers with a ssh keypair. Configured: use the configured user and
+# password. Adminpass: use the injected adminPass. Disabled: avoid
+# using ssh when it is an option. (string value)
#ssh_auth_method = keypair
# How to connect to the instance? fixed: using the first ip belongs
-# the fixed network floating: creating and using a floating ip (string
-# value)
-#ssh_connect_method = fixed
+# the fixed network floating: creating and using a floating ip.
+# (string value)
+#ssh_connect_method = floating
# User name used to authenticate to an instance. (string value)
#ssh_user = root
@@ -286,6 +282,14 @@
# Timeout in seconds to wait for ping to succeed. (integer value)
#ping_timeout = 120
+# The packet size for ping packets originating from remote linux hosts
+# (integer value)
+#ping_size = 56
+
+# The number of ping packets originating from remote linux hosts
+# (integer value)
+#ping_count = 1
+
# Timeout in seconds to wait for authentication to succeed. (integer
# value)
#ssh_timeout = 300
@@ -301,7 +305,8 @@
# Name of the fixed network that is visible to all test tenants. If
# multiple networks are available for a tenant this is the network
# which will be used for creating servers if tempest does not create a
-# network or a network is not specified elsewhere (string value)
+# network or a network is not specified elsewhere. It may be used for
+# ssh validation only if floating IPs are disabled. (string value)
#fixed_network_name = <None>
# Network used for SSH connections. Ignored if
@@ -326,10 +331,6 @@
# Allowed values: public, admin, internal, publicURL, adminURL, internalURL
#endpoint_type = publicURL
-# Path to a private key file for SSH access to remote hosts (string
-# value)
-#path_to_private_key = <None>
-
# Expected device name when a volume is attached to an instance
# (string value)
#volume_device_name = vdb
@@ -746,14 +747,19 @@
# The mask bits for tenant ipv6 subnets (integer value)
#tenant_network_v6_mask_bits = 64
-# Whether tenant network connectivity should be evaluated directly
-# (boolean value)
+# Whether tenant networks can be reached directly from the test
+# client. This must be set to True when the 'fixed' ssh_connect_method
+# is selected. (boolean value)
#tenant_networks_reachable = false
# Id of the public network that provides external connectivity (string
# value)
#public_network_id =
+# Default floating network name. Used to allocate floating IPs when
+# neutron is enabled. (string value)
+#floating_network_name = <None>
+
# Id of the public router that provides external connectivity. This
# should only be used when Neutron's 'allow_overlapping_ips' is set to
# 'False' in neutron.conf. usually not needed past 'Grizzly' release
@@ -796,6 +802,10 @@
# attributes ipv6_ra_mode and ipv6_address_mode (boolean value)
#ipv6_subnet_attributes = false
+# Does the test environment support changing port admin state (boolean
+# value)
+#port_admin_state_change = true
+
[object-storage]
@@ -1071,6 +1081,38 @@
#too_slow_to_test = true
+[validation]
+
+#
+# From tempest.config
+#
+
+# Default IP type used for validation: -fixed: uses the first IP
+# belonging to the fixed network -floating: creates and uses a
+# floating IP (string value)
+# Allowed values: fixed, floating
+#connect_method = floating
+
+# Default authentication method to the instance. Only ssh via keypair
+# is supported for now. Additional methods will be handled in a
+# separate spec. (string value)
+# Allowed values: keypair
+#auth_method = keypair
+
+# Default IP version for ssh connections. (integer value)
+#ip_version_for_ssh = 4
+
+# Timeout in seconds to wait for ping to succeed. (integer value)
+#ping_timeout = 120
+
+# Timeout in seconds to wait for the TCP connection to be successful.
+# (integer value)
+#connect_timeout = 60
+
+# Timeout in seconds to wait for the ssh banner. (integer value)
+#ssh_timeout = 300
+
+
[volume]
#
diff --git a/requirements.txt b/requirements.txt
index bf7471e..0d7fc0d 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -12,7 +12,6 @@
python-glanceclient>=0.15.0
python-cinderclient>=1.1.0
python-heatclient>=0.3.0
-python-swiftclient>=2.2.0
testrepository>=0.0.18
oslo.concurrency>=1.8.0,<1.9.0 # Apache-2.0
oslo.config>=1.9.3,<1.10.0 # Apache-2.0
diff --git a/tempest/api/compute/servers/test_list_server_filters.py b/tempest/api/compute/servers/test_list_server_filters.py
index f33204d..eccd600 100644
--- a/tempest/api/compute/servers/test_list_server_filters.py
+++ b/tempest/api/compute/servers/test_list_server_filters.py
@@ -69,7 +69,10 @@
network = cls.get_tenant_network()
if network:
- cls.fixed_network_name = network['name']
+ if network.get('name'):
+ cls.fixed_network_name = network['name']
+ else:
+ cls.fixed_network_name = None
else:
cls.fixed_network_name = None
network_kwargs = fixed_network.set_networks_kwarg(network)
diff --git a/tempest/api/data_processing/base.py b/tempest/api/data_processing/base.py
index d91fbaa..5a903b7 100644
--- a/tempest/api/data_processing/base.py
+++ b/tempest/api/data_processing/base.py
@@ -12,14 +12,216 @@
# License for the specific language governing permissions and limitations
# under the License.
+from collections import OrderedDict
+
+import six
from tempest_lib import exceptions as lib_exc
from tempest import config
+from tempest import exceptions
import tempest.test
CONF = config.CONF
+"""Default templates.
+There should always be at least a master1 and a worker1 node
+group template."""
+DEFAULT_TEMPLATES = {
+ 'vanilla': OrderedDict([
+ ('2.6.0', {
+ 'NODES': {
+ 'master1': {
+ 'count': 1,
+ 'node_processes': ['namenode', 'resourcemanager',
+ 'hiveserver']
+ },
+ 'master2': {
+ 'count': 1,
+ 'node_processes': ['oozie', 'historyserver',
+ 'secondarynamenode']
+ },
+ 'worker1': {
+ 'count': 1,
+ 'node_processes': ['datanode', 'nodemanager'],
+ 'node_configs': {
+ 'MapReduce': {
+ 'yarn.app.mapreduce.am.resource.mb': 256,
+ 'yarn.app.mapreduce.am.command-opts': '-Xmx256m'
+ },
+ 'YARN': {
+ 'yarn.scheduler.minimum-allocation-mb': 256,
+ 'yarn.scheduler.maximum-allocation-mb': 1024,
+ 'yarn.nodemanager.vmem-check-enabled': False
+ }
+ }
+ }
+ },
+ 'cluster_configs': {
+ 'HDFS': {
+ 'dfs.replication': 1
+ }
+ }
+ }),
+ ('1.2.1', {
+ 'NODES': {
+ 'master1': {
+ 'count': 1,
+ 'node_processes': ['namenode', 'jobtracker']
+ },
+ 'worker1': {
+ 'count': 1,
+ 'node_processes': ['datanode', 'tasktracker'],
+ 'node_configs': {
+ 'HDFS': {
+ 'Data Node Heap Size': 1024
+ },
+ 'MapReduce': {
+ 'Task Tracker Heap Size': 1024
+ }
+ }
+ }
+ },
+ 'cluster_configs': {
+ 'HDFS': {
+ 'dfs.replication': 1
+ },
+ 'MapReduce': {
+ 'mapred.map.tasks.speculative.execution': False,
+ 'mapred.child.java.opts': '-Xmx500m'
+ },
+ 'general': {
+ 'Enable Swift': False
+ }
+ }
+ })
+ ]),
+ 'hdp': OrderedDict([
+ ('2.0.6', {
+ 'NODES': {
+ 'master1': {
+ 'count': 1,
+ 'node_processes': ['NAMENODE', 'SECONDARY_NAMENODE',
+ 'ZOOKEEPER_SERVER', 'AMBARI_SERVER',
+ 'HISTORYSERVER', 'RESOURCEMANAGER',
+ 'GANGLIA_SERVER', 'NAGIOS_SERVER',
+ 'OOZIE_SERVER']
+ },
+ 'worker1': {
+ 'count': 1,
+ 'node_processes': ['HDFS_CLIENT', 'DATANODE',
+ 'YARN_CLIENT', 'ZOOKEEPER_CLIENT',
+ 'MAPREDUCE2_CLIENT', 'NODEMANAGER',
+ 'PIG', 'OOZIE_CLIENT']
+ }
+ },
+ 'cluster_configs': {
+ 'HDFS': {
+ 'dfs.replication': 1
+ }
+ }
+ })
+ ]),
+ 'spark': OrderedDict([
+ ('1.0.0', {
+ 'NODES': {
+ 'master1': {
+ 'count': 1,
+ 'node_processes': ['namenode', 'master']
+ },
+ 'worker1': {
+ 'count': 1,
+ 'node_processes': ['datanode', 'slave']
+ }
+ },
+ 'cluster_configs': {
+ 'HDFS': {
+ 'dfs.replication': 1
+ }
+ }
+ })
+ ]),
+ 'cdh': OrderedDict([
+ ('5.3.0', {
+ 'NODES': {
+ 'master1': {
+ 'count': 1,
+ 'node_processes': ['CLOUDERA_MANAGER']
+ },
+ 'master2': {
+ 'count': 1,
+ 'node_processes': ['HDFS_NAMENODE',
+ 'YARN_RESOURCEMANAGER']
+ },
+ 'master3': {
+ 'count': 1,
+ 'node_processes': ['OOZIE_SERVER', 'YARN_JOBHISTORY',
+ 'HDFS_SECONDARYNAMENODE',
+ 'HIVE_METASTORE', 'HIVE_SERVER2']
+ },
+ 'worker1': {
+ 'count': 1,
+ 'node_processes': ['YARN_NODEMANAGER', 'HDFS_DATANODE']
+ }
+ },
+ 'cluster_configs': {
+ 'HDFS': {
+ 'dfs_replication': 1
+ }
+ }
+ }),
+ ('5', {
+ 'NODES': {
+ 'master1': {
+ 'count': 1,
+ 'node_processes': ['CLOUDERA_MANAGER']
+ },
+ 'master2': {
+ 'count': 1,
+ 'node_processes': ['HDFS_NAMENODE',
+ 'YARN_RESOURCEMANAGER']
+ },
+ 'master3': {
+ 'count': 1,
+ 'node_processes': ['OOZIE_SERVER', 'YARN_JOBHISTORY',
+ 'HDFS_SECONDARYNAMENODE',
+ 'HIVE_METASTORE', 'HIVE_SERVER2']
+ },
+ 'worker1': {
+ 'count': 1,
+ 'node_processes': ['YARN_NODEMANAGER', 'HDFS_DATANODE']
+ }
+ },
+ 'cluster_configs': {
+ 'HDFS': {
+ 'dfs_replication': 1
+ }
+ }
+ })
+ ]),
+ 'mapr': OrderedDict([
+ ('4.0.1.mrv2', {
+ 'NODES': {
+ 'master1': {
+ 'count': 1,
+ 'node_processes': ['CLDB', 'FileServer', 'ZooKeeper',
+ 'NodeManager', 'ResourceManager',
+ 'HistoryServer', 'Oozie']
+ },
+ 'worker1': {
+ 'count': 1,
+ 'node_processes': ['FileServer', 'NodeManager', 'Pig']
+ }
+ },
+ 'cluster_configs': {
+ 'Hive': {
+ 'Hive Version': '0.13',
+ }
+ }
+ })
+ ]),
+}
+
class BaseDataProcessingTest(tempest.test.BaseTestCase):
@@ -28,6 +230,7 @@
super(BaseDataProcessingTest, cls).skip_checks()
if not CONF.service_available.sahara:
raise cls.skipException('Sahara support is required')
+ cls.default_plugin = cls._get_default_plugin()
@classmethod
def setup_credentials(cls):
@@ -43,6 +246,10 @@
def resource_setup(cls):
super(BaseDataProcessingTest, cls).resource_setup()
+ cls.default_version = cls._get_default_version()
+ if cls.default_plugin is not None and cls.default_version is None:
+ raise exceptions.InvalidConfiguration(
+ message="No known Sahara plugin version was found")
cls.flavor_ref = CONF.compute.flavor_ref
# add lists for watched resources
@@ -172,3 +379,100 @@
cls._jobs.append(resp_body['id'])
return resp_body
+
+ @classmethod
+ def _get_default_plugin(cls):
+ """Returns the default plugin used for testing."""
+ if len(CONF.data_processing_feature_enabled.plugins) == 0:
+ return None
+
+ for plugin in CONF.data_processing_feature_enabled.plugins:
+ if plugin in DEFAULT_TEMPLATES.keys():
+ break
+ else:
+ plugin = ''
+ return plugin
+
+ @classmethod
+ def _get_default_version(cls):
+ """Returns the default plugin version used for testing.
+ This is gathered separately from the plugin to allow
+ the usage of plugin name in skip_checks. This method is
+ rather invoked into resource_setup, which allows API calls
+ and exceptions.
+ """
+ if not cls.default_plugin:
+ return None
+ plugin = cls.client.get_plugin(cls.default_plugin)
+
+ for version in DEFAULT_TEMPLATES[cls.default_plugin].keys():
+ if version in plugin['versions']:
+ break
+ else:
+ version = None
+
+ return version
+
+ @classmethod
+ def get_node_group_template(cls, nodegroup='worker1'):
+ """Returns a node group template for the default plugin."""
+ try:
+ plugin_data = (
+ DEFAULT_TEMPLATES[cls.default_plugin][cls.default_version]
+ )
+ nodegroup_data = plugin_data['NODES'][nodegroup]
+ node_group_template = {
+ 'description': 'Test node group template',
+ 'plugin_name': cls.default_plugin,
+ 'hadoop_version': cls.default_version,
+ 'node_processes': nodegroup_data['node_processes'],
+ 'flavor_id': cls.flavor_ref,
+ 'node_configs': nodegroup_data.get('node_configs', {}),
+ }
+ return node_group_template
+ except (IndexError, KeyError):
+ return None
+
+ @classmethod
+ def get_cluster_template(cls, node_group_template_ids=None):
+ """Returns a cluster template for the default plugin.
+ node_group_template_defined contains the type and ID of pre-defined
+ node group templates that have to be used in the cluster template
+ (instead of dynamically defining them with 'node_processes').
+ """
+ if node_group_template_ids is None:
+ node_group_template_ids = {}
+ try:
+ plugin_data = (
+ DEFAULT_TEMPLATES[cls.default_plugin][cls.default_version]
+ )
+
+ all_node_groups = []
+ for ng_name, ng_data in six.iteritems(plugin_data['NODES']):
+ node_group = {
+ 'name': '%s-node' % (ng_name),
+ 'flavor_id': cls.flavor_ref,
+ 'count': ng_data['count']
+ }
+ if ng_name in node_group_template_ids.keys():
+ # node group already defined, use it
+ node_group['node_group_template_id'] = (
+ node_group_template_ids[ng_name]
+ )
+ else:
+ # node_processes list defined on-the-fly
+ node_group['node_processes'] = ng_data['node_processes']
+ if 'node_configs' in ng_data:
+ node_group['node_configs'] = ng_data['node_configs']
+ all_node_groups.append(node_group)
+
+ cluster_template = {
+ 'description': 'Test cluster template',
+ 'plugin_name': cls.default_plugin,
+ 'hadoop_version': cls.default_version,
+ 'cluster_configs': plugin_data.get('cluster_configs', {}),
+ 'node_groups': all_node_groups,
+ }
+ return cluster_template
+ except (IndexError, KeyError):
+ return None
diff --git a/tempest/api/data_processing/test_cluster_templates.py b/tempest/api/data_processing/test_cluster_templates.py
index 8a63c3f..cebf493 100644
--- a/tempest/api/data_processing/test_cluster_templates.py
+++ b/tempest/api/data_processing/test_cluster_templates.py
@@ -15,6 +15,7 @@
from tempest_lib.common.utils import data_utils
from tempest.api.data_processing import base as dp_base
+from tempest import exceptions
from tempest import test
@@ -23,55 +24,30 @@
sahara/restapi/rest_api_v1.0.html#cluster-templates
"""
@classmethod
+ def skip_checks(cls):
+ super(ClusterTemplateTest, cls).skip_checks()
+ if cls.default_plugin is None:
+ raise cls.skipException("No Sahara plugins configured")
+
+ @classmethod
def resource_setup(cls):
super(ClusterTemplateTest, cls).resource_setup()
- # create node group template
- node_group_template = {
- 'name': data_utils.rand_name('sahara-ng-template'),
- 'description': 'Test node group template',
- 'plugin_name': 'vanilla',
- 'hadoop_version': '1.2.1',
- 'node_processes': ['datanode'],
- 'flavor_id': cls.flavor_ref,
- 'node_configs': {
- 'HDFS': {
- 'Data Node Heap Size': 1024
- }
- }
- }
- resp_body = cls.create_node_group_template(**node_group_template)
- node_group_template_id = resp_body['id']
- cls.full_cluster_template = {
- 'description': 'Test cluster template',
- 'plugin_name': 'vanilla',
- 'hadoop_version': '1.2.1',
- 'cluster_configs': {
- 'HDFS': {
- 'dfs.replication': 2
- },
- 'MapReduce': {
- 'mapred.map.tasks.speculative.execution': False,
- 'mapred.child.java.opts': '-Xmx500m'
- },
- 'general': {
- 'Enable Swift': False
- }
- },
- 'node_groups': [
- {
- 'name': 'master-node',
- 'flavor_id': cls.flavor_ref,
- 'node_processes': ['namenode'],
- 'count': 1
- },
- {
- 'name': 'worker-node',
- 'node_group_template_id': node_group_template_id,
- 'count': 3
- }
- ]
- }
+ # pre-define a node group templates
+ node_group_template_w = cls.get_node_group_template('worker1')
+ if node_group_template_w is None:
+ raise exceptions.InvalidConfiguration(
+ message="No known Sahara plugin was found")
+
+ node_group_template_w['name'] = data_utils.rand_name(
+ 'sahara-ng-template')
+ resp_body = cls.create_node_group_template(**node_group_template_w)
+ node_group_template_id = resp_body['id']
+ configured_node_group_templates = {'worker1': node_group_template_id}
+
+ cls.full_cluster_template = cls.get_cluster_template(
+ configured_node_group_templates)
+
# create cls.cluster_template variable to use for comparison to cluster
# template response body. The 'node_groups' field in the response body
# has some extra info that post body does not have. The 'node_groups'
diff --git a/tempest/api/data_processing/test_node_group_templates.py b/tempest/api/data_processing/test_node_group_templates.py
index d7381f4..4068027 100644
--- a/tempest/api/data_processing/test_node_group_templates.py
+++ b/tempest/api/data_processing/test_node_group_templates.py
@@ -19,27 +19,16 @@
class NodeGroupTemplateTest(dp_base.BaseDataProcessingTest):
+
+ @classmethod
+ def skip_checks(cls):
+ super(NodeGroupTemplateTest, cls).skip_checks()
+ if cls.default_plugin is None:
+ raise cls.skipException("No Sahara plugins configured")
+
@classmethod
def resource_setup(cls):
super(NodeGroupTemplateTest, cls).resource_setup()
- cls.node_group_template = {
- 'description': 'Test node group template',
- 'plugin_name': 'vanilla',
- 'hadoop_version': '1.2.1',
- 'node_processes': [
- 'datanode',
- 'tasktracker'
- ],
- 'flavor_id': cls.flavor_ref,
- 'node_configs': {
- 'HDFS': {
- 'Data Node Heap Size': 1024
- },
- 'MapReduce': {
- 'Task Tracker Heap Size': 1024
- }
- }
- }
def _create_node_group_template(self, template_name=None):
"""Creates Node Group Template with optional name specified.
@@ -47,6 +36,10 @@
It creates template, ensures template name and response body.
Returns id and name of created template.
"""
+ self.node_group_template = self.get_node_group_template()
+ self.assertIsNotNone(self.node_group_template,
+ "No known Sahara plugin was found")
+
if not template_name:
# generate random name if it's not specified
template_name = data_utils.rand_name('sahara-ng-template')
diff --git a/tempest/api_schema/response/compute/flavors.py b/tempest/api_schema/response/compute/flavors.py
deleted file mode 100644
index 65f2c28..0000000
--- a/tempest/api_schema/response/compute/flavors.py
+++ /dev/null
@@ -1,80 +0,0 @@
-# Copyright 2014 NEC Corporation. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
-# not use this file except in compliance with the License. You may obtain
-# a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-# License for the specific language governing permissions and limitations
-# under the License.
-
-from tempest.api_schema.response.compute import parameter_types
-
-list_flavors = {
- 'status_code': [200],
- 'response_body': {
- 'type': 'object',
- 'properties': {
- 'flavors': {
- 'type': 'array',
- 'items': {
- 'type': 'object',
- 'properties': {
- 'name': {'type': 'string'},
- 'links': parameter_types.links,
- 'id': {'type': 'string'}
- },
- 'required': ['name', 'links', 'id']
- }
- },
- 'flavors_links': parameter_types.links
- },
- # NOTE(gmann): flavors_links attribute is not necessary
- # to be present always So it is not 'required'.
- 'required': ['flavors']
- }
-}
-
-common_flavor_info = {
- 'type': 'object',
- 'properties': {
- 'name': {'type': 'string'},
- 'links': parameter_types.links,
- 'ram': {'type': 'integer'},
- 'vcpus': {'type': 'integer'},
- 'swap': {'type': 'integer'},
- 'disk': {'type': 'integer'},
- 'id': {'type': 'string'}
- },
- 'required': ['name', 'links', 'ram', 'vcpus',
- 'swap', 'disk', 'id']
-}
-
-common_flavor_list_details = {
- 'status_code': [200],
- 'response_body': {
- 'type': 'object',
- 'properties': {
- 'flavors': {
- 'type': 'array',
- 'items': common_flavor_info
- }
- },
- 'required': ['flavors']
- }
-}
-
-common_flavor_details = {
- 'status_code': [200],
- 'response_body': {
- 'type': 'object',
- 'properties': {
- 'flavor': common_flavor_info
- },
- 'required': ['flavor']
- }
-}
diff --git a/tempest/api_schema/response/compute/hosts.py b/tempest/api_schema/response/compute/hosts.py
deleted file mode 100644
index 2596c27..0000000
--- a/tempest/api_schema/response/compute/hosts.py
+++ /dev/null
@@ -1,85 +0,0 @@
-# Copyright 2014 NEC Corporation. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
-# not use this file except in compliance with the License. You may obtain
-# a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-# License for the specific language governing permissions and limitations
-# under the License.
-
-common_start_up_body = {
- 'type': 'object',
- 'properties': {
- 'host': {'type': 'string'},
- 'power_action': {'enum': ['startup']}
- },
- 'required': ['host', 'power_action']
-}
-
-list_hosts = {
- 'status_code': [200],
- 'response_body': {
- 'type': 'object',
- 'properties': {
- 'hosts': {
- 'type': 'array',
- 'items': {
- 'type': 'object',
- 'properties': {
- 'host_name': {'type': 'string'},
- 'service': {'type': 'string'},
- 'zone': {'type': 'string'}
- },
- 'required': ['host_name', 'service', 'zone']
- }
- }
- },
- 'required': ['hosts']
- }
-}
-
-show_host_detail = {
- 'status_code': [200],
- 'response_body': {
- 'type': 'object',
- 'properties': {
- 'host': {
- 'type': 'array',
- 'item': {
- 'type': 'object',
- 'properties': {
- 'resource': {
- 'type': 'object',
- 'properties': {
- 'cpu': {'type': 'integer'},
- 'disk_gb': {'type': 'integer'},
- 'host': {'type': 'string'},
- 'memory_mb': {'type': 'integer'},
- 'project': {'type': 'string'}
- },
- 'required': ['cpu', 'disk_gb', 'host',
- 'memory_mb', 'project']
- }
- },
- 'required': ['resource']
- }
- }
- },
- 'required': ['host']
- }
-}
-
-update_host_common = {
- 'type': 'object',
- 'properties': {
- 'host': {'type': 'string'},
- 'maintenance_mode': {'enum': ['on_maintenance', 'off_maintenance']},
- 'status': {'enum': ['enabled', 'disabled']}
- },
- 'required': ['host', 'maintenance_mode', 'status']
-}
diff --git a/tempest/api_schema/response/compute/keypairs.py b/tempest/api_schema/response/compute/keypairs.py
deleted file mode 100644
index 2ae410c..0000000
--- a/tempest/api_schema/response/compute/keypairs.py
+++ /dev/null
@@ -1,62 +0,0 @@
-# Copyright 2014 NEC Corporation. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
-# not use this file except in compliance with the License. You may obtain
-# a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-# License for the specific language governing permissions and limitations
-# under the License.
-
-list_keypairs = {
- 'status_code': [200],
- 'response_body': {
- 'type': 'object',
- 'properties': {
- 'keypairs': {
- 'type': 'array',
- 'items': {
- 'type': 'object',
- 'properties': {
- 'keypair': {
- 'type': 'object',
- 'properties': {
- 'public_key': {'type': 'string'},
- 'name': {'type': 'string'},
- 'fingerprint': {'type': 'string'}
- },
- 'required': ['public_key', 'name', 'fingerprint']
- }
- },
- 'required': ['keypair']
- }
- }
- },
- 'required': ['keypairs']
- }
-}
-
-create_keypair = {
- 'type': 'object',
- 'properties': {
- 'keypair': {
- 'type': 'object',
- 'properties': {
- 'fingerprint': {'type': 'string'},
- 'name': {'type': 'string'},
- 'public_key': {'type': 'string'},
- 'user_id': {'type': 'string'},
- 'private_key': {'type': 'string'}
- },
- # When create keypair API is being called with 'Public key'
- # (Importing keypair) then, response body does not contain
- # 'private_key' So it is not defined as 'required'
- 'required': ['fingerprint', 'name', 'public_key', 'user_id']
- }
- },
- 'required': ['keypair']
-}
diff --git a/tempest/api_schema/response/compute/quotas.py b/tempest/api_schema/response/compute/quotas.py
deleted file mode 100644
index 863104c..0000000
--- a/tempest/api_schema/response/compute/quotas.py
+++ /dev/null
@@ -1,46 +0,0 @@
-# Copyright 2014 NEC Corporation. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
-# not use this file except in compliance with the License. You may obtain
-# a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-# License for the specific language governing permissions and limitations
-# under the License.
-
-common_quota_set = {
- 'status_code': [200],
- 'response_body': {
- 'type': 'object',
- 'properties': {
- 'quota_set': {
- 'type': 'object',
- 'properties': {
- 'instances': {'type': 'integer'},
- 'cores': {'type': 'integer'},
- 'ram': {'type': 'integer'},
- 'floating_ips': {'type': 'integer'},
- 'fixed_ips': {'type': 'integer'},
- 'metadata_items': {'type': 'integer'},
- 'key_pairs': {'type': 'integer'},
- 'security_groups': {'type': 'integer'},
- 'security_group_rules': {'type': 'integer'},
- 'server_group_members': {'type': 'integer'},
- 'server_groups': {'type': 'integer'},
- },
- # NOTE: server_group_members and server_groups are represented
- # when enabling quota_server_group extension. So they should
- # not be required.
- 'required': ['instances', 'cores', 'ram',
- 'floating_ips', 'fixed_ips',
- 'metadata_items', 'key_pairs',
- 'security_groups', 'security_group_rules']
- }
- },
- 'required': ['quota_set']
- }
-}
diff --git a/tempest/api_schema/response/compute/v2_1/flavors.py b/tempest/api_schema/response/compute/v2_1/flavors.py
index 76c4cee..725d17a 100644
--- a/tempest/api_schema/response/compute/v2_1/flavors.py
+++ b/tempest/api_schema/response/compute/v2_1/flavors.py
@@ -12,52 +12,86 @@
# License for the specific language governing permissions and limitations
# under the License.
-import copy
-
-from tempest.api_schema.response.compute import flavors
from tempest.api_schema.response.compute import parameter_types
-list_flavors_details = copy.deepcopy(flavors.common_flavor_list_details)
+list_flavors = {
+ 'status_code': [200],
+ 'response_body': {
+ 'type': 'object',
+ 'properties': {
+ 'flavors': {
+ 'type': 'array',
+ 'items': {
+ 'type': 'object',
+ 'properties': {
+ 'name': {'type': 'string'},
+ 'links': parameter_types.links,
+ 'id': {'type': 'string'}
+ },
+ 'required': ['name', 'links', 'id']
+ }
+ },
+ 'flavors_links': parameter_types.links
+ },
+ # NOTE(gmann): flavors_links attribute is not necessary
+ # to be present always So it is not 'required'.
+ 'required': ['flavors']
+ }
+}
-# 'swap' attributes comes as integer value but if it is empty it comes as "".
-# So defining type of as string and integer.
-list_flavors_details['response_body']['properties']['flavors']['items'][
- 'properties']['swap'] = {'type': ['string', 'integer']}
+common_flavor_info = {
+ 'type': 'object',
+ 'properties': {
+ 'name': {'type': 'string'},
+ 'links': parameter_types.links,
+ 'ram': {'type': 'integer'},
+ 'vcpus': {'type': 'integer'},
+ # 'swap' attributes comes as integer value but if it is empty
+ # it comes as "". So defining type of as string and integer.
+ 'swap': {'type': ['integer', 'string']},
+ 'disk': {'type': 'integer'},
+ 'id': {'type': 'string'},
+ 'OS-FLV-DISABLED:disabled': {'type': 'boolean'},
+ 'os-flavor-access:is_public': {'type': 'boolean'},
+ 'rxtx_factor': {'type': 'number'},
+ 'OS-FLV-EXT-DATA:ephemeral': {'type': 'integer'}
+ },
+ # 'OS-FLV-DISABLED', 'os-flavor-access', 'rxtx_factor' and
+ # 'OS-FLV-EXT-DATA' are API extensions. So they are not 'required'.
+ 'required': ['name', 'links', 'ram', 'vcpus', 'swap', 'disk', 'id']
+}
-# Defining 'flavors_links' attributes for V2 flavor schema
-list_flavors_details['response_body'][
- 'properties'].update({'flavors_links': parameter_types.links})
-# NOTE(gmann): flavors_links attribute is not necessary to be
-# present always So it is not 'required'.
-
-# Defining extra attributes for V2 flavor schema
-list_flavors_details['response_body']['properties']['flavors']['items'][
- 'properties'].update({'OS-FLV-DISABLED:disabled': {'type': 'boolean'},
- 'os-flavor-access:is_public': {'type': 'boolean'},
- 'rxtx_factor': {'type': 'number'},
- 'OS-FLV-EXT-DATA:ephemeral': {'type': 'integer'}})
-# 'OS-FLV-DISABLED', 'os-flavor-access', 'rxtx_factor' and 'OS-FLV-EXT-DATA'
-# are API extensions. So they are not 'required'.
+list_flavors_details = {
+ 'status_code': [200],
+ 'response_body': {
+ 'type': 'object',
+ 'properties': {
+ 'flavors': {
+ 'type': 'array',
+ 'items': common_flavor_info
+ },
+ # NOTE(gmann): flavors_links attribute is not necessary
+ # to be present always So it is not 'required'.
+ 'flavors_links': parameter_types.links
+ },
+ 'required': ['flavors']
+ }
+}
unset_flavor_extra_specs = {
'status_code': [200]
}
-create_get_flavor_details = copy.deepcopy(flavors.common_flavor_details)
-
-# 'swap' attributes comes as integer value but if it is empty it comes as "".
-# So defining type of as string and integer.
-create_get_flavor_details['response_body']['properties']['flavor'][
- 'properties']['swap'] = {'type': ['string', 'integer']}
-
-# Defining extra attributes for V2 flavor schema
-create_get_flavor_details['response_body']['properties']['flavor'][
- 'properties'].update({'OS-FLV-DISABLED:disabled': {'type': 'boolean'},
- 'os-flavor-access:is_public': {'type': 'boolean'},
- 'rxtx_factor': {'type': 'number'},
- 'OS-FLV-EXT-DATA:ephemeral': {'type': 'integer'}})
-# 'OS-FLV-DISABLED', 'os-flavor-access', 'rxtx_factor' and 'OS-FLV-EXT-DATA'
-# are API extensions. So they are not 'required'.
+create_get_flavor_details = {
+ 'status_code': [200],
+ 'response_body': {
+ 'type': 'object',
+ 'properties': {
+ 'flavor': common_flavor_info
+ },
+ 'required': ['flavor']
+ }
+}
delete_flavor = {
'status_code': [202]
diff --git a/tempest/api_schema/response/compute/v2_1/hosts.py b/tempest/api_schema/response/compute/v2_1/hosts.py
index 0944792..72d5a07 100644
--- a/tempest/api_schema/response/compute/v2_1/hosts.py
+++ b/tempest/api_schema/response/compute/v2_1/hosts.py
@@ -14,12 +14,70 @@
import copy
-from tempest.api_schema.response.compute import hosts
+list_hosts = {
+ 'status_code': [200],
+ 'response_body': {
+ 'type': 'object',
+ 'properties': {
+ 'hosts': {
+ 'type': 'array',
+ 'items': {
+ 'type': 'object',
+ 'properties': {
+ 'host_name': {'type': 'string'},
+ 'service': {'type': 'string'},
+ 'zone': {'type': 'string'}
+ },
+ 'required': ['host_name', 'service', 'zone']
+ }
+ }
+ },
+ 'required': ['hosts']
+ }
+}
+
+get_host_detail = {
+ 'status_code': [200],
+ 'response_body': {
+ 'type': 'object',
+ 'properties': {
+ 'host': {
+ 'type': 'array',
+ 'item': {
+ 'type': 'object',
+ 'properties': {
+ 'resource': {
+ 'type': 'object',
+ 'properties': {
+ 'cpu': {'type': 'integer'},
+ 'disk_gb': {'type': 'integer'},
+ 'host': {'type': 'string'},
+ 'memory_mb': {'type': 'integer'},
+ 'project': {'type': 'string'}
+ },
+ 'required': ['cpu', 'disk_gb', 'host',
+ 'memory_mb', 'project']
+ }
+ },
+ 'required': ['resource']
+ }
+ }
+ },
+ 'required': ['host']
+ }
+}
startup_host = {
'status_code': [200],
- 'response_body': hosts.common_start_up_body
+ 'response_body': {
+ 'type': 'object',
+ 'properties': {
+ 'host': {'type': 'string'},
+ 'power_action': {'enum': ['startup']}
+ },
+ 'required': ['host', 'power_action']
+ }
}
# The 'power_action' attribute of 'shutdown_host' API is 'shutdown'
@@ -38,5 +96,14 @@
update_host = {
'status_code': [200],
- 'response_body': hosts.update_host_common
+ 'response_body': {
+ 'type': 'object',
+ 'properties': {
+ 'host': {'type': 'string'},
+ 'maintenance_mode': {'enum': ['on_maintenance',
+ 'off_maintenance']},
+ 'status': {'enum': ['enabled', 'disabled']}
+ },
+ 'required': ['host', 'maintenance_mode', 'status']
+ }
}
diff --git a/tempest/api_schema/response/compute/v2_1/keypairs.py b/tempest/api_schema/response/compute/v2_1/keypairs.py
index ec26fa0..ceae6cf 100644
--- a/tempest/api_schema/response/compute/v2_1/keypairs.py
+++ b/tempest/api_schema/response/compute/v2_1/keypairs.py
@@ -12,8 +12,6 @@
# License for the specific language governing permissions and limitations
# under the License.
-from tempest.api_schema.response.compute import keypairs
-
get_keypair = {
'status_code': [200],
'response_body': {
@@ -47,9 +45,56 @@
create_keypair = {
'status_code': [200],
- 'response_body': keypairs.create_keypair
+ 'response_body': {
+ 'type': 'object',
+ 'properties': {
+ 'keypair': {
+ 'type': 'object',
+ 'properties': {
+ 'fingerprint': {'type': 'string'},
+ 'name': {'type': 'string'},
+ 'public_key': {'type': 'string'},
+ 'user_id': {'type': 'string'},
+ 'private_key': {'type': 'string'}
+ },
+ # When create keypair API is being called with 'Public key'
+ # (Importing keypair) then, response body does not contain
+ # 'private_key' So it is not defined as 'required'
+ 'required': ['fingerprint', 'name', 'public_key', 'user_id']
+ }
+ },
+ 'required': ['keypair']
+ }
}
delete_keypair = {
'status_code': [202],
}
+
+list_keypairs = {
+ 'status_code': [200],
+ 'response_body': {
+ 'type': 'object',
+ 'properties': {
+ 'keypairs': {
+ 'type': 'array',
+ 'items': {
+ 'type': 'object',
+ 'properties': {
+ 'keypair': {
+ 'type': 'object',
+ 'properties': {
+ 'public_key': {'type': 'string'},
+ 'name': {'type': 'string'},
+ 'fingerprint': {'type': 'string'}
+ },
+ 'required': ['public_key', 'name', 'fingerprint']
+ }
+ },
+ 'required': ['keypair']
+ }
+ }
+ },
+ 'required': ['keypairs']
+ }
+}
diff --git a/tempest/api_schema/response/compute/v2_1/quota_classes.py b/tempest/api_schema/response/compute/v2_1/quota_classes.py
index a7374df..a0cdaf5 100644
--- a/tempest/api_schema/response/compute/v2_1/quota_classes.py
+++ b/tempest/api_schema/response/compute/v2_1/quota_classes.py
@@ -20,12 +20,12 @@
# NOTE(mriedem): os-quota-class-sets responses are the same as os-quota-sets
# except for the key in the response body is quota_class_set instead of
# quota_set, so update this copy of the schema from os-quota-sets.
-quota_set = copy.deepcopy(quotas.quota_set)
-quota_set['response_body']['properties']['quota_class_set'] = (
- quota_set['response_body']['properties'].pop('quota_set'))
-quota_set['response_body']['required'] = ['quota_class_set']
+get_quota_class_set = copy.deepcopy(quotas.get_quota_set)
+get_quota_class_set['response_body']['properties']['quota_class_set'] = (
+ get_quota_class_set['response_body']['properties'].pop('quota_set'))
+get_quota_class_set['response_body']['required'] = ['quota_class_set']
-quota_set_update = copy.deepcopy(quotas.quota_set_update)
-quota_set_update['response_body']['properties']['quota_class_set'] = (
- quota_set_update['response_body']['properties'].pop('quota_set'))
-quota_set_update['response_body']['required'] = ['quota_class_set']
+update_quota_class_set = copy.deepcopy(quotas.update_quota_set)
+update_quota_class_set['response_body']['properties']['quota_class_set'] = (
+ update_quota_class_set['response_body']['properties'].pop('quota_set'))
+update_quota_class_set['response_body']['required'] = ['quota_class_set']
diff --git a/tempest/api_schema/response/compute/v2_1/quotas.py b/tempest/api_schema/response/compute/v2_1/quotas.py
index 630b227..9141f7e 100644
--- a/tempest/api_schema/response/compute/v2_1/quotas.py
+++ b/tempest/api_schema/response/compute/v2_1/quotas.py
@@ -14,34 +14,49 @@
import copy
-from tempest.api_schema.response.compute import quotas
+update_quota_set = {
+ 'status_code': [200],
+ 'response_body': {
+ 'type': 'object',
+ 'properties': {
+ 'quota_set': {
+ 'type': 'object',
+ 'properties': {
+ 'instances': {'type': 'integer'},
+ 'cores': {'type': 'integer'},
+ 'ram': {'type': 'integer'},
+ 'floating_ips': {'type': 'integer'},
+ 'fixed_ips': {'type': 'integer'},
+ 'metadata_items': {'type': 'integer'},
+ 'key_pairs': {'type': 'integer'},
+ 'security_groups': {'type': 'integer'},
+ 'security_group_rules': {'type': 'integer'},
+ 'server_group_members': {'type': 'integer'},
+ 'server_groups': {'type': 'integer'},
+ 'injected_files': {'type': 'integer'},
+ 'injected_file_content_bytes': {'type': 'integer'},
+ 'injected_file_path_bytes': {'type': 'integer'}
+ },
+ # NOTE: server_group_members and server_groups are represented
+ # when enabling quota_server_group extension. So they should
+ # not be required.
+ 'required': ['instances', 'cores', 'ram',
+ 'floating_ips', 'fixed_ips',
+ 'metadata_items', 'key_pairs',
+ 'security_groups', 'security_group_rules',
+ 'injected_files', 'injected_file_content_bytes',
+ 'injected_file_path_bytes']
+ }
+ },
+ 'required': ['quota_set']
+ }
+}
-quota_set = copy.deepcopy(quotas.common_quota_set)
-quota_set['response_body']['properties']['quota_set']['properties'][
+get_quota_set = copy.deepcopy(update_quota_set)
+get_quota_set['response_body']['properties']['quota_set']['properties'][
'id'] = {'type': 'string'}
-quota_set['response_body']['properties']['quota_set']['properties'][
- 'injected_files'] = {'type': 'integer'}
-quota_set['response_body']['properties']['quota_set']['properties'][
- 'injected_file_content_bytes'] = {'type': 'integer'}
-quota_set['response_body']['properties']['quota_set']['properties'][
- 'injected_file_path_bytes'] = {'type': 'integer'}
-quota_set['response_body']['properties']['quota_set']['required'].extend([
- 'id',
- 'injected_files',
- 'injected_file_content_bytes',
- 'injected_file_path_bytes'])
-
-quota_set_update = copy.deepcopy(quotas.common_quota_set)
-quota_set_update['response_body']['properties']['quota_set']['properties'][
- 'injected_files'] = {'type': 'integer'}
-quota_set_update['response_body']['properties']['quota_set']['properties'][
- 'injected_file_content_bytes'] = {'type': 'integer'}
-quota_set_update['response_body']['properties']['quota_set']['properties'][
- 'injected_file_path_bytes'] = {'type': 'integer'}
-quota_set_update['response_body']['properties']['quota_set'][
- 'required'].extend(['injected_files',
- 'injected_file_content_bytes',
- 'injected_file_path_bytes'])
+get_quota_set['response_body']['properties']['quota_set']['required'].extend([
+ 'id'])
delete_quota = {
'status_code': [202]
diff --git a/tempest/cli/simple_read_only/object_storage/__init__.py b/tempest/cli/simple_read_only/object_storage/__init__.py
deleted file mode 100644
index e69de29..0000000
--- a/tempest/cli/simple_read_only/object_storage/__init__.py
+++ /dev/null
diff --git a/tempest/cli/simple_read_only/object_storage/test_swift.py b/tempest/cli/simple_read_only/object_storage/test_swift.py
deleted file mode 100644
index 7201eab..0000000
--- a/tempest/cli/simple_read_only/object_storage/test_swift.py
+++ /dev/null
@@ -1,110 +0,0 @@
-# Copyright 2014 OpenStack Foundation
-# All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
-# not use this file except in compliance with the License. You may obtain
-# a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-# License for the specific language governing permissions and limitations
-# under the License.
-
-import re
-
-from tempest_lib import exceptions
-
-from tempest import cli
-from tempest import config
-from tempest import test
-
-CONF = config.CONF
-
-
-class SimpleReadOnlySwiftClientTest(cli.ClientTestBase):
- """Basic, read-only tests for Swift CLI client.
-
- Checks return values and output of read-only commands.
- These tests do not presume any content, nor do they create
- their own. They only verify the structure of output if present.
- """
-
- @classmethod
- def resource_setup(cls):
- if not CONF.service_available.swift:
- msg = ("%s skipped as Swift is not available" % cls.__name__)
- raise cls.skipException(msg)
- super(SimpleReadOnlySwiftClientTest, cls).resource_setup()
-
- def swift(self, *args, **kwargs):
- return self.clients.swift(
- *args, endpoint_type=CONF.object_storage.endpoint_type, **kwargs)
-
- @test.idempotent_id('74360cdc-e7ec-493f-8a87-2b65f4d54aa3')
- def test_swift_fake_action(self):
- self.assertRaises(exceptions.CommandFailed,
- self.swift,
- 'this-does-not-exist')
-
- @test.idempotent_id('809ec373-828e-4279-8df6-9d4db81c7909')
- def test_swift_list(self):
- self.swift('list')
-
- @test.idempotent_id('325d5fe4-e5ab-4f52-aec4-357533f24fa1')
- def test_swift_stat(self):
- output = self.swift('stat')
- entries = ['Account', 'Containers', 'Objects', 'Bytes', 'Content-Type',
- 'X-Timestamp', 'X-Trans-Id']
- for entry in entries:
- self.assertTrue(entry in output)
-
- @test.idempotent_id('af1483e1-dafd-4552-a39b-b9d337df808b')
- def test_swift_capabilities(self):
- output = self.swift('capabilities')
- entries = ['account_listing_limit', 'container_listing_limit',
- 'max_file_size', 'Additional middleware']
- for entry in entries:
- self.assertTrue(entry in output)
-
- @test.idempotent_id('29c83a64-8eb7-418c-a39b-c70cefa5b695')
- def test_swift_help(self):
- help_text = self.swift('', flags='--help')
- lines = help_text.split('\n')
- self.assertFirstLineStartsWith(lines, 'Usage: swift')
-
- commands = []
- cmds_start = lines.index('Positional arguments:')
- cmds_end = lines.index('Examples:')
- command_pattern = re.compile('^ {4}([a-z0-9\-\_]+)')
- for line in lines[cmds_start:cmds_end]:
- match = command_pattern.match(line)
- if match:
- commands.append(match.group(1))
- commands = set(commands)
- wanted_commands = set(('stat', 'list', 'delete',
- 'download', 'post', 'upload'))
- self.assertFalse(wanted_commands - commands)
-
- # Optional arguments:
-
- @test.idempotent_id('2026be82-4e53-4414-a828-f1c894b8cf0f')
- def test_swift_version(self):
- self.swift('', flags='--version')
-
- @test.idempotent_id('0ae6172e-3df7-42b8-a987-d42609ada6ed')
- def test_swift_debug_list(self):
- self.swift('list', flags='--debug')
-
- @test.idempotent_id('1bdf5dd0-7df5-446c-a124-2b0703a5d199')
- def test_swift_retries_list(self):
- self.swift('list', flags='--retries 3')
-
- @test.idempotent_id('64eae749-8fbd-4d85-bc7f-f706d3581c6f')
- def test_swift_region_list(self):
- region = CONF.object_storage.region
- if not region:
- region = CONF.identity.region
- self.swift('list', flags='--os-region-name ' + region)
diff --git a/tempest/cmd/javelin.py b/tempest/cmd/javelin.py
index fec3bd4..f84771f 100755
--- a/tempest/cmd/javelin.py
+++ b/tempest/cmd/javelin.py
@@ -77,7 +77,8 @@
- name: javelin_cirros
owner: javelin
file: cirros-0.3.2-x86_64-blank.img
- format: ami
+ disk_format: ami
+ container_format: ami
aki: cirros-0.3.2-x86_64-vmlinuz
ari: cirros-0.3.2-x86_64-initrd
@@ -629,6 +630,15 @@
for image in images:
client = client_for_user(image['owner'])
+ # DEPRECATED: 'format' was used for ami images
+ # Use 'disk_format' and 'container_format' instead
+ if 'format' in image:
+ LOG.warning("Deprecated: 'format' is deprecated for images "
+ "description. Please use 'disk_format' and 'container_"
+ "format' instead.")
+ image['disk_format'] = image['format']
+ image['container_format'] = image['format']
+
# only upload a new image if the name isn't there
if _get_image_by_name(client, image['name']):
LOG.info("Image '%s' already exists" % image['name'])
@@ -636,7 +646,7 @@
# special handling for 3 part image
extras = {}
- if image['format'] == 'ami':
+ if image['disk_format'] == 'ami':
name, fname = _resolve_image(image, 'aki')
aki = client.images.create_image(
'javelin_' + name, 'aki', 'aki')
@@ -651,7 +661,8 @@
_, fname = _resolve_image(image, 'file')
body = client.images.create_image(
- image['name'], image['format'], image['format'], **extras)
+ image['name'], image['container_format'],
+ image['disk_format'], **extras)
image_id = body.get('id')
client.images.store_image(image_id, open(fname, 'r'))
diff --git a/tempest/cmd/verify_tempest_config.py b/tempest/cmd/verify_tempest_config.py
index 3c71e07..b61f286 100755
--- a/tempest/cmd/verify_tempest_config.py
+++ b/tempest/cmd/verify_tempest_config.py
@@ -24,6 +24,7 @@
from six import moves
from tempest import clients
+from tempest.common import credentials
from tempest import config
@@ -254,7 +255,11 @@
}
# Get catalog list for endpoints to use for validation
_token, auth_data = os.auth_provider.get_auth()
- for entry in auth_data['serviceCatalog']:
+ if os.auth_version == 'v2':
+ catalog_key = 'serviceCatalog'
+ else:
+ catalog_key = 'catalog'
+ for entry in auth_data[catalog_key]:
services.append(entry['type'])
# Pull all catalog types from config file and compare against endpoint list
for cfgname in dir(CONF._config):
@@ -329,7 +334,8 @@
CONF_PARSER = moves.configparser.SafeConfigParser()
CONF_PARSER.optionxform = str
CONF_PARSER.readfp(conf_file)
- os = clients.Manager()
+ icreds = credentials.get_isolated_credentials('verify_tempest_config')
+ os = clients.Manager(icreds.get_primary_creds())
services = check_service_availability(os, update)
results = {}
for service in ['nova', 'cinder', 'neutron', 'swift']:
diff --git a/tempest/common/accounts.py b/tempest/common/accounts.py
index 6d376d6..acf6d4f 100644
--- a/tempest/common/accounts.py
+++ b/tempest/common/accounts.py
@@ -19,7 +19,9 @@
from oslo_log import log as logging
import yaml
+from tempest import clients
from tempest.common import cred_provider
+from tempest.common import fixed_network
from tempest import config
from tempest import exceptions
@@ -60,15 +62,18 @@
@classmethod
def get_hash_dict(cls, accounts):
- hash_dict = {'roles': {}, 'creds': {}}
+ hash_dict = {'roles': {}, 'creds': {}, 'networks': {}}
# Loop over the accounts read from the yaml file
for account in accounts:
roles = []
types = []
+ resources = []
if 'roles' in account:
roles = account.pop('roles')
if 'types' in account:
types = account.pop('types')
+ if 'resources' in account:
+ resources = account.pop('resources')
temp_hash = hashlib.md5()
temp_hash.update(str(account))
temp_hash_key = temp_hash.hexdigest()
@@ -91,6 +96,13 @@
CONF.object_storage.reseller_admin_role,
temp_hash_key,
hash_dict)
+ # Populate the network subdict
+ for resource in resources:
+ if resource == 'network':
+ hash_dict['networks'][temp_hash_key] = resources[resource]
+ else:
+ LOG.warning('Unkown resource type %s, ignoring this field'
+ % resource)
return hash_dict
def is_multi_user(self):
@@ -174,7 +186,7 @@
"Account file %s doesn't exist" % CONF.auth.test_accounts_file)
useable_hashes = self._get_match_hash_list(roles)
free_hash = self._get_free_hash(useable_hashes)
- return self.hash_dict['creds'][free_hash]
+ return self._wrap_creds_with_network(free_hash)
@lockutils.synchronized('test_accounts_io', external=True)
def remove_hash(self, hash_string):
@@ -190,8 +202,15 @@
def get_hash(self, creds):
for _hash in self.hash_dict['creds']:
# Comparing on the attributes that are expected in the YAML
- if all([getattr(creds, k) == self.hash_dict['creds'][_hash][k] for
- k in creds.get_init_attributes()]):
+ init_attributes = creds.get_init_attributes()
+ hash_attributes = self.hash_dict['creds'][_hash].copy()
+ if ('user_domain_name' in init_attributes and 'user_domain_name'
+ not in hash_attributes):
+ # Allow for the case of domain_name populated from config
+ domain_name = CONF.identity.admin_domain_name
+ hash_attributes['user_domain_name'] = domain_name
+ if all([getattr(creds, k) == hash_attributes[k] for
+ k in init_attributes]):
return _hash
raise AttributeError('Invalid credentials %s' % creds)
@@ -202,20 +221,16 @@
def get_primary_creds(self):
if self.isolated_creds.get('primary'):
return self.isolated_creds.get('primary')
- creds = self._get_creds()
- primary_credential = cred_provider.get_credentials(
- identity_version=self.identity_version, **creds)
- self.isolated_creds['primary'] = primary_credential
- return primary_credential
+ net_creds = self._get_creds()
+ self.isolated_creds['primary'] = net_creds
+ return net_creds
def get_alt_creds(self):
if self.isolated_creds.get('alt'):
return self.isolated_creds.get('alt')
- creds = self._get_creds()
- alt_credential = cred_provider.get_credentials(
- identity_version=self.identity_version, **creds)
- self.isolated_creds['alt'] = alt_credential
- return alt_credential
+ net_creds = self._get_creds()
+ self.isolated_creds['alt'] = net_creds
+ return net_creds
def get_creds_by_roles(self, roles, force_new=False):
roles = list(set(roles))
@@ -228,11 +243,9 @@
elif exist_creds and force_new:
new_index = str(roles) + '-' + str(len(self.isolated_creds))
self.isolated_creds[new_index] = exist_creds
- creds = self._get_creds(roles=roles)
- role_credential = cred_provider.get_credentials(
- identity_version=self.identity_version, **creds)
- self.isolated_creds[str(roles)] = role_credential
- return role_credential
+ net_creds = self._get_creds(roles=roles)
+ self.isolated_creds[str(roles)] = net_creds
+ return net_creds
def clear_isolated_creds(self):
for creds in self.isolated_creds.values():
@@ -252,6 +265,19 @@
def admin_available(self):
return self.is_role_available(CONF.identity.admin_role)
+ def _wrap_creds_with_network(self, hash):
+ creds_dict = self.hash_dict['creds'][hash]
+ credential = cred_provider.get_credentials(
+ identity_version=self.identity_version, **creds_dict)
+ net_creds = cred_provider.TestResources(credential)
+ net_clients = clients.Manager(credentials=credential)
+ compute_network_client = net_clients.networks_client
+ net_name = self.hash_dict['networks'].get(hash, None)
+ network = fixed_network.get_network_from_name(
+ net_name, compute_network_client)
+ net_creds.set_resources(network=network)
+ return net_creds
+
class NotLockingAccounts(Accounts):
"""Credentials provider which always returns the first and second
@@ -282,8 +308,9 @@
return self.isolated_creds.get('primary')
primary_credential = cred_provider.get_configured_credentials(
credential_type='user', identity_version=self.identity_version)
- self.isolated_creds['primary'] = primary_credential
- return primary_credential
+ self.isolated_creds['primary'] = cred_provider.TestResources(
+ primary_credential)
+ return self.isolated_creds['primary']
def get_alt_creds(self):
if self.isolated_creds.get('alt'):
@@ -291,8 +318,9 @@
alt_credential = cred_provider.get_configured_credentials(
credential_type='alt_user',
identity_version=self.identity_version)
- self.isolated_creds['alt'] = alt_credential
- return alt_credential
+ self.isolated_creds['alt'] = cred_provider.TestResources(
+ alt_credential)
+ return self.isolated_creds['alt']
def clear_isolated_creds(self):
self.isolated_creds = {}
@@ -300,8 +328,8 @@
def get_admin_creds(self):
creds = cred_provider.get_configured_credentials(
"identity_admin", fill_in=False)
- self.isolated_creds['admin'] = creds
- return creds
+ self.isolated_creds['admin'] = cred_provider.TestResources(creds)
+ return self.isolated_creds['admin']
def get_creds_by_roles(self, roles, force_new=False):
msg = "Credentials being specified through the config file can not be"\
diff --git a/tempest/common/cred_provider.py b/tempest/common/cred_provider.py
index 2c6763d..3223027 100644
--- a/tempest/common/cred_provider.py
+++ b/tempest/common/cred_provider.py
@@ -84,6 +84,8 @@
domain_fields = set(x for x in auth.KeystoneV3Credentials.ATTRIBUTES
if 'domain' in x)
if not domain_fields.intersection(kwargs.keys()):
+ # TODO(andreaf) It might be better here to use a dedicated config
+ # option such as CONF.auth.tenant_isolation_domain_name
params['user_domain_name'] = CONF.identity.admin_domain_name
auth_url = CONF.identity.uri_v3
else:
diff --git a/tempest/common/fixed_network.py b/tempest/common/fixed_network.py
index 67fbab1..1557474 100644
--- a/tempest/common/fixed_network.py
+++ b/tempest/common/fixed_network.py
@@ -13,17 +13,70 @@
import copy
from oslo_log import log as logging
+from tempest_lib.common.utils import misc as misc_utils
from tempest_lib import exceptions as lib_exc
-from tempest.common import isolated_creds
from tempest import config
-from tempest import exceptions
CONF = config.CONF
LOG = logging.getLogger(__name__)
+def get_network_from_name(name, compute_networks_client):
+ """Get a full network dict from just a network name
+
+ :param str name: the name of the network to use
+ :param NetworksClientJSON compute_networks_client: The network client
+ object to use for making the network lists api request
+ :return: The full dictionary for the network in question, unless the
+ network for the supplied name can not be found. In which case a dict
+ with just the name will be returned.
+ :rtype: dict
+ """
+ caller = misc_utils.find_test_caller()
+ if not name:
+ network = {'name': name}
+ else:
+ try:
+ resp = compute_networks_client.list_networks(name=name)
+ if isinstance(resp, list):
+ networks = resp
+ elif isinstance(resp, dict):
+ networks = resp['networks']
+ else:
+ raise lib_exc.NotFound()
+ if len(networks) > 0:
+ network = networks[0]
+ else:
+ msg = "Network with name: %s not found" % name
+ if caller:
+ LOG.warn('(%s) %s' % (caller, msg))
+ else:
+ LOG.warn(msg)
+ raise lib_exc.NotFound()
+ # To be consistent with network isolation, add name is only
+ # label is available
+ name = network.get('name', network.get('label'))
+ if name:
+ network['name'] = name
+ else:
+ raise lib_exc.NotFound()
+ except lib_exc.NotFound:
+ # In case of nova network, if the fixed_network_name is not
+ # owned by the tenant, and the network client is not an admin
+ # one, list_networks will not find it
+ msg = ('Unable to find network %s. '
+ 'Starting instance without specifying a network.' %
+ name)
+ if caller:
+ LOG.info('(%s) %s' % (caller, msg))
+ else:
+ LOG.info(msg)
+ network = {'name': name}
+ return network
+
+
def get_tenant_network(creds_provider, compute_networks_client):
"""Get a network usable by the primary tenant
@@ -35,43 +88,25 @@
is the network to be used, and it's not visible to the tenant
:return a dict with 'id' and 'name' of the network
"""
+ caller = misc_utils.find_test_caller()
fixed_network_name = CONF.compute.fixed_network_name
- network = None
- # NOTE(andreaf) get_primary_network will always be available once
- # bp test-accounts-continued is implemented
- if (isinstance(creds_provider, isolated_creds.IsolatedCreds) and
- (CONF.service_available.neutron and
- not CONF.service_available.ironic)):
- # tenant_allow_isolation == True, so network is defined
- network = creds_provider.get_primary_creds().network
- else:
+ net_creds = creds_provider.get_primary_creds()
+ network = getattr(net_creds, 'network', None)
+ if not network or not network.get('name'):
if fixed_network_name:
- try:
- resp = compute_networks_client.list_networks(
- name=fixed_network_name)
- if isinstance(resp, list):
- networks = resp
- elif isinstance(resp, dict):
- networks = resp['networks']
- else:
- raise lib_exc.NotFound()
- if len(networks) > 0:
- network = networks[0]
- else:
- msg = "Configured fixed_network_name not found"
- raise exceptions.InvalidConfiguration(msg)
- # To be consistent with network isolation, add name is only
- # label is available
- network['name'] = network.get('name', network.get('label'))
- except lib_exc.NotFound:
- # In case of nova network, if the fixed_network_name is not
- # owned by the tenant, and the network client is not an admin
- # one, list_networks will not find it
- LOG.info('Unable to find network %s. '
- 'Starting instance without specifying a network.' %
- fixed_network_name)
- network = {'name': fixed_network_name}
- LOG.info('Found network %s available for tenant' % network)
+ msg = ('No valid network provided or created, defaulting to '
+ 'fixed_network_name')
+ if caller:
+ LOG.debug('(%s) %s' % (caller, msg))
+ else:
+ LOG.debug(msg)
+ network = get_network_from_name(fixed_network_name,
+ compute_networks_client)
+ msg = ('Found network %s available for tenant' % network)
+ if caller:
+ LOG.info('(%s) %s' % (caller, msg))
+ else:
+ LOG.info(msg)
return network
@@ -87,5 +122,9 @@
return params
if network:
- params.update({"networks": [{'uuid': network['id']}]})
+ if 'id' in network.keys():
+ params.update({"networks": [{'uuid': network['id']}]})
+ else:
+ LOG.warn('The provided network dict: %s was invalid and did not '
+ ' contain an id' % network)
return params
diff --git a/tempest/common/utils/linux/remote_client.py b/tempest/common/utils/linux/remote_client.py
index b19faef..29fb493 100644
--- a/tempest/common/utils/linux/remote_client.py
+++ b/tempest/common/utils/linux/remote_client.py
@@ -87,10 +87,11 @@
cmd = 'sudo sh -c "echo \\"%s\\" >/dev/console"' % message
return self.exec_command(cmd)
- def ping_host(self, host):
+ def ping_host(self, host, count=CONF.compute.ping_count,
+ size=CONF.compute.ping_size):
addr = netaddr.IPAddress(host)
cmd = 'ping6' if addr.version == 6 else 'ping'
- cmd += ' -c1 -w1 {0}'.format(host)
+ cmd += ' -c{0} -w{0} -s{1} {2}'.format(count, size, host)
return self.exec_command(cmd)
def get_mac_address(self):
diff --git a/tempest/config.py b/tempest/config.py
index 3725f58..bcbe41f 100644
--- a/tempest/config.py
+++ b/tempest/config.py
@@ -187,10 +187,6 @@
default="root",
help="User name used to authenticate to an instance using "
"the alternate image."),
- cfg.StrOpt('image_alt_ssh_password',
- default="password",
- help="Password used to authenticate to an instance using "
- "the alternate image."),
cfg.IntOpt('build_interval',
default=1,
help="Time in seconds between build status checks."),
@@ -205,16 +201,17 @@
cfg.StrOpt('ssh_auth_method',
default='keypair',
help="Auth method used for authenticate to the instance. "
- "Valid choices are: keypair, configured, adminpass. "
- "keypair: start the servers with an ssh keypair. "
- "configured: use the configured user and password. "
- "adminpass: use the injected adminPass. "
- "disabled: avoid using ssh when it is an option."),
+ "Valid choices are: keypair, configured, adminpass "
+ "and disabled. "
+ "Keypair: start the servers with a ssh keypair. "
+ "Configured: use the configured user and password. "
+ "Adminpass: use the injected adminPass. "
+ "Disabled: avoid using ssh when it is an option."),
cfg.StrOpt('ssh_connect_method',
- default='fixed',
+ default='floating',
help="How to connect to the instance? "
"fixed: using the first ip belongs the fixed network "
- "floating: creating and using a floating ip"),
+ "floating: creating and using a floating ip."),
cfg.StrOpt('ssh_user',
default='root',
help="User name used to authenticate to an instance."),
@@ -222,6 +219,14 @@
default=120,
help="Timeout in seconds to wait for ping to "
"succeed."),
+ cfg.IntOpt('ping_size',
+ default=56,
+ help="The packet size for ping packets originating "
+ "from remote linux hosts"),
+ cfg.IntOpt('ping_count',
+ default=1,
+ help="The number of ping packets originating from remote "
+ "linux hosts"),
cfg.IntOpt('ssh_timeout',
default=300,
help="Timeout in seconds to wait for authentication to "
@@ -239,7 +244,8 @@
"tenants. If multiple networks are available for a tenant"
" this is the network which will be used for creating "
"servers if tempest does not create a network or a "
- "network is not specified elsewhere"),
+ "network is not specified elsewhere. It may be used for "
+ "ssh validation only if floating IPs are disabled."),
cfg.StrOpt('network_for_ssh',
default='public',
help="Network used for SSH connections. Ignored if "
@@ -264,9 +270,6 @@
choices=['public', 'admin', 'internal',
'publicURL', 'adminURL', 'internalURL'],
help="The endpoint type to use for the compute service."),
- cfg.StrOpt('path_to_private_key',
- help="Path to a private key file for SSH access to remote "
- "hosts"),
cfg.StrOpt('volume_device_name',
default='vdb',
help="Expected device name when a volume is attached to "
@@ -449,12 +452,16 @@
help="The mask bits for tenant ipv6 subnets"),
cfg.BoolOpt('tenant_networks_reachable',
default=False,
- help="Whether tenant network connectivity should be "
- "evaluated directly"),
+ help="Whether tenant networks can be reached directly from "
+ "the test client. This must be set to True when the "
+ "'fixed' ssh_connect_method is selected."),
cfg.StrOpt('public_network_id',
default="",
help="Id of the public network that provides external "
"connectivity"),
+ cfg.StrOpt('floating_network_name',
+ help="Default floating network name. Used to allocate floating "
+ "IPs when neutron is enabled."),
cfg.StrOpt('public_router_id',
default="",
help="Id of the public router that provides external "
@@ -499,6 +506,10 @@
"the extended IPv6 attributes ipv6_ra_mode "
"and ipv6_address_mode"
),
+ cfg.BoolOpt('port_admin_state_change',
+ default=True,
+ help="Does the test environment support changing"
+ " port admin state"),
]
messaging_group = cfg.OptGroup(name='messaging',
@@ -536,6 +547,37 @@
help='The maximum grace period for a claim'),
]
+validation_group = cfg.OptGroup(name='validation',
+ title='SSH Validation options')
+
+ValidationGroup = [
+ cfg.StrOpt('connect_method',
+ default='floating',
+ choices=['fixed', 'floating'],
+ help='Default IP type used for validation: '
+ '-fixed: uses the first IP belonging to the fixed network '
+ '-floating: creates and uses a floating IP'),
+ cfg.StrOpt('auth_method',
+ default='keypair',
+ choices=['keypair'],
+ help='Default authentication method to the instance. '
+ 'Only ssh via keypair is supported for now. '
+ 'Additional methods will be handled in a separate spec.'),
+ cfg.IntOpt('ip_version_for_ssh',
+ default=4,
+ help='Default IP version for ssh connections.'),
+ cfg.IntOpt('ping_timeout',
+ default=120,
+ help='Timeout in seconds to wait for ping to succeed.'),
+ cfg.IntOpt('connect_timeout',
+ default=60,
+ help='Timeout in seconds to wait for the TCP connection to be '
+ 'successful.'),
+ cfg.IntOpt('ssh_timeout',
+ default=300,
+ help='Timeout in seconds to wait for the ssh banner.'),
+]
+
volume_group = cfg.OptGroup(name='volume',
title='Block Storage Options')
@@ -1088,6 +1130,7 @@
(network_group, NetworkGroup),
(network_feature_group, NetworkFeaturesGroup),
(messaging_group, MessagingGroup),
+ (validation_group, ValidationGroup),
(volume_group, VolumeGroup),
(volume_feature_group, VolumeFeaturesGroup),
(object_storage_group, ObjectStoreGroup),
@@ -1148,6 +1191,7 @@
self.image_feature_enabled = _CONF['image-feature-enabled']
self.network = _CONF.network
self.network_feature_enabled = _CONF['network-feature-enabled']
+ self.validation = _CONF.validation
self.volume = _CONF.volume
self.volume_feature_enabled = _CONF['volume-feature-enabled']
self.object_storage = _CONF['object-storage']
diff --git a/tempest/scenario/manager.py b/tempest/scenario/manager.py
index 65d516f..d2c41f0 100644
--- a/tempest/scenario/manager.py
+++ b/tempest/scenario/manager.py
@@ -309,8 +309,13 @@
if isinstance(server_or_ip, six.string_types):
ip = server_or_ip
else:
- addr = server_or_ip['addresses'][CONF.compute.network_for_ssh][0]
- ip = addr['addr']
+ addrs = server_or_ip['addresses'][CONF.compute.network_for_ssh]
+ try:
+ ip = (addr['addr'] for addr in addrs if
+ netaddr.valid_ipv4(addr['addr'])).next()
+ except StopIteration:
+ raise lib_exc.NotFound("No IPv4 addresses to use for SSH to "
+ "remote server.")
if username is None:
username = CONF.scenario.ssh_user
diff --git a/tempest/scenario/test_load_balancer_basic.py b/tempest/scenario/test_load_balancer_basic.py
index 0d17048..8f37d74 100644
--- a/tempest/scenario/test_load_balancer_basic.py
+++ b/tempest/scenario/test_load_balancer_basic.py
@@ -185,7 +185,7 @@
# Start netcat
start_server = ('while true; do '
'sudo nc -ll -p %(port)s -e sh /tmp/%(script)s; '
- 'done &')
+ 'done > /dev/null &')
cmd = start_server % {'port': self.port1,
'script': 'script1'}
ssh_client.exec_command(cmd)
diff --git a/tempest/scenario/test_network_basic_ops.py b/tempest/scenario/test_network_basic_ops.py
index 8353048..b97ad0b 100644
--- a/tempest/scenario/test_network_basic_ops.py
+++ b/tempest/scenario/test_network_basic_ops.py
@@ -318,11 +318,15 @@
LOG.info(msg)
return
- subnet = self._list_subnets(
- network_id=CONF.network.public_network_id)
- self.assertEqual(1, len(subnet), "Found %d subnets" % len(subnet))
+ # We ping the external IP from the instance using its floating IP
+ # which is always IPv4, so we must only test connectivity to
+ # external IPv4 IPs if the external network is dualstack.
+ v4_subnets = [s for s in self._list_subnets(
+ network_id=CONF.network.public_network_id) if s['ip_version'] == 4]
+ self.assertEqual(1, len(v4_subnets),
+ "Found %d IPv4 subnets" % len(v4_subnets))
- external_ips = [subnet[0]['gateway_ip']]
+ external_ips = [v4_subnets[0]['gateway_ip']]
self._check_server_connectivity(self.floating_ip_tuple.floating_ip,
external_ips)
@@ -586,6 +590,9 @@
@testtools.skipIf(CONF.baremetal.driver_enabled,
'admin_state of instance ports cannot be altered '
'for baremetal nodes')
+ @testtools.skipUnless(CONF.network_feature_enabled.port_admin_state_change,
+ "Changing a port's admin state is not supported "
+ "by the test environment")
@test.attr(type='smoke')
@test.services('compute', 'network')
def test_update_instance_port_admin_state(self):
diff --git a/tempest/services/compute/json/flavors_client.py b/tempest/services/compute/json/flavors_client.py
index 25b1869..2de43cf 100644
--- a/tempest/services/compute/json/flavors_client.py
+++ b/tempest/services/compute/json/flavors_client.py
@@ -16,11 +16,10 @@
import json
import urllib
-from tempest.api_schema.response.compute import flavors as common_schema
from tempest.api_schema.response.compute import flavors_access as schema_access
from tempest.api_schema.response.compute import flavors_extra_specs \
as schema_extra_specs
-from tempest.api_schema.response.compute.v2_1 import flavors as v2schema
+from tempest.api_schema.response.compute.v2_1 import flavors as schema
from tempest.common import service_client
@@ -33,7 +32,7 @@
resp, body = self.get(url)
body = json.loads(body)
- self.validate_response(common_schema.list_flavors, resp, body)
+ self.validate_response(schema.list_flavors, resp, body)
return service_client.ResponseBodyList(resp, body['flavors'])
def list_flavors_with_detail(self, params=None):
@@ -43,13 +42,13 @@
resp, body = self.get(url)
body = json.loads(body)
- self.validate_response(v2schema.list_flavors_details, resp, body)
+ self.validate_response(schema.list_flavors_details, resp, body)
return service_client.ResponseBodyList(resp, body['flavors'])
def get_flavor_details(self, flavor_id):
resp, body = self.get("flavors/%s" % str(flavor_id))
body = json.loads(body)
- self.validate_response(v2schema.create_get_flavor_details, resp, body)
+ self.validate_response(schema.create_get_flavor_details, resp, body)
return service_client.ResponseBody(resp, body['flavor'])
def create_flavor(self, name, ram, vcpus, disk, flavor_id, **kwargs):
@@ -73,13 +72,13 @@
resp, body = self.post('flavors', post_body)
body = json.loads(body)
- self.validate_response(v2schema.create_get_flavor_details, resp, body)
+ self.validate_response(schema.create_get_flavor_details, resp, body)
return service_client.ResponseBody(resp, body['flavor'])
def delete_flavor(self, flavor_id):
"""Deletes the given flavor."""
resp, body = self.delete("flavors/{0}".format(flavor_id))
- self.validate_response(v2schema.delete_flavor, resp, body)
+ self.validate_response(schema.delete_flavor, resp, body)
return service_client.ResponseBody(resp, body)
def is_resource_deleted(self, id):
@@ -137,7 +136,7 @@
"""Unsets extra Specs from the mentioned flavor."""
resp, body = self.delete('flavors/%s/os-extra_specs/%s' %
(str(flavor_id), key))
- self.validate_response(v2schema.unset_flavor_extra_specs, resp, body)
+ self.validate_response(schema.unset_flavor_extra_specs, resp, body)
return service_client.ResponseBody(resp, body)
def list_flavor_access(self, flavor_id):
diff --git a/tempest/services/compute/json/hosts_client.py b/tempest/services/compute/json/hosts_client.py
index de925a9..088e695 100644
--- a/tempest/services/compute/json/hosts_client.py
+++ b/tempest/services/compute/json/hosts_client.py
@@ -15,8 +15,7 @@
import json
import urllib
-from tempest.api_schema.response.compute import hosts as schema
-from tempest.api_schema.response.compute.v2_1 import hosts as v2_schema
+from tempest.api_schema.response.compute.v2_1 import hosts as schema
from tempest.common import service_client
@@ -39,7 +38,7 @@
resp, body = self.get("os-hosts/%s" % str(hostname))
body = json.loads(body)
- self.validate_response(schema.show_host_detail, resp, body)
+ self.validate_response(schema.get_host_detail, resp, body)
return service_client.ResponseBodyList(resp, body['host'])
def update_host(self, hostname, **kwargs):
@@ -54,7 +53,7 @@
resp, body = self.put("os-hosts/%s" % str(hostname), request_body)
body = json.loads(body)
- self.validate_response(v2_schema.update_host, resp, body)
+ self.validate_response(schema.update_host, resp, body)
return service_client.ResponseBody(resp, body)
def startup_host(self, hostname):
@@ -62,7 +61,7 @@
resp, body = self.get("os-hosts/%s/startup" % str(hostname))
body = json.loads(body)
- self.validate_response(v2_schema.startup_host, resp, body)
+ self.validate_response(schema.startup_host, resp, body)
return service_client.ResponseBody(resp, body['host'])
def shutdown_host(self, hostname):
@@ -70,7 +69,7 @@
resp, body = self.get("os-hosts/%s/shutdown" % str(hostname))
body = json.loads(body)
- self.validate_response(v2_schema.shutdown_host, resp, body)
+ self.validate_response(schema.shutdown_host, resp, body)
return service_client.ResponseBody(resp, body['host'])
def reboot_host(self, hostname):
@@ -78,5 +77,5 @@
resp, body = self.get("os-hosts/%s/reboot" % str(hostname))
body = json.loads(body)
- self.validate_response(v2_schema.reboot_host, resp, body)
+ self.validate_response(schema.reboot_host, resp, body)
return service_client.ResponseBody(resp, body['host'])
diff --git a/tempest/services/compute/json/keypairs_client.py b/tempest/services/compute/json/keypairs_client.py
index 722aefa..7fe335b 100644
--- a/tempest/services/compute/json/keypairs_client.py
+++ b/tempest/services/compute/json/keypairs_client.py
@@ -15,7 +15,6 @@
import json
-from tempest.api_schema.response.compute import keypairs as common_schema
from tempest.api_schema.response.compute.v2_1 import keypairs as schema
from tempest.common import service_client
@@ -30,7 +29,7 @@
# servers, etc. A bug?
# For now we shall adhere to the spec, but the spec for keypairs
# is yet to be found
- self.validate_response(common_schema.list_keypairs, resp, body)
+ self.validate_response(schema.list_keypairs, resp, body)
return service_client.ResponseBodyList(resp, body['keypairs'])
def get_keypair(self, key_name):
diff --git a/tempest/services/compute/json/quotas_client.py b/tempest/services/compute/json/quotas_client.py
index 89f4acd..6e38c47 100644
--- a/tempest/services/compute/json/quotas_client.py
+++ b/tempest/services/compute/json/quotas_client.py
@@ -31,7 +31,7 @@
url += '?user_id=%s' % str(user_id)
resp, body = self.get(url)
body = json.loads(body)
- self.validate_response(schema.quota_set, resp, body)
+ self.validate_response(schema.get_quota_set, resp, body)
return service_client.ResponseBody(resp, body['quota_set'])
def get_default_quota_set(self, tenant_id):
@@ -40,7 +40,7 @@
url = 'os-quota-sets/%s/defaults' % str(tenant_id)
resp, body = self.get(url)
body = json.loads(body)
- self.validate_response(schema.quota_set, resp, body)
+ self.validate_response(schema.get_quota_set, resp, body)
return service_client.ResponseBody(resp, body['quota_set'])
def update_quota_set(self, tenant_id, user_id=None,
@@ -105,7 +105,7 @@
post_body)
body = json.loads(body)
- self.validate_response(schema.quota_set_update, resp, body)
+ self.validate_response(schema.update_quota_set, resp, body)
return service_client.ResponseBody(resp, body['quota_set'])
def delete_quota_set(self, tenant_id):
@@ -123,7 +123,7 @@
url = 'os-quota-class-sets/%s' % str(quota_class_id)
resp, body = self.get(url)
body = json.loads(body)
- self.validate_response(classes_schema.quota_set, resp, body)
+ self.validate_response(classes_schema.get_quota_class_set, resp, body)
return service_client.ResponseBody(resp, body['quota_class_set'])
def update_quota_class_set(self, quota_class_id, **kwargs):
@@ -136,5 +136,6 @@
post_body)
body = json.loads(body)
- self.validate_response(classes_schema.quota_set_update, resp, body)
+ self.validate_response(classes_schema.update_quota_class_set,
+ resp, body)
return service_client.ResponseBody(resp, body['quota_class_set'])
diff --git a/tempest/tests/common/test_accounts.py b/tempest/tests/common/test_accounts.py
index 6371e49..b4048ba 100644
--- a/tempest/tests/common/test_accounts.py
+++ b/tempest/tests/common/test_accounts.py
@@ -23,11 +23,13 @@
from tempest import auth
from tempest.common import accounts
+from tempest.common import cred_provider
from tempest import config
from tempest import exceptions
from tempest.services.identity.v2.json import token_client
from tempest.tests import base
from tempest.tests import fake_config
+from tempest.tests import fake_http
from tempest.tests import fake_identity
@@ -37,6 +39,9 @@
super(TestAccount, self).setUp()
self.useFixture(fake_config.ConfigFixture())
self.stubs.Set(config, 'TempestConfigPrivate', fake_config.FakePrivate)
+ self.fake_http = fake_http.fake_httplib2(return_type=200)
+ self.stubs.Set(token_client.TokenClientJSON, 'raw_request',
+ fake_identity._fake_v2_response)
self.useFixture(lockutils_fixtures.ExternalLockFixture())
self.test_accounts = [
{'username': 'test_user1', 'tenant_name': 'test_tenant1',
@@ -64,7 +69,7 @@
{'username': 'test_user12', 'tenant_name': 'test_tenant12',
'password': 'p', 'roles': [cfg.CONF.identity.admin_role]},
]
- self.useFixture(mockpatch.Patch(
+ self.accounts_mock = self.useFixture(mockpatch.Patch(
'tempest.common.accounts.read_accounts_yaml',
return_value=self.test_accounts))
cfg.CONF.set_default('test_accounts_file', 'fake_path', group='auth')
@@ -275,6 +280,31 @@
for i in admin_hashes:
self.assertNotIn(i, args)
+ def test_networks_returned_with_creds(self):
+ test_accounts = [
+ {'username': 'test_user13', 'tenant_name': 'test_tenant13',
+ 'password': 'p', 'resources': {'network': 'network-1'}},
+ {'username': 'test_user14', 'tenant_name': 'test_tenant14',
+ 'password': 'p', 'roles': ['role-7', 'role-11'],
+ 'resources': {'network': 'network-2'}}]
+ # Clear previous mock using self.test_accounts
+ self.accounts_mock.cleanUp()
+ self.useFixture(mockpatch.Patch(
+ 'tempest.common.accounts.read_accounts_yaml',
+ return_value=test_accounts))
+ test_accounts_class = accounts.Accounts('v2', 'test_name')
+ with mock.patch('tempest.services.compute.json.networks_client.'
+ 'NetworksClientJSON.list_networks',
+ return_value=[{'name': 'network-2', 'id': 'fake-id'}]):
+ creds = test_accounts_class.get_creds_by_roles(['role-7'])
+ self.assertTrue(isinstance(creds, cred_provider.TestResources))
+ network = creds.network
+ self.assertIsNotNone(network)
+ self.assertIn('name', network)
+ self.assertIn('id', network)
+ self.assertEqual('fake-id', network['id'])
+ self.assertEqual('network-2', network['name'])
+
class TestNotLockingAccount(base.TestCase):
diff --git a/tempest/tests/common/utils/linux/test_remote_client.py b/tempest/tests/common/utils/linux/test_remote_client.py
index 40b7b32..d6377e6 100644
--- a/tempest/tests/common/utils/linux/test_remote_client.py
+++ b/tempest/tests/common/utils/linux/test_remote_client.py
@@ -100,15 +100,17 @@
self._assert_exec_called_with('cut -f1 -d. /proc/uptime')
def test_ping_host(self):
- ping_response = """PING localhost (127.0.0.1) 56(84) bytes of data.
-64 bytes from localhost (127.0.0.1): icmp_req=1 ttl=64 time=0.048 ms
+ ping_response = """PING localhost (127.0.0.1) 70(98) bytes of data.
+78 bytes from localhost (127.0.0.1): icmp_req=1 ttl=64 time=0.048 ms
+78 bytes from localhost (127.0.0.1): icmp_req=2 ttl=64 time=0.048 ms
--- localhost ping statistics ---
-1 packets transmitted, 1 received, 0% packet loss, time 0ms
+2 packets transmitted, 2 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.048/0.048/0.048/0.000 ms"""
self.ssh_mock.mock.exec_command.return_value = ping_response
- self.assertEqual(self.conn.ping_host('127.0.0.1'), ping_response)
- self._assert_exec_called_with('ping -c1 -w1 127.0.0.1')
+ self.assertEqual(self.conn.ping_host('127.0.0.1', count=2, size=70),
+ ping_response)
+ self._assert_exec_called_with('ping -c2 -w2 -s70 127.0.0.1')
def test_get_mac_address(self):
macs = """0a:0b:0c:0d:0e:0f