Merge remote-tracking branch 'origin/feature/qos' into merge-branch

Note to reviewers: gerrit diff for merge patches is very limited, and
leaving comments in global section won't scale, so please comment here:

https://etherpad.openstack.org/p/qos-merge-back-review

This merge commit introduces QoS feature into Liberty release of
Neutron.

The feature is documented in: doc/source/devref/quality_of_service.rst
included with the merge patch.

It includes:

- QoS API service plugin with QoS policy and QoS bandwidth limit
  (egress) rule support;
- core plugin mechanism to determine supported rule types, with its ML2
  implementation;
- new agent extension manager;
- QoS agent extension with pluggable backend QoS drivers (Open vSwitch
  and SR-IOV support is included).

To extend network and port core resources with qos_policy_id attribute,
a new ML2 extension driver (qos) was introduced that relies on the QoS
core resource extension (the idea is that eventually we'll get a core
resource extension manager that can be directly reused by core plugins).

Agent-server interaction is based on:

- get_device_details() method that is extended with qos_policy_id;
- a new push/pull mechanism that allows agents and servers to
  communicate using oslo.versionedobjects based objects sent on the
  wire.

The merge includes the following types of test coverage:

- unit tests;
- functional tests for OVS agent, QoS agent extension, and low level
  ovs_lib changes;
- API tests to cover port/network qos_policy_id attribute and new QoS
  resources.

The client changes can be found at:

* https://review.openstack.org/189655
* https://review.openstack.org/198277

The team also prepared fullstack test but it needs to wait for client
merge before it can pass in the gate:

* https://review.openstack.org/202492

Gerrit does not show diff for merge changes that did not result in any
conflict, so to facilitate review, rely on the following steps:

- fetch the patch locally
- git fetch origin
- git diff origin/master...

This merge also disables qos extension API tests until the service is
enabled in master gate.

Local changes apart from conflicts:
- updated down_revision for qos migration to reflect master expand head;
- disabled qos API tests with gate_hook.sh until we have it enabled in
  master gate;
- bumped oslo.versionedobjects requirement to reflect what is in
  openstack/requirements' global-requirements.txt

DocImpact
APIImpact
Partially-Implements: blueprint quantum-qos-api
Partially-Implements: blueprint ml2-qos
Partially-Implements: blueprint ml2-qos-ovs-bwlimiting
Partially-Implements: blueprint ml2-sriov-qos-with-bwlimiting
Change-Id: I92916d0e391791187e9a25ff172fb4b3504857b1
diff --git a/neutron/tests/tempest/common/glance_http.py b/neutron/tests/tempest/common/glance_http.py
index 0a6f985..3d8c8aa 100644
--- a/neutron/tests/tempest/common/glance_http.py
+++ b/neutron/tests/tempest/common/glance_http.py
@@ -17,7 +17,6 @@
 
 import copy
 import hashlib
-import json
 import posixpath
 import re
 import socket
@@ -28,6 +27,7 @@
 
 import OpenSSL
 from oslo_log import log as logging
+from oslo_serialization import jsonutils as json
 from six import moves
 from six.moves import http_client as httplib
 from tempest_lib import exceptions as lib_exc
diff --git a/neutron/tests/tempest/config.py b/neutron/tests/tempest/config.py
index c459d76..200b247 100644
--- a/neutron/tests/tempest/config.py
+++ b/neutron/tests/tempest/config.py
@@ -15,7 +15,6 @@
 
 from __future__ import print_function
 
-import logging as std_logging
 import os
 
 from oslo_config import cfg
@@ -1191,7 +1190,7 @@
         register_opts()
         self._set_attrs()
         if parse_conf:
-            cfg.CONF.log_opt_values(LOG, std_logging.DEBUG)
+            cfg.CONF.log_opt_values(LOG, logging.DEBUG)
 
 
 class TempestConfigProxy(object):
@@ -1199,15 +1198,15 @@
     _path = None
 
     _extra_log_defaults = [
-        ('keystoneclient.session', std_logging.INFO),
-        ('paramiko.transport', std_logging.INFO),
-        ('requests.packages.urllib3.connectionpool', std_logging.WARN),
+        ('keystoneclient.session', logging.INFO),
+        ('paramiko.transport', logging.INFO),
+        ('requests.packages.urllib3.connectionpool', logging.WARN),
     ]
 
     def _fix_log_levels(self):
         """Tweak the oslo log defaults."""
         for name, level in self._extra_log_defaults:
-            std_logging.getLogger(name).setLevel(level)
+            logging.getLogger(name).logger.setLevel(level)
 
     def __getattr__(self, attr):
         if not self._config:
diff --git a/neutron/tests/tempest/services/identity/v2/json/identity_client.py b/neutron/tests/tempest/services/identity/v2/json/identity_client.py
index 7efda1f..46e8f87 100644
--- a/neutron/tests/tempest/services/identity/v2/json/identity_client.py
+++ b/neutron/tests/tempest/services/identity/v2/json/identity_client.py
@@ -10,7 +10,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-import json
+from oslo_serialization import jsonutils as json
 from tempest_lib import exceptions as lib_exc
 
 from neutron.tests.tempest.common import service_client
diff --git a/neutron/tests/tempest/services/identity/v2/json/token_client.py b/neutron/tests/tempest/services/identity/v2/json/token_client.py
index 51d9db0..e8b33ea 100644
--- a/neutron/tests/tempest/services/identity/v2/json/token_client.py
+++ b/neutron/tests/tempest/services/identity/v2/json/token_client.py
@@ -12,7 +12,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-import json
+from oslo_serialization import jsonutils as json
 from tempest_lib.common import rest_client
 from tempest_lib import exceptions as lib_exc
 
diff --git a/neutron/tests/tempest/services/identity/v3/json/credentials_client.py b/neutron/tests/tempest/services/identity/v3/json/credentials_client.py
index 4300c0f..07e230a 100644
--- a/neutron/tests/tempest/services/identity/v3/json/credentials_client.py
+++ b/neutron/tests/tempest/services/identity/v3/json/credentials_client.py
@@ -13,7 +13,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-import json
+from oslo_serialization import jsonutils as json
 
 from neutron.tests.tempest.common import service_client
 
diff --git a/neutron/tests/tempest/services/identity/v3/json/endpoints_client.py b/neutron/tests/tempest/services/identity/v3/json/endpoints_client.py
index b60dd26..27ac3e5 100644
--- a/neutron/tests/tempest/services/identity/v3/json/endpoints_client.py
+++ b/neutron/tests/tempest/services/identity/v3/json/endpoints_client.py
@@ -13,7 +13,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-import json
+from oslo_serialization import jsonutils as json
 
 from neutron.tests.tempest.common import service_client
 
diff --git a/neutron/tests/tempest/services/identity/v3/json/identity_client.py b/neutron/tests/tempest/services/identity/v3/json/identity_client.py
index a7db46a..a090acf 100644
--- a/neutron/tests/tempest/services/identity/v3/json/identity_client.py
+++ b/neutron/tests/tempest/services/identity/v3/json/identity_client.py
@@ -13,8 +13,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-import json
-
+from oslo_serialization import jsonutils as json
 from six.moves.urllib import parse
 
 from neutron.tests.tempest.common import service_client
diff --git a/neutron/tests/tempest/services/identity/v3/json/policy_client.py b/neutron/tests/tempest/services/identity/v3/json/policy_client.py
index 2e44185..2d247af 100644
--- a/neutron/tests/tempest/services/identity/v3/json/policy_client.py
+++ b/neutron/tests/tempest/services/identity/v3/json/policy_client.py
@@ -13,7 +13,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-import json
+from oslo_serialization import jsonutils as json
 
 from neutron.tests.tempest.common import service_client
 
diff --git a/neutron/tests/tempest/services/identity/v3/json/region_client.py b/neutron/tests/tempest/services/identity/v3/json/region_client.py
index d2fa53b..0effae8 100644
--- a/neutron/tests/tempest/services/identity/v3/json/region_client.py
+++ b/neutron/tests/tempest/services/identity/v3/json/region_client.py
@@ -13,8 +13,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-import json
-
+from oslo_serialization import jsonutils as json
 from six.moves.urllib import parse
 
 from neutron.tests.tempest.common import service_client
diff --git a/neutron/tests/tempest/services/identity/v3/json/service_client.py b/neutron/tests/tempest/services/identity/v3/json/service_client.py
index 529693e..75a5cf8 100644
--- a/neutron/tests/tempest/services/identity/v3/json/service_client.py
+++ b/neutron/tests/tempest/services/identity/v3/json/service_client.py
@@ -13,7 +13,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-import json
+from oslo_serialization import jsonutils as json
 
 from neutron.tests.tempest.common import service_client
 
diff --git a/neutron/tests/tempest/services/identity/v3/json/token_client.py b/neutron/tests/tempest/services/identity/v3/json/token_client.py
index c60b24c..77ecf84 100644
--- a/neutron/tests/tempest/services/identity/v3/json/token_client.py
+++ b/neutron/tests/tempest/services/identity/v3/json/token_client.py
@@ -12,7 +12,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-import json
+from oslo_serialization import jsonutils as json
 from tempest_lib.common import rest_client
 from tempest_lib import exceptions as lib_exc
 
diff --git a/neutron/tests/tempest/services/network/json/network_client.py b/neutron/tests/tempest/services/network/json/network_client.py
index 9c5ef4a..3fb233e 100644
--- a/neutron/tests/tempest/services/network/json/network_client.py
+++ b/neutron/tests/tempest/services/network/json/network_client.py
@@ -10,11 +10,11 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-import json
 import time
 import urllib
 
 
+from oslo_serialization import jsonutils as json
 from six.moves.urllib import parse
 from tempest_lib.common.utils import misc
 from tempest_lib import exceptions as lib_exc
diff --git a/neutron/tests/tempest/test.py b/neutron/tests/tempest/test.py
index d95174b..3abf826 100644
--- a/neutron/tests/tempest/test.py
+++ b/neutron/tests/tempest/test.py
@@ -15,7 +15,6 @@
 
 import atexit
 import functools
-import json
 import os
 import re
 import sys
@@ -24,6 +23,7 @@
 
 import fixtures
 from oslo_log import log as logging
+from oslo_serialization import jsonutils as json
 from oslo_utils import importutils
 import six
 from six.moves.urllib import parse