QoS - Change the way we measure bw limits
This patch introduces new way of fetching the data.
Instead creating the file, it reads /dev/zero.
/dev/zero is always very fast, so we also break the
previous hard disk limitations.
The test time is limited to 5 seconds. After that we
calculate avg bytes per second value and compare it
to expected one.
Sometimes it is visible that first kilobytes of the
test file are downloaded a little bit faster than the
actual bw limit claims, especially while testing OVN
as a backend.
When it happens the avg bytes per second value that is
measured in the test could be higher than required limit.
It is pretty easy to show the case while testing QoS with iperf3:
Accepted connection from 172.24.5.1, port 59690
[ 5] local 10.1.0.35 port 5201 connected to 172.24.5.1 port 59692
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 5] 0.00-1.00 sec 1.32 MBytes 11.0 Mbits/sec 139 2.62 KBytes
[ 5] 1.00-2.00 sec 628 KBytes 5.15 Mbits/sec 96 10.5 KBytes
[ 5] 2.00-3.00 sec 502 KBytes 4.12 Mbits/sec 84 7.85 KBytes
[ 5] 3.00-4.00 sec 649 KBytes 5.32 Mbits/sec 83 10.5 KBytes
[ 5] 4.00-5.00 sec 643 KBytes 5.26 Mbits/sec 84 3.93 KBytes
[ 5] 5.00-6.00 sec 529 KBytes 4.33 Mbits/sec 73 5.23 KBytes
[ 5] 6.00-7.00 sec 628 KBytes 5.15 Mbits/sec 92 20.9 KBytes
[ 5] 7.00-8.00 sec 534 KBytes 4.37 Mbits/sec 82 18.3 KBytes
[ 5] 8.00-9.00 sec 667 KBytes 5.47 Mbits/sec 110 7.85 KBytes
[ 5] 9.00-10.00 sec 635 KBytes 5.20 Mbits/sec 90 11.8 KBytes
[ 5] 10.00-10.02 sec 0.00 Bytes 0.00 bits/sec 0 11.8 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 5] 0.00-10.02 sec 6.61 MBytes 5.53 Mbits/sec 933 sender
[ 5] 0.00-10.02 sec 6.41 MBytes 5.36 Mbits/sec receiver
-----------------------------------------------------------
We can find out that during first second of the test the bw limit
is exceeded, but after that the traffic is shaped.
In our case when we run the tempest QoS test the avg bytes per second
measured value that we compare with bw limit is impacted.
Closes-Bug: 1866039
Change-Id: I0964464e709baf9958548384933bd000fdee979b
diff --git a/neutron_tempest_plugin/scenario/test_qos.py b/neutron_tempest_plugin/scenario/test_qos.py
index f8f1b03..c847077 100644
--- a/neutron_tempest_plugin/scenario/test_qos.py
+++ b/neutron_tempest_plugin/scenario/test_qos.py
@@ -69,41 +69,27 @@
credentials = ['primary', 'admin']
force_tenant_isolation = False
- FILE_SIZE = 1024 * 1024
TOLERANCE_FACTOR = 1.5
BUFFER_SIZE = 512
- COUNT = FILE_SIZE / BUFFER_SIZE
LIMIT_BYTES_SEC = (constants.LIMIT_KILO_BITS_PER_SECOND * 1024 *
TOLERANCE_FACTOR / 8.0)
- FILE_PATH = "/tmp/img"
-
NC_PORT = 1234
- FILE_DOWNLOAD_TIMEOUT = 120
-
- def _create_file_for_bw_tests(self, ssh_client):
- cmd = ("(dd if=/dev/zero bs=%(bs)d count=%(count)d of=%(file_path)s) "
- % {'bs': self.BUFFER_SIZE, 'count': self.COUNT,
- 'file_path': self.FILE_PATH})
- ssh_client.exec_command(cmd, timeout=5)
- cmd = "stat -c %%s %s" % self.FILE_PATH
- filesize = ssh_client.exec_command(cmd, timeout=5)
- if int(filesize.strip()) != self.FILE_SIZE:
- raise sc_exceptions.FileCreationFailedException(
- file=self.FILE_PATH)
+ DOWNLOAD_DURATION = 5
+ # NOTE(mjozefcz): This makes around 10 retries.
+ CHECK_TIMEOUT = DOWNLOAD_DURATION * 10
def _check_bw(self, ssh_client, host, port, expected_bw=LIMIT_BYTES_SEC):
utils.kill_nc_process(ssh_client)
- cmd = ("(nc -ll -p %(port)d < %(file_path)s > /dev/null &)" % {
- 'port': port, 'file_path': self.FILE_PATH})
+ cmd = ("(nc -ll -p %d < /dev/zero > /dev/null &)" % port)
ssh_client.exec_command(cmd, timeout=5)
# Open TCP socket to remote VM and download big file
start_time = time.time()
- socket_timeout = self.FILE_SIZE * self.TOLERANCE_FACTOR / expected_bw
- client_socket = _connect_socket(host, port, socket_timeout)
+ client_socket = _connect_socket(
+ host, port, constants.SOCKET_CONNECT_TIMEOUT)
total_bytes_read = 0
try:
- while total_bytes_read < self.FILE_SIZE:
+ while time.time() - start_time < self.DOWNLOAD_DURATION:
data = client_socket.recv(self.BUFFER_SIZE)
total_bytes_read += len(data)
@@ -113,10 +99,12 @@
LOG.debug("time_elapsed = %(time_elapsed).16f, "
"total_bytes_read = %(total_bytes_read)d, "
- "bytes_per_second = %(bytes_per_second)d",
+ "bytes_per_second = %(bytes_per_second)d, "
+ "expected_bw = %(expected_bw)d.",
{'time_elapsed': time_elapsed,
'total_bytes_read': total_bytes_read,
- 'bytes_per_second': bytes_per_second})
+ 'bytes_per_second': bytes_per_second,
+ 'expected_bw': expected_bw})
return bytes_per_second <= expected_bw
except socket.timeout:
LOG.warning('Socket timeout while reading the remote file, bytes '
@@ -207,16 +195,13 @@
self.os_admin.network_client.update_network(
self.network['id'], qos_policy_id=bw_limit_policy_id)
- # Create file on VM
- self._create_file_for_bw_tests(ssh_client)
-
# Basic test, Check that actual BW while downloading file
# is as expected (Original BW)
utils.wait_until_true(lambda: self._check_bw(
ssh_client,
self.fip['floating_ip_address'],
port=self.NC_PORT),
- timeout=self.FILE_DOWNLOAD_TIMEOUT,
+ timeout=self.CHECK_TIMEOUT,
sleep=1)
# As admin user update QoS rule
@@ -233,7 +218,7 @@
self.fip['floating_ip_address'],
port=self.NC_PORT,
expected_bw=QoSTest.LIMIT_BYTES_SEC * 2),
- timeout=self.FILE_DOWNLOAD_TIMEOUT,
+ timeout=self.CHECK_TIMEOUT,
sleep=1)
# Create a new QoS policy
@@ -256,7 +241,7 @@
ssh_client,
self.fip['floating_ip_address'],
port=self.NC_PORT),
- timeout=self.FILE_DOWNLOAD_TIMEOUT,
+ timeout=self.CHECK_TIMEOUT,
sleep=1)
# As admin user update QoS rule
@@ -271,6 +256,7 @@
utils.wait_until_true(lambda: self._check_bw(
ssh_client,
self.fip['floating_ip_address'],
- port=self.NC_PORT, expected_bw=QoSTest.LIMIT_BYTES_SEC * 3),
- timeout=self.FILE_DOWNLOAD_TIMEOUT,
+ port=self.NC_PORT,
+ expected_bw=QoSTest.LIMIT_BYTES_SEC * 3),
+ timeout=self.CHECK_TIMEOUT,
sleep=1)