Backport cvp-sanity from master to 2019.2.0
Related-Prod; #PROD-29210(PROD:29210)
Squashed commit of the following:
commit 8c05e2703aa328d9e22bc09360ea30723dc0dd74
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Wed Apr 24 11:47:31 2019 +0300
Add test steps into the stdout if test failed.
Related-Prod:#PROD-29995(PROD:29995)
Change-Id: Ie0a03d4d8896c7d7836cfd57736778f3896bcb87
commit a14488d565790992e8453d643a6fbea14bb25311
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Tue Apr 30 15:08:33 2019 +0300
Fix incorrect counting of backends
Split tests for cinder services into two tests
Change-Id: I74137b4cc31a82718fc2a17f5abfd117aacf9963
Fix-Issue:#PROD-29913(PROD:29913)
commit 10e2db4420d74db51259f55cc5b98482b53b116b
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Thu May 2 13:17:00 2019 +0300
Run opencontrain tests for OpenStack-type deployment only
Change-Id: I2b36bf33c4d3fde3fac37d669a4a2e8e449d4caf
Fix-Prod: #PROD-27782(PROD:27782
commit 1db3888a0df328e8c41f3f465c9ed28bb1f95763
Merge: 80514de 50a2167
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Wed May 1 20:43:52 2019 +0000
Merge "test_oss launched if cicd node available"
commit 80514de4b630141ba42e6f4bb85bf5f6e0a15f72
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Thu Apr 25 12:33:28 2019 +0300
Exclude kdt-nodes from test_mounts
Change-Id: I1cb9c2521fff6e9cffe8d4d86c0abf149233c296
Related-Prod: #PROD-29774(PROD:29774)
commit 864f2326856b128aacad5ccba13227938541ce78
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Mon Apr 29 15:48:12 2019 -0500
[CVP] Add ext_net parameter
Change-Id: Ie0d80d86b6d527f5593b9525cf22bc8343b84839
Related-PROD: PROD-26972
commit dd17609d8f4e3a6a080b6cc1858139a0d3cf5057
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Fri Apr 26 15:29:10 2019 -0500
[CVP] Fix parameter name in test_check_services
Change-Id: I338bea5bb180ef9999d22b5acefc5af74f877ba3
Related-PROD: PROD-29928
commit 10b360319fafb711391884af9f2b484a15412c0d
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Wed Apr 24 18:16:43 2019 -0500
[CVP] Add sanity test to check vip address presence for groups
Change-Id: I8b26a8e30de7eadf76254f35afb0e2621b73ea52
Related-PROD: PROD-29845
commit 577453f143d140353d8d62f6bd2f51a4b7011888
Merge: bcb27cd 4a79efd
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Tue Apr 30 22:52:31 2019 +0000
Merge "Added tests to check Drivetrain on K8s"
commit bcb27cd48482ba8daee5a2466482d2d9a30d0091
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Tue Apr 23 17:04:20 2019 -0500
[CVP] Disable public url check for gerrit and jenkins on k8s envs
Change-Id: Iab1637d234e8d597635758c886f7a40165928597
Related-PROD: PROD-28324
commit 50a2167b35f743c27432e6ac6a4dc3634c3b6acb
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Thu Apr 25 12:20:52 2019 +0300
test_oss launched if cicd node available
Change-Id: Ief119e18851b5ec39103195ca183db1d82fc5eb8
Related-Prod: #PROD-29775(PROD:29775)
commit 67aaec97464e5750388d760cb5d35672fd194419
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Mon Apr 15 18:05:13 2019 -0500
[CVP] Do not skip test_jenkins_jobs_branch by default
Change-Id: I2b636e089d77d17833f4839f55808369e1f1ebce
Related-PROD: PROD-29505
commit 4a79efda8e8151760cd54f2cc4b0561aaf536bc0
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Wed Apr 24 11:12:55 2019 +0300
Added tests to check Drivetrain on K8s
Change-Id: I86b9bbccf771cee6d6d294bb76f0c3979e269e86
Related-Prod: #PROD-29625(PROD:29625)
commit 4bfd2ee3f0e1b83ebb6928ea5a490be19b4c9166
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Wed Apr 10 21:56:58 2019 -0500
[CVP] Refactor salt client class
Change-Id: I91cfffe1c8d5df0224657ce9e36be9063b56f0b3
Related-PROD: PROD-28981
Related-PROD: PROD-28729
Related-PROD: PROD-28624
Related-PROD: PROD-29286
commit b7e866cfa45c2887c7b3671463774c3dc78cab26
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Wed Apr 10 13:49:56 2019 +0300
Set requirement for existed cicd-nodes in drivetrain tests
Related-Task: #PROD-28514(PROD:28514)
Change-Id: I95268fae93cb1fe0eed5276468d0e8e1512c92d2
commit 45ae6b65ca436867fcf5b6ac7144e9f837299ad3
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Tue Mar 5 18:52:44 2019 +0300
Added test to check mounted file systems on control plane VMs
Problem: sometimes it happens that after KVM host is rebooted,
its VMs come up with missing mounts (e.g. ctl node can have
keystone volumes missed). We had such an issue. It can happen
because of the hardware / host system performance issue or
network misconfiguration, but such issue can appear after
e.g. HA testing, or when KVM is rebooted. So, such test
will detect the inconsistent mounts.
Created the test to check that mounted file systems are
consistent on the virtual control plane nodes (e.g. on ctl,
prx, etc nodes). The nodes like kvm, cmp, ceph OSD, nodes
with docker (like k8s nodes, cid, mon) are skipped.
To skip other nodes if needed, add the node or group in the
config (skipped_nodes, skipped_groups).
Change-Id: Iab5311060790bd2fdfc8587e4cb8fc63cc3a0a13
Related-PROD: PROD-28247
commit 835b0cb957748e49e21bafd43c0ca9da60707e92
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Wed Apr 10 17:10:20 2019 +0300
[test_cinder_services] Verify backend existence before testing
Added docstring to the method
Change-Id: I511b9876e5a65f21a4cc823e616a29166b5b9cb4
Fixes-bug:#PROD-28523(PROD:28523)
commit 16a8f414ac8cc8d43404995b2002d3a943f893ca
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Mon Apr 8 17:10:38 2019 +0300
Move test_drivetrain_jenkins_job to the end of drivetrain tests queue to
avoid the failing of the test_drivetrain_jenkins_job
Additional changes:
* added docstring to test methods
* fixed pep8
Related-Bug: #PROD-27372(PROD:27372)
Change-Id: I34c679d66e483c107e6dda583b3c2e1ceeca5ced
commit b91c3147e20eb00e5429beefbb8e9a2e157bd3c0
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Tue Mar 26 16:49:44 2019 -0500
[CVP] Fix test_drivetrain_components_and_versions for new MCP updates logic
Related-PROD: PROD-28954
Change-Id: I9ea99b36115834da7d7110de8811730d11df4da4
commit cbf1f3ae648129b26fdd5183878ce7abab9cc794
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Tue Apr 9 20:02:10 2019 +0300
[cvp-spt] change adm_tenant in the create_subnet function
Change-Id: I8ebf04b658d5f17846c23f13670b7b63c1e9c771
Fixes-Issue: #PROD-29311(PROD:29311)
commit d52b5fe2722ea50eac65c5f8f2a55bab9f1db583
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Thu Mar 28 11:11:35 2019 -0500
[CVP] Fix test_jenkins_jobs_branch according to new MCP updates logic
Related-PROD: PROD-28954
Change-Id: I023fd6f57ac5f52642aa963cef5cbc9fc1a74264
commit ab919649a64e1a379e11d84d3c21604d027e9645
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Wed Mar 27 20:05:38 2019 +0200
[test_check_services] Change logic to check services
Change-Id: I1eb0ff077d497f95a0004bfd8ff4f25538acbfd6
Fix-bug: #PROD-26431(PROD:26431)
commit 8fd295c1a4b037b9aad5c1fe485351d4f9ed457c
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Thu Mar 7 13:46:43 2019 +0200
Add possibility to define list of services/modules/packages to skip
Change-Id: Ice289221e6e99181682ddf9155f390c388e590ad
Related-Prod: #PROD-27215(PROD:27215)
commit f139db45e7fe0cc9b62178dc8cd1f799344723a1
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Tue Mar 5 11:18:48 2019 +0300
Added test to check nova services, hosts are consistent
Added test to check nova hosts are consistent in nova-services,
openstack hosts and hypervisors. While deploying clouds, we faced
several times when nova hosts were inconsistent after deployment
(due to incorrect deployment steps), in this case hypervisor list
has some computes missing, but they are present in nova-services.
So, there can be some issues like "host is not mapped to any cell",
or boot VM error. So, it is better to check these nova lists are
consistent.
Related-PROD: PROD-28210
Change-Id: I9705417817e6075455dc4ccf5e25f2ab3439108c
commit 04ac2000016338fa283b9c34931ec3e96c595302
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Fri Mar 1 13:12:41 2019 +0200
cvp-spt, size check image to check Glance upload/download speed can be
changed using env var
It set to 2000 MB by default (because of free space on cid* nodes)
Test vm2vm gracefully skips test if no image found
Change-Id: I3aa5f50bf75b48df528de8c4196ae51c23de4b9e
Fixes-bug: #PROD-27763(PROD:27763)
commit 1ee3a651d10d6b32e1b34adef8c703e2036ffae1
Merge: 90ed2ea c4f520c
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Fri Mar 1 23:33:22 2019 +0000
Merge "Remove accidentally added file"
commit c4f520c98136b8aa35d3ec02f93244bb090da5c3
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Tue Feb 26 17:58:48 2019 -0600
Remove accidentally added file
Related-PROD: PROD-28153
Change-Id: Iee4c7b8fc59fd88fb148ed24776cae1af54998f1
commit 90ed2eadd9c1ce0f2f83d70b34a37144dc0da791
Merge: d006dbf 9b74486
Author: harhipova <harhipova@mirantis.com>
Date: Fri Mar 1 15:39:05 2019 +0000
Merge "Add ntp_skipped_nodes parameter, do not compare time on nodes with salt master"
commit d006dbf457567ab7e43d165e751cae5bf9fe64ff
Merge: 6661b23 24b71aa
Author: harhipova <harhipova@mirantis.com>
Date: Fri Mar 1 15:38:22 2019 +0000
Merge "Do not add node without virbr0* interfaces for comparison"
commit 6661b2332faad465a3e50bd6bf38f05731a95c9d
Merge: 5a0d02b 25215d9
Author: harhipova <harhipova@mirantis.com>
Date: Fri Mar 1 15:37:37 2019 +0000
Merge "Add more public url tests for UIs"
commit 24b71aa285748e8912fd780673f321f32e09a8c8
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Wed Feb 27 17:02:05 2019 -0600
Do not add node without virbr0* interfaces for comparison
Related-PROD: PROD-27217
Change-Id: I704290f5b0708b96e03cbbb96674fc4355639723
commit 9b74486023b04708c9db2ee45ba4d0f0f6410c6b
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Tue Feb 26 17:33:43 2019 -0600
Add ntp_skipped_nodes parameter, do not compare time on nodes with salt master
Related-PROD: PROD-21993
Related-PROD: PROD-27182
Change-Id: Id8247d0b28301d098569f2ae3bd08ff7cfcad154
commit 5a0d02b3f0dfc8e525e2bd49736a352a1e101d06
Merge: e792be5 90fdfb5
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Fri Feb 22 18:10:30 2019 +0000
Merge "Added test to check nodes status in MAAS"
commit 25215d9ededf612f3e9354e9a6232eea6b958bc6
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Thu Jan 31 16:35:57 2019 -0600
Add more public url tests for UIs
Related-PROD: PROD-23746
Change-Id: Ie680d0d934cf36f4147b9d9a079f53469d26eccc
commit 90fdfb5e3cccbba22f8fe60a2fe119cab7308b37
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Sun Jan 27 23:01:07 2019 +0300
Added test to check nodes status in MAAS
MAAS should have the nodes in 'Deployed' status. At the same time,
QA engineer can add some nodes in the skipped_nodes list and ignore
checking them.
Change-Id: I5407523f700fd76bb88cd5383c73cfce55cdd907
commit e792be50fa47222389e2e55f7d46e01b59a88e52
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Wed Feb 13 13:28:11 2019 +0200
Fix version parsing in test_drivetrain_components_and_versions
Change-Id: I3f036a7e3c324be8c50d6c5d7071ee12a5b3127e
Fixes-Bug: #PROD-27454(PROD:27454)
Closes-Task: #PROD-27253(PROD:27253)
commit 6baf78783bad9dbdf1fb1928077507f5f9a70a1a
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Fri Jan 25 19:09:30 2019 +0300
Added test to check all packages are latest
Added test to check that all packages on the nodes are latest and
are not upgradable. Added the possibility to skip some packages
in global_config if they should not be upgraded.
Problem description:
The 'test_check_package_versions' checks that the versions are
consistent across the nodes of the same group. But we have no
test that the actual versions are really correct and from correct
MCP release. I had the cloud that had some packages installed
from the wrong repositories, not from the required MCP release, so
the fix was to upgrade packages. So there is the need to have
the test to check that the packages are latest.
At the same time if several packages should not be upgraded and
have correct version even if Installed!=Candidate, there is
possibility to skip packages.
Currently the test is skipped by default ("skip_test": True in
global_config.yaml file). Set False to run the test.
Change-Id: Iddfab8b3d7fb4e72870aa0791e9da95a66f0ccfd
commit c48585ffded98bf907b98a69b61635829c48f2c4
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Mon Feb 4 19:38:54 2019 +0300
Added test to check ntpq peers state
The existing test 'test_ntp_sync' check the time is equal across
the nodes. Sometimes there can be some NTP issue, but the time
can be still correct. For example, Contrail can have "NTP state
unsynchronized" when noone from remote peers is chosen. So there
is some need to check "ntpq -pn" on the nodes and check the peers
state.
The new test gets ntpq peers state and check the system peer is
declared.
Change-Id: Icb8799b2323a446a3ec3dc6db54fd1d9de0356e5
commit ae0e72af8b63a65fb9e1fcfb7a626532da4c14b1
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Tue Feb 12 13:57:26 2019 +0200
Disabled test result for test_duplicate_ips. It reacts to ens3 networks
Relalet-Bug: #PROD-27449(PROD:27449)
Change-Id: Ia28dcf09a89b4a6bee8a746a7ce1a069b74ce8cf
commit 47e42daa5287c858daefbab8eeefe2d8f406feb5
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Tue Feb 12 11:49:26 2019 +0200
Disabled test results for test_cinder_services. It affects to test_drivetrain job voting
Fixes-Bug:#PROD-27436(PROD:27436)
Change-Id: I0da3365d7f51a8863b10d9450321c7f5119b842e
commit f9a95caa34f0eb1043e2c9655d096d0d69a6d4c2
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Wed Jan 30 15:47:00 2019 +0200
Add stacklight tests from stackligth-pytest repo
Change-Id: I2d2ea6201b6495c35bed57d71450b30b0e0ff49f
Relates-Task: #PROD-21318(PROD:21318)
commit f2660bdee650fa0240a3e9b34ca2b92f7d1d1e00
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Fri Feb 8 17:25:39 2019 +0200
Retry test for docker services replicas
Change-Id: Id4b983fe575516f33be4b401a005b23097c0fe96
Fixes-Bug: #PROD-27372(PROD:27372)
commit 6f34fbbfcb424f99e2a6c81ac4eb73ac4e40ce6b
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Fri Feb 8 11:19:41 2019 +0200
Change jenkins_jobs_branch test to check release branches
Change-Id: I2d0551f6291f79dc20b2d031e4e669c4009d0aa3
commit 42ed43a37b96846cddb1d69985f1e15780c8a697
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Sun Jan 27 23:58:35 2019 +0300
Added timeout in iperf command for Vm2Vm test
Added timeout in iperf command for Vm2Vm test for having some
statistics: sometimes 10s timeout is not enough when the network
speed is unstable.
Change-Id: I4912ccf8ba346a8b427cf6bd6181ce6e6c180fb2
commit 7c5f3fdef6477ac08dec4ace6630662b8adfe458
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Tue Feb 5 18:01:33 2019 +0300
Added test for K8S dashboard availability
In MCP the K8S dashboard is enabled by default. Need to check that
the dashboard is available.
Change-Id: I5b94ecce46d5f43491c9cf65a15a50461214e9c4
commit b8ec40e14917ec3b69dfcfe6ddcf36500dbc4754
Merge: 6dc2b00 ac4a14e
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Thu Jan 31 18:37:51 2019 +0000
Merge "Add a new test to check for duplicate IPs in an env"
commit 6dc2b00bc4b059968daa3d49775ec77e00b903ed
Merge: 09b1ae8 03af292
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Tue Jan 29 22:47:00 2019 +0000
Merge "Small fix: test nodes in ElasticSearch will get > than 500 nodes"
commit ac4a14e24f5bc1f53096627ae7d4f4cb60183ea0
Author: Dmitriy Kruglov <dkruglov@mirantis.com>
Date: Wed Jan 23 09:37:13 2019 +0100
Add a new test to check for duplicate IPs in an env
Change-Id: I08ad6b22f252a0f8ea5bc4a4edd2fe566826868b
Closes-PROD: #PROD-24347
commit 09b1ae88229cb8055a6c291097b3f6b0e0eb63c8
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Mon Jan 28 13:06:01 2019 +0300
Added StackLight UI tests for public endpoints
Change-Id: Ib60f278b77cc6673394c70b6b0ab16f74bc74366
commit df243ef14cbb7ab2707d2e7a2c292863f5010760
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Thu Nov 8 18:17:17 2018 +0300
Added UI tests for Alerta: internal and public addresses
The related fix https://gerrit.mcp.mirantis.com/#/c/34661
adds tests for public endpoints for the rest of
StackLight UI.
Change-Id: Ie94ea242b19e30b7ed7143e01444125182fb6305
commit ac850455f686f1092077e2c95c9ab0d466f099c6
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Sun Jan 27 22:31:38 2019 +0300
Added test to check minions status
The CVP Sanity tests skip the nodes automatically if this minion
does not respond in 1 sec (salt_timeout in config) time.
Sometimes all the tests can pass, but some KVM nodes along with its
Control plane VMs can be down and CVP tests will not test this and
will not inform about this.
The tests check that all minions are up.
Change-Id: Ib8495aeb043448b36aea85bb31ee2650d655075e
commit 03af292569edc29db72bbdf97a331eceab3dc05c
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Mon Jan 28 15:55:02 2019 +0300
Small fix: test nodes in ElasticSearch will get > than 500 nodes
Some big Production clouds have more than 500 nodes in total.
So the test is not valid for such cloud: it will fetch only
500 nodes instead of all nodes of the cloud. Changing the request
to fetch 1000 nodes.
Change-Id: I58493fc55e1deb2c988d61e7c8a4f8ed971a60d4
commit 16e93fb7375fdfb87901b4a074f17ef09e722e56
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Wed Jan 23 19:03:01 2019 +0200
Renamed folder with tests to make them consistent with cvp-runner.groovy
and CVP jobs in cluster Jenkins
Return rsync service into inconsistency_rule
Related-Task: #PROD-23604(PROD:23604)
Change-Id: I94afe350bd1d9c184bafe8e9e270aeb4c6c24c50
commit 27a41d814cc9d4f5bbc7f780a3d9e6042a6aaa4c
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Thu Jan 17 17:40:40 2019 +0200
Check kubectl on kubernetes:master only
Change-Id: I8ae308eb903694feffe65b14f7f857dfaf6b689c
Fixes-Bug: #PROD-26555(PROD:26555)
commit 55cc129f3e93a3801a4abf620b40c1e5d7c53fe7
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Tue Jan 8 14:22:18 2019 +0200
Common Dockerfile for CVP-Sanity and CVP-SPT
Related-Task: #PROD-26312(PROD:26312)
Change-Id: I457a8d5c6ff73d944518f6b0c2c568f8286728a9
commit 753a03e19780b090776ce5e2c27d74c44c5750a3
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Tue Jan 15 17:35:25 2019 -0600
[CVP] Add checks for docker_registry, docker_visualizer and cvp jobs version
Related-PROD: PROD-21801
Change-Id: I79c8c5eb0833aca6d077129e3ec81ff3afb06143
commit 7b70537c2b7bfe29d1dc84915a21da5238f120f0
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Tue Jan 15 18:40:29 2019 -0600
[CVP] Get drivetrain_version parameter from reclass
Related-PROD: PROD-21801
Change-Id: I628480b053e7b03c09c55d5b997e9dc74aa98c90
commit aaa8e6e95861e4e3f51c4d28dc7fcb0ed8ab8578
Merge: c0a7f0c 30bd90c
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Fri Jan 11 16:45:10 2019 +0000
Merge "Add assert for return code"
commit 30bd90c986234343aabf03ec5f174026d02d4988
Author: Tatyana Leontovich <tleontovich@mirantis.com>
Date: Fri Jan 11 16:26:32 2019 +0200
Add assert for return code
* Add assertion that reposne was sucess
before process response.text()
* Add header to request to avoid 406 error
Change-Id: If41598e8c1ef5d9bf36847a750008d1203b4ed84
Closes-Prod: PROD-26423
commit c0a7f0c01adc6a391f662dc306902cde138658ce
Author: Tatyana Leontovich <tleontovich@mirantis.com>
Date: Fri Jan 11 16:08:50 2019 +0200
Remove rsync service from inconsistency_rule
Rsync service exists on all kvm nodes, so that
remove it from inconsistency_rule to avail false -negative results
Change-Id: I25ce5db2990645992c8fa7fb6cc33f082903b295
Closes-PROD: PROD-26431
commit 5d965b230b4b5348d425510dc4667ced0c7e8ec3
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Wed Jan 9 16:29:31 2019 -0600
Fix ceph tests filename
Change-Id: I67ffc9f4da27d8b64c0334f3a6ae3f8f05dcd3b2
commit 0763a040044b20f8229292b547798d4ed99ca7e3
Merge: b8d04d5 f77b50b
Author: Oleksii Zhurba <ozhurba@mirantis.com>
Date: Wed Jan 9 21:47:45 2019 +0000
Merge "Added Ceph health test"
commit b8d04d575b1cb8b05aace823de64d74e0d0e4c48
Author: Hanna Arhipova <harhipova@mirantis.com>
Date: Fri Dec 28 13:19:17 2018 +0200
Remove ldap_server component from test_drivetrain_components_and_versions
Change-Id: Idc8e8581f828334db511b0ca2149ad812e71a6c3
Fixes-Bug: #PROD-26151(PROD:26151)
commit f77b50bdb2b1fb2b747ac8e1b1262ee88fdfd2ed
Author: Ievgeniia Zadorozhna <izadorozhna@mirantis.com>
Date: Wed Dec 12 19:41:15 2018 +0300
Added Ceph health test
Change-Id: If2f318169e841cdd278c76c237f040b09d7b87ea
Change-Id: I9687de3dbdae9d5dc3deb94dbd2afcd5e7f0ec7d
diff --git a/test_set/cvp-sanity/utils/__init__.py b/test_set/cvp-sanity/utils/__init__.py
new file mode 100644
index 0000000..62ccae7
--- /dev/null
+++ b/test_set/cvp-sanity/utils/__init__.py
@@ -0,0 +1,195 @@
+import os
+import yaml
+import requests
+import re
+import sys, traceback
+import time
+
+
+class AuthenticationError(Exception):
+ pass
+
+
+class salt_remote:
+ def __init__(self):
+ self.config = get_configuration()
+ self.skipped_nodes = self.config.get('skipped_nodes') or []
+ self.url = self.config['SALT_URL'].strip()
+ if not re.match("^(http|https)://", self.url):
+ raise AuthenticationError("Salt URL should start \
+ with http or https, given - {}".format(url))
+ self.login_payload = {'username': self.config['SALT_USERNAME'],
+ 'password': self.config['SALT_PASSWORD'], 'eauth': 'pam'}
+ # TODO: proxies
+ self.proxies = {"http": None, "https": None}
+ self.expires = ''
+ self.cookies = []
+ self.headers = {'Accept': 'application/json'}
+ self._login()
+
+ def _login (self):
+ try:
+ login_request = requests.post(os.path.join(self.url, 'login'),
+ headers={'Accept': 'application/json'},
+ data=self.login_payload,
+ proxies=self.proxies)
+ if not login_request.ok:
+ raise AuthenticationError("Authentication to SaltMaster failed")
+ except Exception as e:
+ print ("\033[91m\nConnection to SaltMaster "
+ "was not established.\n"
+ "Please make sure that you "
+ "provided correct credentials.\n"
+ "Error message: {}\033[0m\n".format(e.message or e))
+ traceback.print_exc(file=sys.stdout)
+ sys.exit()
+ self.expire = login_request.json()['return'][0]['expire']
+ self.cookies = login_request.cookies
+ self.headers['X-Auth-Token'] = login_request.json()['return'][0]['token']
+
+ def cmd(self, tgt, fun='cmd.run', param=None, expr_form=None, tgt_type=None, check_status=False, retries=3):
+ if self.expire < time.time() + 300:
+ self.headers['X-Auth-Token'] = self._login()
+ accept_key_payload = {'fun': fun, 'tgt': tgt, 'client': 'local',
+ 'expr_form': expr_form, 'tgt_type': tgt_type,
+ 'timeout': self.config['salt_timeout']}
+ if param:
+ accept_key_payload['arg'] = param
+
+ for i in range(retries):
+ request = requests.post(self.url, headers=self.headers,
+ data=accept_key_payload,
+ cookies=self.cookies,
+ proxies=self.proxies)
+ if not request.ok or not isinstance(request.json()['return'][0], dict):
+ print("Salt master is not responding or response is incorrect. Output: {}".format(request))
+ continue
+ response = request.json()['return'][0]
+ result = {key: response[key] for key in response if key not in self.skipped_nodes}
+ if check_status:
+ if False in result.values():
+ print(
+ "One or several nodes are not responding. Output {}".format(json.dumps(result, indent=4)))
+ continue
+ break
+ else:
+ raise Exception("Error with Salt Master response")
+ return result
+
+ def test_ping(self, tgt, expr_form='pillar'):
+ return self.cmd(tgt=tgt, fun='test.ping', param=None, expr_form=expr_form)
+
+ def cmd_any(self, tgt, param=None, expr_form='pillar'):
+ """
+ This method returns first non-empty result on node or nodes.
+ If all nodes returns nothing, then exception is thrown.
+ """
+ response = self.cmd(tgt=tgt, param=param, expr_form=expr_form)
+ for node in response.keys():
+ if response[node] or response[node] == '':
+ return response[node]
+ else:
+ raise Exception("All minions are down")
+
+ def pillar_get(self, tgt='salt:master', param=None, expr_form='pillar', fail_if_empty=False):
+ """
+ This method is for fetching pillars only.
+ Returns value for pillar, False (if no such pillar) or if fail_if_empty=True - exception
+ """
+ response = self.cmd(tgt=tgt, fun='pillar.get', param=param, expr_form=expr_form)
+ for node in response.keys():
+ if response[node] or response[node] != '':
+ return response[node]
+ else:
+ if fail_if_empty:
+ raise Exception("No pillar found or it is empty.")
+ else:
+ return False
+
+
+def init_salt_client():
+ local = salt_remote()
+ return local
+
+
+def list_to_target_string(node_list, separator, add_spaces=True):
+ if add_spaces:
+ separator = ' ' + separator.strip() + ' '
+ return separator.join(node_list)
+
+
+def calculate_groups():
+ config = get_configuration()
+ local_salt_client = init_salt_client()
+ node_groups = {}
+ nodes_names = set ()
+ expr_form = ''
+ all_nodes = set(local_salt_client.test_ping(tgt='*',expr_form=None))
+ print all_nodes
+ if 'groups' in config.keys() and 'PB_GROUPS' in os.environ.keys() and \
+ os.environ['PB_GROUPS'].lower() != 'false':
+ nodes_names.update(config['groups'].keys())
+ expr_form = 'compound'
+ else:
+ for node in all_nodes:
+ index = re.search('[0-9]{1,3}$', node.split('.')[0])
+ if index:
+ nodes_names.add(node.split('.')[0][:-len(index.group(0))])
+ else:
+ nodes_names.add(node)
+ expr_form = 'pcre'
+
+ gluster_nodes = local_salt_client.test_ping(tgt='I@salt:control and '
+ 'I@glusterfs:server',
+ expr_form='compound')
+ kvm_nodes = local_salt_client.test_ping(tgt='I@salt:control and not '
+ 'I@glusterfs:server',
+ expr_form='compound')
+
+ for node_name in nodes_names:
+ skipped_groups = config.get('skipped_groups') or []
+ if node_name in skipped_groups:
+ continue
+ if expr_form == 'pcre':
+ nodes = local_salt_client.test_ping(tgt='{}[0-9]{{1,3}}'.format(node_name),
+ expr_form=expr_form)
+ else:
+ nodes = local_salt_client.test_ping(tgt=config['groups'][node_name],
+ expr_form=expr_form)
+ if nodes == {}:
+ continue
+
+ node_groups[node_name]=[x for x in nodes
+ if x not in config['skipped_nodes']
+ if x not in gluster_nodes.keys()
+ if x not in kvm_nodes.keys()]
+ all_nodes = set(all_nodes - set(node_groups[node_name]))
+ if node_groups[node_name] == []:
+ del node_groups[node_name]
+ if kvm_nodes:
+ node_groups['kvm'] = kvm_nodes.keys()
+ node_groups['kvm_gluster'] = gluster_nodes.keys()
+ all_nodes = set(all_nodes - set(kvm_nodes.keys()))
+ all_nodes = set(all_nodes - set(gluster_nodes.keys()))
+ if all_nodes:
+ print ("These nodes were not collected {0}. Check config (groups section)".format(all_nodes))
+ return node_groups
+
+
+def get_configuration():
+ """function returns configuration for environment
+ and for test if it's specified"""
+ global_config_file = os.path.join(
+ os.path.dirname(os.path.abspath(__file__)), "../global_config.yaml")
+ with open(global_config_file, 'r') as file:
+ global_config = yaml.load(file)
+ for param in global_config.keys():
+ if param in os.environ.keys():
+ if ',' in os.environ[param]:
+ global_config[param] = []
+ for item in os.environ[param].split(','):
+ global_config[param].append(item)
+ else:
+ global_config[param] = os.environ[param]
+
+ return global_config