Make runPepper more resistant against SaltReqTimeoutError
* In rare condition, salt zmq threads may flap with raise
SaltReqTimeoutError. During those period, salt-api may return
50X answer and drop whole deployment process.
Unfortunatly, in salt no option to increase SaltReqTimeoutError
for all threads, (default 3 for many different threads).
Salt itself allow to configure only few *_tries option for exact
threads(like auth or job render).
* Those patch add simply crutch with retry, in case 50X error
has been detected in stderr from pepper call
* Misc:
- Extend mk.Python.runVirtualenvCommand
- Extend mk.Common.runPepperCommand
- add failover for 50X and retry
- Refactor mirantis.mk.Common.shCmdStatus
Prod-relaeted: PROD-30839(PROD:30839)
Change-Id: I18b152c5f22c8fb602a21a34ea06a4c543d8ae26
diff --git a/src/com/mirantis/mk/Python.groovy b/src/com/mirantis/mk/Python.groovy
index c326bf6..8d9d19b 100644
--- a/src/com/mirantis/mk/Python.groovy
+++ b/src/com/mirantis/mk/Python.groovy
@@ -53,22 +53,27 @@
/**
* Run command in specific python virtualenv
*
- * @param path Path to virtualenv
- * @param cmd Command to be executed
+ * @param path Path to virtualenv
+ * @param cmd Command to be executed
* @param silent dont print any messages (optional, default false)
+ * @param flexAnswer return answer like a dict, with format ['status' : int, 'stderr' : str, 'stdout' : str ]
*/
-def runVirtualenvCommand(path, cmd, silent = false) {
+def runVirtualenvCommand(path, cmd, silent = false, flexAnswer = false) {
def common = new com.mirantis.mk.Common()
-
- virtualenv_cmd = "set +x; . ${path}/bin/activate; ${cmd}"
+ def res
+ def virtualenv_cmd = "set +x; . ${path}/bin/activate; ${cmd}"
if (!silent) {
common.infoMsg("[Python ${path}] Run command ${cmd}")
}
- output = sh(
- returnStdout: true,
- script: virtualenv_cmd
- ).trim()
- return output
+ if (flexAnswer) {
+ res = common.shCmdStatus(virtualenv_cmd)
+ } else {
+ res = sh(
+ returnStdout: true,
+ script: virtualenv_cmd
+ ).trim()
+ }
+ return res
}
/**