tree c15c7d53ec2d8defdda74f35eb85ddb15eb1a7ac
parent a64e5b2574b9470e398f6921b318e8c7ab7fb7d1
author azvyagintsev <azvyagintsev@mirantis.com> 1560422344 +0300
committer azvyagintsev <azvyagintsev@mirantis.com> 1560497583 +0300

Make runPepper more resistant against SaltReqTimeoutError

 * In rare condition, salt zmq threads may flap with raise
   SaltReqTimeoutError. During those period, salt-api may return
   50X answer and drop whole deployment process.
   Unfortunatly, in salt no option to increase SaltReqTimeoutError
   for all threads, (default 3 for many different threads).
   Salt itself allow to configure only few *_tries option for exact
   threads(like auth or job render).
 * Those patch add simply crutch with retry, in case 50X error
   has been detected in stderr from pepper call
 * Misc:
   - Extend mk.Python.runVirtualenvCommand
   - Extend mk.Common.runPepperCommand
     - add failover for 50X and retry
   - Refactor mirantis.mk.Common.shCmdStatus

Prod-relaeted: PROD-30839(PROD:30839)

Change-Id: I18b152c5f22c8fb602a21a34ea06a4c543d8ae26