Extend wait_for maas.py, wait_for_* attempts arg
maas.py: Extend wait_for states with timeout param
Extend the wait_for states with a timeout parameter.
The timeout value is taken from reclass pillar data if
defined. Oterwise, the states use the default value.
Based on Ting's PR [1], slightly refactored.
[1] https://github.com/salt-formulas/salt-formula-maas/pull/34
Signed-off-by: ting wu <ting.wu@enea.com>
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
maas.py: wait_for_*: Add attempts arg
Introduce a new parameter that allows a maximum number of automatic
recovery attempts for the common failures w/ machine operations.
If not present in pillar data, it defaults to 0 (OFF).
Common error states, possible cause and automatic recovery pattern:
* New
- usually indicates issues with BMC connectivity (no network route,
but on rare occassions it happens due to MaaS API being flaky);
- fix: delete the machine, (re)process machine definitions;
* Failed commissioning
- various causes, usually a simple retry works;
- fix: delete the machine, (re)process machine definitions;
* Failed testing
- incompatible hardware, missing drivers etc.
- usually consistent and board-specific;
- fix: override failed testing
* Allocated
- on rare ocassions nodes get stuck in this state instead 'Deploy';
- fix: mark-broken, mark-fixed, if it failed at least once before
perform a fio test (fixes another unrelated spurious issue with
encrypted disks from previous deployments), (re)deploy machines;
* Failed deployment
- various causes, usually a simple retry works;
- fix: same as for nodes stuck in 'Allocated';
Relateed: PROD-28390(PROD:28390)
Change-Id: Ifb7dd9f8fcfbbed557e47d8fdffb1f963604fb15
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
(cherry picked from commit 4fa108e39fbf4da924f0bcbf01ff1625f13910a1)
diff --git a/README.rst b/README.rst
index a82ef15..0010113 100644
--- a/README.rst
+++ b/README.rst
@@ -772,12 +772,16 @@
machines:
- kvm01
- kvm02
- timeout: 1200 # in seconds
+ timeout: {{ region.timeout.ready }}
+ attempts: {{ region.timeout.attempts }}
req_status: "Ready"
- require:
- cmd: maas_login_admin
...
+The timeout setting is taken from the reclass pillar data.
+If the pillar data is not defined, it will use the default value.
+
If module run w/\o any extra paremeters,
``wait_for_machines_ready`` will wait for defined in salt
machines. In this case, it is usefull to skip some machines:
@@ -792,7 +796,8 @@
module.run:
- name: maas.wait_for_machine_status
- kwargs:
- timeout: 1200 # in seconds
+ timeout: {{ region.timeout.deployed }}
+ attempts: {{ region.timeout.attempts }}
req_status: "Deployed"
ignore_machines:
- kvm01 # in case it's broken or whatever