Log collector module
New:
- [Done] multiple namespace selector
- [Done] keyword-based pod selector
- [Done] per-pod logs syntax detection and parsing
- [Differed] in-place filtering for shorter logs
- [Done] individual logs timestamp detection
- [Done] Unix time bases Timestamp sorting
- [Done] Single file logs output using common format
- [Done] add all log types from all MOS namespaces and pods
Update:
- resource preparation can be skipped per module
- updated log collection using multiple threads
- new setting LOG_COLLECT_THREADS
Fixes:
- Network MTU fix
- Faster cmd execution on single pod
- Ceph benchmark validations
- Ceph benchmark report sorting
- Daemonset deployment with nodes skipped
- Network tree debugging script
- Tree depth limiter, i.e. stackoverflow prevention
Related-PROD: PROD-36845
Change-Id: Icf229ac62078c6418ab4dbdff12b0d27ed42af1d
diff --git a/cfg_checker/common/settings.py b/cfg_checker/common/settings.py
index 08d4163..27482ff 100644
--- a/cfg_checker/common/settings.py
+++ b/cfg_checker/common/settings.py
@@ -209,6 +209,7 @@
self.mcp_host = _get_env_value('MCP_ENV_HOST', None)
self.salt_port = _get_env_value('MCP_SALT_PORT', '6969')
self.threads = int(_get_env_value('MCP_THREADS', "5"))
+ self.sage_threads = int(_get_env_value('LOG_COLLECT_THREADS', "15"))
self.script_execution_timeout = int(
_get_env_value('MCP_SCRIPT_RUN_TIMEOUT', "300")
)
@@ -489,3 +490,5 @@
# Init vars that is specific to detected envs only
logger_cli.debug("... loading detected environment type vars")
self._init_env_values()
+ # Set internal resource preparation flag default
+ self.prepare_qa_resources = True