tree 5e4af595b59395d4203d7aadb91a92f201f9df48
parent 90c9348f2cabdc9c4f3debb222350ba12ec808ea
author Oleksii Molchanov <omolchanov@mirantis.com> 1644853129 +0200
committer Oleksiy Molchanov <omolchanov@mirantis.com> 1644853171 +0000

SRE-78: Implement Multiworker Configuration

- In the booking.com clouds, we find that fluentd cannot keep up with
volume of logs it is expected to process.
- Research into https://docs.fluentd.org/deployment/multi-process-workers
has provided us with the option to configure a multiworker setup, which has
largely resolved the problem.
- MCP largely uses the systemd and tail inputs, which do not support
operating on multiple workers. However, we can still assign them to separate
workers on a multiworker setup. Since all workers will process outputs, we can
easily separate concerns this way and remove CPU bottlenecks.
- By default, we configure these inputs to work on worker 0.

Related-Prod: PROD-36710
Change-Id: I68af85f1a62da72fa2bf1f996e52465af3caa5a5