blob: eb8a67bc1bb0e82f06afb59e4f6d7c532a0a23c2 [file] [log] [blame]
Filip Pytloun5163dfb2015-10-06 16:28:31 +02001
2=============
Ales Komarekdd881a62015-12-08 11:03:49 +01003Elasticsearch
Filip Pytloun5163dfb2015-10-06 16:28:31 +02004=============
5
Ales Komarekdd881a62015-12-08 11:03:49 +01006Elasticsearch provides a distributed, multitenant-capable full-text search engine with a HTTP web interface and schema-free JSON documents.
7
Filip Pytloun5163dfb2015-10-06 16:28:31 +02008Sample pillars
9==============
10
Filip Pytloun63c8c252016-05-05 16:14:02 +020011Single-node elasticsearch with clustering disabled:
12
Filip Pytloun5163dfb2015-10-06 16:28:31 +020013.. code-block:: yaml
14
15 elasticsearch:
16 server:
17 enabled: true
Filip Pytloun5163dfb2015-10-06 16:28:31 +020018 bind:
19 address: 0.0.0.0
20 port: 9200
Filip Pytloun63c8c252016-05-05 16:14:02 +020021 cluster:
22 multicast: false
23 index:
24 shards: 1
25 replicas: 0
26
Filip Pytloundf5209c2016-09-23 11:55:56 +020027Setup shared repository for snapshots:
28
29.. code-block:: bash
30
31 elasticsearch:
32 server:
33 snapshot:
34 reponame:
35 path: /var/lib/glusterfs/repo
36 compress: true
37
Filip Pytloun63c8c252016-05-05 16:14:02 +020038Cluster with manually defined members:
39
40.. code-block:: yaml
41
42 elasticsearch:
43 server:
44 enabled: true
45 bind:
46 address: 0.0.0.0
47 port: 9200
48 cluster:
49 multicast: false
50 members:
51 - host: elastic01
52 port: 9300
53 - host: elastic02
54 port: 9300
55 - host: elastic03
56 port: 9300
57 index:
58 shards: 5
59 replicas: 1
Filip Pytloun5163dfb2015-10-06 16:28:31 +020060
Filip Pytloun911588c2016-06-29 16:27:53 +020061Common definition for curator:
62
63.. code-block:: yaml
64
65 elasticsearch:
66 server:
67 curator:
68 timeout: 900
69 logfile: /var/log/elasticsearch/curator.log
70 logformat: json
Simon Pasquier0b138142016-12-02 09:56:08 +010071 master_only: true
Filip Pytloun911588c2016-06-29 16:27:53 +020072 actions:
73 - action: delete_indices
74 description: >-
75 Delete indices older than 45 days (based on index name).
76 Ignore the error if the filter does not result in an actionable
77 list of indices (ignore_empty_list) and exit cleanly.
78 options:
79 ignore_empty_list: True
80 continue_if_exception: False
81 disable_action: False
82 filters:
83 - filtertype: pattern
84 kind: regex
85 value: '.*\-\d\d\d\d\.\d\d\.\d\d$'
86 - filtertype: age
87 source: name
88 direction: older
89 timestring: '%Y.%m.%d'
90 unit: days
91 unit_count: 90
92 - action: replicas
93 description: >-
94 Reduce the replica count to 0 for indices older than 30 days
95 (based on index creation_date)
96 options:
97 count: 0
98 wait_for_completion: False
99 continue_if_exception: False
100 disable_action: False
101 filters:
102 - filtertype: pattern
103 kind: regex
104 value: '.*\-\d\d\d\d\.\d\d\.\d\d$'
105 - filtertype: age
106 source: creation_date
107 direction: older
108 unit: days
109 unit_count: 30
110 - action: forcemerge
111 description: >-
112 forceMerge indices older than 2 days (based on index
113 creation_date) to 2 segments per shard. Delay 120 seconds
114 between each forceMerge operation to allow the cluster to
115 quiesce.
116 This action will ignore indices already forceMerged to the same
117 or fewer number of segments per shard, so the 'forcemerged'
118 filter is unneeded.
119 options:
120 max_num_segments: 2
121 delay: 120
122 continue_if_exception: False
123 disable_action: True
124 filters:
125 - filtertype: pattern
126 kind: regex
127 value: '.*\-\d\d\d\d\.\d\d\.\d\d$'
128 - filtertype: age
129 source: creation_date
130 direction: older
131 unit: days
132 unit_count: 2
133
Guillaume Thouvenind39b3522016-11-17 10:49:58 +0100134Client setup
135------------
136
Guillaume Thouvenin126eef82016-11-17 15:59:28 +0100137Client with host and port:
Guillaume Thouvenind39b3522016-11-17 10:49:58 +0100138
139.. code-block:: yaml
140
141 elasticsearch:
142 client:
143 enabled: true
144 server:
145 host: elasticsearch.host
146 port: 9200
147
Volodymyr Stoikoce2d5ca2017-06-06 09:56:56 +0300148Client where you download an index template that is stored in the directory
Guillaume Thouvenin126eef82016-11-17 15:59:28 +0100149*files/*:
150
151.. code-block:: yaml
152
153 elasticsearch:
154 client:
155 enabled: true
156 server:
157 host: elasticsearch.host
158 port: 9200
159 index:
160 my_index:
161 enabled: true
162 template: elasticsearch/files/my_index_template.json
163
Volodymyr Stoikoce2d5ca2017-06-06 09:56:56 +0300164Client where you download an index template from the metadata definition and force index creation:
165
166.. code-block:: yaml
167
168 elasticsearch:
169 client:
170 enabled: true
171 server:
172 host: elasticsearch.host
173 port: 9200
174 index:
175 my_index:
176 enabled: true
177 force_operation: true
178 definition:
179 template: notifications
180 settings:
181 number_of_shards: 5
182 number_of_replicas: 1
183 mappings:
184 notification:
185 properties:
186 applicationId:
187 type: long
188 content:
189 type: text
190 fields:
191 keyword:
192 type: keyword
193 ignore_above: 256
194
Filip Pytloun5163dfb2015-10-06 16:28:31 +0200195Read more
196=========
197
Ales Komarekdd881a62015-12-08 11:03:49 +0100198
199* https://www.elastic.co/
Filip Pytloun5163dfb2015-10-06 16:28:31 +0200200* http://alex.nederlof.com/blog/2012/11/19/installing-elasticsearch-with-jenkins-on-ubuntu/
201* http://websightdesigns.com/wiki/Setting_up_Centralized_Event_Parsing_on_Ubuntu_12.04
202* https://gist.github.com/wingdspur/2026107
Filip Pytlounee64bd12017-02-02 13:02:03 +0100203
204Documentation and Bugs
205======================
206
207To learn how to install and update salt-formulas, consult the documentation
208available online at:
209
210 http://salt-formulas.readthedocs.io/
211
212In the unfortunate event that bugs are discovered, they should be reported to
213the appropriate issue tracker. Use Github issue tracker for specific salt
214formula:
215
216 https://github.com/salt-formulas/salt-formula-elasticsearch/issues
217
218For feature requests, bug reports or blueprints affecting entire ecosystem,
219use Launchpad salt-formulas project:
220
221 https://launchpad.net/salt-formulas
222
223You can also join salt-formulas-users team and subscribe to mailing list:
224
225 https://launchpad.net/~salt-formulas-users
226
227Developers wishing to work on the salt-formulas projects should always base
228their work on master branch and submit pull request against specific formula.
229
230 https://github.com/salt-formulas/salt-formula-elasticsearch
231
232Any questions or feedback is always welcome so feel free to join our IRC
233channel:
234
235 #salt-formulas @ irc.freenode.net