Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 1 | |
Aleš Komárek | 7215285 | 2017-04-11 13:48:48 +0200 | [diff] [blame] | 2 | ============ |
| 3 | Nova Formula |
| 4 | ============ |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 5 | |
Jakub Pavlik | fcf34f8 | 2016-05-20 09:35:51 +0200 | [diff] [blame] | 6 | OpenStack Nova provides a cloud computing fabric controller, supporting a wide |
| 7 | variety of virtualization technologies, including KVM, Xen, LXC, VMware, and |
| 8 | more. In addition to its native API, it includes compatibility with the |
| 9 | commonly encountered Amazon EC2 and S3 APIs. |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 10 | |
Aleš Komárek | 7215285 | 2017-04-11 13:48:48 +0200 | [diff] [blame] | 11 | Sample Pillars |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 12 | ============== |
| 13 | |
| 14 | Controller nodes |
| 15 | ---------------- |
| 16 | |
| 17 | Nova services on the controller node |
| 18 | |
| 19 | .. code-block:: yaml |
| 20 | |
| 21 | nova: |
| 22 | controller: |
| 23 | version: juno |
| 24 | enabled: true |
| 25 | security_group: true |
Lachlan Evenson | b72de50 | 2016-01-20 15:34:04 -0800 | [diff] [blame] | 26 | cpu_allocation_ratio: 8.0 |
| 27 | ram_allocation_ratio: 1.0 |
Jiri Konecny | 9344a37 | 2016-03-21 19:25:48 +0100 | [diff] [blame] | 28 | disk_allocation_ratio: 1.0 |
Jiri Konecny | b5a80e4 | 2016-03-22 11:51:01 +0100 | [diff] [blame] | 29 | workers: 8 |
Jakub Pavlik | 617a896 | 2016-09-04 18:50:06 +0200 | [diff] [blame] | 30 | report_interval: 60 |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 31 | bind: |
| 32 | public_address: 10.0.0.122 |
| 33 | public_name: openstack.domain.com |
| 34 | novncproxy_port: 6080 |
| 35 | database: |
| 36 | engine: mysql |
| 37 | host: 127.0.0.1 |
| 38 | port: 3306 |
| 39 | name: nova |
| 40 | user: nova |
| 41 | password: pwd |
| 42 | identity: |
| 43 | engine: keystone |
| 44 | host: 127.0.0.1 |
| 45 | port: 35357 |
| 46 | user: nova |
| 47 | password: pwd |
| 48 | tenant: service |
| 49 | message_queue: |
| 50 | engine: rabbitmq |
| 51 | host: 127.0.0.1 |
| 52 | port: 5672 |
| 53 | user: openstack |
| 54 | password: pwd |
| 55 | virtual_host: '/openstack' |
| 56 | network: |
| 57 | engine: neutron |
| 58 | host: 127.0.0.1 |
| 59 | port: 9696 |
Jakub Pavlik | 617a896 | 2016-09-04 18:50:06 +0200 | [diff] [blame] | 60 | extension_sync_interval: 600 |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 61 | identity: |
| 62 | engine: keystone |
| 63 | host: 127.0.0.1 |
| 64 | port: 35357 |
| 65 | user: neutron |
| 66 | password: pwd |
| 67 | tenant: service |
| 68 | metadata: |
| 69 | password: password |
Petr Michalec | aa23dc0 | 2016-11-29 16:30:25 +0100 | [diff] [blame] | 70 | audit: |
| 71 | enabled: false |
Simon Pasquier | 8683b7a | 2017-02-03 16:00:16 +0100 | [diff] [blame] | 72 | osapi_max_limit: 500 |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 73 | |
Jiri Konecny | e31f2c5 | 2016-04-14 17:16:02 +0200 | [diff] [blame] | 74 | |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 75 | Nova services from custom package repository |
| 76 | |
| 77 | .. code-block:: yaml |
| 78 | |
| 79 | nova: |
| 80 | controller: |
| 81 | version: juno |
| 82 | source: |
| 83 | engine: pkg |
| 84 | address: http://... |
| 85 | .... |
| 86 | |
Jiri Konecny | e31f2c5 | 2016-04-14 17:16:02 +0200 | [diff] [blame] | 87 | |
| 88 | Client-side RabbitMQ HA setup |
| 89 | |
| 90 | .. code-block:: yaml |
| 91 | |
| 92 | nova: |
| 93 | controller: |
| 94 | .... |
| 95 | message_queue: |
| 96 | engine: rabbitmq |
| 97 | members: |
| 98 | - host: 10.0.16.1 |
| 99 | - host: 10.0.16.2 |
| 100 | - host: 10.0.16.3 |
| 101 | user: openstack |
| 102 | password: pwd |
| 103 | virtual_host: '/openstack' |
| 104 | .... |
| 105 | |
| 106 | |
Petr Michalec | aa23dc0 | 2016-11-29 16:30:25 +0100 | [diff] [blame] | 107 | Enable auditing filter, ie: CADF |
| 108 | |
| 109 | .. code-block:: yaml |
| 110 | |
| 111 | nova: |
| 112 | controller: |
Simon Pasquier | 6a3c8f7 | 2016-12-19 15:37:24 +0100 | [diff] [blame] | 113 | audit: |
Petr Michalec | aa23dc0 | 2016-11-29 16:30:25 +0100 | [diff] [blame] | 114 | enabled: true |
| 115 | .... |
| 116 | filter_factory: 'keystonemiddleware.audit:filter_factory' |
| 117 | map_file: '/etc/pycadf/nova_api_audit_map.conf' |
| 118 | .... |
| 119 | |
| 120 | |
Ondrej Smola | 25b53cb | 2017-04-28 10:56:19 +0200 | [diff] [blame] | 121 | Enable CORS parameters |
| 122 | |
| 123 | .. code-block:: yaml |
| 124 | |
| 125 | nova: |
| 126 | controller: |
| 127 | cors: |
| 128 | allowed_origin: https:localhost.local,http:localhost.local |
| 129 | expose_headers: X-Auth-Token,X-Openstack-Request-Id,X-Subject-Token |
| 130 | allow_methods: GET,PUT,POST,DELETE,PATCH |
| 131 | allow_headers: X-Auth-Token,X-Openstack-Request-Id,X-Subject-Token |
| 132 | allow_credentials: True |
| 133 | max_age: 86400 |
| 134 | |
Dmitry Ukov | 3562a08 | 2017-05-04 00:00:48 +0400 | [diff] [blame] | 135 | Configuration of policy.json file |
| 136 | |
| 137 | .. code-block:: yaml |
| 138 | |
| 139 | nova: |
| 140 | controller: |
| 141 | .... |
| 142 | policy: |
| 143 | context_is_admin: 'role:admin or role:administrator' |
| 144 | 'compute:create': 'rule:admin_or_owner' |
| 145 | # Add key without value to remove line from policy.json |
| 146 | 'compute:create:attach_network': |
Ondrej Smola | 25b53cb | 2017-04-28 10:56:19 +0200 | [diff] [blame] | 147 | |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 148 | Compute nodes |
| 149 | ------------- |
| 150 | |
| 151 | Nova controller services on compute node |
| 152 | |
| 153 | .. code-block:: yaml |
| 154 | |
| 155 | nova: |
| 156 | compute: |
| 157 | version: juno |
| 158 | enabled: true |
| 159 | virtualization: kvm |
Jiri Broulik | 70d9e3f | 2017-02-15 18:37:13 +0100 | [diff] [blame] | 160 | availability_zone: availability_zone_01 |
Damian Szeluga | e192241 | 2017-04-18 16:36:46 +0200 | [diff] [blame] | 161 | aggregates: |
| 162 | - hosts_with_fc |
| 163 | - hosts_with_ssd |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 164 | security_group: true |
Petr Michalec | f03e488 | 2017-04-10 10:26:18 +0200 | [diff] [blame] | 165 | resume_guests_state_on_host_boot: False |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 166 | bind: |
| 167 | vnc_address: 172.20.0.100 |
| 168 | vnc_port: 6080 |
| 169 | vnc_name: openstack.domain.com |
| 170 | vnc_protocol: http |
| 171 | database: |
| 172 | engine: mysql |
| 173 | host: 127.0.0.1 |
| 174 | port: 3306 |
| 175 | name: nova |
| 176 | user: nova |
| 177 | password: pwd |
| 178 | identity: |
| 179 | engine: keystone |
| 180 | host: 127.0.0.1 |
| 181 | port: 35357 |
| 182 | user: nova |
| 183 | password: pwd |
| 184 | tenant: service |
| 185 | message_queue: |
| 186 | engine: rabbitmq |
| 187 | host: 127.0.0.1 |
| 188 | port: 5672 |
| 189 | user: openstack |
| 190 | password: pwd |
| 191 | virtual_host: '/openstack' |
| 192 | image: |
| 193 | engine: glance |
| 194 | host: 127.0.0.1 |
| 195 | port: 9292 |
| 196 | network: |
| 197 | engine: neutron |
| 198 | host: 127.0.0.1 |
| 199 | port: 9696 |
| 200 | identity: |
| 201 | engine: keystone |
| 202 | host: 127.0.0.1 |
| 203 | port: 35357 |
| 204 | user: neutron |
| 205 | password: pwd |
| 206 | tenant: service |
| 207 | qemu: |
| 208 | max_files: 4096 |
| 209 | max_processes: 4096 |
| 210 | |
Dmitry Stremkovskiy | 3cd6ba8 | 2017-07-25 17:15:36 +0300 | [diff] [blame^] | 211 | Group membership for user nova (upgrade related) |
| 212 | |
| 213 | .. code-block:: yaml |
| 214 | |
| 215 | nova: |
| 216 | compute: |
| 217 | enabled: true |
| 218 | ... |
| 219 | user: |
| 220 | groups: |
| 221 | - libvirt |
| 222 | |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 223 | Nova services on compute node with OpenContrail |
| 224 | |
| 225 | .. code-block:: yaml |
| 226 | |
| 227 | nova: |
| 228 | compute: |
| 229 | enabled: true |
| 230 | ... |
| 231 | networking: contrail |
| 232 | |
Jiri Konecny | e31f2c5 | 2016-04-14 17:16:02 +0200 | [diff] [blame] | 233 | |
Filip Pytloun | 4a72d79 | 2015-10-06 16:28:32 +0200 | [diff] [blame] | 234 | Nova services on compute node with memcached caching |
| 235 | |
| 236 | .. code-block:: yaml |
| 237 | |
| 238 | nova: |
| 239 | compute: |
| 240 | enabled: true |
| 241 | ... |
| 242 | cache: |
| 243 | engine: memcached |
| 244 | members: |
| 245 | - host: 127.0.0.1 |
| 246 | port: 11211 |
| 247 | - host: 127.0.0.1 |
| 248 | port: 11211 |
| 249 | |
Jiri Konecny | e31f2c5 | 2016-04-14 17:16:02 +0200 | [diff] [blame] | 250 | |
| 251 | Client-side RabbitMQ HA setup |
| 252 | |
| 253 | .. code-block:: yaml |
| 254 | |
| 255 | nova: |
| 256 | controller: |
| 257 | .... |
| 258 | message_queue: |
| 259 | engine: rabbitmq |
| 260 | members: |
| 261 | - host: 10.0.16.1 |
| 262 | - host: 10.0.16.2 |
| 263 | - host: 10.0.16.3 |
| 264 | user: openstack |
| 265 | password: pwd |
| 266 | virtual_host: '/openstack' |
| 267 | .... |
| 268 | |
maxstack | 39e6aca | 2016-05-04 13:50:13 +0000 | [diff] [blame] | 269 | |
| 270 | Nova with ephemeral configured with Ceph |
| 271 | |
| 272 | .. code-block:: yaml |
| 273 | |
| 274 | nova: |
| 275 | compute: |
| 276 | enabled: true |
| 277 | ... |
| 278 | ceph: |
| 279 | ephemeral: yes |
| 280 | rbd_pool: nova |
| 281 | rbd_user: nova |
| 282 | secret_uuid: 03006edd-d957-40a3-ac4c-26cd254b3731 |
| 283 | |
| 284 | |
Jiri Broulik | 0ce9fc9 | 2017-02-01 23:10:40 +0100 | [diff] [blame] | 285 | Client role |
| 286 | ----------- |
| 287 | |
Dmitry Stremkovskiy | 665c728 | 2017-07-05 17:36:27 +0300 | [diff] [blame] | 288 | Nova configured with NFS |
| 289 | |
| 290 | .. code-block:: yaml |
| 291 | |
| 292 | nova: |
| 293 | compute: |
| 294 | instances_path: /mnt/nova/instances |
| 295 | |
| 296 | linux: |
| 297 | storage: |
| 298 | enabled: true |
| 299 | mount: |
| 300 | nfs_nova: |
| 301 | enabled: true |
| 302 | path: ${nova:compute:instances_path} |
| 303 | device: 172.31.35.145:/data |
| 304 | file_system: nfs |
| 305 | opts: rw,vers=3 |
| 306 | |
Jiri Broulik | 0ce9fc9 | 2017-02-01 23:10:40 +0100 | [diff] [blame] | 307 | Nova flavors |
| 308 | |
| 309 | .. code-block:: yaml |
| 310 | |
| 311 | nova: |
| 312 | client: |
| 313 | enabled: true |
| 314 | server: |
| 315 | identity: |
| 316 | flavor: |
Jiri Broulik | 70d9e3f | 2017-02-15 18:37:13 +0100 | [diff] [blame] | 317 | flavor1: |
Jiri Broulik | 0ce9fc9 | 2017-02-01 23:10:40 +0100 | [diff] [blame] | 318 | flavor_id: 10 |
| 319 | ram: 4096 |
| 320 | disk: 10 |
| 321 | vcpus: 1 |
Jiri Broulik | 70d9e3f | 2017-02-15 18:37:13 +0100 | [diff] [blame] | 322 | flavor2: |
| 323 | flavor_id: auto |
| 324 | ram: 4096 |
| 325 | disk: 20 |
| 326 | vcpus: 2 |
Jiri Broulik | 0ce9fc9 | 2017-02-01 23:10:40 +0100 | [diff] [blame] | 327 | identity1: |
| 328 | flavor: |
| 329 | ... |
| 330 | |
Jiri Broulik | 70d9e3f | 2017-02-15 18:37:13 +0100 | [diff] [blame] | 331 | |
| 332 | Availability zones |
| 333 | |
| 334 | .. code-block:: yaml |
| 335 | |
| 336 | nova: |
| 337 | client: |
| 338 | enabled: true |
| 339 | server: |
| 340 | identity: |
| 341 | availability_zones: |
| 342 | - availability_zone_01 |
| 343 | - availability_zone_02 |
| 344 | |
Damian Szeluga | 5dca0f0 | 2017-04-13 17:27:15 +0200 | [diff] [blame] | 345 | |
| 346 | |
| 347 | Aggregates |
| 348 | |
| 349 | .. code-block:: yaml |
| 350 | |
| 351 | nova: |
| 352 | client: |
| 353 | enabled: true |
| 354 | server: |
| 355 | identity: |
| 356 | aggregates: |
| 357 | - aggregate1 |
| 358 | - aggregate2 |
| 359 | |
Petr Jediný | d855ef2 | 2017-03-06 22:24:33 +0100 | [diff] [blame] | 360 | SR-IOV |
Jakub Pavlik | 39a0594 | 2017-02-13 23:03:08 +0100 | [diff] [blame] | 361 | ------ |
| 362 | |
| 363 | Add PciPassthroughFilter into scheduler filters and NICs on specific compute nodes. |
| 364 | |
| 365 | .. code-block:: yaml |
| 366 | |
| 367 | nova: |
| 368 | controller: |
| 369 | sriov: true |
| 370 | scheduler_default_filters: "DifferentHostFilter,RetryFilter,AvailabilityZoneFilter,RamFilter,CoreFilter,DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,PciPassthroughFilter" |
| 371 | |
| 372 | nova: |
| 373 | compute: |
| 374 | sriov: |
| 375 | nic_one: |
| 376 | devname: eth1 |
| 377 | physical_network: physnet1 |
| 378 | |
Jakub Pavlik | 26fb85c | 2017-02-16 22:29:22 +0100 | [diff] [blame] | 379 | CPU pinning & Hugepages |
| 380 | ----------------------- |
| 381 | |
| 382 | CPU pinning of virtual machine instances to dedicated physical CPU cores. |
| 383 | Hugepages mount point for libvirt. |
| 384 | |
| 385 | .. code-block:: yaml |
| 386 | |
| 387 | nova: |
| 388 | controller: |
| 389 | scheduler_default_filters: "DifferentHostFilter,RetryFilter,AvailabilityZoneFilter,RamFilter,CoreFilter,DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,NUMATopologyFilter,AggregateInstanceExtraSpecsFilter" |
| 390 | |
| 391 | nova: |
| 392 | compute: |
| 393 | vcpu_pin_set: 2,3,4,5 |
| 394 | hugepages: |
| 395 | mount_points: |
| 396 | - path: /mnt/hugepages_1GB |
| 397 | - path: /mnt/hugepages_2MB |
Jiri Broulik | 0ce9fc9 | 2017-02-01 23:10:40 +0100 | [diff] [blame] | 398 | |
Michel Nederlof | 171c7ac | 2017-04-13 12:54:14 +0200 | [diff] [blame] | 399 | Custom Scheduler filters |
| 400 | ------------------------ |
| 401 | |
| 402 | If you have a custom filter, that needs to be included in the scheduler, then you can include it like so: |
| 403 | |
| 404 | .. code-block:: yaml |
| 405 | |
| 406 | nova: |
| 407 | controller: |
| 408 | scheduler_custom_filters: |
| 409 | - my_custom_driver.nova.scheduler.filters.my_custom_filter.MyCustomFilter |
| 410 | |
| 411 | # Then add your custom filter on the end (make sure to include all other ones that you need as well) |
| 412 | scheduler_default_filters: "DifferentHostFilter,RetryFilter,AvailabilityZoneFilter,RamFilter,CoreFilter,DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,PciPassthroughFilter,MyCustomFilter" |
| 413 | |
Michel Nederlof | eb566f6 | 2017-04-21 15:37:47 +0200 | [diff] [blame] | 414 | Hardware Trip/Unmap Support |
| 415 | --------------------------- |
| 416 | |
| 417 | To enable TRIM support for ephemeral images (thru nova managed images), libvirt has this option. |
| 418 | |
| 419 | .. code-block:: yaml |
| 420 | |
| 421 | nova: |
| 422 | compute: |
| 423 | libvirt: |
| 424 | hw_disk_discard: unmap |
| 425 | |
| 426 | In order to actually utilize this feature, the following metadata must be set on the image as well, so the SCSI unmap is supported. |
| 427 | |
| 428 | .. code-block:: bash |
| 429 | |
| 430 | glance image-update --property hw_scsi_model=virtio-scsi <image> |
| 431 | glance image-update --property hw_disk_bus=scsi <image> |
Filip Pytloun | 5bc9e9f | 2017-02-02 13:05:40 +0100 | [diff] [blame] | 432 | |
Thom Gerdes | ec00afd | 2017-04-07 18:06:59 +0000 | [diff] [blame] | 433 | libvirt CPU mode |
| 434 | ---------------- |
| 435 | |
| 436 | Allow setting the model of CPU that is exposed to a VM. This allows better |
| 437 | support live migration between hypervisors with different hardware, among other |
| 438 | things. Defaults to host-passthrough. |
| 439 | |
| 440 | .. code-block:: yaml |
| 441 | |
| 442 | nova: |
| 443 | compute: |
| 444 | cpu_mode: host-model |
| 445 | |
Michel Nederlof | f7eefb2 | 2017-07-10 11:14:33 +0200 | [diff] [blame] | 446 | Nova compute workarounds |
| 447 | ------------------------ |
| 448 | |
| 449 | Live snapshotting is disabled by default in nova. To enable this, it needs a manual switch. |
| 450 | |
| 451 | From manual: |
| 452 | |
| 453 | .. code-block:: yaml |
| 454 | |
| 455 | # When using libvirt 1.2.2 live snapshots fail intermittently under load |
| 456 | # (likely related to concurrent libvirt/qemu operations). This config |
| 457 | # option provides a mechanism to disable live snapshot, in favor of cold |
| 458 | # snapshot, while this is resolved. Cold snapshot causes an instance |
| 459 | # outage while the guest is going through the snapshotting process. |
| 460 | # |
| 461 | # For more information, refer to the bug report: |
| 462 | # |
| 463 | # https://bugs.launchpad.net/nova/+bug/1334398 |
| 464 | |
| 465 | Configurable pillar data: |
| 466 | |
| 467 | .. code-block:: yaml |
| 468 | |
| 469 | nova: |
| 470 | compute: |
Michel Nederlof | e322ebb | 2017-07-10 12:29:21 +0200 | [diff] [blame] | 471 | workaround: |
Michel Nederlof | f7eefb2 | 2017-07-10 11:14:33 +0200 | [diff] [blame] | 472 | disable_libvirt_livesnapshot: False |
| 473 | |
Michel Nederlof | b51a514 | 2017-06-27 08:31:35 +0200 | [diff] [blame] | 474 | Config drive options |
| 475 | -------------------- |
| 476 | |
| 477 | See example below on how to configure the options for the config drive. |
| 478 | |
| 479 | .. code-block:: yaml |
| 480 | |
| 481 | nova: |
| 482 | compute: |
| 483 | config_drive: |
| 484 | forced: True # Default: True |
| 485 | cdrom: True # Default: False |
| 486 | format: iso9660 # Default: vfat |
| 487 | inject_password: False # Default: False |
| 488 | |
Thom Gerdes | ec00afd | 2017-04-07 18:06:59 +0000 | [diff] [blame] | 489 | |
Filip Pytloun | 5bc9e9f | 2017-02-02 13:05:40 +0100 | [diff] [blame] | 490 | Documentation and Bugs |
| 491 | ====================== |
| 492 | |
| 493 | To learn how to install and update salt-formulas, consult the documentation |
| 494 | available online at: |
| 495 | |
| 496 | http://salt-formulas.readthedocs.io/ |
| 497 | |
| 498 | In the unfortunate event that bugs are discovered, they should be reported to |
| 499 | the appropriate issue tracker. Use Github issue tracker for specific salt |
| 500 | formula: |
| 501 | |
| 502 | https://github.com/salt-formulas/salt-formula-nova/issues |
| 503 | |
| 504 | For feature requests, bug reports or blueprints affecting entire ecosystem, |
| 505 | use Launchpad salt-formulas project: |
| 506 | |
| 507 | https://launchpad.net/salt-formulas |
| 508 | |
| 509 | You can also join salt-formulas-users team and subscribe to mailing list: |
| 510 | |
| 511 | https://launchpad.net/~salt-formulas-users |
| 512 | |
| 513 | Developers wishing to work on the salt-formulas projects should always base |
| 514 | their work on master branch and submit pull request against specific formula. |
| 515 | |
| 516 | https://github.com/salt-formulas/salt-formula-nova |
| 517 | |
| 518 | Any questions or feedback is always welcome so feel free to join our IRC |
| 519 | channel: |
| 520 | |
| 521 | #salt-formulas @ irc.freenode.net |