Initial version of a Salt adapter
Signed-off-by: martin f. krafft <madduck@madduck.net>
diff --git a/README.Salt b/README.Salt
new file mode 100644
index 0000000..c1702ff
--- /dev/null
+++ b/README.Salt
@@ -0,0 +1,168 @@
+=============================================================
+ reclass — recursive external node classification
+=============================================================
+reclass is © 2007–2013 martin f. krafft <madduck@madduck.net>
+and available under the terms of the Artistic Licence 2.0
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+Please make sure to read the generic information in the README file first, or
+alongside this document.
+
+Quick start with Salt
+~~~~~~~~~~~~~~~~~~~~~
+The following steps should get you up and running quickly. You will need to
+decide for yourself where to put your reclass inventory. This can be
+/etc/reclass, or it could be /srv/salt, for instance if /srv/salt/states is
+where your Salt file_roots live. The following shall assume the latter.
+
+Or you can also just look into ./examples/salt of your reclass checkout,
+where the following steps have already been prepared.
+
+/…/reclass refers to the location of your reclass checkout.
+
+ 1. Symlink /…/reclass/adapters/ansible to /etc/ansible/hosts (or
+ ./hacking/hosts)
+
+ 2. Copy the two directories 'nodes' and 'classes' from the example
+ subdirectory in the reclass checkout to /etc/ansible
+
+ If you prefer to put those directories elsewhere, you can create
+ /etc/ansible/reclass-config.yml with contents such as
+
+ storage_type: yaml_fs
+ nodes_uri: /srv/reclass/nodes
+ classes_uri: /srv/reclass/classes
+
+ Note that yaml_fs is currently the only supported storage_type, and it's
+ the default if you don't set it.
+
+ 3. Check out your inventory by invoking
+
+ ./hosts --list
+
+ which should return 5 groups in JSON-format, and each group has exactly
+ one member 'localhost'.
+
+ 4. See the node information for 'localhost':
+
+ ./hosts --host localhost
+
+ This should print a set of keys and values, including a greeting,
+ a colour, and a sub-class called 'RECLASS'.
+
+ 5. Execute some ansible commands, e.g.
+
+ ansible -i hosts \* --list-hosts
+ ansible -i hosts \* -m ping
+ ansible -i hosts \* -m debug -a 'msg="${greeting}"'
+ ansible -i hosts \* -m setup
+ ansible-playbook -i hosts test.yml
+
+ 6. You can also invoke reclass directly, which gives a slightly different
+ view onto the same data, i.e. before it has been adapted for Ansible:
+
+ /…/reclass.py --pretty-print --inventory
+ /…/reclass.py --pretty-print --nodeinfo localhost
+
+Integration with Salt
+~~~~~~~~~~~~~~~~~~~~~
+The integration between reclass and Ansible is performed through an adapter,
+and needs not be of our concern too much.
+
+However, Ansible has no concept of "nodes", "applications", "parameters", and
+"classes". Therefore it is necessary to explain how those correspond to
+Ansible. Crudely, the following mapping exists:
+
+ nodes hosts
+ classes groups
+ applications playbooks
+ parameters host_vars
+
+reclass does not provide any group_vars because of its node-centric
+perspective. While class definitions include parameters, those are inherited
+by the node definitions and hence become node_vars.
+
+reclass also does not provide playbooks, nor does it deal with any of the
+related Ansible concepts, i.e. vars_files, vars, tasks, handlers, roles, etc..
+
+ Let it be said at this point that you'll probably want to stop using
+ host_vars, group_vars and vars_files altogether, and if only because you
+ should no longer need them, but also because the variable precedence rules
+ of Ansible are full of surprises, at least to me.
+
+reclass' Ansible adapter massage the reclass output into Ansible-usable data,
+namely:
+
+ - Every class in the ancestry of a node becomes a group to Ansible. This is
+ mainly useful to be able to target nodes during interactive use of
+ Ansible, e.g.
+
+ ansible debiannode@wheezy -m command -a 'apt-get upgrade'
+ → upgrade all Debian nodes running wheezy
+
+ ansible ssh.server -m command -a 'invoke-rc.d ssh restart'
+ → restart all SSH server processes
+
+ ansible mailserver -m command -a 'tail -n1000 /var/log/mail.err'
+ → obtain the last 1,000 lines of all mailserver error log files
+
+ The attentive reader might stumble over the use of singular words, whereas
+ it might make more sense to address all 'mailserver*s*' with this tool.
+ This is convention and up to you. I prefer to think of my node as
+ a (singular) mailserver when I add 'mailserver' to its parent classes.
+
+ - Every entry in the list of a host's applications might well correspond to
+ an Ansible playbook. Therefore, reclass creates a (Ansible-)group for
+ every application, and adds '_hosts' to the name. This postfix can be
+ configured with a CLI option (--applications-postfix) or in the
+ configuration file (applications_postfix).
+
+ For instance, the ssh.server class adds the ssh.server application to
+ a node's application list. Now the admin might create an Ansible playbook
+ like so:
+
+ - name: SSH server management
+ hosts: ssh.server_hosts ← SEE HERE
+ tasks:
+ - name: install SSH package
+ action: …
+ …
+
+ There's a bit of redundancy in this, but unfortunately Ansible playbooks
+ hardcode the nodes to which a playbook applies.
+
+ It's now trivial to apply this playbook across your infrastructure:
+
+ ansible-playbook ssh.server.yml
+
+ My suggested way to use Ansible site-wide is then to create a 'site'
+ playbook that includes all the other playbooks (which shall hopefully be
+ based on Ansible roles), and then to invoke Ansible like this:
+
+ ansible-playbook site.yml
+
+ or, if you prefer only to reconfigure a subset of nodes, e.g. all
+ webservers:
+
+ ansible-playbook site.yml --limit webserver
+
+ Again, if the singular word 'webserver' puts you off, change the
+ convention as you wish.
+
+ And if anyone comes up with a way to directly connect groups in the
+ inventory with roles, thereby making it unnecessary to write playbook
+ files (containing redundant information), please tell me!
+
+ - Parameters corresponding to a node become host_vars for that host.
+
+It is possible to include Jinja2-style variables like you would in Ansible,
+in parameter values. This is especially powerful in combination with the
+recursive merging, e.g.
+
+ parameters:
+ motd:
+ greeting: Welcome to {{ ansible_fqdn }}!
+ closing: This system is part of {{ realm }}
+
+Now you just need to specify realm somewhere. The reference can reside in
+a parent class, while the variable is defined e.g. in the node.
diff --git a/TODO b/TODO
index ca51150..4fead5a 100644
--- a/TODO
+++ b/TODO
@@ -1,7 +1,7 @@
reclass TODO
============
-- Salt and Puppet adapters, whoever might want them.
+- Puppet adapter, whoever might want that…???
- Tests for outputters
- Improve testing of yaml_fs, maybe with more realistic examples
- Configurable file extension (.yaml/.yml, or support both)
diff --git a/adapters/.gitignore b/adapters/.gitignore
index a6246db..cfa52e9 100644
--- a/adapters/.gitignore
+++ b/adapters/.gitignore
@@ -1 +1,2 @@
/ansible
+/salt
diff --git a/adapters/salt.in b/adapters/salt.in
new file mode 100644
index 0000000..212368e
--- /dev/null
+++ b/adapters/salt.in
@@ -0,0 +1,98 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+#
+# salt-adapter — adapter between Salt and reclass
+#
+# Note that this file is not really necessary and exists mostly for debugging
+# purposes and admin joys. Have a look at README.Salt for proper integration
+# between Salt and reclass.
+#
+# Copyright © 2007–13 martin f. krafft <madduck@madduck.net>
+# Released under the terms of the Artistic Licence 2.0
+#
+__name__ = 'salt-reclass'
+__description__ = 'reclass adapter for Salt'
+__version__ = '1.0'
+__author__ = 'martin f. krafft <madduck@madduck.net>'
+__copyright__ = 'Copyright © 2007–13 ' + __author__
+__licence__ = 'Artistic Licence 2.0'
+
+import sys, os, posix
+
+# In order to be able to use reclass as modules, manipulate the search
+# path, starting from the location of the adapter. Realpath will make
+# sure that symlinks are resolved.
+realpath = os.path.realpath(sys.argv[0] + '/../../')
+sys.path.insert(0, realpath)
+
+import os, sys, posix
+import reclass.config
+from reclass.output import OutputLoader
+from reclass.storage import StorageBackendLoader
+from reclass.errors import ReclassException, InvocationError
+from reclass import output
+from reclass.adapters.salt import ext_pillar, top
+
+def _error(msg, rc):
+ print >>sys.stderr, msg
+ sys.exit(rc)
+
+try:
+ if len(sys.argv) == 1:
+ raise InvocationError('Need to specify --top or --pillar.',
+ posix.EX_USAGE)
+
+ # initialise options and try to read ./reclass-config.yml, which is
+ # expected to sit next to the salt-reclass CLI
+ options = {'storage_type': 'yaml_fs',
+ 'pretty_print' : True,
+ 'output' : 'yaml'
+ }
+ basedir = os.path.dirname(sys.argv[0])
+ config_path = os.path.join(basedir, 'reclass-config.yml')
+ if os.path.exists(config_path) and os.access(config_path, os.R_OK):
+ options.update(reclass.config.read_config_file(config_path))
+
+ nodes_uri, classes_uri = reclass.config.path_mangler(options.get('inventory_base_uri'),
+ options.get('nodes_uri'),
+ options.get('classes_uri'))
+ options['nodes_uri'] = nodes_uri
+ options['classes_uri'] = classes_uri
+
+ if sys.argv[1] in ('--top', '-t'):
+ if len(sys.argv) > 2:
+ raise InvocationError('Unknown arguments: ' + \
+ ' '.join(sys.argv[2:]), posix.EX_USAGE)
+ node = False
+
+ elif sys.argv[1] in ('--pillar', '-p'):
+ if len(sys.argv) < 3:
+ raise InvocationError('Missing hostname.', posix.EX_USAGE)
+ elif len(sys.argv) > 3:
+ raise InvocationError('Unknown arguments: ' + \
+ ' '.join(sys.argv[3:]), posix.EX_USAGE)
+ node = sys.argv[2]
+
+ else:
+ raise InvocationError('Unknown mode (--top or --pillar required).',
+ posix.EX_USAGE)
+
+ if not node:
+ # we want master_tops behaviour
+ # so we first hack:
+ opts = {'master_tops': {'reclass': options}}
+ data = top(salt={}, opts=opts, grains={})
+
+ else:
+ opts={}; salt={}; grains={}; pillar={}
+ data = ext_pillar(opts, salt, grains, pillar,
+ options.get('storage_type'),
+ options.get('inventory_base_uri'),
+ options.get('nodes_uri'),
+ options.get('classes_uri'))
+
+ print output(data, options.get('output'), options.get('pretty_print'))
+ sys.exit(posix.EX_OK)
+
+except ReclassException, e:
+ _error(e.message, e.rc)
diff --git a/examples/salt/reclass b/examples/salt/reclass
new file mode 120000
index 0000000..0d84784
--- /dev/null
+++ b/examples/salt/reclass
@@ -0,0 +1 @@
+../../adapters/salt
\ No newline at end of file
diff --git a/reclass/adapters/salt.py b/reclass/adapters/salt.py
new file mode 100644
index 0000000..7e0f306
--- /dev/null
+++ b/reclass/adapters/salt.py
@@ -0,0 +1,77 @@
+#
+# -*- coding: utf-8 -*-
+#
+# This file is part of reclass (http://github.com/madduck/reclass)
+#
+# Copyright © 2007–13 martin f. krafft <madduck@madduck.net>
+# Released under the terms of the Artistic Licence 2.0
+#
+from reclass import config, get_data
+from reclass.errors import InvocationError
+
+def _check_storage_params(storage_type, inventory_base_uri, nodes_uri,
+ classes_uri):
+ nodes_uri, classes_uri = config.path_mangler(inventory_base_uri,
+ nodes_uri, classes_uri)
+
+ if nodes_uri is None:
+ raise InvocationError('missing nodes_uri or inventory_base_uri parameters')
+
+ if classes_uri is None:
+ raise InvocationError('missing classes_uri or inventory_base_uri parameters')
+
+ if storage_type is None:
+ storage_type = 'yaml_fs' # TODO: should be obtained from config
+
+ return storage_type, nodes_uri, classes_uri
+
+
+def _get_data(storage_type, inventory_base_uri, nodes_uri, classes_uri, node):
+
+ storage_type, nodes_uri, classes_uri = _check_storage_params(storage_type,
+ inventory_base_uri,
+ nodes_uri,
+ classes_uri)
+ return get_data(storage_type, nodes_uri, classes_uri, node)
+
+
+def ext_pillar(pillar, storage_type=None, inventory_base_uri=None,
+ nodes_uri=None, classes_uri=None):
+
+ node = opts.get('id')
+ if node is None:
+ raise InvocationError('no node ID provided')
+
+ data = _get_data(storage_type, inventory_base_uri, nodes_uri, classes_uri,
+ node)
+ params = data.get('parameters', {})
+ params['__reclass__'] = {}
+ params['__reclass__']['applications'] = data['applications']
+ params['__reclass__']['classes'] = data['classes']
+
+ # TODO: template interpolation?
+ return params
+
+
+def top(salt, opts, grains):
+ reclass_opts = opts.get('master_tops', {}).get('reclass')
+ if reclass_opts is None:
+ raise InvocationError('no configuration provided')
+
+ storage_type = reclass_opts.get('storage_type')
+ inventory_base_uri = reclass_opts.get('inventory_base_uri')
+ nodes_uri = reclass_opts.get('nodes_uri')
+ classes_uri = reclass_opts.get('classes_uri')
+
+ data = _get_data(storage_type, inventory_base_uri, nodes_uri, classes_uri,
+ node=None)
+ env = 'base'
+ top = {env: {}}
+ # TODO: node environments
+ for node_id, node_data in data['nodes'].iteritems():
+ #env = data.environment
+ #if env not in top:
+ # top[env] = {}
+ top[env][node_id] = node_data['applications']
+
+ return top