Jenkins job templating
Change-Id: I4babee33801e8f939051a47d473c96c2974db4e9
diff --git a/README.rst b/README.rst
index 5b6f7ea..e86ae05 100755
--- a/README.rst
+++ b/README.rst
@@ -1,58 +1,19 @@
-=======
-Jenkins
-=======
+===============
+Jenkins formula
+===============
-Jenkins is an application that monitors executions of repeated jobs, such as building a software project or jobs run by cron.
+Jenkins is an application that monitors executions of repeated jobs, such as
+building a software project or jobs run by cron.
-Available states
-================
+Setup jenkins client, works with Salt 2016.3+, supports pipeline workflow
+projects only now.
-.. contents::
- :local:
-
-``jenkins.master``
-------------------
-
-Setup jenkins master.
-
-``jenkins.slave``
------------------
-
-Setup jenkins slave.
-
-``jenkins.job_builder``
------------------------
-
-Setup jenkins job builder.
-
-``jenkins.client``
-------------------
-
-Setup jenkins client, works with Salt 2016.3+, supports pipeline workflow projects only now.
-
-
-Available metadata
-==================
-
-.. contents::
- :local:
-
-``metadata.jenkins.master.single``
-----------------------------------
-
-Setup single-node master
-
-
-``metadata.jenkins.slave.single``
----------------------------------
-
-Setup Jenkins slave
Sample pillars
==============
-Jenkins master
---------------
+Master role
+-----------
Simple master with reverse proxy
@@ -98,7 +59,7 @@
- name: rebuild
- name: test-stability
-Jenkins with experimental plugin source support
+Jenkins master with experimental plugin source support
.. code-block:: yaml
@@ -107,9 +68,47 @@
enabled: true
update_site_url: 'http://updates.jenkins-ci.org/experimental/update-center.json'
+SMTP server settings
-Agent (former slave)
---------------------
+.. code-block:: yaml
+
+ jenkins:
+ master:
+ email:
+ engine: "smtp"
+ host: "smtp.domain.com"
+ user: "user@domain.cz"
+ password: "smtp-password"
+ port: 25
+
+Script approvals
+
+.. code-block:: yaml
+
+ jenkins:
+ master:
+ approved_scripts:
+ - method groovy.json.JsonSlurperClassic parseText java.lang.String
+
+User enforcement
+
+.. code-block:: yaml
+
+ jenkins:
+ master:
+ user:
+ admin:
+ api_token: xxxxxxxxxx
+ password: admin_password
+ email: admin@domain.com
+ user01:
+ api_token: xxxxxxxxxx
+ password: user_password
+ email: user01@domain.com
+
+
+Agent (slave) role
+------------------
.. code-block:: yaml
@@ -131,8 +130,9 @@
-----BEGIN PGP PRIVATE KEY BLOCK-----
...
-Client
-------
+
+Client role
+-----------
Simple client with workflow job definition
@@ -194,7 +194,7 @@
description: multi-liner
default: default_text
-Inline Groovy script samples
+Inline Groovy scripts
.. code-block:: yaml
@@ -233,7 +233,7 @@
}
-GIT controlled groovy script samples
+GIT controlled groovy scripts
.. code-block:: yaml
@@ -323,6 +323,52 @@
keep_num: 6
keep_days: 6
+
+Using job templates in similar way as in jjb. For now just 1 defined param is
+supported.
+
+.. code-block:: yaml
+
+ jenkins:
+ client:
+ job_template:
+ test_workflow_template:
+ name: test-{{formula}}-workflow
+ template:
+ type: workflow
+ display_name: Test jenkins {{name}} workflow
+ param:
+ repo_param:
+ type: string
+ default: repo/{{formula}}
+ script:
+ repository: base
+ file: workflows/test_formula_workflow.groovy
+ param:
+ formula:
+ - aodh
+ - linux
+ - openssh
+
+Interpolating parameters for job templates.
+
+ _param:
+ salt_formulas:
+ - aodh
+ - git
+ - nova
+ - xorg
+ jenkins:
+ client:
+ job_template:
+ test_workflow_template:
+ name: test-{{formula}}-workflow
+ template:
+ ...
+ param:
+ formula: ${_param:salt_formulas}
+
+
Purging undefined jobs from Jenkins
.. code-block:: yaml
@@ -330,6 +376,9 @@
jenkins:
client:
purge_jobs: true
+ job:
+ my-amazing-job:
+ type: workflow
Plugins management from client
@@ -452,66 +501,53 @@
.. code-block:: yaml
jenkins:
- client:
- node:
- node01:
- remote_home: /remote/home/path
- desc: node-description
- num_executors: 1
- node_mode: Normal
- ret_strategy: Always
- labels:
- - example
- - label
- launcher:
- type: jnlp
+ client:
+ node:
+ node01:
+ remote_home: /remote/home/path
+ desc: node-description
+ num_executors: 1
+ node_mode: Normal
+ ret_strategy: Always
+ labels:
+ - example
+ - label
+ launcher:
+ type: jnlp
Node enforcing from client using SSH launcher
.. code-block:: yaml
jenkins:
- client:
- node:
- node01:
- remote_home: /remote/home/path
- desc: node-description
- num_executors: 1
- node_mode: Normal
- ret_strategy: Always
- labels:
- - example
- - label
- launcher:
- type: ssh
- host: test-launcher
- port: 22
- username: launcher-user
- password: launcher-pass
+ client:
+ node:
+ node01:
+ remote_home: /remote/home/path
+ desc: node-description
+ num_executors: 1
+ node_mode: Normal
+ ret_strategy: Always
+ labels:
+ - example
+ - label
+ launcher:
+ type: ssh
+ host: test-launcher
+ port: 22
+ username: launcher-user
+ password: launcher-pass
Setting node labels
.. code-block:: yaml
jenkins:
- client:
- label:
- node-name:
- lbl_text: label-offline
- append: false # set true for label append instead of replace
-
-SMTP server settings from master
-
-.. code-block:: yaml
-
- jenkins:
- master:
- email:
- engine: "smtp"
- host: "smtp.domain.com"
- user: "user@domain.cz"
- password: "smtp-password"
- port: 25
+ client:
+ label:
+ node-name:
+ lbl_text: label-offline
+ append: false # set true for label append instead of replace
SMTP server settings from client
@@ -527,15 +563,6 @@
ssl: false
reply_to: reply_to@address.com
-Jenkins script approvals
-
-.. code-block:: yaml
-
- jenkins:
- master:
- approved_scripts:
- - method groovy.json.JsonSlurperClassic parseText java.lang.String
-
Slack plugin configuration
.. code-block:: yaml
@@ -543,28 +570,13 @@
jenkins:
client:
slack:
- team_domain: example.com
- token: slack-token
- room: slack-room
- token_credential_id: cred_id
- send_as: Some slack user
+ team_domain: example.com
+ token: slack-token
+ room: slack-room
+ token_credential_id: cred_id
+ send_as: Some slack user
-Users enforcing from master
-
-.. code-block:: yaml
-
- jenkins:
- user:
- admin:
- api_token: xxxxxxxxxx
- password: admin_password
- email: admin@domain.com
- user01:
- api_token: xxxxxxxxxx
- password: user_password
- email: user01@domain.com
-
Usage
=====
@@ -576,11 +588,13 @@
Place in the configuration ``salt:hashpassword``.
-Read more
-=========
+
+External links
+==============
* https://wiki.jenkins-ci.org/display/JENKINS/Use+Jenkins
+
Documentation and Bugs
======================
diff --git a/jenkins/client/init.sls b/jenkins/client/init.sls
index 2017ca7..e5e1552 100644
--- a/jenkins/client/init.sls
+++ b/jenkins/client/init.sls
@@ -14,6 +14,9 @@
{%- if client.job is defined %}
- jenkins.client.job
{%- endif %}
+{%- if client.job_template is defined %}
+ - jenkins.client.job_template
+{%- endif %}
{%- if client.credential is defined %}
- jenkins.client.credential
{%- endif %}
diff --git a/jenkins/client/job.sls b/jenkins/client/job.sls
index 7fe0b0f..3822ca5 100644
--- a/jenkins/client/job.sls
+++ b/jenkins/client/job.sls
@@ -45,7 +45,29 @@
{%- endfor %}
{%- if client.get('purge_jobs', False) %}
+
+{%- set jobs = client.get('job', {}).keys() %}
+
+{%- for job_template_name, job_template in client.get('job_template', {}).iteritems() %}
+
+{%- if job_template.get('enabled', true) %}
+
+{%- for param_name, params in job_template.param.iteritems() %}
+
+{%- set replacer = "{{" + param_name + "}}" %}
+
+{%- for param in params %}
+
+{%- set job_name = job_template.name|replace(replacer, param) %}
+
+{%- endfor %}
+
+{%- endfor %}
+
+{%- endif %}
+
jenkins_clean_undefined_jobs:
jenkins_job.cleanup:
- - jobs: {{ client.get('job', {}).keys()|yaml }}
+ - jobs: {{ jobs|yaml }}
+
{%- endif %}
\ No newline at end of file
diff --git a/jenkins/client/job_template.sls b/jenkins/client/job_template.sls
new file mode 100644
index 0000000..43d1f1d
--- /dev/null
+++ b/jenkins/client/job_template.sls
@@ -0,0 +1,49 @@
+{% from "jenkins/map.jinja" import client with context %}
+
+include:
+ - jenkins.client
+
+{%- for job_template_name, job_template in client.get('job_template', {}).iteritems() %}
+
+{%- if job_template.get('enabled', true) %}
+
+{# now just 1 defined param is supported #}
+
+{%- if job_template.param|length == 1 %}
+
+{%- for param_name, params in job_template.param.iteritems() %}
+
+{%- set replacer = "{{" + param_name + "}}" %}
+
+{%- for param in params %}
+
+{%- set job_name = job_template.name|replace(replacer, param) %}
+{%- set job = job_template.template|yaml|replace(replacer, param) %}
+
+jenkins_job_{{ job_name }}_definition:
+ file.managed:
+ - name: {{ client.dir.jenkins_jobs_root }}/{{ job_name }}.xml
+ - source: salt://jenkins/files/jobs/{{ job.type }}.xml
+ - mode: 400
+ - template: jinja
+ - defaults:
+ job_name: {{ job_name }}
+ job: {{ job }}
+ - require:
+ - file: jenkins_client_dirs
+
+jenkins_job_{{ job_name }}_present:
+ jenkins_job.present:
+ - name: {{ job_name }}
+ - config: {{ client.dir.jenkins_jobs_root }}/{{ job_name }}.xml
+ - watch:
+ - file: jenkins_job_{{ job_name }}_definition
+ - file: /etc/salt/minion.d/_jenkins.conf
+
+{%- endfor %}
+
+{%- endfor %}
+
+{%- endif %}
+
+{%- endfor %}
diff --git a/jenkins/files/jobs/workflow.xml b/jenkins/files/jobs/workflow.xml
index ebf8365..65a4e83 100644
--- a/jenkins/files/jobs/workflow.xml
+++ b/jenkins/files/jobs/workflow.xml
@@ -1,5 +1,7 @@
{%- from "jenkins/map.jinja" import client with context -%}
+{%- if job is not defined %}
{%- set job = salt['pillar.get']('jenkins:client:job:'+job_name) -%}
+{%- endif %}
{%- macro load_groovy_file() -%}
{%- set groovy_file=client.dir.jenkins_source_root+'/'+job.script.repository+'/'+job.script.file -%}