Setup of jenkins.debian.net
===========================
:Author:           Holger Levsen
:Authorinitials:   holger
:EMail:            holger@layer-acht.org
:Status:           working, in progress
:lang:             en
:Doctype:          article
:Licence:	   GPLv2

== About jenkins.debian.net

See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian.net"].

== Setup

=== Installed software that makes jenkins.debian.net run

link:https://jenkins.debian.net[jenkins.debian.net] runs Debian wheezy.

=== Jenkins

Currently running the long term support .deb from jenkins-ci.org.

Jenkins is installed using 'deb http://pkg.jenkins-ci.org/debian-stable binary/' into '/etc/apt/sources.list.d/jenkins.list' and `apt-get update && apt-get install jenkins`.

I would prefer to run a jenkins package from wheezy-backports.

==== Jenkins plugins

* Jenkins GIT plugin
* Jenkins Subversion Plug-in

* Email-ext Plugin
* Log Parser Plugin
* HTML Publisher plugin
* Static Analysis Utilities

* Read-only configurations
* SCM Sync Configuration Plugin
* Dependency Graph Viewer Plugin
* View Job Filters
* Console Column Plugin
* Priority Sorter Plugin
* Throttle Concurrent Builds Plugin
* Next Executions Plugin
* Build Token Root Plugin
* Embeddable Build Status Plugin
* Plot Plugin

* Sidebar Link
* Green Balls
* Compact Columns
* Console Tail Plugin
* Cron Column
* Simple Theme Plugin
* LiveScreenshot Plugin
* Locale plugin
* Jenkins Translation Assistance plugin

=== updating jenkins.debian.net with 'update_jdn.sh'

'update_jdn.sh' is a small hackish script which will install all needed packages, copy their configuration in place, generate the documentation webpages and update all jenkins jobs if needed.

As 'root' run:
----
cd
git clone git://git.debian.org/git/qa/jenkins.debian.net.git
cd jenkins.debian.net
./update_jdn.sh
----

'jenkins-job-builder', 'vncdotool' and (currently) 'kgb-client' are the only software which is not covered by this script.

=== Installing jenkins-job-builder

----
pip install python-jenkins

git clone git://git.openstack.org/openstack-infra/jenkins-job-builder
cd jenkins-job-builder 
python setup.py install
----

Though actually https://github.com/h01ger/jenkins-job-builder is used atm.

==== Using jenkins-job-builder

'update_jdn.sh' takes care of this too, so this is just for quick documentation. The real documentation is at http://ci.openstack.org/jenkins-job-builder/

* run `cd /srv/jenkins/job-cfg ; jenkins-jobs update .` to update jobs
* for testing: `cd /srv/jenkins/job-cfg ; rm test-output/ -r ; mkdir test-output ; jenkins-jobs test . -o test-output/ ; ls test-output/`

=== Installing kgb-client

Rebuild the 'kgb-bot' source package from sid on whezzy and then install the 'kbg-client' package. Configuration is done through 'update_jdn.sh'.

==== IRC notification configuration

The jenkins jobs are configured to send email to 'jenkins+$IRC_CHANNEL' (like 'jenkins+debian-qa'), this is parsed by a script ('/srv/jenkins/bin/email2irc.sh') run through 'procmail' which then in turn notifies 'kgb-client', which notfies 'kgb-server'(s) on the internet, which are the bots notifying the IRC channels. 'kbg-client' needs to be '>= 1.24'.

The Jenkins EMail notification plugin is used as its state tracking is best (and the Jenkins IRC plugin is way too powerful).

=== Installing vncdotool

----
apt-get install python-twisted python-imaging
git clone https://github.com/sibson/vncdotool
cd vncdotool/
python setup.py
----

=== munin

link://munin-monitoring[Munin] is installed, running with link:https://jenkins.debian.net/munin/[jenkins plugins] not yet published elsewhere.

=== Backup

Currently only the configuration is backed up, but neither the jenkins results nor system logs. But that's at least sufficient to re-setup the system if needed.

* The _etckeeper_ package is installed and automatically keeps '/etc' in '/etc/.git'.
* The _SCM Sync Configuration Plugin_ is installed in Jenkins, putting all Jenkins configuration into '/srv/jenkins.debian.net-scm-sync.git'.
** (This is not ideal yet, because it also puts the generated job configuration there...)

I've cloned these two git repos on my desktop and manually run `git pull`. This is certainly not ideal but good enough to re-setup the system anytime, which is something.

// vim: set filetype=asciidoc: