1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
|
Setup of jenkins.debian.net
===========================
:Author: Holger Levsen
:Authorinitials: holger
:EMail: holger@layer-acht.org
:Status: in progress
:lang: en
:Doctype: article
:Licence: GPLv2
== About jenkins.debian.net
See link:http://jenkins.debian.net/userContent/about.html["about jenkins.debian.net"].
== Setup
=== Installed software that makes jenkins.debian.net run
link:http://jenkins.debian.net[jenkins.debian.net] runs Debian wheezy.
=== Jenkins
Currently running the long term support .deb from jenkins-ci.org.
Jenkins is installed using 'deb http://pkg.jenkins-ci.org/debian-stable binary/' into '/etc/apt/sources.list.d/jenkins.list' and `apt-get update && apt-get install jenkins`.
I would prefer to run a jenkins package from wheezy-backports.
==== Jenkins plugins
* Jenkins GIT plugin
* Jenkins Subversion Plug-in
* Log Parser Plugin
* HTML Publisher plugin
* Static Analysis Utilities
* Read-only configurations
* SCM Sync Configuration Plugin
* Dependency Graph Viewer Plugin
* View Job Filters
* Console Column Plugin
* Sidebar Link
* Green Balls
* Compact Columns
* Simple Theme Plugin
* Locale plugin
* Jenkins Translation Assistance plugin
=== updating jenkins.debian.net with 'update_jdn.sh'
'update_jdn.sh' is a small hackish script which will install all needed packages, copy their configuration in place, generate the documentation webpages and update all jenkins jobs if needed.
As 'root' run:
----
cd
git clone git://git.debian.org/git/users/holger/jenkins.debian.net.git
cd jenkins.debian.net
./update_jdn.sh
----
'jenkins-job-builder', 'vncdotool' and (currently) 'kgb-client' are the only software which is not covered by this script.
=== Installing jenkins-job-builder
----
pip install python-jenkins
git clone https://github.com/openstack-ci/jenkins-job-builder
cd jenkins-job-builder
python setup.py install
----
Though actually https://github.com/h01ger/jenkins-job-builder is used atm.
==== Using jenkins-job-builder
'update_jdn.sh' takes care of this too, so this is just for quick documentation. The real documentation is at http://ci.openstack.org/jenkins-job-builder/
* run `cd /srv/jenkins/job-cfg ; jenkins-jobs update .` to update jobs
* for testing: `cd /srv/jenkins/job-cfg ; rm test-output/ -r ; mkdir test-output ; jenkins-jobs test . -o test-output/ ; ls test-output/`
=== Installing kgb-client
Rebuild the 'kgb-bot' source package from sid on whezzy and then install the 'kbg-client' package. Configuration is done through 'update_jdn.sh'.
==== IRC notification configuration
The jenkins jobs are configured to send email to 'jenkins+$IRC_CHANNEL' (like 'jenkins+debian-qa'), this is parsed by a script ('/srv/jenkins/bin/email2irc.sh') run through 'procmail' which then in turn notifies 'kgb-client', which notfies 'kgb-server'(s) on the internet, which are the bots notifying the IRC channels. 'kbg-client' needs to be '>= 1.24'.
The Jenkins EMail notification plugin is used as its state tracking is best (and the Jenkins IRC plugin is way too powerful).
=== Installing vncdotool
----
apt-get install python-twisted python-imaging
git clone https://github.com/sibson/vncdotool
cd vncdotool/
python setup.py
----
=== munin
link://munin-monitoring[Munin] is installed, running with link:http://jenkins.debian.net/munin/[jenkins plugins] not yet published elsewhere.
=== Backup
Currently only the configuration is backed up, but neither the jenkins results nor system logs. But that's at least sufficient to re-setup the system if needed.
* The _etckeeper_ package is installed and automatically keeps '/etc' in '/etc/.git'.
* The _SCM Sync Configuration Plugin_ is installed in Jenkins, putting all Jenkins configuration into '/srv/jenkins.debian.net-scm-sync.git'.
** (This is not ideal yet, because it also puts the generated job configuration there...)
I've cloned these two git repos on my desktop and manually run `git pull`. This is certainly not ideal but good enough to re-setup the system anytime, which is something.
== Debugging jobs without jenkins
*FIXME*: To debug certain jobs, a jenkins setup is actually not needed. Make this easy and describe here.
*This is work in progress too and is not done yet!*
FIXME: Describe which packages need to be installed, too. No, don't. Package it properly. update_jdn.sh is a hack and needs to die.
FIXME: svn/git using test-scripts need to learn checkout out - currently this is always done by jenkins.
=== Debugging g-i-installation jobs
as user run:
----
mkdir workspace ; cd workspace # test-script should check if they are in a directory called workspace and if not, mkdir it and cd into it.
/srv/jenkins/bin/cd_tester.sh 2 squeeze-test-debian-edu-standalone 20 http://ftp.skolelinux.org/cd-squeeze-test-amd64-i386-netinst/debian-edu-amd64-i386-NETINST-1.iso
----
See the *fourth line* on *any* log to find out how to debug any job. Point this out at the end of any log.
When packaging this, two packages should be created:
- jenkins.debian.net - turn a machine into jenkins.$localdomain (not policy complient package), probably jenkins.d.n-seteup or such is a better package name
- jenkins.d.n-debug - or such to set an environment to run specific jobs easily
|