summaryrefslogtreecommitdiffstats
path: root/TODO
blob: 1dd44220c9c2e6ceb950d8a9d11b63d2e304a746 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
ToDo for jenkins.debian.net
===========================
:Author:           Holger Levsen
:Authorinitials:   holger
:EMail:            holger@layer-acht.org
:Status:           in progress
:lang:             en
:Doctype:          article
:Licence:	   GPLv2

== About jenkins.debian.net

See link:http://jenkins.debian.net/userContent/about.html["about jenkins.debian.net"].

== General ToDo

* keep git+svn urls, and use commit hooks. also better for test setups of this setup as well as overall stability.
** create d-i_svn_trigger job to trigger sub-set jobs (like manual)
** create d-i git hooks
* get rid of some reduncacy in job-cfg/*.yaml
** FIXME: sometimes true and some True used in yaml files
* proper git repo url, outside users/holger
** same for /srv/jenkins.debian.net-scm-sync.git
* fully automate backup and backup /var/lib/jenkins/jobs /var/lib/munin /var/log /root/ too
* try awstats
* put kgb-client.conf in git and sed passwords from filesystem into it...
* push jenkins munin plugins upstream

== Debian Packaging related

* /usr/sbin/jenkins.debian.net-setup needs to be written
* what update-j.d.n.sh does, needs to be put elsewhere...
* debian/copyright is incorrect about some licenses: a.) the profitbricks+debian+jenkins logos b.) the preseeding files
* switch to scm trigger instead of scm pulling before releasing this as 0.1...

=== jenkins-job-builder related

* support for gitweb in jobs 
** needs clean merge of my patches to jenkins-job-builder first...
* cleanup my patches (eg add documentation) and send pull requests on github:
** publisher:logparse
** publisher:htmlpublisher
** publisher:imagegallery
** svn:scm
** properties: sidebar

== Improve existing tests

=== d-i_manual*

* svn:trunk/manual/po triggers the full build, should trigger language specific builds.
* svn:trunk/manual is all thats needed, not whole svn:trunk
* inform translators (and debian-boot for the package)

=== d-i_build*

* run scripts/digress/
* bubulle wrote: "Another interesting target would be d-i builds *including non uploaded packages* (something like "d-i from git repositories" images). That would in some way require to create a quite specific image, with all udebs (while netboot only has udebs needed before one gets a working network setup).
* build branches? (which?)
* inform debian-boot@ ?

=== chroot-installation_*

* inform debian-devel@
* "Hash Sum Mismatch" problem with squid, see http://jenkins.debian.net/job/chroot-installation_squeeze_install_developer_upgrade_to_wheezy/13/console
** pere says this is supposed to be fixed in squid3
* chroot-installation: only trigger (=really run) jobs if available+used packages have changed (save packages in db and compare)

----
<pabs> h01ger: how about all of the blends?
<h01ger> pabs, if you could give me concrete (meta-)package names to install, that would be great+very helpful
<h01ger> for ^education* and ^debian-edu* i can find them myself
<pabs> h01ger: hmm, doesn't seem to be easy to find that list, but here are a few: ezgo-* med-* science-* junior-* gis-*
----

=== webchecks*

* inform debian-www@

=== g-i-installation_*:

more languages

* babelbox.git has a nice langlist
* $name-preseed.base -> sed .cfg (LANG)
* for edu: pick LANG from predefined list at random - if last build was not successful or unstable fall back to English
* for d-i: create jobs for something like Tamil, Traditional Chinese, Japanese, Hebrew, some Arabic and maybe some more Indian languages would be sufficient
** these jobs would not need to do an install, just booting them in rescue mode is probably enough

post-install

* boot system
** scp -r vm:/var/log/ results/log  || mount .img and copy logs from there
** login graphically
** run vim in xterm
** run emacs in xterm
** run iceweasel www.debian.org / www.skolelinux.org
** su -c poweroff

general

* use jigdo to download images - but no edu jigdo images.
* test live images from http://live.debian.net/cdimage/release/current-next and .../current
* use https://wiki.jenkins-ci.org/display/JENKINS/Image+Gallery+Plugin
* still havent got http://www.os-autoinst.org/ out of my mind

== Add more tests

=== Test more Debian Edu related things

* manual - by language and full
* build a cd-image from all edu packages (even (and especially) UNRELEASED ones and do a g-i-installation test with it
** this probably better is done after the edu packages has been switched to git and when edu cd are (also?) build on pettersson.d.o
** also because it's probably better to do this with unmodified d-i first :-)

=== Test tasks:

* test installations with (at least) more language tasks enabled, though tasksel doesn't work like this:

----
<h01ger> how do i install a task outside d-i?
<h01ger> what interesting tasks are there?
<jcristau> tasksel install <task>
<daemonkeeper> h01ger:  http://packages.debian.org/source/sid/tasksel
----

Help explaining how to test tasks (ie all the language tasks) very much welcome!

=== Tests using autopkgtest

* see https://wiki.ubuntu.com/AutomatedTesting and http://developer.ubuntu.com/packaging/html/auto-pkg-test.html 
* finding packages with autopkgtests: http://lists.debian.org/debian-qa/2012/11/msg00012.html
** this only lists a few packages, so a very first test could be to compare this list against this: `wget -q http://http.debian.net/debian/dists/unstable/main/Contents-source.gz -O- | zgrep -m1 -E '^debian/tests/control\s'`, make a job out of it and make it UNSTABLE if the diff is non-zero. And as a second step, run those tests...
* also see http://dep.debian.net/deps/dep8/
* also from that thread on debian-qa@l.d.o:
----
Subject: Re: automated tests or builds for d-i
From: Martin Pitt <mpitt@debian.org>

  Stefano Zacchiroli [2012-11-04 14:51 +0100]:
> AFAIK in Ubuntu they've glue code that hooks autopkgtest tests into
> jenkins. I don't know where the code is or how Ubuntu-specific it is,
> but it is worth investigating. It'd be awesome to make it as
> distro-agnostic as possible, enabling all derivatives to run their own
> autopkgtest tests: it'd be a nice way to spot regressions introduced by
> distro-specific patches and more.

Indeed, our goal is to get a lot more of them. We currently have some
20 [1], but at the last UDS we had a little competition which got us
another 11 [2], which we'll upload and forward to Debian in the next
days.

Our scripts for creating VMs, running the tests in those, and the
Jenkins integration are in "bzr branch lp:auto-package-testing", which
you can browse at [3]. In the doc/ directory there is documentation
how the Jenkins integration works. I can't say much about it I'm
afraid, as I have never touched a Jenkins job so far; this has been
created by Jean-Baptiste Lallement (in CC:); he's on holiday for the
next week, though.

Please let me know if this is useful. I can poke at the jobs in our
actual Jenkins instance (jenkins.qa.u.c. is just a readonly public
mirror) if you need to know something specific, but we try to avoid
having any manual job configuration; these are all autogenerated
through scripts through python-jenkins.

Martin

[1] https://jenkins.qa.ubuntu.com/view/Quantal/view/AutoPkg%20Test/
[2] https://bugs.launchpad.net/ubuntu/+bugs?field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=FIXRELEASED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.tag=udstestcompetition
[3] http://bazaar.launchpad.net/~auto-package-testing-dev/auto-package-testing/trunk
----

=== Test co-installability

----
<jwilk> Install everything with priority >= optional? These packages are not supposed to conflict with each other. </wishful-thinking>
<h01ger> jwilk, there are tools to detect these package sets just based on debian/control information. no need to test this every day by installing them :)
<h01ger> running these tools daily OTOH...
<h01ger> http://coinst.irill.org/
----