1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
|
ToDo for jenkins.debian.net
===========================
:Author: Holger Levsen
:Authorinitials: holger
:EMail: holger@layer-acht.org
:Status: in progress
:lang: en
:Doctype: article
:Licence: GPLv2
== About jenkins.debian.net
See link:http://jenkins.debian.net/userContent/about.html["about jenkins.debian.net"].
== General ToDo
* keep git+svn urls, and use commit hooks. also better for test setups of this install :)
* run calamaris, visitors etc
* proper git repo url, outside users/holger
** same for /srv/jenkins.debian.net-scm-sync.git
* fully automate backup and backup /var/lib/jenkins/jobs /var/lib/munin /var/log /root/ too
* try awstats
* put kgb-client.conf in git and sed passwords from filesystem into it...
=== jenkins-job-builder related
* support for gitweb in jobs
** needs clean merge of my patches to jenkins-job-builder first...
* get rid of some reduncacy in job-cfg/*.yaml
* add documentation for jenkins-job-builder and send pull requests on github:
** publisher:logparse
** publisher:htmlpublisher
** svn:scm
** properties: sidebar
* file a bug about jenkins/jenkins-job-builder sometimes "forgetting" about triggers
== Improve existing tests
=== d-i_manual*
* svn:trunk/manual/po triggers the full build, should trigger language specific builds.
* svn:trunk/manual is all thats needed, not whole svn:trunk
* inform translators (and debian-boot for the package)
=== d-i_build*
* run scripts/digress/
* bubulle wrote: "Another interesting target would be d-i builds *including non uploaded packages* (something like "d-i from git repositories" images). That would in some way require to create a quite specific image, with all udebs (while netboot only has udebs needed before one gets a working network setup).
* build branches? (which?)
* inform debian-boot@ ?
=== chroot-test_*
* inform debian-devel@
* chroot-tests: only trigger (=really run) jobs if available+used packages have changed (save packages in db and compare)
----
<pabs> h01ger: how about all of the blends?
<h01ger> pabs, if you could give me concrete (meta-)package names to install, that would be great+very helpful
<h01ger> for ^education* and ^debian-edu* i can find them myself
<pabs> h01ger: hmm, doesn't seem to be easy to find that list, but here are a few: ezgo-* med-* science-* junior-* gis-*
----
=== webchecks*
* inform debian-www@
=== cd-test_*:
* $name-preseed.base -> sed .cfg (LANG)
** pick LANG from predefined list at random
* boot system
** scp -r vm:/var/log/ results/log
** ssh root@vm shutdown
* use jigdo to download images - but no edu jigdo images.
* still havent got http://www.os-autoinst.org/ out of my mind
== Add more tests
=== Test more Debian Edu related things
* manual - by language and full
* build a cd-image from all edu packages (even (and especially) UNRELEASED ones and do a g-i-installation test with it
** this probably better is done after the edu packages has been switched to git and when edu cd are (also?) build on pettersson.d.o
** also because it's probably better to do this with unmodified d-i first :-)
=== Test tasks:
* test installations with (at least) more language tasks enabled, though tasksel doesn't work like this:
----
<h01ger> how do i install a task outside d-i?
<h01ger> what interesting tasks are there?
<jcristau> tasksel install <task>
<daemonkeeper> h01ger: http://packages.debian.org/source/sid/tasksel
----
Help explaining how to test tasks (ie all the language tasks) very much welcome!
=== Test co-installability
----
<jwilk> Install everything with priority >= optional? These packages are not supposed to conflict with each other. </wishful-thinking>
<h01ger> jwilk, there are tools to detect these package sets just based on debian/control information. no need to test this every day by installing them :)
<h01ger> running these tools daily OTOH...
<h01ger> http://coinst.irill.org/
----
=== Tests using autopkgtest
* I aware but also lost, after looking at /usr/share/doc/autopkgtest/ - wanna help?
* https://wiki.ubuntu.com/AutomatedTesting has more information, but help is still welcome to implement this.
* finding packages with autopkgtests: http://lists.debian.org/debian-qa/2012/11/msg00012.html
** this only lists a few packages, so a very first test could be to compare this list against this: `wget -q http://http.debian.net/debian/dists/unstable/main/Contents-source.gz -O- | zgrep -m1 -E '^debian/tests/control\s'`, make a job out of it and make it UNSTABLE if the diff is non-zero. And as a second step, run those tests...
* also see http://dep.debian.net/deps/dep8/
* also from that thread on debian-qa@l.d.o:
----
Subject: Re: automated tests or builds for d-i
From: Martin Pitt <mpitt@debian.org>
Stefano Zacchiroli [2012-11-04 14:51 +0100]:
> AFAIK in Ubuntu they've glue code that hooks autopkgtest tests into
> jenkins. I don't know where the code is or how Ubuntu-specific it is,
> but it is worth investigating. It'd be awesome to make it as
> distro-agnostic as possible, enabling all derivatives to run their own
> autopkgtest tests: it'd be a nice way to spot regressions introduced by
> distro-specific patches and more.
Indeed, our goal is to get a lot more of them. We currently have some
20 [1], but at the last UDS we had a little competition which got us
another 11 [2], which we'll upload and forward to Debian in the next
days.
Our scripts for creating VMs, running the tests in those, and the
Jenkins integration are in "bzr branch lp:auto-package-testing", which
you can browse at [3]. In the doc/ directory there is documentation
how the Jenkins integration works. I can't say much about it I'm
afraid, as I have never touched a Jenkins job so far; this has been
created by Jean-Baptiste Lallement (in CC:); he's on holiday for the
next week, though.
Please let me know if this is useful. I can poke at the jobs in our
actual Jenkins instance (jenkins.qa.u.c. is just a readonly public
mirror) if you need to know something specific, but we try to avoid
having any manual job configuration; these are all autogenerated
through scripts through python-jenkins.
Martin
[1] https://jenkins.qa.ubuntu.com/view/Quantal/view/AutoPkg%20Test/
[2] https://bugs.launchpad.net/ubuntu/+bugs?field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=FIXRELEASED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.tag=udstestcompetition
[3] http://bazaar.launchpad.net/~auto-package-testing-dev/auto-package-testing/trunk
----
|