path: root/TODO

ToDo for jenkins.debian.net
===========================
:Author:           Holger Levsen
:Authorinitials:   holger
:EMail:            holger@layer-acht.org
:Status:           in progress
:lang:             en
:Doctype:          article
:Licence:	   GPLv2

== About jenkins.debian.net

See link:http://jenkins.debian.net/userContent/about.html["about jenkins.debian.net"].

== General ToDo

* switch git:// urls to http:// and thus use proxy. increase polling frequency to 2min.
* general housekeeping job:
** run calamaris, visitors etc
** check for stale processes
** check for apt-upgrades
** check for jobs forgetting their triggers
* proper git repo url, outside users/holger
** same for /srv/jenkins.debian.net-scm-sync.git
* chroot-tests: only trigger (=really run) jobs if available+used packages have changed (save packages in db and compare)
* update jdk
* fully automate backup and full backup /var/lib/jenkins/jobs and /var/lib/munin and /var/log too
* try awstats
* put kgb-client.conf in git and sed passwords from filesystem into it...

=== jenkins-job-builder related

* support for gitweb in jobs 
** needs clean merge of my patches to jenkins-job-builder first...
* get rid of some reduncacy in job-cfg/*.yaml 
* add documentation for jenkins-job-builder and send pull requests on github:
** publisher:logparse
** publisher:htmlpublisher
** svn:scm
** properties: sidebar
* file a bug about jenkins/jenkins-job-builder sometimes "forgetting" about triggers

== More tests to be run

=== d-i_manual*

* svn:trunk/manual/po triggers the full build, should trigger language specific builds.
* inform translators (and debian-boot for the package)

=== d-i_build*

* run scripts/digress/
* bubulle wrote: "Another interesting target would be d-i builds *including non uploaded packages* (something like "d-i from git repositories" images). That would in some way require to create a quite specific image, with all udebs (while netboot only has udebs needed before one gets a working network setup).
* build branches? (which?)
* inform debian-boot@ ?

=== chroot-test_*

* inform debian-devel@

----
<pabs> h01ger: how about all of the blends?
<h01ger> pabs, if you could give me concrete (meta-)package names to install, that would be great+very helpful
<h01ger> for ^education* and ^debian-edu* i can find them myself
<pabs> h01ger: hmm, doesn't seem to be easy to find that list, but here are a few: ezgo-* med-* science-* junior-* gis-*
----

=== webchecks*

* inform debian-www@

=== Test cd-images:

* NIH:
** weekly edu
** $IMAGE is in results
** script $4 qcow image size=
** edu-wheezy-preseed.base -> sed .cfg (LANG)
*** pick LANG from predefined list at random
** wheezy-preseed.base for normal d-i
** scp -r vm:/var/log/ results/log
** ssh root@vm shutdown
** d-i weekly sid: cat /mnt/isolinux/lxde/gtk.cfg
* try http://www.os-autoinst.org/
* start client VMs using the profitbricks API: https://www.profitbricks.com/us/media/docs/PB_PublicAPI_EN.pdf (and dhcp)
** or probably just run kvm...
* to test (at first) weekly images of amd64 std,kde,lxde+xfce weekly and the first sid-di daily
** http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-cd/
** http://cdimage.debian.org/cdimage/daily-builds/sid_d-i/20121020-1/amd64/
* use jigdo to download those images

=== Test tasks:

* test installations with (at least) more language tasks enabled, though tasksel doesn't work like this:

----
<h01ger> how do i install a task outside d-i?
<h01ger> what interesting tasks are there?
<jcristau> tasksel install <task>
<daemonkeeper> h01ger:  http://packages.debian.org/source/sid/tasksel
----

Help explaining how to test tasks (ie all the language tasks) very much welcome!

=== Test co-installability

----
<jwilk> Install everything with priority >= optional? These packages are not supposed to conflict with each other. </wishful-thinking>
<h01ger> jwilk, there are tools to detect these package sets just based on debian/control information. no need to test this every day by installing them :)
<h01ger> running these tools daily OTOH...
<h01ger> http://coinst.irill.org/
----

=== Tests using autopkgtest

* I aware but also lost, after looking at /usr/share/doc/autopkgtest/ - wanna help?
* https://wiki.ubuntu.com/AutomatedTesting has more information, but help is still welcome to implement this.
* finding packages with autopkgtests: http://lists.debian.org/debian-qa/2012/11/msg00012.html
** this only lists a few packages, so a very first test could be to compare this list against this: `wget -q http://http.debian.net/debian/dists/unstable/main/Contents-source.gz -O- | zgrep -m1 -E '^debian/tests/control\s'`, make a job out of it and make it UNSTABLE if the diff is non-zero. And as a second step, run those tests...
* also see http://dep.debian.net/deps/dep8/
* also from that thread on debian-qa@l.d.o:
----
Subject: Re: automated tests or builds for d-i
From: Martin Pitt <mpitt@debian.org>

  Stefano Zacchiroli [2012-11-04 14:51 +0100]:
> AFAIK in Ubuntu they've glue code that hooks autopkgtest tests into
> jenkins. I don't know where the code is or how Ubuntu-specific it is,
> but it is worth investigating. It'd be awesome to make it as
> distro-agnostic as possible, enabling all derivatives to run their own
> autopkgtest tests: it'd be a nice way to spot regressions introduced by
> distro-specific patches and more.

Indeed, our goal is to get a lot more of them. We currently have some
20 [1], but at the last UDS we had a little competition which got us
another 11 [2], which we'll upload and forward to Debian in the next
days.

Our scripts for creating VMs, running the tests in those, and the
Jenkins integration are in "bzr branch lp:auto-package-testing", which
you can browse at [3]. In the doc/ directory there is documentation
how the Jenkins integration works. I can't say much about it I'm
afraid, as I have never touched a Jenkins job so far; this has been
created by Jean-Baptiste Lallement (in CC:); he's on holiday for the
next week, though.

Please let me know if this is useful. I can poke at the jobs in our
actual Jenkins instance (jenkins.qa.u.c. is just a readonly public
mirror) if you need to know something specific, but we try to avoid
having any manual job configuration; these are all autogenerated
through scripts through python-jenkins.

Martin

[1] https://jenkins.qa.ubuntu.com/view/Quantal/view/AutoPkg%20Test/
[2] https://bugs.launchpad.net/ubuntu/+bugs?field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=FIXRELEASED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.tag=udstestcompetition
[3] http://bazaar.launchpad.net/~auto-package-testing-dev/auto-package-testing/trunk
----