summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorHolger Levsen <holger@layer-acht.org>2012-11-20 00:27:36 +0100
committerHolger Levsen <holger@layer-acht.org>2012-11-20 00:27:36 +0100
commit0dfb874f39279e3efeb6cc076f51cfd8b2781c35 (patch)
treedf3801c5166bf3f1e39ea7595a6e06a21c1babaf
parentfd972c12328219914e56a931c0e570df41744082 (diff)
downloadjenkins.debian.net-0dfb874f39279e3efeb6cc076f51cfd8b2781c35.tar.xz
update documentation and todo
-rw-r--r--INSTALL16
-rw-r--r--README2
-rw-r--r--TODO76
-rw-r--r--etc/shorewall/rules2
4 files changed, 79 insertions, 17 deletions
diff --git a/INSTALL b/INSTALL
index 63374761..ea87cbcd 100644
--- a/INSTALL
+++ b/INSTALL
@@ -60,7 +60,7 @@ cd jenkins.debian.net
./update_jdn.sh
----
-'jenkins-job-builder' is the only software which is not covered by this script.
+'jenkins-job-builder' and (currently) 'kgb-client' is the only software which is not covered by this script.
=== Installing jenkins-job-builder
@@ -76,13 +76,23 @@ python setup.py install
Though actually https://github.com/h01ger/jenkins-job-builder is used atm.
-=== Using jenkins-job-builder
+==== Using jenkins-job-builder
-'update_jdn.sh' takes care of this too, so this is just for quick documentation. The real documentation is at link:http://ci.openstack.org/jenkins-job-builder/
+'update_jdn.sh' takes care of this too, so this is just for quick documentation. The real documentation is at http://ci.openstack.org/jenkins-job-builder/
* run `cd /srv/jenkins/job-cfg ; jenkins-jobs update .` to update jobs
* for testing: `cd /srv/jenkins/job-cfg ; rm test-output/ -r ; mkdir test-output ; jenkins-jobs test . -o test-output/ ; ls test-output/`
+=== Installing kgb-client
+
+Rebuild the 'kgb-bot' source package from sid on whezzy and then install the 'kbg-client' package. Configuration is done through 'update_jdn.sh'.
+
+==== IRC notification configuration
+
+The jenkins jobs are configured to send email to 'jenkins+$IRC_CHANNEL' (like 'jenkins+debian-qa'), this is parsed by a script ('/srv/jenkins/bin/email2irc.sh') run through 'procmail' which then in turn notifies 'kgb-client', which notfies 'kgb-server'(s) on the internet, which are the bots notifying the IRC channels. 'kbg-client' needs to be '>= 1.24'.
+
+The Jenkins EMail notification plugin is used as its state tracking is best (and the Jenkins IRC plugin is way too powerful).
+
=== munin
link://munin-monitoring[Munin] is installed, running with link:http://jenkins.debian.net/munin/[jenkins plugins] not yet published elsewhere.
diff --git a/README b/README
index 8167e173..c9c0130c 100644
--- a/README
+++ b/README
@@ -68,7 +68,7 @@ Each job pdebuilds the master branch of its git repo on every git push in a sid
* d_i_manual - builds the full installation-guide package with pdebuild in sid on every commit to svn://svn.debian.org/svn/d-i/ matching these patterns:
'/trunk/manual/debian/.\*', '/trunk/manual/po/.\*', '/trunk/manual/doc/.\*' and '/trunk/manual/scripts/.\*'
** while this job are triggered on commits, the SCM is only polled every 30min to see if there are new commits.
-* d_i_manual_* - builds a language (on wheezy) on every commit of svn/trunk/manual/$LANG with "make languages=$LANG architectures=amd64 formats=html"
+* d_i_manual_*_html - builds a language (on wheezy) on every commit of svn/trunk/manual/$LANG with "make languages=$LANG architectures=amd64 formats=html". On successful build, d_i_manual_$lang_pdf is triggered.
** while these jobs are triggered on commits, the SCM is only polled every 30min to see if there are new commits.
* d_i_parse_build_logs - parses logs from http://d-i.debian.org/daily-images/build-logs.html daily, to give them a bit more exposure.
** this job is run daily.
diff --git a/TODO b/TODO
index 243ace15..87b14bb9 100644
--- a/TODO
+++ b/TODO
@@ -14,11 +14,25 @@ See link:http://jenkins.debian.net/userContent/about.html["about jenkins.debian.
== General ToDo
+* switch git:// and svn:// urls to http:// and thus use proxy. increase polling frequency to 2min.
* general housekeeping job:
** run calamaris, visitors etc
** check for stale processes
** check for apt-upgrades
** check for jobs forgetting their triggers
+* proper git repo url, outside users/holger
+** same for /srv/jenkins.debian.net-scm-sync.git
+* chroot-tests: only trigger (=really run) jobs if available+used packages have changed (save packages in db and compare)
+* update jdk
+* fully automate backup and full backup /var/lib/jenkins/jobs and /var/lib/munin and /var/log too
+* try awstats
+
+=== Bugs
+
+* html_publisher doesn't make the results of d-i_manual_* available
+
+=== jenkins-job-builder related
+
* support for gitweb in jobs
** needs clean merge of my patches to jenkins-job-builder first...
* get rid of some reduncacy in job-cfg/*.yaml
@@ -28,28 +42,28 @@ See link:http://jenkins.debian.net/userContent/about.html["about jenkins.debian.
** svn:scm
** properties: sidebar
* file a bug about jenkins/jenkins-job-builder sometimes "forgetting" about triggers
-* mail notifications (to where?)
-* proper git repo url, outside users/holger
-** same for /srv/jenkins.debian.net-scm-sync.git
-* only trigger (=really run) jobs if available+used packages have changed (save packages in db and compare)
-* update jdk
-* fully automate backup and full backup /var/lib/jenkins/jobs and /var/lib/munin and /var/log too
-* try awstats
== More tests to be run
=== d-i_manual*
-* html_publisher doesn't make the results of d-i_manual_* available
-* svn:trunk:packages/po/* triggers nothing
-* notify #debian-boot about jenkins jobs results
+* svn:trunk/manual/po triggers the full build, should trigger language specific builds.
+----
+As documented in manual/doc/translations_po.txt:
+ Use the following commands:
+ 1. ./scripts/merge_xml en
+ 2. ./scripts/update_pot
+ 3. ./scripts/update_po <your language>
+ 4. ./scripts/revert_pot
+ 5. ./scripts/create_xml <your language>
+----
+* inform translators (and debian-boot for the package)
=== d-i_build*
* run scripts/digress/
* bubulle wrote: "Another interesting target would be d-i builds *including non uploaded packages* (something like "d-i from git repositories" images). That would in some way require to create a quite specific image, with all udebs (while netboot only has udebs needed before one gets a working network setup).
-* notify #debian-boot about jenkins jobs results
-* run scripts/digress/
+* inform debian-boot@ ?
=== chroot-test_*
@@ -89,3 +103,41 @@ Help explaining how to test tasks (ie all the language tasks) very much welcome!
* finding packages with autopkgtests: http://lists.debian.org/debian-qa/2012/11/msg00012.html
** this only lists a few packages, so a very first test could be to compare this list against this: `wget -q http://http.debian.net/debian/dists/unstable/main/Contents-source.gz -O- | zgrep -m1 -E '^debian/tests/control\s'`, make a job out of it and make it UNSTABLE if the diff is non-zero. And as a second step, run those tests...
* also see http://dep.debian.net/deps/dep8/
+* also from that thread on debian-qa@l.d.o:
+----
+Subject: Re: automated tests or builds for d-i
+From: Martin Pitt <mpitt@debian.org>
+
+ Stefano Zacchiroli [2012-11-04 14:51 +0100]:
+> AFAIK in Ubuntu they've glue code that hooks autopkgtest tests into
+> jenkins. I don't know where the code is or how Ubuntu-specific it is,
+> but it is worth investigating. It'd be awesome to make it as
+> distro-agnostic as possible, enabling all derivatives to run their own
+> autopkgtest tests: it'd be a nice way to spot regressions introduced by
+> distro-specific patches and more.
+
+Indeed, our goal is to get a lot more of them. We currently have some
+20 [1], but at the last UDS we had a little competition which got us
+another 11 [2], which we'll upload and forward to Debian in the next
+days.
+
+Our scripts for creating VMs, running the tests in those, and the
+Jenkins integration are in "bzr branch lp:auto-package-testing", which
+you can browse at [3]. In the doc/ directory there is documentation
+how the Jenkins integration works. I can't say much about it I'm
+afraid, as I have never touched a Jenkins job so far; this has been
+created by Jean-Baptiste Lallement (in CC:); he's on holiday for the
+next week, though.
+
+Please let me know if this is useful. I can poke at the jobs in our
+actual Jenkins instance (jenkins.qa.u.c. is just a readonly public
+mirror) if you need to know something specific, but we try to avoid
+having any manual job configuration; these are all autogenerated
+through scripts through python-jenkins.
+
+Martin
+
+[1] https://jenkins.qa.ubuntu.com/view/Quantal/view/AutoPkg%20Test/
+[2] https://bugs.launchpad.net/ubuntu/+bugs?field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=FIXRELEASED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.tag=udstestcompetition
+[3] http://bazaar.launchpad.net/~auto-package-testing-dev/auto-package-testing/trunk
+----
diff --git a/etc/shorewall/rules b/etc/shorewall/rules
index db08726c..4b95dd6f 100644
--- a/etc/shorewall/rules
+++ b/etc/shorewall/rules
@@ -26,7 +26,7 @@ Ping(DROP) net $FW
ACCEPT $FW net icmp
-# http and ssh are allowed
+# incoming http and ssh are allowed
ACCEPT net $FW tcp 80
ACCEPT net $FW tcp 22