diff options
| -rw-r--r-- | INSTALL | 16 | ||||
| -rw-r--r-- | README | 2 | ||||
| -rw-r--r-- | TODO | 76 | ||||
| -rw-r--r-- | etc/shorewall/rules | 2 |
4 files changed, 79 insertions, 17 deletions
@@ -60,7 +60,7 @@ cd jenkins.debian.net ./update_jdn.sh ---- -'jenkins-job-builder' is the only software which is not covered by this script. +'jenkins-job-builder' and (currently) 'kgb-client' is the only software which is not covered by this script. === Installing jenkins-job-builder @@ -76,13 +76,23 @@ python setup.py install Though actually https://github.com/h01ger/jenkins-job-builder is used atm. -=== Using jenkins-job-builder +==== Using jenkins-job-builder -'update_jdn.sh' takes care of this too, so this is just for quick documentation. The real documentation is at link:http://ci.openstack.org/jenkins-job-builder/ +'update_jdn.sh' takes care of this too, so this is just for quick documentation. The real documentation is at http://ci.openstack.org/jenkins-job-builder/ * run `cd /srv/jenkins/job-cfg ; jenkins-jobs update .` to update jobs * for testing: `cd /srv/jenkins/job-cfg ; rm test-output/ -r ; mkdir test-output ; jenkins-jobs test . -o test-output/ ; ls test-output/` +=== Installing kgb-client + +Rebuild the 'kgb-bot' source package from sid on whezzy and then install the 'kbg-client' package. Configuration is done through 'update_jdn.sh'. + +==== IRC notification configuration + +The jenkins jobs are configured to send email to 'jenkins+$IRC_CHANNEL' (like 'jenkins+debian-qa'), this is parsed by a script ('/srv/jenkins/bin/email2irc.sh') run through 'procmail' which then in turn notifies 'kgb-client', which notfies 'kgb-server'(s) on the internet, which are the bots notifying the IRC channels. 'kbg-client' needs to be '>= 1.24'. + +The Jenkins EMail notification plugin is used as its state tracking is best (and the Jenkins IRC plugin is way too powerful). + === munin link://munin-monitoring[Munin] is installed, running with link:http://jenkins.debian.net/munin/[jenkins plugins] not yet published elsewhere. @@ -68,7 +68,7 @@ Each job pdebuilds the master branch of its git repo on every git push in a sid * d_i_manual - builds the full installation-guide package with pdebuild in sid on every commit to svn://svn.debian.org/svn/d-i/ matching these patterns: '/trunk/manual/debian/.\*', '/trunk/manual/po/.\*', '/trunk/manual/doc/.\*' and '/trunk/manual/scripts/.\*' ** while this job are triggered on commits, the SCM is only polled every 30min to see if there are new commits. -* d_i_manual_* - builds a language (on wheezy) on every commit of svn/trunk/manual/$LANG with "make languages=$LANG architectures=amd64 formats=html" +* d_i_manual_*_html - builds a language (on wheezy) on every commit of svn/trunk/manual/$LANG with "make languages=$LANG architectures=amd64 formats=html". On successful build, d_i_manual_$lang_pdf is triggered. ** while these jobs are triggered on commits, the SCM is only polled every 30min to see if there are new commits. * d_i_parse_build_logs - parses logs from http://d-i.debian.org/daily-images/build-logs.html daily, to give them a bit more exposure. ** this job is run daily. @@ -14,11 +14,25 @@ See link:http://jenkins.debian.net/userContent/about.html["about jenkins.debian. == General ToDo +* switch git:// and svn:// urls to http:// and thus use proxy. increase polling frequency to 2min. * general housekeeping job: ** run calamaris, visitors etc ** check for stale processes ** check for apt-upgrades ** check for jobs forgetting their triggers +* proper git repo url, outside users/holger +** same for /srv/jenkins.debian.net-scm-sync.git +* chroot-tests: only trigger (=really run) jobs if available+used packages have changed (save packages in db and compare) +* update jdk +* fully automate backup and full backup /var/lib/jenkins/jobs and /var/lib/munin and /var/log too +* try awstats + +=== Bugs + +* html_publisher doesn't make the results of d-i_manual_* available + +=== jenkins-job-builder related + * support for gitweb in jobs ** needs clean merge of my patches to jenkins-job-builder first... * get rid of some reduncacy in job-cfg/*.yaml @@ -28,28 +42,28 @@ See link:http://jenkins.debian.net/userContent/about.html["about jenkins.debian. ** svn:scm ** properties: sidebar * file a bug about jenkins/jenkins-job-builder sometimes "forgetting" about triggers -* mail notifications (to where?) -* proper git repo url, outside users/holger -** same for /srv/jenkins.debian.net-scm-sync.git -* only trigger (=really run) jobs if available+used packages have changed (save packages in db and compare) -* update jdk -* fully automate backup and full backup /var/lib/jenkins/jobs and /var/lib/munin and /var/log too -* try awstats == More tests to be run === d-i_manual* -* html_publisher doesn't make the results of d-i_manual_* available -* svn:trunk:packages/po/* triggers nothing -* notify #debian-boot about jenkins jobs results +* svn:trunk/manual/po triggers the full build, should trigger language specific builds. +---- +As documented in manual/doc/translations_po.txt: + Use the following commands: + 1. ./scripts/merge_xml en + 2. ./scripts/update_pot + 3. ./scripts/update_po <your language> + 4. ./scripts/revert_pot + 5. ./scripts/create_xml <your language> +---- +* inform translators (and debian-boot for the package) === d-i_build* * run scripts/digress/ * bubulle wrote: "Another interesting target would be d-i builds *including non uploaded packages* (something like "d-i from git repositories" images). That would in some way require to create a quite specific image, with all udebs (while netboot only has udebs needed before one gets a working network setup). -* notify #debian-boot about jenkins jobs results -* run scripts/digress/ +* inform debian-boot@ ? === chroot-test_* @@ -89,3 +103,41 @@ Help explaining how to test tasks (ie all the language tasks) very much welcome! * finding packages with autopkgtests: http://lists.debian.org/debian-qa/2012/11/msg00012.html ** this only lists a few packages, so a very first test could be to compare this list against this: `wget -q http://http.debian.net/debian/dists/unstable/main/Contents-source.gz -O- | zgrep -m1 -E '^debian/tests/control\s'`, make a job out of it and make it UNSTABLE if the diff is non-zero. And as a second step, run those tests... * also see http://dep.debian.net/deps/dep8/ +* also from that thread on debian-qa@l.d.o: +---- +Subject: Re: automated tests or builds for d-i +From: Martin Pitt <mpitt@debian.org> + + Stefano Zacchiroli [2012-11-04 14:51 +0100]: +> AFAIK in Ubuntu they've glue code that hooks autopkgtest tests into +> jenkins. I don't know where the code is or how Ubuntu-specific it is, +> but it is worth investigating. It'd be awesome to make it as +> distro-agnostic as possible, enabling all derivatives to run their own +> autopkgtest tests: it'd be a nice way to spot regressions introduced by +> distro-specific patches and more. + +Indeed, our goal is to get a lot more of them. We currently have some +20 [1], but at the last UDS we had a little competition which got us +another 11 [2], which we'll upload and forward to Debian in the next +days. + +Our scripts for creating VMs, running the tests in those, and the +Jenkins integration are in "bzr branch lp:auto-package-testing", which +you can browse at [3]. In the doc/ directory there is documentation +how the Jenkins integration works. I can't say much about it I'm +afraid, as I have never touched a Jenkins job so far; this has been +created by Jean-Baptiste Lallement (in CC:); he's on holiday for the +next week, though. + +Please let me know if this is useful. I can poke at the jobs in our +actual Jenkins instance (jenkins.qa.u.c. is just a readonly public +mirror) if you need to know something specific, but we try to avoid +having any manual job configuration; these are all autogenerated +through scripts through python-jenkins. + +Martin + +[1] https://jenkins.qa.ubuntu.com/view/Quantal/view/AutoPkg%20Test/ +[2] https://bugs.launchpad.net/ubuntu/+bugs?field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=FIXRELEASED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.tag=udstestcompetition +[3] http://bazaar.launchpad.net/~auto-package-testing-dev/auto-package-testing/trunk +---- diff --git a/etc/shorewall/rules b/etc/shorewall/rules index db08726c..4b95dd6f 100644 --- a/etc/shorewall/rules +++ b/etc/shorewall/rules @@ -26,7 +26,7 @@ Ping(DROP) net $FW ACCEPT $FW net icmp -# http and ssh are allowed +# incoming http and ssh are allowed ACCEPT net $FW tcp 80 ACCEPT net $FW tcp 22 |