diff options
| -rw-r--r-- | TODO | 30 | ||||
| -rw-r--r-- | etc/apache2/conf.d/munin | 86 | ||||
| -rw-r--r-- | etc/apache2/sites-available/jenkins.debian.net | 42 | ||||
| -rw-r--r-- | etc/munin/munin.conf | 141 | ||||
| -rw-r--r-- | job-cfg/d-i.yaml | 2 | ||||
| -rwxr-xr-x | update_jdn.sh | 3 |
6 files changed, 296 insertions, 8 deletions
@@ -12,9 +12,9 @@ About jenkins.debian.net == About jenkins.debian.net * This is pretty much work in progress, (hopefully) doing something useful since October 15th 2012 -* Join #debian-qa on irc.debian.org to report bugs or give other feedback! +* Join #debian-qa on irc.debian.org to report bugs or give other feedback! Use debian-qa@lists.debian.org if you prefer mail. * git clone git://git.debian.org/git/users/holger/jenkins.debian.net.git -* (virtual) hardware sponsored by http://www.profitbricks.com +* (virtual) hardware sponsored by http://www.profitbricks.com - currently it's just a server, but in future there will be installation and tests clients started on demand as well. == General ToDo @@ -28,10 +28,12 @@ About jenkins.debian.net === More minor stuff +* make job configuration visible to un-authenticated users * squid has problems with SO_FAIL * split TODO and about.txt? +* setup munin monitoring ** use debian stylesheet for about.html -** use job description to create .txt file to be included in about.txt/TODO? +** add sidebar links for all jobs * proper git repo url, outside users/holger * only trigger (=really run) jobs if available+used packages have changed (save packages in db and compare) @@ -47,6 +49,8 @@ About jenkins.debian.net === Test tasks: +* test installation with (at least) more language tasks enabled + ---- <h01ger> how do i install a task outside d-i? <h01ger> what interesting tasks are there? @@ -56,6 +60,20 @@ About jenkins.debian.net (FIXME: tasksel doesn't work like this) +=== Tests for supporting d-i development: + +* build the manual on every svn commit +** later: notify debian-boot@l.d.o and #debian-boot +* run scripts/digress/ +* bubulle wrote: "Another interesting target would be d-i builds *including non uploaded > packages* (something like "d-i from git repositories" images). That would in some way require to create a quite specific image, with all udebs (while netboot only has udebs needed before one gets a working network setup). + +=== Tests using autopkgtest + +* I aware but also lost, after looking at /usr/share/doc/autopkgtest/ - wanna help? +* https://wiki.ubuntu.com/AutomatedTesting has more information, but help is still welcome to implement this. +* finding packages with autopkgtests: http://lists.debian.org/debian-qa/2012/11/msg00012.html +* also see http://dep.debian.net/deps/dep8/ + == Installed software that makes jenkins.debian.net run * jenkins.debian.net runs Debian wheezy. @@ -64,12 +82,12 @@ About jenkins.debian.net ** update_jdn.sh is available in the git repo * package configuration is kept in git and deployed by update_jdn.sh as well -=== installing jenkins-job-builder +=== Installing jenkins-job-builder * rebuild python-jenkinsapi from sid for wheezy * pip install python-jenkins * git clone https://github.com/openstack-ci/jenkins-job-builder ; cd jenkins-job-builder ; python setup.py install -=== using jenkins-job-builder +=== Using jenkins-job-builder * then run 'cd /srv/jenkins/job-cfg ; jenkins-jobs update .' to update jobs * testing: 'cd /srv/jenkins/job-cfg ; rm test-output/ -r ; mkdir test-output ; jenkins-jobs test . -o test-output/ ; ls test-output/' @@ -77,4 +95,4 @@ About jenkins.debian.net == To backup * /etc/.git -* /var/lib/jenkins/config.xml as well as its plugins etc +* /var/lib/jenkins/config.xml as well as its plugins diff --git a/etc/apache2/conf.d/munin b/etc/apache2/conf.d/munin new file mode 100644 index 00000000..2c77e185 --- /dev/null +++ b/etc/apache2/conf.d/munin @@ -0,0 +1,86 @@ +# Enable this for template generation +Alias /munin /var/cache/munin/www + +# Enable this for cgi-based templates +#Alias /munin-cgi/static /var/cache/munin/www/static +#ScriptAlias /munin-cgi /usr/lib/munin/cgi/munin-cgi-html +#<Location /munin-cgi> +# Order allow,deny +# Allow from localhost 127.0.0.0/8 ::1 +# AuthUserFile /etc/munin/munin-htpasswd +# AuthName "Munin" +# AuthType Basic +# require valid-user +#</Location> + +<Directory /var/cache/munin/www> + Order allow,deny + Allow from all + Options None + + # This file can be used as a .htaccess file, or a part of your apache + # config file. + # + # For the .htaccess file option to work the munin www directory + # (/var/cache/munin/www) must have "AllowOverride all" or something + # close to that set. + # + + # AuthUserFile /etc/munin/munin-htpasswd + # AuthName "Munin" + # AuthType Basic + # require valid-user + + # This next part requires mod_expires to be enabled. + # + + # Set the default expiration time for files to 5 minutes 10 seconds from + # their creation (modification) time. There are probably new files by + # that time. + # + + <IfModule mod_expires.c> + ExpiresActive On + ExpiresDefault M310 + </IfModule> + +</Directory> + +# Enables fastcgi for munin-cgi-html if present +#<Location /munin-cgi> +# <IfModule mod_fastcgi.c> +# SetHandler fastcgi-script +# </IfModule> +#</Location> + +#<Location /munin-cgi/static> +# SetHandler None +#</Location> + +# Enables fastcgi for munin-cgi-graph if present +ScriptAlias /munin-cgi/munin-cgi-graph /usr/lib/munin/cgi/munin-cgi-graph +<Location /munin-cgi/munin-cgi-graph> + Order allow,deny + Allow from localhost 127.0.0.0/8 ::1 + # AuthUserFile /etc/munin/munin-htpasswd + # AuthName "Munin" + # AuthType Basic + # require valid-user + <IfModule mod_fastcgi.c> + SetHandler fastcgi-script + </IfModule> +</Location> + +ScriptAlias /munin-cgi/munin-cgi-html /usr/lib/munin/cgi/munin-cgi-html +<Location /munin-cgi/munin-cgi-html> + Order allow,deny + Allow from localhost 127.0.0.0/8 ::1 + # AuthUserFile /etc/munin/munin-htpasswd + # AuthName "Munin" + # AuthType Basic + # require valid-user + <IfModule mod_fastcgi.c> + SetHandler fastcgi-script + </IfModule> +</Location> + diff --git a/etc/apache2/sites-available/jenkins.debian.net b/etc/apache2/sites-available/jenkins.debian.net new file mode 100644 index 00000000..dc2d5b59 --- /dev/null +++ b/etc/apache2/sites-available/jenkins.debian.net @@ -0,0 +1,42 @@ +<VirtualHost *:80> + ServerAdmin webmaster@localhost + + DocumentRoot /var/www + <Directory /> + Options FollowSymLinks + AllowOverride None + </Directory> + <Directory /var/www/> + Options Indexes FollowSymLinks MultiViews + AllowOverride None + Order allow,deny + allow from all + </Directory> + + ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ + <Directory "/usr/lib/cgi-bin"> + AllowOverride None + Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch + Order allow,deny + Allow from all + </Directory> + + ProxyRequests Off + <Proxy *> + Order deny,allow + Allow from all + </Proxy> + ProxyPreserveHost on + # proxy everything but /munin + ProxyPass /munin ! + ProxyPass / http://localhost:8080/ + + + ErrorLog ${APACHE_LOG_DIR}/error.log + + # Possible values include: debug, info, notice, warn, error, crit, + # alert, emerg. + LogLevel warn + + CustomLog ${APACHE_LOG_DIR}/access.log combined +</VirtualHost> diff --git a/etc/munin/munin.conf b/etc/munin/munin.conf new file mode 100644 index 00000000..0bedd5a8 --- /dev/null +++ b/etc/munin/munin.conf @@ -0,0 +1,141 @@ +# Example configuration file for Munin, generated by 'make build' + +# The next three variables specifies where the location of the RRD +# databases, the HTML output, logs and the lock/pid files. They all +# must be writable by the user running munin-cron. They are all +# defaulted to the values you see here. +# +#dbdir /var/lib/munin +#htmldir /var/cache/munin/www +#logdir /var/log/munin +#rundir /var/run/munin + +# Where to look for the HTML templates +# +#tmpldir /etc/munin/templates + +# Where to look for the static www files +# +#staticdir /etc/munin/static + +# temporary cgi files are here. note that it has to be writable by +# the cgi user (usually nobody or httpd). +# +# cgitmpdir /var/lib/munin/cgi-tmp + +# (Exactly one) directory to include all files from. +includedir /etc/munin/munin-conf.d + +# You can choose the time reference for "DERIVE" like graphs, and show +# "per minute", "per hour" values instead of the default "per second" +# +#graph_period second + +# Graphics files are generated either via cron or by a CGI process. +# See http://munin-monitoring.org/wiki/CgiHowto2 for more +# documentation. +# Since 2.0, munin-graph has been rewritten to use the cgi code. +# It is single threaded *by design* now. +# +#graph_strategy cron + +# munin-cgi-graph is invoked by the web server up to very many times at the +# same time. This is not optimal since it results in high CPU and memory +# consumption to the degree that the system can thrash. Again the default is +# 6. Most likely the optimal number for max_cgi_graph_jobs is the same as +# max_graph_jobs. +# +#munin_cgi_graph_jobs 6 + +# If the automatic CGI url is wrong for your system override it here: +# +#cgiurl_graph /munin-cgi/munin-cgi-graph + +# max_size_x and max_size_y are the max size of images in pixel. +# Default is 4000. Do not make it too large otherwise RRD might use all +# RAM to generate the images. +# +#max_size_x 4000 +#max_size_y 4000 + +# HTML files are normally generated by munin-html, no matter if the +# files are used or not. You can change this to on-demand generation +# by following the instructions in http://munin-monitoring.org/wiki/CgiHowto2 +# +# Notes: +# - moving to CGI for HTML means you cannot have graph generated by cron. +# - cgi html has some bugs, mostly you still have to launch munin-html by hand +# +#html_strategy cron + +# munin-update runs in parallel. +# +# The default max number of processes is 16, and is probably ok for you. +# +# If set too high, it might hit some process/ram/filedesc limits. +# If set too low, munin-update might take more than 5 min. +# +# If you want munin-update to not be parallel set it to 0. +# +#max_processes 16 + +# RRD updates are per default, performed directly on the rrd files. +# To reduce IO and enable the use of the rrdcached, uncomment it and set it to +# the location of the socket that rrdcached uses. +# +#rrdcached_socket /var/run/rrdcached.sock + +# Drop somejuser@fnord.comm and anotheruser@blibb.comm an email everytime +# something changes (OK -> WARNING, CRITICAL -> OK, etc) +#contact.someuser.command mail -s "Munin notification" somejuser@fnord.comm +#contact.anotheruser.command mail -s "Munin notification" anotheruser@blibb.comm +# +# For those with Nagios, the following might come in handy. In addition, +# the services must be defined in the Nagios server as well. +#contact.nagios.command /usr/bin/send_nsca nagios.host.comm -c /etc/nsca.conf + +# a simple host tree +[jenkins.debian.net] + address 127.0.0.1 + use_node_name yes + +# +# A more complex example of a host tree +# +## First our "normal" host. +# [fii.foo.com] +# address foo +# +## Then our other host... +# [fay.foo.com] +# address fay +# +## Then we want totals... +# [foo.com;Totals] #Force it into the "foo.com"-domain... +# update no # Turn off data-fetching for this "host". +# +# # The graph "load1". We want to see the loads of both machines... +# # "fii=fii.foo.com:load.load" means "label=machine:graph.field" +# load1.graph_title Loads side by side +# load1.graph_order fii=fii.foo.com:load.load fay=fay.foo.com:load.load +# +# # The graph "load2". Now we want them stacked on top of each other. +# load2.graph_title Loads on top of each other +# load2.dummy_field.stack fii=fii.foo.com:load.load fay=fay.foo.com:load.load +# load2.dummy_field.draw AREA # We want area instead the default LINE2. +# load2.dummy_field.label dummy # This is needed. Silly, really. +# +# # The graph "load3". Now we want them summarised into one field +# load3.graph_title Loads summarised +# load3.combined_loads.sum fii.foo.com:load.load fay.foo.com:load.load +# load3.combined_loads.label Combined loads # Must be set, as this is +# # not a dummy field! +# +## ...and on a side note, I want them listen in another order (default is +## alphabetically) +# +# # Since [foo.com] would be interpreted as a host in the domain "com", we +# # specify that this is a domain by adding a semicolon. +# [foo.com;] +# node_order Totals fii.foo.com fay.foo.com +# diff --git a/job-cfg/d-i.yaml b/job-cfg/d-i.yaml index 97d16613..b7113418 100644 --- a/job-cfg/d-i.yaml +++ b/job-cfg/d-i.yaml @@ -6,7 +6,7 @@ url: svn://svn.debian.org/svn/d-i/trunk basedir: . workspaceupdater: update - includedRegions: 'trunk/manual/.*' + includedRegions: 'manual/.*' excludedRegions: '' excludedUsers: '' excludedRevprop: '' diff --git a/update_jdn.sh b/update_jdn.sh index 675a5cc5..039193d6 100755 --- a/update_jdn.sh +++ b/update_jdn.sh @@ -22,7 +22,7 @@ asciidoc -a numbered -a data-uri -a iconsdir=/etc/asciidoc/images/icons -a scrip # install packages we need # (more or less grouped into more-then-nice-to-have, needed-while-things-are-new, needed) # -sudo apt-get install vim screen less etckeeper curl mtr-tiny dstat devscripts bash-completion shorewall shorewall6 cron-apt apt-listchanges \ +sudo apt-get install vim screen less etckeeper curl mtr-tiny dstat devscripts bash-completion shorewall shorewall6 cron-apt apt-listchanges munin \ build-essential python-setuptools \ debootstrap sudo figlet graphviz apache2 python-yaml python-pip mr subversion subversion-tools vnstat @@ -41,6 +41,7 @@ fi sudo chown root.root /etc/sudoers.d/jenkins ; sudo chmod 700 /etc/sudoers.d/jenkins sudo ln -sf /etc/apache2/sites-available/jenkins.debian.net /etc/apache2/sites-enabled/000-default sudo service apache2 reload +cd /etc/munin/plugins ; sudo rm rm postfix_* open_inodes df_inode interrupts diskstats # # run jenkins-job-builder to update jobs if needed |