Steady as a rock

Ubuntu-nl has outgrown its server. To reduce load on it, I’ve moved the USN and -changes feeds to feedburner. Here are the updated locations:

• http://feeds.ubuntu-nl.org/UbuntuSecurityNotices
• http://feeds.ubuntu-nl.org/DapperChanges
• http://feeds.ubuntu-nl.org/EdgyChanges
• http://feeds.ubuntu-nl.org/FeistyChanges
• http://feeds.ubuntu-nl.org/GutsyChanges
• http://feeds.ubuntu-nl.org/HardyChanges
• http://feeds.ubuntu-nl.org/IntrepidChanges (still empty of course :))

Inspired by the latest launchpad news, here is a better version of launchpadduser.py. It does more sanity checking, can be used as a module and allows you to use a different name locally than on launchpad (eg sudo ./launchpadduser.py –launchpad-name kiko christian) Enjoy!

In related news: It’s now happened too often that I had to walk a user through the boot with live cd/find partition/chroot/passwd hoops after they forget the password for their account. So I whipped up a shellscript that one can wget and run from a live cd. It searches for partitions with linux installs on them and allows you to the password for any account on them that has a usable password. So now these hoops are reduced to:

• Boot from a live cd
• Go to applications » accessories » terminal
• Type this: wget http://media.kaarsemaker.net/reset_passwords && bash reset_passwords
• Reboot

xs4all, the best ISP in the Netherlands, now offers free wifi at KPN hotspots to its members. This is excellent, since 3 stations I pass when traincoding now have wifi I can use, so I can use documentation when hacking and not look like an arse because I don’t read docs.

The latest quick hack didn’t work properly at the 2nd station, so I looked up documentation, found my error and now I present: nm_unset - a very simple tool with which you can remove wifi networks for network manager from gconf. I simply got tired of having to use gconftool-2 –recursive-unset /system/networking/wireless/networks/$ssid all the time (and of having to explain how to do this to new users on #ubuntu). I’m really hoping that they soon provide something in n-m for this so I can toss this out again.

Back in 2005, when Ubuntu was still young and #ubuntu small (less than 300 users), I became Ubuntu member and IRC operator. At the latest Ubuntu release the channel peaked at over 1600 and usually has more than 1200 users. It’s been fantastic to see this growth in #ubuntu and related channels.

In the past 3 years it’s been a fun ride from Ubuntu user/channel visitor in 2004, through IRC op, and then representative of Ubuntu on freenode. Earlier this year we have worked on a more formal IRC governance, resulting in an IRC council of which I have been a proud member.

But today that has all come to an end, I have resigned from the council and have placed the responsibility for Ubuntu’s presence on freenode in the caring hands of the other council members. The IRC community is a vital part of the community and is in good shape, it is time for me to do the same for my other big love inside the Ubuntu community: the dutch locoteam.

Founded in 2004, the dutch locoteam it is one of the oldest locoteams around and has seen a steady growth over the past years. We have many forum contributors and a good crew of active people, it is time to turn this enthousiastic group of people into a professional team of Ubuntu supporters.

We’re halfway there already, the active members all have great ideas on how to move forward and lots of progress has been made already. The best part so far of this is the AWESOME release party we had last saturday, we estimate the number of visitors at over 400!

On to 2008, the year of Ubuntu-NL!

Seeing no changes coming in in my rss reader, I realized that I forgot to create a changes feed for Hardy. That negligence has now been corrected

You can find it at the usual place: http://media.ubuntu-nl.org/rss/hardy.xml

The infamous ‘ubotu’ IRC bot is very important to the Ubuntu IRC team and the users of ubuntu-related channels on freenode. Since I administer the bot, it is my task to keep it running and check its status. However, being a lazy git, I’d rather not do too much to accomplish that, hence I let nagios do that for me now. If you are in the same position, find a nagios plugin here, and here are the needed nagios config snippets:

define command{
        command_name    check_nick
        command_line    /usr/lib/nagios/plugins/check_nick $ARG1$ $ARG2$ $ARG3$
}

define service {
        use                             generic-service
        name                            ubotu
        service_description             ubotu
        check_command                   check_nick!irc.freenode.net!ubotu!ubuntu/bot/ubotu
        host                            ubuntu-nl
}

A very important feature in the beta 3 release of falcon (which will happen when I finish the documentation) is automated package building. Yes, falcon is now becoming a full fledged repository manager where you can dput your package to and it will build your package and install it in its database.

Here’s what it looks like when run manually, only one package in the queue:

dennis@blackbird:/data1/src/falcon$ bin/falcon-build-queue
Falcon repository manager 2.0.0~beta3 (C)2005-2007 Dennis Kaarsemaker
*  Building casper_1.87.dsc on Blackbird, Starfreighter
*  Trying to build on Blackbird
*  Executing the build command, logging to buildlog_Seveas-feisty-i386.casper_1.87_BUILDING.txt
*  Downloading ubiquity-casper_1.87_all.deb
*  Downloading casper_1.87_i386.deb
*  Succeeded building casper 1.87 on i386 buildd Blackbird
*  Uploaded casper 1.87 for building on amd64 buildd Starfreighter
*  Build result for casper 1.87
*  ============================
*    amd64 Starfreighter        PENDING
*     i386 Blackbird            OK
*  Run falcon-build-queue to check for finished builds
dennis@blackbird:/data1/src/falcon$ ssh starfreighter ./falcon-build-local-queue
* Building casper_1.87.dsc, log will be written to /pbuilder/result/buildlog_Seveas-feisty-amd64.casper_1.87_BUILDING.txt
dennis@blackbird:/data1/src/falcon$ bin/falcon-build-queue
Falcon repository manager 2.0.0~beta3 (C)2005-2007 Dennis Kaarsemaker
*  Downloading casper_1.87_amd64.deb
*  Succeeded building casper 1.87 on amd64 buildd Starfreighter
*  Build result for casper 1.87
*  ============================
*     i386 Blackbird            OK
*    amd64 Starfreighter        OK

When this is run via cron, it will be fully automatic.

Almost a year after I released the last version (1.5.3) of the falcon repository manager, beta 1 of version 2 is finally here! In the past months I’ve completely rewritten it, so if you need to manage a repository of .deb files, here’s a good application for you

New features in this release:

• Scanning code no longer uses apt-ftparchive
• All internal data is modeled using django, making it very flexible
• The templates for html indices are als django-based
• It is translatable!
• Configuration is now even easier, using an interactive config editor
• Numerous fixes and improvements
• The beginning of a plugin system is created (completion in beta 2)
• Same for an automatic building system (completion in beta 2)
• You can now easily install single source packages and binaries into the archive
• Having all data internally cached makes the code much less fragile
• Support for sha1/sha256 checksums in Packages and Sources files
• Compis with the new python policy, so compatible with edgy/feisty

Of course all the original features are still there:

• Support for multiple releases & components
• GPG signed repository
• Themable HTML indices
• No hassle with incoming if you don’t want to use it (but now you can easily use it!)
• Quick and easy creation of .iso images

Grab a deb at my repo or download the source with bzr from http://blackbird.kaarsemaker.net/code/falcon

Django is of course an excellent framework for building web applications. But since it’s quit modular, it is also possible to use only parts of it in non-web products. Being fed up with the cheetah templating engine (which basically reimplements python, and is no fun to use) and sqlobject (where’s the documentation?), I decided that for falcon I would use django’s excellent object-relational mapper and templating engine.

Since I made that switch, hacking on falcon finally became fun again and the next release (albeit beta) is imminent!

But no matter how nice django is, there are some caveats when you want to use and extend django the way I did.

• You need to call django.conf.settings.configure manually, so I ran into an interesting bootstrapping problem.
• Changing configuration later needs setattr()
• Using your own templatetags without actually having what django calls an application is actually simple, but you do need to know how it works: you need a folder foo/templatetags, where foo is in sys.path and in django’s INSTALLED_APPS, then put your tags in foo/templatetags/bar.py
• Automatically initializing database tables is not a feature. But since getting the creation statements is a feature and access to the database connection is another one, that problem can be solved by looking at the django code, this is how I did it:

from django.core import management
management.disable_termcolors()
all_models = (pocket.Component, pocket.MetaComponent, package.SourcePackage, package.BinaryPackage)
all_sql = []
for model in all_models:
    sql, ref = management._get_sql_model_create(model, all_models)
    all_sql += sql
cursor = connection.cursor()
for s in all_sql:
    cursor.execute(s.replace('CREATE TABLE', 'CREATE TABLE IF NOT EXISTS'))

• Adding a new field type is surprisingly hard. Sqlobject has a PickleColumn, for which data is automatically pickled. I wanted to add this to django, without modifying django itself. After creating the field type (a subclass of django.db.models.Field), I only got errors. Adding a type mapping to what django calls the creation module, those errors were gone. Automatic pickling is also possible, if you override the get_db_prep_save function. But then it broke, since automatic unpickling is not possible, there is no ‘data thawing’ functionality in django. So I ended up letting the constructors of the models that use the PickleField do this.

After those things were solved, the excellent django documentation helped me write the rest of the surrounding code and falcon now is faster and better than ever. Now to fix the remaining bugs and then release it, it’s been too long since I made a release.

Lord Jono Duck Bacon is a liar, I am not the bastard child of James and Scott, and he does fancy ducks.

I pity the fowl…

I’m fixing my moms pc, she’s been infected by a nasty msn worm virus thingie.Fixing it of course means replacing windows with Ubuntu, making the total number of machines running Ubuntu in my apartment (60 m²) seven. Lots of Ubuntu for a small apartment! This is what my work area currently looks like:

Some annoucments about things I run for the Ubuntu community:

• Breezy-changes rss feed has been discontinued
• Gutsy-changes rss feed is now available
• feisty-seveas is available, sadly without falcon since falcon 1.5 needs fixes for feisty and 2.0 isn’t available yet. Working on it. Best thing in feisty-seveas is the latest xmoto version. Xmoto rocks

On Feisty release day, #ubuntu had 1600 people in it, #ubuntu-release-party got to 800 and ubotu (the Ubuntu irc bot) still didn’t break. I’m glad that we’re now back to a slightly saner amount of people though (1340 atm)  Many thanks to all IRC operators for keeping the Ubunu IRC channels a nice place to hang out in!

My fiancee used to not run Ubuntu on her PC. That of course was suboptimal, so after a lot of convincing she switched to Ubuntu a few weeks ago and so far she felt ok with it. “felt ok”… hmm… I should be able to do better…

Enter the new pc: I installed MPD on it and hooked it up to the stereo. Me playing with sonata on my laptops and glurp on the nokia770 made her curious, so I installed sonata on her PC as well and she’s really liking it. Phase 2 completed, preparing for world domination

Sonata really is the killer app for today!

In other news: next week I’ll be in paris for the first time in over 6 years. Let’s hope my french is still not too rusty.

At work there was a spare desktop-size-but-server-hardware machine which no one was using. *was* using indeed, since it’s now in my kitchen, replacing an old an noisy macine that was my home server until now. It’s damn quiet,even the fridge makes more noise! And the specs are good as wel: p4 2.8 GHz HT, 1 GB ram, 2×80GB SATA.

It’s already running ssh, bind, apache, openvpn, postfix, mpd, apt-cacher and some fresh pbuilder and falcon instances. Now to build a load of feisty packages

Update: some instructions were missing.

The client side of an OpenVPN setup is dead-easy now with the n-m in feisty. Simply sudo apt-get install network-manager-openvpn and fill in the forms. The server side of a forward-everything-over-the-VPN setup is slightly more difficult, but this should do the trick:

sudo -i
apt-get install openvpn dnsmasq openssl
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE

Add this last line to /etc/rc.local before exit 0, uncomment the ipv4 forwarding line in /etc/sysctl.conf and grab my openvpn config from kaarsemaker.net (save it as /etc/openvpn/home.conf). This config is pretty standard except for the following

• It expects all traffic to be routed through the VPN, I use this because I mainly use the VPN for safe remote working over unprotected wifi
• It uses a tcp connection instead of udp because the vpn server is behind nat and I want to test it from behind the same nat (yes, the test setup does 3 nat steps before traffic goes out to the internet!)
• It uses a local DNS server on the VPN host (that’s why dnsmasq was being installed earlier)

Read the config and if you use 10.8.0.0/24 in your network, change the IP addresses. Then place it in /etc/openvpn and edit /etc/default/openvpn so the “home” VPN is started automatically.

With the config in place, you only have to generate the PKI for the VPN. Fortunately openvpn makes this all very easy for you with the easy-rsa scripts. First you edit /etc/openvpn/easy-rsa/vars and then you can simply run a few of the easy-rsa scripts:

cp -r /usr/share/doc/openvpn/examples/easy-rsa /etc/openvpn
cd /etc/openvpn/easy-rsa
source ./vars
./clean-all
./build-ca
./build-key-server server
./build-dh
./build-key client1

If you want to support more than one client, repeat the last command with different names. Now copy the keys/certificates to the appropriate places:

cp /etc/openvpn/easy-rsa/keys/{ca.crt,server.crt,server.key,dh1024.pem} /etc/openvpn

Make sure that each client gets its .key and .crt file and the ca.crt file. Now restart openvpn and you’re ready to connect to it!

To connect to your freshly created openvpn server from another machine which uses network-manager, simply rightclick on the n-m icon, select “VPN Connections” and “Configure VPN”. The following screenshots are my setup, which works with the example OpenVPN setup and the freshly generated keys:

And now I can safely use unprotected wifi, with 2 mouseclicks n-m can switch the VPN connection on and off. Yes, OpenVPN and network-manager rock!