Managing vhosts on Ubuntu with puppet
On the Ubuntu NL servers, we use puppet for centralized configuration management. Some would say it is overkill fpr only a few hosts but keeping systems consistent makes life easier. Here’s how we integrated the debian-style vhost management with puppet:
There’s a class called apache in modules/apache/init.pp, it defines which packages are needed for an apache setup and which vhosts run where:
class apache {
$packages = [ "libapache2-mod-php5", "libapache2-mod-wsgi", "php5-gd", "php5-mysql", "php5-ldap", "python-ldap", "python-mysqldb","libnet-ldap-perl"]
$vhosts = $hostname ? {
caterpillar => ["intranet","proxy","archive","default"],
bumblebee => ["mail","lists","otrs","people","feeds","default"],
dragonfly => ["blogs","chat","forum","passwd","planet","wiki","www","default","kaart"]
}
Also make sure all packages are installed, and some general files as well:
package{$packages:
ensure => installed,
notify => Service["apache2"],
}
file{”/etc/logrotate.d/apache2″:
owner => “root”,
group => “root”,
mode => “0444″,
source => “puppet://$servername/apache/apache2.logrotate”,
}
file{”/etc/apache2/force-https.include”:
owner => “root”,
group => “root”,
mode => “0444″,
source => “puppet://$servername/apache/force-https.include”,
notify => Service["apache2"],
}
file{”/var/www/index.php”:
owner => “root”,
group => “root”,
mode => “0444″,
ensure => “/srv/bzr/503/index.php”,
}
file{”/var/www/index.html”:
ensure => absent,
}
The logrotate config is because each vhost logs into its own directory. All these logs need to be rotated too of course :)
Speaking of logs, here’s more magic for them:
dir{$vhosts:
dir => "/var/log/apache2",
}
define dir($dir){
file{"$dir/$name":
ensure => directory,
owner => root,
group => adm,
before => Service["apache2"],
}
}
Just making sure that all logdirs get created before apache starts. The real fun comes next though, with the vhosts. This handles installing configfiles, doing the equivalent to a2enmod en also handles SSL vhosts (the ssl key/cert is handled by a different module).
First there’s a small separate script that autogenerates ssl vhosts (we use a wildcard certificate, so we can run several ssl vhosts on one address. We also use mod_wsgi for MoinMoin, so we need some mangling there as well
#!/usr/bin/python
import os, re
root = '/etc/puppet/modules/apache/files/vhosts'
ssl_snippet = """
SSLEngine on
SSLCertificateFile /etc/ssl/certs/ubuntu-nl.org.crt
SSLCertificateKeyFile /etc/ssl/private/ubuntu-nl.org.key"""
for f in os.listdir(root):
if f.endswith('-ssl'):
continue
conf = open(os.path.join(root, f)).read()
conf = conf.replace('*:80','*:443')
conf = conf.replace('\n<VirtualHost>',ssl_snippet + '\n<VirtualHost>')
conf = re.sub(r'(WSGI(?:DaemonProcess|ProcessGroup)\s+\S+)', r'\1-ssl', conf)
fd = open(os.path.join(root, f + '-ssl'),'w')
fd.write(conf)
fd.close()
So after creating a vhost in the puppet root, I run this script to autogenerate the https vhost. DRY. When I want to force a specific vhost to use SSl, it includes the force-https.include, which contains this:
RewriteEngine on
RewriteCond %{HTTPS} off
RewriteRule /?(.*) https://%{HTTP_HOST}/$1
So we can still autogenerate the https vhost based on the http vhost :). Here’s the last snippet for puppet to finish our class apache.
vhost{$vhosts: }
define vhost(){
$linkname = $name ? {
"default" => "000-default",
default => $name
}
file{"/etc/apache2/sites-available/$name":
owner => root,
group => root,
mode => 0444,
source => "puppet://$servername/apache/vhosts/$name",
notify => Service["apache2"],
before => File["/etc/apache2/sites-enabled/$linkname"],
}
file{”/etc/apache2/sites-enabled/$linkname”:
owner => root,
group => root,
mode => 0444,
ensure => “/etc/apache2/sites-available/$name”,
notify => Service["apache2"],
}
file{”/etc/apache2/sites-available/$name-ssl”:
owner => root,
group => root,
mode => 0444,
source => “puppet://$servername/apache/vhosts/$name-ssl”,
notify => Service["apache2"],
before => File["/etc/apache2/sites-enabled/$linkname-ssl"],
}
file{”/etc/apache2/sites-enabled/$linkname-ssl”:
owner => root,
group => root,
mode => 0444,
ensure => “/etc/apache2/sites-available/$name-ssl”,
notify => Service["apache2"],
}
}
}
So we install a vhost config for each vhost, both http and https. Then we link them similar to what a2ensite does, including prepending 000- to the default vhost. This default vhost is pretty special too, to take a vhost down for maintenance, I simply stop puppetd on the relevant webserver and a2dissite the vhost. Requests for that vhost will then be caught by the default one, which has a RewriteRule for all nonexisting files to index.php (so all urls are caught by it). The index.php will actually emit an ‘HTTP/1.1 503 Temporarily Unavailable’ response so errorpages are not caught by indexers.After the maintenance, puppet is started again and the first thing it does is reenabling that vhost.
Great, great post. Appreciate any and all stuff about Puppet. Trying to use it myself, and hoping to overcome the initial learning curve.
beautiful post i am gonna try this one thanx………..alot
Thanks for the tips, I’ll be cribbing heavily from this while doing our LAMP stack configs.
Just curious : is there any reason you’re not using Puppets templating? Seems like a powerful way to have one config file and turn on bits conditionally depending whether you need SSL or not.
Is it just you’d rather use Python than (E)Ruby?
I use puppet templating for other bits, but not here as many vhosts have special per-vhosts things, making it more convoluted to template than it is to write manually.
Here’s one example: network config
auto lo
iface lo inet loopback
<%
i = 0
eth0_addresses.each do |address|
-%>
auto eth0<% if i != 0 %>:<%=i-%><% end %>
iface eth0<% if i != 0 %>:<%=i-%><% end %> inet static
address <%= address %>
netmask 255.255.255.240
gateway 82.94.254.97
<% i+=1; end -%>
<%
i = 0
eth1_addresses.each do |address|
-%>
auto eth1<% if i != 0 %>:<%=i-%><% end %>
iface eth1<% if i != 0 %>:<%=i-%><% end %> inet static
address <%= address %>
netmask 255.255.255.0
<% if i == 0 && address != “10.42.4.1″ -%>
up route add -net 10.42.0.0/16 gw 10.42.4.1
<% end -%>
<% i+=1; end -%>
Thanks Dennis.
Having a bit more experience now, seems to me definitions are going to be more powerful than
templates for vhosts. Thanks for the head start.