Subversion/Drupal distributed multisite project
(please excuse the note form of this article)
Background
I am involved in a number of drupal projects, both new and legacy, some as paid projects, some volunteer.
I am involved or want to help with the maintainence of the code base (security patching, upgrades etc). However not all of the projects are on my own server. And as I am not a hosting company that is how things should be.
However, patching and upgrading is a pain, especially over many sites. How could this be managed efficiently and cooperatively?
The Solution in general
Make a designated server for subversion. Create a "master" drupal instance that is not actually used to publish anything, and from this create a subversion repository that will house the drupal "core" codebase, AND a seperate but related repository that can house the files specific to each drupal instance. Install each drupal instance that is to be maintained in this way via svn checkout on their own server, wherever it is. Lets call these drupal instances client instances.
When patches and upgrades come out, do these on the master instance and commit the changes to the repository to create a new revision. To upgrade each client instance simply run svn update on each server, possibly followed by running the drupal update.php script.
The Solution in more detail (NB. this is not a how-to yet ;)
We need to adhere to the conventions of drupal multisite installations to make this work nicely, even though the sites are most probably on different servers so this is not actually a multisite installation.
As we need to combine two code projects (the drupal core files and the files specific to each client instance) to create each instance, we make use of the svn:externals property. The master drupal files will be "external" to the domain specific files.
Basically, the drupal core repository will store all of the files, including the core modules EXCEPT anything under the sites/directory.
The sites/ directory is EXCLUDED from the drupal core repository, except for sites/all/modules, where we can keep modules that are NOT core but are used in pretty much every client.
To complete the picture we create seperate svn projects for each client instance. these projects will house the following files and directories:
sites/www.thisparticularone.com/settings.php
sites/www.thisparticularone.com/files
sites/www.thisparticularone.com/modules
sites/www.thisparticularone.com/themes
sites/www.thisparticularone.com/tmp
(where www.thisparticularone.com is the base url of each instance - this adheres to the convention of drupal multisite installations and drupal will look here for the files)
When we install a new drupal we do it like this:
(NB these commands currently untested as i've changed things a bit since last version)
#install drupal "core"
svn checkout https://svn.southspace.org/subversion/drupal/thisparticularone/trunk/ .
#add the externals property
svn propedit svn:externals .
(which will launch an editor)
#add a line like this:
. https://svn.southspace.org/subversion/drupal/master/trunk/
#get the new items from the external project, ie drupal mater
svn update
(to update your drupal including core ie. master project files run)
svn update
an ordinary
svn commit
will NOT commit to the external files, ie. the drupal "master". this is what we want ;)
though should committ files in /sites/www.thisparticularone.com
this is the behavour we want.
What i've done already
Installed a xen virtual machine (debian etch) on mediaartprojects.org.uk specificaly for subversion and master instances
Made the repositories available via https
Apache access control only so far
Installed webSVN for viewing repositories (https)
Created master drupal repro, and some "test" projects
Done lots of tests with checking out, setting externals property updating etc using master and projects
refs:
Subversion Manual
http://svnbook.red-bean.com/
http://svnbook.red-bean.com/en/1.1/ch07s04.html
http://drupal.org/node/53705
http://agaricdesign.com/note/agaric-wants-version-control-lets-drupal-co...
i'm not the only one working on this, by the way ;)
Comments
subversion, git, and... and...
Thanks jaromil,
thats really interesting and the repro is impressive.
and to add more... heres a bzr vs. git article..
http://bazaar-vcs.org/BzrVsGit
from subversion to git
Hi Lisa,
I was using CVS back in 2000, had switched to Subversion (SVN) in 2003 and one week ago i switched again all my code repository to GIT.
After all that, the good thing is to have intact the *whole history* of software projects: no revision history should be lost between the conversion. And so it is in my case: my GIT repository holds intact information that was submitted 8 years ago using CVS. Considered the typical incidents occurring in similar situations with closed source software and compatibility wars between competitors, I find this result amazing; what holds true for all those systems, CVS SVN and GIT, is that they are open source and that people can develop them to be compatible with each other.
About the difference between GIT and SVN you are absolutely right, the main one being that GIT is *distributed* where SVN is *centralized*. Seen from the point of view of an open source programmer, decentralization of repositories is a valuable feature.
Another functional difference i see between the two systems is the quality of tools to operate them: probably because of its early adoption in a mission critical environment as it is the Linux kernel development, GIT became more pleasant to use than all its ancestors by providing an amazing set of interfaces to handle local and remote repositories and, last but not least, visualizing amazing graphs which we all love :) see the 'gitk' utility for instance.
After a week of switching to the new system and coding on it my opinion about GIT is more than positive: branching and merging is finally made *really* possible and practical, which is a vital feature for a project with more than one full-time active coder.
Anyway, maybe i should wait a bit more before going on with this apology of GIT, since i'm just too enthusiastic about it right now :) will just point you http://git.dyne.org which is the web-interface to our new repository, where recently especially thank to GIT more people can get involved continuing a development that, as you can see from the history, goes on now for long...
ciao
subversion and git
i have been looking at git to see if it has advantages to use in this case.
so far, i haven't concluded anything ;) . the answer is still "maybe".
the main difference between git and subversion is that git is distributed and subversion is client-server. on first thoughts, it seems that client-server is definately what we want.. but maybe not. perhaps there are advantages to each "client" drupal checking out and holding their own entire repository? i don't know yet.
thoughts from others welcome.