Mirroring GNOME SVN to Git

Here lie some of the scripts i'm using to mirror svn.gnome.org to git-mirror.gnome.org. Thanks to James Sharpe for the know how, i'm just scripting it and running on the whole of GNOME SVN.

Problems

Git currently breaks on the following:

accerciser
fast-user-switch-applet

Gtk+ seems to work if i use my local rsync of Gtk svn, but chokes on http:// (This is a know bug in git-svn due to there being a + character in the url - it is fixed in git head and will apparently appear in releases >= 1.6; James Sharpe)

Getting git-svn to work again after cloning any of the GNOME Git repositories is proving painful.

Using a mirror

git clone git://gnomegit.unrouted.co.uk/git/cheese
cd cheese/
git svn init --stdlayout svn+ssh://svn.gnome.org/svn/cheese --prefix origin/svn/
git svn rebase

git-create.sh

This takes the name of the module and clones it to /srv/git/$MODULENAME.

echo $1

# Make a directory for our mirror
mkdir -p /srv/git/$1
cd /srv/git/$1

# Create a new repo and pull in the GNOME module
git --bare init
git --bare svn init --stdlayout --rewrite-root svn+ssh://svn.gnome.org/svn/$1 file:///srv/svn/$1
git --bare svn fetch

# The packing isn't very agressive right now, so repack it like hell
# (I'm told this is wrong, too, and i can pack far more aggressively. Maybe when I can clone)
git --bare gc --prune --aggressive

# Point at svn.gnome.org - we'll pull from there in future
git --bare config svn-remote.svn.url http://svn.gnome.org/svn/$1

post-update

This makes the tags and branches make sense, and runs the command so that cloning over HTTP works. From Tim Janik, via James Sharpe. Not using it until i've got the "easy" stuff working.

#
# An example hook script to prepare a packed repository for use over
# dumb transports.
#
# To enable this hook, make this file executable by "chmod +x post-update".
#!/bin/bash
# Update .git/refs/{heads|tags}/svn/* from refs/remotes/*

SET_ECHO="set -x" # enable echo
SET_ECHO="set +x" # disable echo

# update all branches and tags
git for-each-ref "--format=%(refname)" refs/remotes |
  sed 's,/\([^/]*\)$, \1,' | while read trunk ref ; do
        case "$trunk" in
        */tags) ($SET_ECHO ; git update-ref refs/tags/svn/$ref  $trunk/$ref ) ;;
        *)      ($SET_ECHO ; git update-ref refs/heads/svn/$ref $trunk/$ref ) ;;
        esac
done

# delete stale branches
git for-each-ref "--format=%(refname)" refs/heads/svn |
  sed 's,refs/heads/svn/,,' | while read ref ; do
        git rev-parse "refs/remotes/$ref" >/dev/null 2>&1 ||
                ($SET_ECHO ; git update-ref -d "refs/heads/svn/$ref" "refs/heads/svn/$ref" )
done

# delete stale tags
git for-each-ref "--format=%(refname)" refs/tags/svn |
  sed 's,refs/tags/svn/,,' | while read ref ; do
        git rev-parse "refs/remotes/tags/$ref" >/dev/null 2>&1 ||
                ($SET_ECHO ; git update-ref -d "refs/tags/svn/$ref" "refs/tags/svn/$ref" )
done

exec git-update-server-info

Keeping it up to date

Every time a mail is fired to svn-commits-list, we look at the module name and do equivalent to:

cd /srv/git/$MODULE
git --bare svn fetch

Patch

From a581675dc67394679b7e51179e941946729a4db2 Mon Sep 17 00:00:00 2001
From: =?utf-8?q?Jan=20Kr=C3=BCger?= <jk@jk.gs>
Date: Thu, 19 Jun 2008 23:24:32 +0200
Subject: [PATCH] git-svn: make rev_map rebuilder respect rewriteRoot option
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

Suppose someone fetches git-svn-ified commits from another repo and then
attempts to initialize git-svn with rewriteRoot. Using git svn rebase after
that will attempt (but fail to) rebuild the rev_map for every single commit.
This can take a long time without yielding any useful result.

By using Git::SVN->metadata_url rather than ...->full_url, the rebuild works
even if rewriteRoot is enabled.

Signed-off-by: Jan Krüger <jk@jk.gs>
---
 git-svn.perl |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index a54979d..4c9c59b 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -2577,8 +2577,8 @@ sub rebuild {
        my ($log, $ctx) =
            command_output_pipe(qw/rev-list --pretty=raw --no-color --reverse/,
                                $self->refname, '--');
-       my $full_url = $self->full_url;
-       remove_username($full_url);
+       my $metadata_url = $self->metadata_url;
+       remove_username($metadata_url);
        my $svn_uuid = $self->ra_uuid;
        my $c;
        while (<$log>) {
@@ -2596,7 +2596,7 @@ sub rebuild {
                # if we merged or otherwise started elsewhere, this is
                # how we break out of it
                if (($uuid ne $svn_uuid) ||
-                   ($full_url && $url && ($url ne $full_url))) {
+                   ($metadata_url && $url && ($url ne $metadata_url))) {
                        next;
                }
 
-- 
1.5.5.1