Debugging Subversion Repository Hooks in Windows

Posted by Dan on Feb 5, 2008 @ 10:54 AM

I've been working on a post-commit hook for our Subversion install and was running in to a number of issues. The post-commit.bat file would run fine from command line, but I just could get things to work as I expected from SVN. After much debugging and scouring Google for answers, I've found a few tips that will hopefully help you to troubleshoot your own SVN repository hooks.

1) Subversion executes all hook programs with an empty environment

This was the biggest issue I was running in to, because I was expecting the my script to be able to find any programs in my %PATH% statement. That's the main reason my scripts were working fine from command line, but were breaking when executing from my SVN hook.

This is actually covered in the documentation in the Implementing Repository Hooks section:

For security reasons, the Subversion repository executes hook programs with an empty environment—that is, no environment variables are set at all, not even $PATH (or %PATH%, under Windows). Because of this, many administrators are baffled when their hook program runs fine by hand, but doesn't work when run by Subversion. Be sure to explicitly set any necessary environment variables in your hook program and/or use absolute paths to programs.

This was something I overlooked and it cost me many hours of debugging my script.

2) Hooks don't execute from the hooks folder

One of the things I kept trying to do to debug my hook was to write some ECHO statements to a log file in order to see what things were doing. Using the syntax ECHO I am here > debug.log I was expecting the debug.log file to be written to the hooks folder, after all this was the behavior I was seeing when running from the command line.

However, when SVN executed the hook the file would seemingly never get written to disk. This misled me into thinking there was a permission problem on the folder. After changing the permissions to give Everyone full control and things still weren't working, I realized the problem was something else.

I finally thought to search the drive for the debug.log file and sure enough it was on the drive—just not in the directory I expected. The files were getting written to my Apache installation folder by default. This was definitely not the behavior I expected.

If you plan on writing files to your hooks folder, you can use the repository argument that's supplied to your hook: SET DIR=%1/hooks. You can then use this environmental variable when writing to a file: ECHO I am here > %DIR%/debug.log. This will cause the file to be written to your hooks folder instead of context folder that Subversion is running under.

3) Save a log file of your hook's execution output

Because I was really struggling trying to figure out what my hook was actually doing I really wanted to try and capture the output of my hook script to see what was going on. I kept searching for some way to turn on some kind of debug log for SVN, but I ended up finding a nifty tip from an e-mail from Nathan Kidd.

In order to capture your post-commit scripts output to a log file, rename your current post-commit.bat file to post-commit-run.bat.

Now change create the following file as your post-commit.bat file:

call "%~dp0post-commit-run.bat" %* > %1/hooks/post-commit.log 2>&1

Using this little trick all the output from your post-commit-run.bat will be saved to the post-commit.log file. I modified Nathan's original example in order to save the log file to the hooks folder from which it's running (by using the repository path supplied by SVN.)

This makes troubleshooting any problems with your hook program extremely easy to find, because you can see exactly what Subversion is running.

Hopefully these tips will help out some other people. I think with understanding these three tips should make understanding and debugging hook programs much easier. If I would have understood all of this yesterday, it would have only taken me minutes to write my hook script instead of hours.

Categories: JavaScript, SQL, HTML/ColdFusion, Flex/Flash, Java, Source Code

63 Comments

  • Thanks for your information, above all for Tip #3. I definetely solved problems with hook scripts, I wasn't able to run.
  • @Cecilia:

    I'm glad this information was able to help. I know I spent way too much time to get a few simple tasks accomplished. Now that I understand things, it's been much easier to add new hooks.
  • THANK YOU
  • Any idea how to get around an authentication realm issue? My log shows this:

    Authentication realm: <svn://www.domain.net:3690>; a5e31e42-3360-484f-8ddb-a161e99888bf
    svn: Can't open stdin: The handle is invalid.

    Thanks Dan!
  • I figured it out. I had to change the SVN service to logon as a user with correct permissions to write files.
  • daan verbaarschott's Gravatar
    daan verbaarschott
    just what i was looking for! thx
  • Hi Dan,

    Thanks a lot for this post. It was point 1 that was the problem for me and point 3 saved my day!

    Strange though is that when I echoed my %PATH% it did show to path to my svn tools. But I still needed to supply to complete path to get it to work.

    Anyway, it's working now!
  • Unfortunally I see this only after I had done my own troubleshooting :-((
    If I had seen this before I doesn't spend lot of time to figure out why it doesn't work.
    The 'Log-Method' described here is verry nice. I use a similar, but not so elegant way.
    The way shown here is verry generic and easy. (will use it in the future..)

    But now I had another problem:
    I try to use 'stdin' inside a 'pre-revprop-change.bat'...
    My script 'hangs' :-(
    I tink this has something to do how stdin is written too by svn.
    I use:
    set LogFile= %~dp0\..\..\__Log\PreRevpropChange.log
    type con: >>%LogFile%
    or
    copy con: >>%LogFile%

    Because both (DOS) commands expect stdin to be terminated with '^Z' (EOF), my script hangs.
    How to copy stdin to a (log) file when it isn't terminated by EOF ?
  • My script is hanging too :(

    I even tried setting up absolute paths and everything. I added a file-write command before and after the svn update command. After the script finally times out, i open up the log fine and i see the "starting update" text in the file, but I don't see the "finished update".
    My script works fine when i just execute the batch file, I don't get it? I've been working on this for hours and I'm stuck.

    heres the contents of my post-commit.bat:


    ECHO starting update %DATE% %TIME% >>C:\subversionrep\hooks\updatelog.txt
    C:\subversion\bin\svn update C:\sites\elac
    ECHO finshed update %DATE% %TIME% >>C:\subversionrep\hooks\updatelog.txt
  • @Elac:

    Do exactly what my Step 3 does to log the output--don't try rolling your own. The code I provided will dump out everything that appears in the console, so you'll be able to see why your update statement isn't working like you think it should.
  • My log change using TSVN still hangs. And when it hangs the log file does not get written.

    I have followed Step 3 and when I run the above test it hangs with no log written.
    So I run the batch file manually on windows I get a log that looks like this.

    C:\svn_repository\hooks>REPOS="$1"
    'REPOS' is not recognized as an internal or external command,
    operable program or batch file.

    C:\svn_repository\hooks>REV="$2"
    'REV' is not recognized as an internal or external command,
    operable program or batch file.

    C:\svn_repository\hooks>USER="$3"

    C:\svn_repository\hooks>PROPNAME="$4"
    'PROPNAME' is not recognized as an internal or external command,
    operable program or batch file.

    C:\svn_repository\hooks>ACTION="$5"
    'ACTION' is not recognized as an internal or external command,
    operable program or batch file.
    "$ACTION" was unexpected at this time.
    C:\svn_repository\hooks>if [ "$ACTION" = "M" -a "$PROPNAME" = "svn:log" ]; then exit 0; fi

    I am stuck and need a Super Hero. Anyone out there can can suggest some steps here.
  • @ViperMaul:

    Look here for a working post-commit.bat file:
    http://blog.pengoworks.com/index.cfm/2008/2/5/SVN-...

    It looks like your syntax is off. The lines like "REPOS=$1" should be "SET REPOS=$1"
  • Hi Dan, saved my life this script it did :)
    Got a question though. I need to exclude sending mail for a commit that has been done on the root of the repository but rather on its branch. Can it be done through this batch script?? So I would need to pick only commits on the branch and send an notification email.

    Thanks
  • @Mario:

    See this blog entry I wrote a while back on selectively running commands in a post-commit depending on which folder was executed:

    http://blog.pengoworks.com/index.cfm/2008/5/15/Sel...
  • Hi again Dan. Eternally greatfull :) This did what I needed. Had to modify it a bit because filtering folder_path did not give a verbose description of svn log in my mail but just the path of the commit. I just added another svn log line under "if error" line.

    Thank you again Dan. You have been a big help.
  • @Mario:

    Glad this was able to help you out!
  • Hi,
    Thanks for all your help dan I figured out the issue I was not running apache and svn as a the same user that owned all of the files.
  • Hi Dan,

    Thanks for your most helpful insights.

    I try to do a post-commit svnsync using a svn+ssh url.
    I think the empty environment might prevent Putty from working and tip #3 does not return any log.

    Would you have any idea on that?

    Thanks!

    Manu
  • @Manu:

    If you're not seeing output, it's probably because there's nothing in the stdout stream. Try simplifying things and see if you can just echo some output from a batch file.
  • Daniel Fredericks's Gravatar
    Daniel Fredericks
    Dan,
    can you help me a bit here: this below is what i get in my log file when it runs:

    Authentication realm: <http://serverName:8080>; Subversion Repositories
    Password for 'SubVersion':

    The hangs up my commit and copy? I see things above that may help me, but I am not sure...Apache and SVN need to use the same username/passoword, yes/no?
    I am using a system account with SVN that has read/write, so that should not be the issue.

    as you notice I am not using https, we were having way to many problems with https setup, so we figured if we used just http, it would work.

    Can you help me please????

    thanks so much
    Dan
  • Don't call the working copy from the URL, but use the local file path instead. There's no reason to try accessing the repository over HTTP when you can access it via the file system.
  • Daniel Fredericks's Gravatar
    Daniel Fredericks
    Is this what you are talking about above, I am not exactly sure where you say not to call working copy via http (this is part of the post-commit-run.bat file):

    REM * define the path to the working copy of your code     *
    REM *                              *
    REM * the default working_copy folder would be:         *
    REM * file:///%REPOS%                      *
    REM *************************************************************
    SET WORKING_COPY=e:\cf8dev1\Test2\

    REM *************************************************************
    REM * update the local working copy               *
    REM * this will sync the new changes to your servers copy    *
    REM *************************************************************
    svn update %WORKING_COPY%


    I am not sure exactly what you mean by don't use url for working copy...i'm sorry, and appreciate any help you can give me.

    dan
  • @Dan F:

    The use of http/https should not affect a post-hook at all. Since you are access svn locally, you may need to add your username/password information to the SVN command line. Here's the docs on using svn update:

    http://svnbook.red-bean.com/en/1.0/re28.html

    So you'll want to do something like:
    svn update %WORKING_COPY% --username USER --password PASS
  • Daniel Fredericks's Gravatar
    Daniel Fredericks
    Hey,
    thanks so much for walking me through this...after you have me the code above, it seemed to work. I am going to do more testing, but the first 2 tests worked.

    Dan
  • THANK YOU!

    I'd spent the better part of two hours wondering desperately what was wrong with my svn installation not calling hooks. Thank you so much for the post_commit_run.bat logging idea.
  • Thanks Dan for making this available.
    However to get it to work for me I had to add quotes to cope with spaces in the path, i.e.:
    call "%~dp0pre-revprop-change-run.bat" %* > %1/hooks/pre-revprop-change.log 2>&1
  • @David:

    That's a good modification--I'll update the post. Thanks!
  • Thanks a lot, it helped me find out my pre-commit hook was crashing because of a *double quote* in the comment...
  • Thank you for this. This blog post saved my rear tonight, and will get me closer to SVN commit messages updating issues @ RIAForge.
  • I used your tip 3 to debug a pre-commit hook, but it seems that hook is not launched. I testest the .bat file locally and it works but when I used a real commit command, nothing happens, no log file...Could you help me? Any ideas?
    Thanks.
    Sylvia.
  • I have the same problem as Sylvia above. I can run the batch script from the command line and recieve an email, but it does not work on real commits. Might you be able to provide any leads? Your tip 3 was helpful in getting this to work, but there are no more errors in thd standard output. BTW - I'm using the URL as a working directory. I couldn't get the direct path to work i.e. C:\path\to\repo did not work. I am using https://server/path/to/repo/trunk. Any assitance would be greatly appreciated.

    Many thanks,
    Eric
  • @Eric:

    I wonder if it's the use of SSL. Is your SSL cert a commercial cert or a self-signed cert? I really don't have any other helpful tips. Any problems I've ever had I've documented here and this isn't something I work with every day. I haven't had to touch my hooks in a long time.
  • Thanks a million! You've just saved a lot of my time =)
  • I am new to Subversion, so please pardon my ignorance. I have Visual SVN running on Apache on a Windows Server. I have a development server on a different box. I am trying to copy the modified files when a developer commits to SVN from their respective machines. So, I am using the post-commit.bat to implement this. By the way, I am running the Visual SVN as "local system". So, when I run the following script, it hangs. I tried writing it to a log file, which displayed a line where it seems to be waiting for me to put a password. I have posted this on various other forums, but have yet to get a response. Can somebody here please help me?

    @ECHO OFF & setLocal EnableDelayedExpansion
    SET REPOS=%1
    SET REV=%2

    SET repoURL=http://subversion.bkvi.com/svn/BKV
    SET webSiteRoot=\\10.0.1.162\websites
    SET logDir=%REPOS%/hooks
    SET PATH=%PATH%
    svnlook changed %1

    for /F "eol=¬ delims=¬" %%A in ('svnlook dirs-changed %1 -r %2') do svn export "%repoURL%/%%A" "c:/svn_exports/%%A" --force

    SET SMTP_SERVER="mailserver.bkv.com"

    VisualSVNServerHooks.exe ^
      commit-notification %REPOS% -r %REV% --no-diffs ^
      --from "sharma.vedula@bkv.com" --to "sharma.vedula@bkv.com" ^
      --smtp-server %SMTP_SERVER%
  • @Sharma:

    If you're using a URL to the repository, you'll most likely need to supply your credentials to the SVN command line tool. However, instead of using the remote URL, change the repoURL to you local file system. That should bypass the need to use credentials and will be much faster.
  • Thanks Dan for the prompt response. But when you say, "..change the repoURL to your local file system", do you mean change it to local file system on the Subversion server? If so, where do the files reside? I thought they were stored in the Subversion DB. Sorry if this is a stupid question. Just to clarify, the post-commit.bat is triggered when a developer commits his/her changes to SVN.

    Thanks

    Sharma
  • @Sharma:

    You need to change the repoURL to point to the location on the local file system (which is the SVN server's file system) to where the repository db leaves--this is the local repository. Commands will work against the local file system just as they would if you were executing them against the URL, but you remove a lot of overhead and you shouldn't need to pass credentials to execute commands against the local repository.
  • Sorry, but I am still not very clear as to what this path would be. Is there a place on the Visual SVN config that would tell me the local file system where these files would be stored on the SVN server?
  • @Sharma:

    It would be the root directory of where your hooks folder is.
  • Okay, I tried that. You are right, it did not prompt me for my password. But here is what I got in my log file:

    U  nxtorders.com/trunk/ROOT/default.jsp
    svn: 'S:\Repositories\BKV\nxtorders.com\trunk' is not a working copy
    svn: 'S:\Repositories\BKV\nxtorders.com\trunk' does not exist
  • @Sharma:

    Make sure you're paying attention to the docs:

    http://svnbook.red-bean.com/en/1.5/svn.ref.svn.c.e...

    The syntax for the local system would be:

    svn export file:///var/svn/repos my-export

    So, the syntax you're looking for would look like:

    svn export file:///S:/Repositories/BKV/nxtorders.com/trunk

    (Provided the /trunk folder is actually the parent folder for you hooks, db, etc folders.)

    I would recommend testing your command directly from a command prompt to make sure you're getting the intended results from each command--and then working them into your hook.
  • Thanks Dan for all your help. That seems to be working great. Now, I just have to modify my script to export to a different server where we have our DEV and QA running. But thanks for your prompt responses. I do appreciate it.

    Sharma
  • Hi Dan, is there a way for me to get the project from the svnlook dirs-changed command? I am trying to modify my post-commit.bat to copy certain folders from SVN to our DEV server. But for that, I need to know the project name, each time a developer commits his/her changes to SVN, which results in the post-commit.bat getting triggered. I tried using the SED command to strip the project name, but did not succeed.

    Thanks

    Sharma
  • @Sharma:

    What do you mean by "get the project"?

    Do you really need to know the project, or is knowing the directory good enough? I would assume you can figure out the "project" by the folder.

    What I've done in the past when I wanted to update something when the content of a specific folder was updated, was:

    SET WATCH_FOLDER="/path/to/watch"

    svn log %LOCAL_REPOS% -v -r %REV% | find /i %WATCH_FOLDER% > nul
    if errorlevel 1 goto end
        svn update C:\some\local\workspace

    :end

    What this is doing is just looking through the log to see if a string path to the folder exists and if it does, then it's doing some operation.
  • Knowing the directory is good enough. Here is what I am trying to do in my post-commit.bat. I want to copy only certain folders under the project folder to my DEV server when either the project is imported or any changes are committed. And I have many projects in the same repository. So, as the post-commit.bat is triggered when I import a new project into the SVN repository, it currently copies every folder and file. For me to copy certain folders to my DEV server, I need to know the project folder, so that I can define the correct path in m svn export. Hope that makes sense.

    Thanks

    Sharma
  • We have found that SVN swallows STDERR and STDOUT. Our current process in CVS provides feedback to the end user via the hook scripts during commits and tagging to let the user know the status during the transaction as well as afterward. SVN, does now allow this. It redirects both STDOUT and STDERR, and only in case of hook or other transaction failures, does it displays STDERR.

    is it possible to display STDOUT on user screen ?
  • @Madhusudhan:

    I'm honestly not sure. I suggest you direct your question to one of the SVN mailing lists.
  • Paul Brookfield's Gravatar
    Paul Brookfield
    Cheers for this blog - in a roundabout sort of way it helped me figure out what was going south with my post-commit hook on Linux. Strange, but true.
  • Great post! Although I'm running a Subversion repository on Linux, this post helped me a lot (I've just spent the whole day trying to figure it out, and didn't knew that Subversion was running in an empty environment). Thank you!
  • Thanks for the summary! That environment variables are not working is the main point. Therefore, the execution of my Java program did not work in my post-commit hook. Now I know why! Thank you.
  • I am running win 2008 server and looks like all env. vars are being set just fine

    windows weirdness?
  • My start-commit.bat is not properly working.My intention is to copy a file from a certain folder to my working copy. No error was generated from the log.
  • OK, the batch file is working and I receive the mail. The problem is that I receive the following error when invoking svn log:
    Error validating server certificate for 'https://10.10.10.11:443" target="_blank">https://10.10.10.11:443':
    - The certificate is not issued by a trusted authority. Use the
      fingerprint to validate the certificate manually!
    - The certificate hostname does not match.
    Certificate information:
    - Hostname: svnedge.collab.net
    - Valid: from Fri, 27 Aug 2010 12:05:48 GMT until Mon, 24 Aug 2020 12:05:48 GMT
    - Issuer: CollabNet Subversion Edge, CollabNet, Inc., Brisbane, CA, US
    - Fingerprint: 9d:63:2e:1b:53:31:00:ba:c9:ef:f6:45:92:7a:c2:37:d1:de:b4:2c
    (R)eject, accept (t)emporarily or accept (p)ermanently? svn: OPTIONS of 'https://10.10.10.11/svn/cardo_all': Server certificate verification failed: certificate issued for a different hostname, issuer is not trusted (https://10.10.10.11)


    When I do svn log from my PC (client) the error message appears only once and then I choose permanently. And there is no more error message.
    Same on the server. After choosing permanently the first time there is no problem.

    Any idea ?
    Thanks.
  • @Michel:

    Make sure to use the %1 environment variable that's passed to the hook script. This should be the local path to the repository--which avoids any HTTP issues and SSL cert issues.
  • I don't have a working copy on the server, so I invoke svn log with the URL.
    I cannot use %1 for svn log
  • I tried to keep a working copy on the server and to invoke the svn log with the working copy path (not the URL) but I have the same problem
  • @Michel:

    You shouldn't need a working copy. You should be able to just point to the repository location on your local hard drive--instead of trying to access the HTTP URL. That should work the same.
  • the command
    svn log %REPOS% -r %REV%
    returns the error
    svn: '%REPOS%l' is not a working copy
  • @Michel:

    Did you follow step 3 in the direction above ("Save a log file of your hook's execution output")?

    That should allow you to view the actual svn log statement that's being run.

    If so, what's actually getting executed?

    Is %REPOS% defined? Does %REPOS% need to be wrapped in quotes because the path has spaces in it?
  • I did the step 3 and the %REPOS% (actually this is %1) is correct, without spaces.
    I don't understand when you say "You should be able to just point to the repository location on your local hard drive"
    svn log needs url or working copy and %REPOS% is not a working copy.
  • @Michel:

    My mistake, you need to add "file:///" before the path to the local repository. This should work:

    svn log "file:///%REPOS%" -r %REV%
  • I got the following error:
    svn: Unable to open an ra_local session to URL
    svn: Unable to open repository 'file:///C:%5Ccsvn%5Cdata%5Crepositories%5Ccardo_all'

    I found another way. I send the content of the file %REPOS%\db\revprops\%REV_FIRST_DIGIT%\%REV%

    this is not a good way but we get the info
  • Appreciating the commitment you put into your site and
    detailed information you offer. It's awesome to come across a blog
    every once in a while that isn't the same out of date rehashed
    information. Fantastic read! I've bookmarked your site and I'm
    including your RSS feeds to my Google account.

Comments for this entry have been disabled.