Version Control

From VisItusers.org

Jump to: navigation, search

Contents

[edit] Why Subversion

Subversion was selected as the repository system because of its popularity among open source projects; it is a known quantity and the broader community's general knowledge of the product will reduce barriers. Further, it has the requisite feature set: distributed development, branching, etc, that are required for a project like VisIt.

[edit] How VisIt uses Subversion

At some level, VisIt uses Subversion just like any other project. However, we chose to augment Subversion with a set of scripts. The purpose of these scripts is three-fold:

  1. Reduce the amount of Subversion commands a VisIt developer needs to know
  2. Ensure that developers use the Subversion features consistently (e.g. branches located in the same place, etc)
  3. Manage the merge process, which can be cumbersome.

Here's some information about more pages that describe how to use Subversion that you may find useful:

  • Doing VisIt development requires using a mix of scripts and Subversion commands. This page describes which scripts and which commands to use.
  • The VisIt repository is laid out like many other Subversion repositories, with trunk, branches, and tags directories. This page describes how the repository is laid out in detail.
  • Our project approves of development both directly on the trunk and also with using branches. To decide which you want to do, see here.
  • Many developers have made small customizations to their Subversion environment. See Customizing SVN for more information.

[edit] Repository location

VisIt's source is controlled by a subversion server, edison.nersc.gov, currently residing at NERSC. The path is svn+ssh://${SVN_NERSC_NAME}@edison.nersc.gov/project/projectdirs/visit/svn/visit. A sponsoring developer will help you obtain access to the repository -- see GettingSVNAccess more information.

There are also scripts to make access easier. You can learn more about the scripts here. These scripts live in the src/svn_bin directory. To retrieve them into a common location, like ~/svn_bin where you can add them to your path permanently, cd into your home directory and type:

svn co svn+ssh://USERNAME@edison.nersc.gov/project/projectdirs/visit/svn/visit/trunk/src/svn_bin

You might want to periodically cd into ~/svn_bin and type svn update.

[edit] Examples of Subversion development

[edit] Warning about commits

Committing your changes goes beyond issuing an svn commit. There are two separate pages that describe the actions to be performed when committing changes. See here and here for more information. (These two pages should be merged, but I'm not going to do it now.)

[edit] TEMPORARY: Warning about merges from rctrunk to trunk

We have encountered a problem with merges from rctrunk to trunk (merge_rctrunk_to_trunk) script. Until the problem is corrected, if you plan to run the merge_rctrunk_to_trunk script, be sure your rctrunk checkout is an all mode checkout. If you wind up doing a merge from an rctrunk that was NOT an all mode checkout, then the merge_rctrunk_to_trunk script will inadvertently record the fact of a merge for the whole trunk, fooling others who attempt a merge after you but on a part of the trunk different from your rctrunk checkout, into thinking that their part of the trunk got merged.

So, if you want to checkout the rctrunk without using the all mode, that is fine. You can even commit your changes to the rctrunk from your working copy. But, let someone else who has an all mode rctrunk checkout do the actual merge to the main trunk for you.

Leave this warning text in until the scripts are corrected.

[edit] Online Subversion documentation

The online documentation for Subversion is superb. It contains information ranging from basic repository concepts to syntax for each and every Subversion command. As you get going, you will probably want to take a look. You can find it here.

[edit] Hooks

The VisIt project makes use of several subversion hooks. Detailed information on how hooks work is available in the subversion red book. We utilize both pre-commit and post-commit hooks.

In addition to the standard uses of commit hooks, the VisIt project takes advantage of some undocumented but useful behavior of recent subversion installations. Notably, though it is explicitly documented that output and error code information from a post-commit hook are ignored, recent releases (1.4.2, perhaps others) of the official subversion client, with our subversion server, appear to receive post-commit's error output. This behavior is capitalized on to check for and produce warnings which we feel are 'red flags' but not sufficient criterion for aborting a commit outright.

The issue did receive some previous discussion/inquiry. The subversion community recommends emails for these such warnings. The VisIt team did not like this approach, and decided that warnings were a non-critical feature anyway. Therefore it was acceptable if not all clients received the warning messages, and so relying on the undocumented behavior was agreed upon.

[edit] Adding a Pre-Commit Hook

To add a new pre-commit hook to VisIt's repo, you need to do the following

  1. create a shell or python script that implements the hook and add it to the repo in src/svn_bin/hooks
  2. edit src/svn_bin/pre-commit to call the script you added in a)
  3. install the script to VisIt's repo. This is described in more detail below.

To see examples of some scripts, look in src/svn_bin/hooks. The script check_autoconf_version.sh does two things. First, it checks to ensure that any commit in which configure.in is changed, configure is also changed. Second, it ensures that the correct version of autoconf was used to generate configure. The script hook_vars.sh contains global enviornment variable definitions used by various hook scripts. If you need to add some new variables for a given script, consider adding them to the hook_vars.sh script. Finally, any pre-commit script that returns non-zero exit status will prevent the commit.

The way a commit operation in svn works is that the files to be commited are transmitted to the machine where the repository is located. There, they are stored in a special place by the svn server awaiting the final step of the commit process. The whole operation is given a transaction id. This transaction id and the path to the repository are the only arguments passed by svn to pre-commit hook scripts. For post-commit hook scripts, since the files have already been committed at the time the hook script is executing, instead of getting a transaction id, you get the revision number that the files were committed in.

For these reasons, within a hook script, you must use the special svnlook command to examine the files to be committed (for a pre-commit hook) or just committed (for a post-commit hook). The svnlook command has a number of subcommands to, for example, get the list of files that are being committed or to cat the contents of a particular file. Note that you cannot use any of the ordinary svn commands (e.g. svn cat) commands that would attempt to modify the repository during the commit. Remember, inside the execution of a commit hook script, the repository is already in the middle of a modification.

The practice of placing the hook scripts themselves in the repository as we do for VisIt in src/svn_bin/hooks is actually completely optional and not necessary as far as hook functionality goes. In order to affect hook functionality, the only place a hook script actually needs to exist is in the repository's adminstrative hooks directory (see below).

[edit] Installing Hooks

We've added a hook script that will, in theory, automatically install your hook after you have committed it to the repository with a succesful 'svn commit' command. The name of this hook script is hooks_update.sh. It is the first post-commit hook to get fired and should be able to install your added/modified hook script (or un-install your deleted hook script) automatically. So, in theory, you should not have to worry about the following. If your hook does not appear to be getting triggered, it may be that something went wrong with this automatic install feature. In that case, read on...

Now, after you've written the commit hook (step a) above) and added a call to the hook from the pre-commit meta-script (step b) above), you still need to install the script to VisIt's repository so that it will actually execute during svn operations. Basically, this means you need to copy the files to the hooks subdirectory where the svn repository is actually housed. On the portal-auth.nersc.gov system where VisIt's repository was housed as of this writing, that would be the directory /svn/visit/hooks. Also, when copying the files there, you need make sure to

  1. copy both the new hook script you wrote and the modified pre-commit meta-script that calls your new hook script.
  2. ensure permissions on both scripts are rwx for user and group (e.g. 775)
  3. ensure the group is all visit developers and not just yourself.

[edit] Hooks getting triggered on files they should not be?

Occasionally, you may encounter a situation where you are doing a commit and, for some unknown reason, a file in your commit is triggering a pre-commit hook failure. For example, maybe you've added a new library tarball, libfoobar.tar.gz, to the third_party_builtin dir and the hook to check for tabs in files is getting triggered and the .tar.gz file is failing the test. We try to design the hooks to minimize the occurance of this sort of thing by including logic to skip files with certain svn properties, or to skip files in certain parts of the source code tree, or with certain extensions, etc. However, since that logic can sometimes depend on the structure of the source code tree, as the tree structure changes (e.g. files or dirs are added, etc.), the skipping logic may need to be updated.

If you find yourself in such a situation, here is what to do. First, identify the hook that is failing. The error message from the pre-commit failure should be enough to indicate which hook is failing. Then, go to the src/svn_bin/hooks directory and edit the skipping logic of the appropriate <hook-code>.sh file. Typically, there is a loop over all the files in the commit with a big switch statement in it and a bunch of cases that continue the loop if the file name matches certain criteria. So, you would update this logic to handle your case(s). But, just take care to do it in such a way that it winds up matching the smallest number of files to handle your case. Otherwise, you can inadvertently wind up nullifying the hook for files that it really should get triggered on.

Once you've updated the skipping logic in the appropate hook, then svn commit just the hook (which should automatically install it for you, see above) and then try the commit you were orginally doing again.

[edit] Pre-commits are now broken and you're stuck because you can't commit anything?

You can get into this situation if you've created or changed pre-commit hooks in such a way that they always fail. In this case, you will not be able to rely upon the automatic hook install behavior described in #Installing Hooks. In this case, you will have to manually update the hooks in the adminstrative directories of the repository. As of this writing, those directories are located on portal-auth.nersc.gov/project/projectdirs/visit/svn/visit/hooks. If you log into portal-auth.nersc.gov and cd /project/projectdirs/visit/svn/visit/hooks, you should be able to edit the files you see there. to fix the problem. Alternatively, you can edit them locally and then just use secure copy (scp) to copy them to the right place (e.g. scp my_fixed_hook.sh miller@portal-auth.nersc.gov/project/projectdirs/visit/svn/visit/hooks/.) After all is said and done, however, remember to ensure that the VisIt repo itself has the updated code for the hook script.

Personal tools