Outreach Move

VisIt is planning to move to the SciDAC Outreach center for both its subversion and bug tracking.

Outreach Center

The main impetus for moving our existing development setup to the center is transparency. Users (or potential developers) can grab our latest and greatest source code without having to go through any human barriers. We will be able to point to source code which is accessible via the web, which can be useful in development discussions.

Concerns

There is a laundry list of concerns which have been raised:

Issue Resolution
Some RHEL4 and RHEL5 kernel versions cannot do a full checkout of the trunk on GForge using SVN.

Add the following to /etc/sysctl.conf and either run "/sbin/sysctl -p /etc/sysctl.conf" or just reboot -- the change will be permanent.

# network performance tuning
net.core.rmem_default = 256960
net.core.rmem_max = 256960
net.core.wmem_default = 256960
net.core.wmem_max = 256960
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1

This should also fix other network related issues, like downloads in a web browser hanging.

What is the backup policy at the Outreach center? Andrew Uselton: "If we lose California you may have a problem."

Backups are done hourly to a companion server. Every day, a backup is transferred to NERSC's HPSS system, which is housed in a building across town.

What happens to our hook scripts? Will our pre-commit hooks still work? Yes, they should.
What happens to our hook scripts? Will 'commit warnings' work? Maybe. These rely on a subversion bug, and we are unsure whether it is a client bug, a server bug, or a combination. We may have to switch to emails for these.
What happens to our hook scripts? How do we modify them? Actively being pursued. Current status: our 'hook-that-updates-a-hook' mechanism seems like it will be viable; NERSC contacts are looking into it. Worst case involves requesting an Outreach admin to update the hooks.
What about our NERSC accounts? There is no reason to relinquish your existing NERSC account. However, you will no longer need it to access the subversion repository. You can still use that account for testing VisIt, for example on davinci.nersc.gov.
What about our NERSC accounts? Cron jobs -- automated testing? This issue isn't really related to the switch. We might have to change a line or two of the existing scripts, if they hard-code in the svn repository URI.
What about our scripts in svn_bin? These will need to be updated. (Jeremy notes that not all switching is just a hostname, e.g. the username to email mapping will change based on new user ids.)
There's at least one file which is in our history but cannot be, for licensing reasons. Now seems like the time to rewrite history to fix this. This will be addressed, see the 'Schedule'.
Should every existing developer be an 'Admin' on the Outreach center's GForge? Or maybe a 'Senior Developer'? What is the difference? Andrew found this link for us: http://wiki.evolvis.org/evolvis/index.php/GForge_roles

which suggests we probably want most people as "Senior Developers".

Will we have a passwordless equivalent to our current passwordless ssh access over the new https protocol repo? This appears to be the default with recent versions of subversion. The subversion book has a section on this. Verified on subversion 1.5.1. Do note that using this mechanism stores your Outreach center password on disk, in plaintext. This is no less secure than using public/private keys without a pass phrase.

Of course, anonymous access would never require a password.

How do we switch an existing checkout tree to the new server? First, to tell where a checkout is pointing, run svn info:
src$ svn info | grep URL
URL: svn+ssh://user@svn.nersc.gov/svn/visit/trunk/src

Assuming it points to svn.nersc.gov, and you want to point to the outreach center, run this at the root of your checkout:

svn switch --relocate                  \
   svn+ssh://oldusername@svn.nersc.gov \
   https://newusername@outreach.scidac.gov

where oldusername is the user name you had been using at svn.nersc.gov, i.e. the old value of $SVN_NERSC_NAME in your environment, and newusername is the one you use at the Outreach Center. This will work no matter that subtree, e.g. trunk, or a branch, or test, that you had checked out, because the path after the hostname is the same for both.

When I try to commit, it uses my local username instead of the one I use for the outreach center. How do I fix this? Either (a) just hit ENTER at the password prompt, and it will ask for a new username, or (b) add the --usermame=outreach-name flag to svn commit. With credential caching, it will remember your username for future commits, so this should happen very rarely.
Re-use $SVN_NERSC_NAME or make a new variable? Good news, everyone! I think we can just get rid of this concept entirely. With HTTPS, the username is ignored, and with anonymous checkouts, you don't need it for many operations. For the operations where you do need it, it will let you enter the right username if you enter a blank password, or you can put --username=NAME on the command line of your first commit. And due to credential caching, it will remember the correct username for the future on its own. So there appears to be no value in having this string anymore.
It complains every time I commit about 'approving my mailing list' post! Shouldn't happen. Tom is pursuing with NERSC contact.

Schedule

  1. Create a list of issues and keep track of their solutions (this!)
  2. Rewrite history to remove 'illegal' files. Optionally, simply totally destroy and re-create from scratch the top-level third_party directory taking care not to include the 'illegal' files in the re-creation.
  3. Developers must get accounts at the Outreach center by Thursday, 3/19.
  4. We make a copy of the repository to the Outreach center on Thursday, 3/19.
  5. Developers checkout the Outreach repo. They have a week to try a commit, test to see if things work, and in general broach any complaints.
    • If there are significant issues, this might be extended to two weeks.
  6. On 4/06/2009, the repository is 'switched'. Commits to the svn.nersc.gov repository are disabled, such that they will fail with an error.