Friday, May 30, 2008

Managing Multiple Repositories with Mercurial

Unfortunately my interest has shifted to other projects, so I will not be continuing my series on different distributed revision control systems at the moment. I may pick it up again at some point in the future. However, since I currently use Mercurial, I will be posting information on how I use it. For now I'll give an overview of different ways to use Mercurial to manage multiple projects.

From my personal experience as well as sources that I have read, it seems that there are three main ways to use Mercurial to manage multiple projects:
  • Single Repository - There is a root directory called something like "projects", which is the Mercurial repository. Each project has a subfolder, but all of the tracked files are in the main repository.
  • Multiple Repositories - Each project has its own repository. They are most likely all grouped under the same root folder, but all have different histories and are managed independently.
  • Multiple Repositories with the ForestExtension - Each project has it's own repository, however they are all "nested" repositories in one large Mercurial repository. The ForestExtension provides commands to manage all the repositories in a loose group.
Single Repository
This is the setup I currently use. I have a main "Projects" directory, containing folders for various categories of projects. Each of those in turn has a folder for each project in that category. The advantage is that to back up the repository I can deal with everything at one time. In addition, there are a few tricks to deal with the projects independently.

For example, to view the history only affecting files associated with a certain project, list that project's subdirectory after the "log" command:
  • hg log /projects/school/project1
It is also possible to clone only part of the repository (eg, a particular project), and merge it back in using the ConvertExtension. You must create a temporary file that includes the path of the files you would like to extract. For example, you would create a file testconvertmap.txt that contains the text: include "school/project1". Then run the command:
  • hg convert --filemap testconvertmap.txt /projects /subproject
The new folder "subproject" will contain the subset of the repository with all the files in the directory that you specified, with full history concerning those files. This is useful for testing a feature in a particular project, or providing someone with only the part of the repository they need. You can merge your changes back into the main repository, however depending on the nature of the repository and the changes you may need to employ the method outlined in Merging Unrelated Repositories.

Multiple Repositories
The advantage to this method is that each project is actually independent. No special methods are needed to clone a project to test a new feature or only extract the information for a particular project. However if you later want to combine two projects (eg, they are closely related), you will need to employ the method in the Merging Unrelated Repositories document mentioned in the previous section.

Multiple Repositories with the ForestExtension
I have not used this extensively, I have a small group of test repositories in a "forest". However, at least conceptually, this seems to be the most convenient method for handling multiple projects. Each project is in its own repository, so cloning individual projects is just as easy as with the Multiple Repositories method. However, all of these repositories are in one larger repository. The ForestExtension provides commands to operate on all of the repositories at once, such as fpull, fpush, fupdate, fstatus, etc. This makes backups and bundling of all the repositories similar to the Single Repository method. There is also a recommendation to include the functionality of this extension into the core of Mercurial with NestedRepositories. I will be working with this method more in the future and will expand on it when I know more.

Each of these methods has advantages and disadvantages, just like anything else. I would recommend trying each with a small number of projects to see which you prefer, then employ that for all of your projects. Since it is possible to convert between them, whichever you choose is not set in stone.