Plastic SCM 3.0 to 4.0 migration notes

Plastic SCM 3.0 to 4.0 migration notes

Wednesday, January 04, 2012 Ruben migration 3 Comments

This guide will give you some hints about how to perform a migration from Plastic SCM 3.0 to version 4.0.

Background

Plastic SCM 1.0 was first released in November 2006, and since then the basic underlying design has been evolving incrementally. During the development of version 4.0 we had to make some modifications to the core server structure in order to enable better distributed support, increased performance and an improved merge mechanism.

These core changes didn’t have a huge impact on the underlying database structure (if you look carefully you’ll note that the only difference is about the usage of the old “childrenitem” table, which entries were modified (visible on beta) and now have disappeared completely.

This important change made direct database upgrade (as we always did so far: just run the server and during start-up it upgrades the database) no longer feasible.

On the other hand, we have changed our “import strategy” and instead of developing specific “importers” (Subversion, CVS, Visual Source Safe) we’ve now standardized on the “fast-import / fast-export command suite”. This new “fast-export/import” allows us to provide import capabilities from a wider set of SCMs and also enable an easier “way out” for teams who need to feel they’ve a “way back” in case something fails during the move to a new SCM. It also allows us to provide a much better git interop.

We implemented the “fast-export” support for Plastic SCM 3.0 too (available on the latest releases) and then we’ve just decided to stick to this migration path to enable the move to 4.0.

In short:

Upgrade your 3.0 installation to the latest (3.0.187.33 -> for migration purposes only, don't use it for normal production).

Upgrade your 4.0 installation to 4.0.237.7 (or newer).

Fast-export your repository from 3.0

Fast-import it on 4.0

How to migrate from 3.0 to 4.0

Perform the fast-export of the 3.0 target repository. This command will create a file that will be used by the fast-import. The first argument is the spec of the target repository. The second one is the output file.

$ cm fast-export repo@localhost:8084 repo.fast-export 

Perform the fast-import of the exported repository. This command will create a 4.0 database with the imported data (previously exported from 3.0). The first argument is the spec of the target repository (this should be a non-existing repository or a recently created, empty one). The second one is the output file with the 3.0 data.

$ cm fast-import repo@localhost:8087 repo.fast-export 

Restrictions due to branch inheritance

There are some important changes between 3.0 and 4.0 underlying structure which will force the 3.0 fast-export command to “discard” some branches.

The reason is tightly related to the way in which 3.0 dynamic branch inheritance worked. In 4.0 each branch points to a changeset “head”, the latest changeset on the branch. So, switching to a branch is always equivalent to switching to the last changeset on the branch. Each changeset is statically resolved which means it simply points to a given source code tree, starting on the root directory of your repository and then to all its content (it doesn’t mean each changeset contains an entire copy of the repo!! Don’t worry about storage!).

Things were a little bit different with 3.0, where the selector rules were necessary in order to resolve a given tree. Remember you could have branches pointing to “last” which means switching to it would load your content on the branch and “combine it” (not merge, just combine the non-overlapping entries) with the latest on the parent branch.

There is a big difference between the changesets in 3.0 and 4.0. In 4.0, each changeset points to an entire, statically resolved tree. It wasn’t true in 3.0 for the changesets on branches inheriting from “last” since “last” would dynamically move.

What does it mean? Basically: branches inheriting from “last” won’t be exported by the 3.0 fast-export command and hence won’t be imported in 4.0. If this is a problem for you and prevents you to migrate to 4.0, please let us know.

Merge tracking changes

Merge tracking has been deeply modified in 4.0. Merge tracking was implemented at the “item” level prior to 4.0 and has now been moved to “changeset” level.

Item level merge tracking: means that each time you merge a file or directory, there’s a new merge link created between the specific file or directory revisions. It provides some advanced functionalities (like partial merging: you can merge just part of a changeset and then merge the rest later keeping full tracking) but paying a high price for it: performance is poorer. Each item has its own “merge tree” that needs to be calculated and checked individually. Merging a big number of items tends to be slow.

Changeset level merge tracking: means that the merge links are created between changesets instead of items. (Note: this is the way in which all the new DVCS work: Git, Hg, Plastic SCM…). Performance is much higher since merge calculation doesn’t depend on the number of items to be merged. Understanding the evolution is also simpler because you only have to take a look at the branch explorer to understand what was merged and what was not, instead of having to check each item individually. In 4.0 we didn’t only modify the “tracking” but also greatly improved the underlying merging mechanism supporting many merge cases that were simply not doable before (divergent move, change-delete conflicts and many more).

This important structural changes (tree + merge tracking) are the reasons why a “direct” migration wasn’t feasible so far, and also the reason why we decided to go through a “export-import” path.

The main risk migrating from 3.0 to 4.0 (and we’re currently working on it) is when a “partial merge link” is wrongly migrated as a full merge link (the only ones supported in 4.0) leading to potential merge calculation errors later on.

Merge between branches with different bases

Check the following merge scenario created with 3.0:

The merge from /main/task001 to /main/task002 will only propose /src/foo.c to be merged, because it was the only item modified in the branch. It means that “/doc/plan.pdf” won’t be proposed to be merged to “task002”. This was an important limitation on 3.0 and has been fixed in 4.0. But, if the merge was performed in 3.0, the merge link will be there but “task002” won’t contain the modified “plan.pdf”.

Now you migrate from 3.0 to 4.0 and decide to merge “task002” to “main”. The new merge system will check the differences between the source changeset and the base changeset and it will find that “plan.pdf” was modified (because “task002”, despite of the 3.0 merge, will still contain the “old plan.pdf”, and 4.0 doesn’t have (so far) a good way to know whether it was incorrectly merged on 3.0 or really modified.

At the end of the day, merging task002 will put the old “plan.pdf” on “main”, incorrectly, because the merge link between task001 and task002 is set by 3.0, and 4.0 can’t handles merge differently.

(The case is a little bit dense so feel free to contact us if you need further info). This is not a problem with the following type of merges:

It has been detected as branch rebase (the base of the branch was changed by the switchbranchbase command).

The source and destination of the merge are in the same branch.

The source of the merge is the main branch.

The destination branch of the merge is the branch base of the source branch.

The source branch of the the merge is the branch base of the destination branch.

The source & destination branches have the same changeset/label as branch base.