This project has moved. For the latest updates, please go here.

Complex branching (extended)

Dec 7, 2010 at 1:12 PM

I have a project that requires a more complex branching structure than what the TFS branching guide describes and I'd like know how to set this up in TFS and if TFS will be able to handle the current setup without much pain on the deverloper/integrators side.

The best that I can describe the project is as a shared code base, which has a part-time team maintaining the mainline of the project. This team makes official releases of the project every few months, by harvesting and refactoring changes made by teams using the code base. Each team that uses the code base branches of from an official main_release and does its work on a team specific branch, subsequently each team will create its own release(s) of the shared code base project which is included in their software.

main -> hot fix -> main_release -> team1 -> hot fix -> team1_release1

                                                                              -> hot fix -> team1_release2

                                                            -> team2 -> hot fix -> team2_release

                                                            -> team3 -> hot fix -> team3_release

                                            .....

                                                            -> team8 -> hot fix -> team8_release

The first step that adds complexity is when team2 finds a bug (or 'cool' feature) that team1 fixed, the preferred method for team2 to get this change is by integrating the change set(s) from team1's branch to team2's branch, rather than emails etc. ;) Challenge lies in the cherry picking that team2 will have to perform and the support from TFS for such actions, getting to little code (easier to detect) is less of a problem than too much code (harder to detect).

As there are 8 teams doing this the number of cross integrations between the various team branches can be substantial especially if the central team doesn't do a roll up release in a while.

For the central team the biggest challange is that when they start looking at integrating / refactoring the code from the 8 teams the last thing they need is a flood of conflicing merge warnings from the tool.

Dec 7, 2010 at 3:06 PM

I would recommend looking at the feature crew model that's talked about in the guidance. In the mean time I might also suggest looking at this presentation: http://www.msteched.com/2010/Europe/DPR304

Developer
Dec 7, 2010 at 7:27 PM

I strongly suggest looking at branching for development separately from branching for release.

With this perspective, there should be a distinction between new feature *development* and hotfixes to *released* code. I recommend using Feature Team branches on the development side:

Main -> Feature Team 1 and Main -> Feature Team 2 (making each of the Feature Team branches full-children of Main and siblings of each other)

I also recommend trying to avoid doing cherry picking on the development side. When Feature Team 1 is ready to release (or share) their feature with another team, they should merge their feature (Reverse Integration) with Main and the other feature teams will pick up this feature the next time they merge (Forward Integration) the Main branch with their Feature branch. Note, that before Feature Team 1 merges their feature with Main, they need to integrate Main with their Feature branch and test this integration. Then they are ready to merge their feature to Main.

Cherry picking changes runs the risk that you will not get all of the changes necessary to merge a complete feature. This is a primary reason why I recommend merging the latest version of a branch rather than making cherry picking a regular part of the process on the development side.

There is no good way to completely avoid possible merge conflicts when you are integrating changes from eight (8) different teams. As I am sure you know, merge conflicts most often are the result of having a changes made to the same file(s) in both the source and the targe of a merge. It is necessary in this situation for a person (or possibly the tool) to identify and resolve conflicting changes made, on a file-by-file basis.

One way that merge conflicts are reduced is that we recommend:

  • Do frequent (daily) merges from Main to *each* of the development feature branches. If you merge Main to the feature branches on a daily basis (after a good build in Main), you will avoid the *big bang* problem where you have a large number of changes that have accumulated over time that are merged into each branch.
  • Integrate Main with the Feature branch *in* the feature branch first, before merging the feature branch to Main. Only merge the Feature branch to Main when it is ready to be integrated with Main and the integration has been done first and tested in the feature branch.
  • Avoid cherry picking if possible. Merge the latest version of a branch if possible. Typically there are so many interdepencies with the code developed in a feature branch, that trying to cherry pick changes to share with another team is bound to be challenging if not disasterous

Rather than having each feature team *release* independently of other teams, consider having each feature team integrate their changes with Main. Then stabilize Main and release it when it passes QA quality gates. If two teams want to participate in a single release, have each team merge their changes into Main (after testing the integration in their feature branches first), Main as I stated earlier would release when it is ready, with whatever feature changes have been merged to that point.

The complexity for the branching structure to support feature teams should be isolated to the development side of the branching structure. In other words, try to avoid having team-specific release branches.

Regards,
Bill Heys
VS ALM Ranger

 

 

Dec 8, 2010 at 9:16 AM
AllenFeinberg wrote:

I would recommend looking at the feature crew model that's talked about in the guidance. In the mean time I might also suggest looking at this presentation: http://www.msteched.com/2010/Europe/DPR304

The way FI & RI is presented has me worried, because it sounds like not doing this properly in TFS will result in baseless merges, which I hope is not the case because that would make TFS as usefull in the real world as Subversion. :(

Antoni

Dec 8, 2010 at 9:28 AM

@Bill

what you say all makes sense, in an ideal world where all teams deliver flawless code and everything passes QA this will work, and if you are lucky enough to work in an environment where this is the case then I am glad for you.

However reallity is what it is and the setup you describe will not work in my environment for various reasons, foremost because teams do not want to be constrained in not making their release window because of another team's changes not passing QA for example, which means merging back to main for a release will not fly.

Which brings me back to the question of how well TFS will support the branch model I described, is it at all possible to setup? What issues will it cause with the tool? Will cherry picked changes when merge up to main cause conflicts?

Antoni

Dec 8, 2010 at 3:14 PM
Edited Dec 8, 2010 at 3:30 PM

Tekkie,

          Please understand that the branching guidance works in real world teams where flawed code is delivered and where code does not pass QA. The branching models outlined in the branching guidance help to prevent the team from hurting themselves with regressions, lack of quality, and loss of codeline stability. This guidance has been battle tested and the gotchas identified and worked around.

          You can setup whatever branching structure, no matter how convoluted, you want. All we are trying to do is make you aware that your life will be exponentially harder and more painful if you choose to go and create your own branching structure. Go in with your eyes open and realize that if you creat own branching structure you are putting your own life in your hands. [MSFT gives you the rope...either play jump rope and have a good time...or go hang yourself...your choice]. :-)

From what you described I created the following in TFS:

 

 

Just an FYI...for our organization we have a rule creating a new branch costs the team $10,000. I won't get into specifics but this model your suggesting would cost a mimum of $90,000. If it's worth the cost to you for your efforts...go at it...but remember go in with your eyes open branching is neither FREE nor PAINLESS. oh and be prepared to spend your weekends, holidays, and precious sleep supporting your customized branching structure when things go bump in the night and some developer can't release their code because "the TFS merge window is showing 8234 conflicts". I've been there...and we became believers in the branching guidance.

Dec 8, 2010 at 3:50 PM

I am more than aware of the pain and issues that branching can create (and take away), however a situation that must be avoid at all costs is a restriction in the branch model because a tool cannot cope with a certain branch model, I am fine with not doing certain branch models because of cost, overhead, complexity etc. but not because a tool can't handle it.

For example Subversion cannot handle a branch model as described, its merge tracking is so poor that if you try to do this, every single merge will be conflicting even if only a single change set was 'integrated' between the various branches and you then wanted to integrate them all into main.

From your answer I am assuming that TFS is much better at tracking changes across branches and that when I create a single change set on team1 branch and integrate this from team1 -> team2..8, when at the end I integrate team1..8 -> main TFS will not flag this as a conflicting change because it was able to track this single change properly.... right? yes? no?

Developer
Dec 8, 2010 at 6:30 PM

I am not aware of any statements made in either the guidance or on this forum to suggest that Team Foundation Server 2010 cannot cope with the branching structures you have described. What we try to achieve in our guidance and in our advice here in this forum is a combination of a sound branching structure and a sound source code management process (including branches, merges, continuous integration, daily builds, etc.). Combined, these will give you a reasonable and sound branching strategy.

It is wrong to suggest that simply becuase Subversion cannot handle a branching model such as you describe, that TFS cannot handle it. In fact, and as Allen illustrates above, TFS is quite capable of handling most any logical branching structure you might propose. By logical, I am suggesting that you can probably propose a hypothetical branching structure that makes no sense whatsoever (such as recursive parent-child relationships, nested branches, etc.) There maybe some restrictions that TFS specficially imposes. But TFS 2010 is in use at some very large companies with some very complex branching requirements.

Rather than assume a tool cannot handle your requirements, I suggest that you think through your requirements and pose a complete statement of these requirements. We can work with you to recommend a branching strategy that meets your requirements, or help you modify your requirements if necessary to work better with the best practices approaches we suggest.

Specifically to your statement "when I create a single change set on team1 branch ..." I will not make a blanket assertion that TFS will always be able to integrate this change through multiple layers of branching hierarchy without encountering a merge conflict. I believe this to be the case. But rather than make a blanket assertion, I would prefer to help you understand why, in TFS, merge conflicts most commonly occur.

A merge conflict most commonly occurs during a merge of a file from a source branch to a target branch (or branched folders) when there have been changes made to this file in BOTH the source and target branches. In some cases the tool is capable of reconciling these changes automatically without manual intervention. But often a merge conflict requires a person to look at the changes made to the file in the source branch, along with the changes made to the file in the target branch, and to make a decision (keep the target, keep the source, or combine the changes from both files). There may be multiple *conflicting changes* that might need to be resolved in this way.

Assuming in you example, that you make a change to a file in the source branch and merge this change to the file in the target branch there should NOT be a merge conflict ASSUMING there were no changes made to the file in the target branch since the last successful merge.

TFS 2010 does not generally have ANY problems tracking this type of change through multiple layers of branches. In fact I have never used Subversion, but I would be surprised if it had difficulty tracking this change properly as well. I am assuming we are talking about source code files (essentially text files) where changes can be tracked by a tool on a line-by-line basis. Changes to binary files or images or other types of non-text files would, of course, be a different situation.

Regards,
Bill Heys
VS

Developer
Dec 8, 2010 at 6:38 PM
Edited Dec 8, 2010 at 6:44 PM
Tekkie wrote:

@Bill

what you say all makes sense, in an ideal world where all teams deliver flawless code and everything passes QA this will work, and if you are lucky enough to work in an environment where this is the case then I am glad for you.

However reallity is what it is and the setup you describe will not work in my environment for various reasons, foremost because teams do not want to be constrained in not making their release window because of another team's changes not passing QA for example, which means merging back to main for a release will not fly.

Which brings me back to the question of how well TFS will support the branch model I described, is it at all possible to setup? What issues will it cause with the tool? Will cherry picked changes when merge up to main cause conflicts?

Antoni

 Our branching guidance is based upon real-world experience, not simply ideal world scenarios. In every real world one needs to cope with the issues you raise (code is NOT flawless, code does NOT always pass QA - at least initially). Nobody has suggested that this ideal world is the basis for the best practice guidance we have published, or the advice we offer here. I take exception to your suggestion that this is the case.

There are reasonable branching strategies, combinations of sound branching structures and sound branching practices, that you can employ to deal with your scenarios. You are NOT the only organization that needs to deal with constraints such as one team being ready to release (within the window) and another team being not ready becuase their features or changes have not passed QA testing. Do not assume, however that your solution (merging back to main) is the ONLY possible solution to this constraint.

I have worked with MANY organizations that have to deal with this issue. I am happy to work with you to recommend best practices for both a branching structure and a branching process to help you minimize your pain. However that does not necessarily mean that the branching model you describe is one that I (or the VS ALM Ranger team) endorse or recommend.

Regards,
Bill Heys
VS ALM Ranger

Developer
Dec 8, 2010 at 6:43 PM
Tekkie wrote:
AllenFeinberg wrote:

I would recommend looking at the feature crew model that's talked about in the guidance. In the mean time I might also suggest looking at this presentation: http://www.msteched.com/2010/Europe/DPR304

The way FI & RI is presented has me worried, because it sounds like not doing this properly in TFS will result in baseless merges, which I hope is not the case because that would make TFS as usefull in the real world as Subversion. :(

Antoni

 Antoni,

I am not sure how you have concluded that simply becuase you do not do merges properly (as recommended) that you will end up with baseless merges. You may end up with a flawed process that needs to be corrected (by doing forward and reverse integrations according to the best practices advice offered in the guidance and here on this forum). A bad process does not make the tools bad. TFS 2010 is perfectly capable at handling your requirements (as far as I understand them). But if you choose not to understand and embrace a sound branching process, NO tool will keep you happy!

Regards,
Bill Heys
VS ALM Ranger