Relative project references break when merging

Mar 14, 2011 at 7:23 PM

Greetings,

Based on the wonderful TFS Branching Guide 2010 (thanks ALM Rangers), I have created a team project structure for a common code library team project with a single dependent team project (a portal application).  I followed the vanilla basic branch scheme pretty closely.  Here is what it looks like:

TeamProject Portal
|
->BuildProcessTemplates
|
->Development
  |
  ->DevMain
    |
    ->Source
|
->Main
  |
  ->Source
|
->Releases
  |
  ->PortalV1
    |
    ->Source
|
->Shared
  ->Framework
    ->Bin
    ->Source


TeamProject Framework
|
->BuildProcessTemplates
|
->Development
  |
  ->DevMain
    |
    ->Bin
    |
    ->Source
|
->Main
  |
  ->Bin
  |
  ->Source
|
->Releases
  |
  ->FrameworkV1
    |
    ->Source

This structure feels pretty robust and mostly works well.  However, I have a major issue that I cannot seem to resolve.  The "Shared" Framework folder is a branch of the Framework team project's Main folder, as described in the MSDN article from Feb 2011.

When I add project references in the "DevMain" project of the Portal team project, I am adding the reference to the project in the Shared Folder local to the Portal team project.  It works great, and I can compile.

Then, I try to merge from the Portal team project's DevMain folder into the same project's Main folder for testing/demo.  Upon opening the solution file for that project, I am presented with the error message that "Projects have recently been added to this solution. Do you want to get them from source control?".  I click "Yes" and then receive the message "One or more projects were not loaded correctly.  Please see the output window for details".

I can see in the DevMain solution file that the paths are RELATIVE.  I also see that, following the merge from DevMain -> Main of the Portal Team Project, that the solution file's paths CHANGE to point somewhere that doesn't exist.

Am I missing something with the project references approach when merging from my DevMain branch to my Main branch??  How do you configure them?

Help ALM rangers!!  Thanks for your impeding rescue.

SixDArray

Developer
Mar 14, 2011 at 9:03 PM

I just wrote a blog on this issue and some possible solutions:

http://blogs.msdn.com/b/billheys/archive/2011/03/14/project-dependencies-will-break-with-branching-if-not-done-properly.aspx

Regards,
Bill Heys
VS ALM Ranger

Mar 17, 2011 at 5:33 PM

Thanks for your response Bill, and your patience with my questions.  Based on the Feb 2011 MSDN article and your blog post, I am currently able to get a functional setup in the following manner:

Team Project = Framework
|
->Dev (Branched from $/Framework/Main)
    |
    ->Source
->Main (Branch)
    |
    ->Source

Team Project = Portal
|
->Dev (Branched from $/Portal/Main)
    |
    ->Source
->Main (Branch)
    |
    ->Source
->Shared (Folder)
    |
    -> Framework (Branched from $/Framework/Main)

This is working correctly with relative project references when merging from Dev->Main.  Unfortunately, what this breaks is having the subfolders such as "Releases" with branches point to specific releases - those would not work with relative references since the Release subfolder causes the subsequent branches to be at a different depth.  It was also nicer before to have a folder to hold all the development branches (Development with a Dev subfolder).

How do you recommend handling release branches?  Just as branches at the same depth as the Dev and Main branches?

Also, do you think maybe another way around this is a first layer of folders, with a layer of branches underneath?  It seems like a lot of folders but, at least it might work to keep the references relative.

For example:


Team Project = Framework
|
->Dev Folder
   |
   ->Dev (Branched from $/Framework/Main)
       |
       ->Source
->Main Folder
    |
    ->Main (Branch)
        |
        ->Source
->Release Folder
    |
    ->ReleaseV1 (Branched from $/Framework/Main)
        |
        ->Source


Team Project = Portal
|
->Dev Folder
    |
    ->Dev Branch (Branched from $/Portal/Main)
        |
        ->Source
->Main Folder
    |
    ->Main Branch (Branch)
        |
        ->Source
->Release Folder
    |
    ->ReleaseV1 (Branched from $/Portal/Main)
        |
        ->Source
->Shared (Folder)
    |
    -> Framework (Branched from $/Framework/Main)

Thanks again for your advice.

 

SixDArray

Mar 17, 2011 at 5:35 PM

Arg, I just realized I have a few typos in my example above (the path for the ReleaseV1 in the Portal Team Project should have been $/Portal/Main/Main I think).  But you get the idea.

SixDArray

Developer
Mar 17, 2011 at 5:50 PM

I think you have two choices for the Release branches. Keep them at the same folder-depth as Main and Dev (maybe having Main and Dev in their own containders)

OR

put all the dependent code in a folder subordinate to Main (and Dev, etc). This will force it to be at the same relative depth regardless of the depth that Main, Dev, and Release are within th Team Project. You can either branch or copy the code, but however it gets there you need to keep the dependent (referenced) projects at the same relative postion to the solution / project.

Regards,
Bill Heys
VS ALM Ranger

 

 

Developer
Mar 17, 2011 at 5:51 PM

I like your last option best, I think.

Regards,
Bill

 

Mar 17, 2011 at 8:07 PM
Edited Mar 17, 2011 at 8:09 PM

Just to follow up, I restructured my source tree as in the last example, and that works well now. 

It also opens up two possibilities for linking the shared framework for the Project References:

  1. You can reference the shared framework project "in situ" (wherever it got checked out on your hard drive) - this prevents making a copy of the project and the VS2010 TFS plugin still recognizes that I made changes to the shared framework.
  2. You can use a full branch into the Shared folder, and then reference the copied shared framework project in the dependent project.  That works just fine, but requires a copy of the project.

As for which of the shared framework "branches" to link to the dependent project, I am finding that your recommendation to link the Main branch is probably the best, although I could see situations where you might want only want to link a stable release branch, or an actual separately branched development branch (in the original shared project) also.

Any guidance as to which is the best branch of a shared framework to link to in a dependent project?  For now I will stick to Main.

Also, I'll try to write up and post my final solution to my blog and link it here once I'm done with it.

SixDArray

Developer
Mar 17, 2011 at 8:47 PM

Let me restate your question to: "Which is the best branch of the shared source project to branch from?"

Branch from Main in the source Team Project if you think you want changes to be merged from the target team project (the dependent project), back to the source team project. You never want to merge changes *into* a release branch.

Branch from a Release branch in the source Team Project if you do NOT want to merge changes from the target project back to the source project and / or if you need to branch from a prior release of the source project.

An important consideration for when you reference the shared source project "in situ" is that to make sure that you do not break a build. The Build workspace matching will need to match your local workspace mapping or it might break. A benefit of referencing the shared source project *in situ* is that any changes you make to the source are immediately visible to the dependent (referencing) projects. Since you did not branch from the source team project, you do not need to merge any changes into the dependent projects. This may not seem like a big deal, but in the Visual Studio developer division, where there are many dependencies between dozens or tens of dozens of projects, branching rather than referencing *in situ* could be a merge nightmare.

Regards,
Bill Heys

 

Mar 18, 2011 at 12:55 AM

I'm investigating a very similar folder structure design question. I haven't broken any references yet, but I have the opportunity to avoid (or cause) relative path issues starting next week.

QUESTION: Are there any significant drawbacks to having the Main branch in a subfolder to avoid folder depth reference problems?
This is the second bullet at the end of Bill's (awesome) blog. Excerpt from http://blogs.msdn.com/b/billheys/archive/2011/03/14/project-dependencies-will-break-with-branching-if-not-done-properly.aspx

  • (#1) Keep all dependent assemblies or projects in a folder that is subordinate to the branch, but the downside is that you will be copying the assemblies and / or the project source to each child branch.
  • (#2) Keep all dependent assemblies or projects in a folder within the team project that is outside of the branch, but the downside is that all branches must be at the same folder depth within the team project. This may not be a problem if a person who is responsible for creating branches is familiar with this dependency issue. The risk is that a branch might be created at a different folder level (folder depth) within the Team Project. This will then break relative project or assembly references for solutions under that branch.

My thoughts:

PRO: All developers can create relative references "inside" or "outside" the branch folder without worrying about breaks due to merging to or from Main. Relative references to files in other Team Projects (or other path not under the Branch folder) won't break just by merging to Main.
PRO: You can do both (#1) and (#2) or just (#2) and not break relative references. The same does NOT go for just doing (#1). If you can keep all references "inside" the branch folder (#1) then you're pretty safe regardless of branch folder depth differences (if any).
PRO: You don't have to move any references or worry about introducing new breaks if you already have all branches at the same folder depth.
PRO/CON: You won't see builds break in the Main branch when "outside" relative references are encountered.

This might be a CON if you want to enforce only "inside" relative references. Having all references contained within the branch makes it less likely to accidentally miss part of the files needed to build that branch in the future. For example, if your archive process is to move Team Projects to another Team Project Collection then delete the Team Project then relative references between Team Projects may be very undesirable. Of course you need build automation in the Main branch to identify relative reference breaks.
Otherwise it is a PRO because you avoid a build break and/or avoid needing customized build steps just for the Main branch.

CON: Having the Main branch in a subfolder is not the normal "industry standard" location. 

If you follow (#2) then all branch creators should be aware to follow same-depth pattern. However if you have Main one folder down then every developer creating relative references must be aware of breaking references when integrating and building from Main (or other branch at different folder depth).

My Situation:

1. Our current branch structure has all branches at same folder depth.
2. I suspect we have very few "outside" relative references currently, but we may soon choose to reference shared source projects "in situ" for same reason mentioned in Bill's previous post today.
3. It was proposed that we move our Main down one folder from $/TeamProject/QA/Main to $/TeamProject/Main.  This would match the folder structure patterns shown in the TFSBranchingGuideIII Scenarios diagram and several other common practice references.  There was not much discussion for keeping folder depth same because impact to relative references were not considered.
4. This Friday I implement our plan on the $/Platform Team Project. (!)


Example below shows 3 folders in each Team Project (Dev, QA, Release). "-" dashes indicate branch depth from Main, brown=shared code folders, grey=descriptions)
$/Platform   (Contains our recent shared code and shared services for our new SOA-based platform architecture)
    /Dev
        -Int    (Child of Main Most devs work directly in Int branch)
        --Story1001    (Short-lived child of Int branch, ideally <3 weeks before merge to Int & close)
        --Platform1.0Hotfix1234   (Short-lived child of Platform1.0 branch to resolve complex fix for Production bug/WI#1234)
    /QA
        Main
            \Lib\Logger
            \Services\
               \Lib
              
\ThirdParty

               \...
               \Microsoft
                   \EnterpriseLibrary5.0
                   \...
    /Release
        -Platform1.0  
        -Platform1.5

$/ProjectA
    /Dev
        -Int    (Child of Main Most devs work directly in Int branch)
        --Story1001    (Short-lived child of Int branch, ideally <3 weeks before merge to Int & close)
        --LoggerRefactor1 (Short-lived child of Int branch, ideally <3 weeks before merge to Int & close)
        --ProjectA2.5Hotfix2343   (Short-lived child of ProjectA2.5 branch to resolve complex fix for Production bug/WI#2343)
    /QA
        Main
    /Release
        -ProjectA2.5

As a side note I'm a little perplexed about having Main at a different folder depth requiring only "inside" relative references but TFS Branch Folders feature not allowing a child branch of Shared to be nested in a subfolder of a branch. I feel like have missed something and need to re-read through the relative reference posts another couple times.

Thanks for your time and any feedback.  -Zephan

Mar 18, 2011 at 1:29 AM
Edited Mar 18, 2011 at 1:30 AM

Hi Zephan,

One initial reaction - might be careful moving your Platform Main down from QA - that might mean that the references in the Platform Main solution file would now be relative at one level LOWER, which might throw stuff off.  I felt the same about "missing" something - but I can't find what it is I missed!

It's counter-intuitive (to me too), but your QA folder may be helping keep relative references at the same depth, no matter whether you are in Dev, QA or Release folders.  The other option seems to be to just keep all the branches at the same level at the root.  That works but seems like a flat structure might just get messy fast, and the subfolders help keep things organized.

Unless Bill or somebody from Microsoft tells us it's a horrible idea, I don't think there is any reason you can't keep your Main branch nested in a folder.  It seems excessive, maybe Microsoft missed out not just managing the relative references for us (could they have an "add team project reference" feature or something and have the VS2010 TFS plugin do the work for us?).  After all, it's your source tree, you can have it the way you like (or need!) :)

If this helps any, here is the exact structure I have working right now:

$/Platform
   /Dev
      /Feature1
        /Bin
        /Docs
        /Source
        SolutionFile.sln
   /Mainline (like your QA folder, but mine is just named badly)
      /Main
        /Bin
        /Docs
        /Source
        SolutionFile.sln
   /Releases
      /Release1
        /Bin
        /Docs
        /Source
        SolutionFile.sln

$/ProjectA
   /Dev
      /Feature1
        /Bin
        /Docs
        /Source
        SolutionFile.sln
   /Mainline
      /Main
        /Bin
        /Docs
        /Source
        SolutionFile.sln
   /Releases
      /Release1
        /Bin
        /Docs
        /Source
        SolutionFile.sln

I am using the "in-situ" project references, and ensuring that my local workspace mapping is always mapping the references correctly.  So far it is working ok, and I haven't noticed any major problems yet.

Hope that helps - your original question stands for someone above my pay-grade - is it really any problem having MAIN in a subfolder?

SixDArray

Developer
Apr 15, 2011 at 3:28 PM

I like the idea of having each of your branches at the same relative folder depth within a Team Project. It might minimize broken relative references when you branch.
Regards,
Bill Heys
VS ALM Ranger

Apr 15, 2011 at 4:46 PM

It's working great in practice too - I've been using this method (since I posted last) for all of my development projects, and I haven't had any issues at all.

Thanks for your help Bill! :)

Jon