vsarbranchingguide Discussions Rss Feed

New Post: Structure for large amounts of customisations per product

j_machale — Wed, 19 Jun 2013 14:04:32 GMT

Hi,

I am working with a customer who is building 1 (or more) core products using TFS2012 which they will "supply" to another department within the organisation. This other department will, in turn then customise the product (including code changes if necessary) for individual customers. These customisation projects could number in the 100's. The organisation would like to share all of the artifacts from the core product with the customisation project (including test cases) so that they can be used for regression testing etc. The process template is a customised version of CMMI.

I am wondering if anyone has any opinions on what I see as my two options (or if there are options I have not considered):

Create new team projects branched from the core product.
Shared work items build resources
Ability to push bug fixes
May hit PC project limit
Managing security
Create a separate project collection and create a custom application based on the TFS integration platform to create and update team projects as a copy of the core project.
No problem PC project limit
Managing security
Work Items only a copy
No current support for Test cases in TFS IP as well as limited support for TFS2012 from what I can see on the blogs.

Thanks in advance for any guidance!

John.

New Post: Identify an artifact or Multiple Artifacts for a Project

davismc — Tue, 18 Jun 2013 04:26:31 GMT

If you are using .NET you must get all of them because like you said, the solution and the project files for a build.

However, actually they are .NET files and makes perfect sense if you get the common items 1 time and 1 time for all the projects, maybe per release, and put it in your workspace. This is essentially the baseline version or the version of the last release. You obviously branch from the main and then you get the 3 files from the branch and do a get latest off of those and have the workspace of the branch be the same as the workspace for baseline.

The references of the projects in the .sln would all have to be in your workplace. You even do an MKLINK to make a symbolic link to make the referencing easlier with the .sln.

That may actually work-out, kinda thinking and typing here.

One that situation would have 1 hit on time to take the 10 minutes to get that latest from the common and 1 for all the projects. Then after that you still have the branch and if you only need to modify those 3 files. Then you have only those 3 files from the branch in your workspace.

You can still build and compile the .NET using those 3 changed versions. The other projects would use the same baseline version in the workspace but hmmm...wouldn't isolate like the Project 1, Project 2, etc because those change from each Project would go into the base the the projects really aren't isolated, per say. It would be like a unit test system test.

Let me think about this...there are pro's and con's.

I will play with this tomorrow. I will let you know the outcome.

Thanks for the dicussion davidkallan. It has me thinking.

Again, I'll let you know how that works out or if it is even within reason considering.

Thanks!

New Post: Identify an artifact or Multiple Artifacts for a Project

davidkallen — Tue, 18 Jun 2013 03:47:04 GMT

no. I've not seen anything about creating a branch on just specific artifacts.

But here is an idea. I just tried it. Make a full branch of all 17,890 files. It only takes a few seconds, right? Because it just makes pointers on the server. Then if you know you want to just change 3 files, get latest on just the files you want, at the time you want them. Just in time editing. Then change them. Of course, that makes no sense if they are normal .Net projects because you want the solution and project files and all supporting files or you cannot build. But if you want to build the solution then you want the whole thing so I don't see the point in what you are trying to do in the first place. BUt if you really just want to touch 3 files out of thousands, you can get just those files. Nobody says you must get latest on all of them.

As a radical idea, I know TFS and Visual Studio support GIT now. It is a totally different paradigm. If you and your team are willing to learn a totally new approach to version control, it might be a better. But I cannot say as I have zero experience using it.

New Post: Identify an artifact or Multiple Artifacts for a Project

davismc — Tue, 18 Jun 2013 03:20:37 GMT

Whenever we create a new project we have it on a branch. So, how often really is based on how many projects we have going on that is scheduled for a release. You could associate it to a feature branch as that is essentially what they are. Then the SYSTEST will be overwritten by merges to it and then a release branch is created per release.

I could have sworn is takes me about 15 minutes then again, I am doing other things at the same time too which I'm sure that could slow things down as well. I have a an adequate pipe of about 40 MB/SEC taking into account VPN, obviously faster without. That's just me though and we have others who are geographically separated.

I could only imagine it would be quicker on the changes as your just dealing with the local workspace or changes that are identified with that; just like with SVN. The merges would be the same.

I just keep thinking there has to be a better way and not have to deal with the 15 minutes or what have you. I mean it only makes sense. If I have 17,890 files or you have 27,000 files each one is not being touched and only some are. Why not just get them once and create a branch based on just the needed artifacts (with the option to add or remove)? That option or possibility does not exist does it? I have never seen in documentation that mentioned it.

New Post: Identify an artifact or Multiple Artifacts for a Project

davidkallen — Tue, 18 Jun 2013 01:47:04 GMT

How often do you create a branch?

As a reference for performance, it takes me only 7 minutes to download a complete new branch with 2 GB and 27,000 files. And my source code is on the internet in the Microsoft TFS cloud service. So if you are getting 15 minutes, you may have network or server issues.

But regardless, I admit that is an annoying delay. But you only have to do that once, when a branch is created.. Later, during development, when you get latest version, it should be much quicker for just changes.

At work, we only branch once a month. So waiting 10 minutes once a month for a developer is not a problem. You just go get a drink of water, or go to a meeting. And you are done when you return. How often do you branch? That is the key to how big an issue this is in real life.

New Post: Identify an artifact or Multiple Artifacts for a Project

davismc — Tue, 18 Jun 2013 01:04:59 GMT

We are currently using SVN. But yep, familiar with how to get the size on the development machine or local machine.

The 2,000 artifacts was just an example and very close to the disk size but here are the numbers:

Size: 2 GB
Artifacts: 17,890
Folders: 1,400

Now, that is for 1 project. For instance, I am currently involved in 3 and each on it's own branch off the main/trunk. So that's about 6 GB in disk space. 53,670 artifacts, 4,200 folders.

It took a good 15 minutes to download locally for each one of those branches. As may know, SVN downlaods the file and meta for the .svn to track each artifact. Much like TFS would with it's local workspaces. (TFS 2012 with the predominant local workspace.)

The really sad thing, is on 1 project I had a total of 10 artifact changed, the 2nd - about 22, the 3rd - 2.

I want to come up with a plan to not require a download of all that yet keep the identification of each project and artifacts comprising the project for isolation merges. So, that if let's say the release upcoming was initially scheduled for the 3 projects turned out in the midst of system testing that only 2 of those 3 were going to go in; I need to be able to remove the 1 of those identified to NOT go in and remove all the elements. Yielding the system and only those 2, rebuilding and having those in just system test.

I thought about the use of version labels. That would avoid the need of branches and all BUT labels in TFS and in VSS are risky as the reason: they are free-form text on the name which means one developer could label something "Test" and the other developer labeling it "test" (really meaning "Test"). I have used things like PVCS which had the promotion group that was staticly named which enforced consistency and they had the version label as well that was not static. However it was somewhat slow, a little outdated and did not integrate well with VS as TFS does, and work with a fully integrated bug tracking and build aspect as well and the other features of TFS.

I have figured a way to remove about 2,000 of the 17,900 which comprise common elements like the DAL and Placeholder logic as well as other "common" items. I would put them in a separate tree structure on the same TKC and using the builds for a reference. As well, the projects downloaded would use a reference to the static locations as well. That would require the "common" to be downloaded onces into the workspace and each project having a reference to that workspace. However, that still leaves about 15,900. By the way, not of those artifacts in the 3 projects are touching the common anyway.

I'm sorry about all the detail. Just wanted to do my best to explain what I'm dealing with and what I'm trying to avoid in the use with TFS. Again, I haven't really found any good information on any of the materials and documents that outlines a good way of dealing with something like this.

Any information, ideas, suggestions you have would be greatly appreciated.

Thanks!

New Post: Identify an artifact or Multiple Artifacts for a Project

davidkallen — Tue, 18 Jun 2013 00:00:13 GMT

How much disk space is consumed by the entire collection of 2,000 artifacts? It should be easy to obtain by looking at your development machine.

New Post: Identify an artifact or Multiple Artifacts for a Project

davismc — Mon, 17 Jun 2013 21:07:07 GMT

Hi Davidkallen,

So sorry about the delay.

Yes, I believe it's not taking much space on the server as I believe that is just the meta stored on there. However, you are correct and that's what I'm getting at. If a developer is getting out the latest on a branch they are getting everything and probably 99.9% of that they will not even use. If they are working simultaneously on 2 branched projects and they need to keep the branches isolated they would then need to get the latest on the other project as well which means they would get all that yet again and 99.9% of those they would not use either. So, yes, you are correct in that it's not taking much space on the server but rather on the local copy.

I see what you're saying on if the applications are independent and all making each of them a trunk (or main). That would be 1 TKC per application would it not or all in the same collection just a different heirachy structure per application? However, the release would entail everything for each of those trunk's.

I am looking through several different documentation sets and PDF's. All great information but little to point a finger at and say I really need to look at this and take a deep-dive into.

That's sort of why I am posting the question.

Thanks,

New Post: Identify an artifact or Multiple Artifacts for a Project

davidkallen — Wed, 05 Jun 2013 00:44:30 GMT

First, it's important to understand the facts about TFS branching and disk storage. If you have a trunk (MAIN) with 2,000 artifacts, and you branch it all, very little extra space is used on the server. However, when a developer gets latest on that branch, they will have a local copy. So their local disk space gets used. But that is not normally a problem. It is only if you change some of the files that space is made for the changed copy. The algorithm used varies depending on the change.

If the 2,000 artifacts represent several different applications, and they are independent, then you might make each of them a trunk and branch off of them. That reduces local disk use since you only need to download the project you are working on.

I would encourage you to skim the entire branching guide that is available in the downloads section of this project. It has some great information that will help you see alternatives. When you see something of interest, you can slow down and study it carefully.

New Post: Identify an artifact or Multiple Artifacts for a Project

davismc — Tue, 04 Jun 2013 14:04:04 GMT

Hi all,

In TFS I understanding there is the branching and the labeling but it also seems with TFS that it's too much information or risky, respectively.

Assuming we have a multiple applications comprised in 1 team foundation collection (TKC) and they total comprise 2000 artifacts:

If we have a project that let's say only changes 6 of those 2000 artifacts and 2 developers working them:

Creating a branch works BUT it branches off the main and thereby makes a copy of the 2000 artifacts so the 6 can be changed and isolated. That would utilize a lof of unnecessary space for changing just 6 artifacts BUT it would work.
Creating a label works BUT labels are free-form text and meaning each developer could label it with a different name.

A code-promotion type branching model sounds reasonable off the Main to QA and release for stabilization but what is the best method to isolate and identify the changes to those 6 artifacts in this example?

If there was a way to promote and identify without a label (due to the risk) that is ideal but I am not seeing it with TFS nor did I with VSS.

Any assistance and inforamtion on this would be greatly appreciated.

New Post: Basic plan confusion about number of release branches and dev branches?

davidkallen — Fri, 10 May 2013 01:43:05 GMT

There is a way to reparent an existing branch to an integration branch, but to do what you are asking is very tricky. The built-in reparent feature is not intended for this scenario. You may not like this solution much. If you fear this scenario, then make your integration branch at the beginning.

So I tried an experiment. You can experiment too. Here is what I did:

I Created Dev1, Dev2, Dev3 off of Main.

Then I "decided" to create an integration branch off of Main.

I then created Dev1new and Dev3new off of Main.

I then checked out Dev1, Dev3, Dev1new, Dev3new.

I then copied the old Dev1 folder over Dev1new.

I then copied the old Dev3 folder over Dev3new.

I then checked in the Dev1new and Dev3new.

I then undid checkout on Dev1 and Dev3.

I then deleted Dev1 and Dev3 as I no longer needed them.

In summary,

I created the new branching structure I desired.

I checked out every project involved in the reorganization (both old and new

Using windows explorer, I then copied the old project files over the new ones.

I verified it all worked.

I checked in the new projects (that are now off the Integration branch).

I undid checkout on the old projects as they were the source and did not get altered. The only reason I checked them out in the first place was to remove the read-only locks on them so that when they were dropped over checked out files, they would be in the same state as the files they replaced.

As you can see, this is possible. I figured did the whole demo in 15. But it is not pretty. It is so complex that there is a risk you will make a mistake. You can also use baseless merges instead, but I am unfamiliar with them in practice and cannot give any advice on their value.

And lets keep this in perspective. You would do all this just to reduce the risk that your Dev1 and Dev3 teams would not step on each other when they merge their changes in the common code module? The safer thing is to have a single sub-team responsible for changing the common sub-module on behalf of both other teams. Or, you could leave Dev1, Dev2, and Dev3 branches as they are. Then, the first one(let's assume the Dev1 team) who checks in their tested code has it easy. The second team (Dev2) is responsible for getting the latest version, merging from Main to Dev2, then ensuring all tests pass before checking it in.

If the separate projects are so different or mutually dependent on one another that they really NEED an integration branch, this should be obvious from the very beginning. And thus, you can design an integration branch from the start.

Sorry for the complicated answer. But branching cannot magically solve all problems of development. All the pieces are interrelated - the branching approach, the environments available, the way the teams are organized, the way the work is organized, the nature of the changes made, etc...

New Post: Rolling Release as a component of a Periodic Release

petescott — Thu, 09 May 2013 14:37:53 GMT

First, I'd like to thank you for the excellent documentation. It's proved invaluable lately.

We're responsible for deploying a web application that contains customizations (via a single compiled DLL). That is, many of our customers have very unique business rules which they cannot change, but which we do not wish to directly support in our product due to the added complexity and/or the limited marketability.

Our web application is released about every two months (periodic release), but these customizations are developed, tested, deployed, fixed, tested again, redeployed (etc) as often as ten in one day.

The branch structure we're using is (somewhat simplified):
/Main
-/Source
--/BusinessObjectLayer
--/WebApplication
--/Customizations

We're currently using the standard branching plan in this fashion:
Feature Dev (opt) <- Dev <- Main -> Release
(we will likely add the Service Pack strategy soon)

Each of those branches is a branch of the Main/Source, so each branch contains its own copy of the Customizations project. That means Release1 has its own copy of Customizations, Release2 has its own copy, &c.

This structure is causing some issues.

One issue is that, if a customer is on Release1 and discovers an issue with their customization code, developers are modifying that code IN THE RELEASE1 BRANCH. This isn't a huge deal, since we're a small team and they're responsible enough to not modify anything outside of that project, but I feel that it is only a matter of time before somebody slips up. Note that the Customizations project does NOT undergo QA.

Another issue is the merging problems. If a developer adds a new class file (customizations for our clients are stored in a unique namespace, classes separate the customizable bits by feature) to Release1, that results in a change to Customizations.csproj in that release. If another developer adds a new class file to the Customizations project in Release2, we now have multiple .csproj edits... often further confounded by some of the developers leaving the .csproj checked out for too long. When merging these changes (RI to main, then FI to latest release) Visual Studio often wants to delete items that shouldn't be deleted (I'm not at all sure what the root cause of that is). I think that disabling shared checkout on .csproj might alleviate much of this issue.

Finally, we occasionally make edits (signature changes, etc) to the Customizations project and related objects in the other areas. That essentially ties the Customizations to a minimum version of the entire solution (I'm not certain that makes sense, please let me know if additional clarification is needed). This is not a problem currently (since each branch contains its own copy of the Customizations project), but will be important to consider when suggesting different approaches.

Any thoughts on structuring this differently? Or could this be handled more easily with a bit of developer training and TFS configuration?

[edited for spelling/grammar typos]

New Post: Basic plan confusion about number of release branches and dev branches?

tandars — Thu, 09 May 2013 14:31:41 GMT

Ah, it's all starting to make a lot of sense.

So if I have 3 DEV branches, and initially, I thought they would be independent from each other, but as development progresses, I notice that DEV 1 and DEV 3 both had to heavily modify a common part of the code base, and I hadn't anticipated it, is it possible in TFS to create the integration branch afterwards? Or is it really something that I need to anticipate from the get go to be able to use?

Thank you again.

New Post: Basic plan confusion about number of release branches and dev branches?

davidkallen — Thu, 09 May 2013 02:39:55 GMT

1. Yes, the technique to "Branch by feature" can be added on to any of the plans.

2. It sounds like you understand. But there are some tricky spots. It is not merely the presence of multiple dev branches that motivates the use of an integration branch. It is the fact that they must each have their code integrated and tested before you have confidence they are stable. So if you have multiple dev branches but they are completely independent of one another, then there is not need to merge them to an integration branch first. By definition, the risk is low that one will break the other.

I'm not sure an integration branch makes much sense between Main and Release branches. Technically, you can do it. But release branches are used for a different purpose than development branches.

David

New Post: Basic plan confusion about number of release branches and dev branches?

tandars — Wed, 08 May 2013 16:47:39 GMT

Hi,

I read over the Basic Plan again, and I understand it a lot better now. I still have two more questions.

1) Is the Branch by Feature Plan an add-on to the Basic, Standard and Advance plan? It sounds like so too me. Since the basic plan says:

Work in DEV branches can be isolated by feature, organization, or temporary collaboration.

I see this as if you isolate your DEV branches by feature, that is, you create a dev branch for each feature, you are adding the Branch by Feature plan to the basic one, no?

2) From what I understand about the Integration line pattern, it should be used whenever you are going to be RI more than one branch back into Main. So in the case of the Basic plan where I would have more than one DEV branch, I should use the Integration Line pattern right? That means I should branch main into Integration, and Branch Integration into all of my DEV branches, am I still right?

Similarly, if I have many Release Branches, I should have an Integration branch from Main to Integration to All my Release branches?

Thank you very much for taking the time to answer me. I love this project, it is helping me a lot.

New Post: Basic plan confusion about number of release branches and dev branches?

davidkallen — Wed, 08 May 2013 00:19:17 GMT

My comments are based on the Friday, August 24, 2012 version of the Branching and Merging Guide.

I recommend you re-read page 40. Most of your questions are answered there. I pasted the first part below so you can see that it answers most of your questions. Notice that it describes several variants such as :one development area vs multiple development branches; or one release branch (reused) vs one per release. After you re-read this a couple of times. If you still have questions, let us know.

From Page 40
Basic Branch Plan
The basic branch plan with a main, dev, and release branch enables concurrent development for your next release, a stable MAIN branch for testing and a release branch for any ship blocking bug fixes.
Multiple development areas are supported by creating additional development branches from MAIN. These are peers to each other and children of MAIN.
Additional releases are supported by creating additional release branches for each product release. Each release branch is a child of MAIN and a peer to each other (e.g. release 2.0 branch is peer to release 3.0 and both are children of MAIN). If supporting only a single release in production at a time, you may consider a single release branch, and make bug fixes directly on this branch as depicted in the diagram below.
Once the release branch is created MAIN and the development branches can start taking changes approved for the next product release. An even simpler version of this plan for smaller teams might only include a MAIN and RELEASE branch. Consider this if the isolation required only required two branches such as for smaller teams and projects.

New Post: Basic plan confusion about number of release branches and dev branches?

tandars — Tue, 07 May 2013 19:45:54 GMT

I just read through the Guide, and I'm a bit confused about the Basic plan and the role of the integration branch.

Tell me if I understood it correctly:

You have a Main branch that will always contain the latest stable changes from everywhere. So initially, you have a Main branch that you branch out into a Dev branch. At that point, you're development of features happen in the Dev branch until it reaches a stable point, at which point you RI into the Main branch. You keep doing that until you have the features that match release 1. Once it does, you branch the Main, which will contain the stable latest features, into a Release 1 branch.

At that moment, you can continue developing new features in the dev branch, while stabilizing the Release 1 branch, until you feel it is ready to be delivered. All fixes made to the Release 1 branch must be RI into the Main branch, and FI from the Main branch into the Dev branch. Since Release 1 has been stabilized, you change it to read-only, and package your Release 1 from the Release 1 branch. This is your first deliverable.

All the meanwhile, work is being done in the Dev branch, and being RI into the Main branch every time new stable features are made. Similarly, once enough new stable features are RI into the Main branch to meet your Release 2 feature set, you branch out Main into a Release 2 branch, where it will be stabilized, and then made read-only once delivered to the clients.

Here I get a bit confused.

First, the basic plan talk about having 3 branch, Dev, Main and Release, yet it seems to me like basic will have a Dev, a Main and as many release branch as you have releases, am I correct to believe that here?

Second, what if 2 weeks later, we find more bugs inside Release 1. Where do I fix those bugs? Is that asking too much for Basic, to be able to fix bug in an already released branch? Is this where I am supposed to have a Service 1 branch which branches out into a locked Release 1 branch. This way, I can keep maintaining Service 1 branch, and I can keep releasing those fixes with new locked branches from Service 1 into Release 1.1, Release 1.2, etc. ? Or I'm wrong about having one release branch per release in Basic, and it's that the release branch actually never becomes read-only, we just keep fixing bugs over it. And once Release 2 comes out, instead of being branched form Main into Release 2, it's FI from Main into the Release branch?

This is my first confusion.

My second confusion is with the Dev branch and is kind of similar. Do I have only 1 dev branch? What if we are developing one risky feature, while also adding in small and frequent but not as risky features. Do we do it all on the Dev branch given the Basic plan? Or do we branch the Dev branch into a Feature 1 branch, where we can work on our big feature, while using the Dev branch for the smaller feature development. In this case, would the Dev branch be like our Integration branch?

Which lead to my last confusion. Is the Dev branch the Integration branch? If it says we should use an integration branch to integrate on, so that Main remains stable and functional, is that branch our Dev branch? If not, is my Dev supposed to be a child of the integration branch, and the integration branch a child of Main? Or are both the Integration and Dev branch a child of Main? Where does the Integration branch lies in the Basic plan?

I know this is a lot of questions at once, but I feel they all somewhat relates.

Thank you.

New Post: Code Promotion Branch Plan Limitations?

mlhoop — Tue, 30 Apr 2013 16:18:09 GMT

As with most things branching... "it depends"... and there are options. You could make the hotfix to the prod branch and get it out the door, then move it back to Test and Main (if they needed it). You could cut a branch from prod to address the specific "fixes". You have many paths. In general if this is happening all the time (every release) you might introduce a fixes branch from the beginning or choose a different model.

New Post: Code Promotion Branch Plan Limitations?

bdeem — Tue, 30 Apr 2013 16:13:44 GMT

I suppose I can answer my own question.

Looking at "Branching and Merging Guidance - Strategies" it's clear the it does not support a hot fix.

New Post: Code Promotion Branch Plan Limitations?

bdeem — Tue, 30 Apr 2013 16:10:32 GMT

I've been looking at the Code Promotion Branch Plan and I'm not clear as to how to apply a hot fix to a production release.

Main -> Test -> Prod

For example, if we are working on vNext in Main and QA we cannot apply the fix to those branches. If we apply the fix to the Production branch then are we forced to RI into QA and Main? What if we don't want to RI the change? We just entered another problem as we changed Prod branch directly without proper testing. I suppose we could use Shevesets, but that feels like a cruch.

What's the best way to implement this scenario or would hot fixes not be supported?