Handling Content Changes Selectively Released To Production

Mar 8, 2011 at 8:56 PM

Hi All,

I have a scenario I'd like some best practice advice on.

We have a large public facing website.  We have 3 development teams working on the website, each in their own project/feature branch, merged from a Main (or as well call it Trunk).

This is all working well and apart from a project which does major re-organisation of the codebase the merge's are not to bad.  We only merge from a development branch to the Main when that project is the next to be released to production.

From Main we also have a Production branch, which represents what is currently in Production allowing us to perform hotfixes etc.

The issue we have is we also have a UI team which make daily changes to content, javascript, css etc on the website.

From our Production branch we have created a child branch called UI.

The UI team work on TFS workitems and check in their code against a workitem.  When a particular workitem is ready for production, all the files with all changesets associated with that workitem are released into production.

The problem we have is then merging these changesets back to the Production branch.  We fall into the scenario of cherry picking changesets and all the associated problems.  Especially when multiple changesets linked to the same workitem have the same file.

We cannot just select all UI branch changes and merge to production as not all of the inprogress work items actually have been released to production yet.

Does anyone have any thoughts or suggestions on how best we should handle this scenario?



Mar 17, 2011 at 10:03 PM
Edited Mar 17, 2011 at 10:39 PM

I can't give you best practice advice, but I can help move this conversation along a bit.

From the description it sounds like your problem area is the UI branch contains several short/tactical changes that are ready to release to Production at different times. How would you organize this if one of your feature (product) branches had the same situation (overlapping release changes in the same branch)?

Option #1: Separate "hotfix" UIChangeN child branches off Production

My first thought is to move away from having a monolithic "UI" branch for all UI changes. If you have several concurrent UI changes in progress that overlap (or have risk of overlapping) then each should have their own branch to remain isolated until it is ready for release. This looks very similar to having multiple short-lived "hotfix" branches. This is a mirror of the "Multi-Feature Teams Scenario" (from TFSBranchingGuideIII :-)) pattern you described for your project/feature branches. Only difference is UI "hotfix/livesite" branches are children of Production rather than Trunk.
Example: If UIChange1 and UIChange2 are branches off the Production branch then UIChange1 or UIChange2 could go to production without shipping parts of the other change.
               IDEA: Consider naming branch after Work Item number (WI12345, etc.)
----WI12345  (Where WI12345 is Work Item #12345)

PRO: Support parallel development on same source files. Also it mirrors pattern your teams are already familiar with for pre-release development (and consistency is a GOOD thing).
CON: Build automation, Forward Integration of any changes (from other hotfix releases), and other branch management is multiplied by number of concurrent fixes. Also short-lived branches typically have unique names so you can't re-use the same build definitions and workspace mappings

What if UIChange1 and UIChange2 have shared changes they both need before either can be released?
Ohh, that's tougher. You probably cannot integrate through the parent branch because you said parent is Production. Ideally you can stabilize, test,  and release the common parts first (either cherry-picked or preferrably from a third UIChangeCommon branch). Then Forward Integrate to all active UIChangeN branches to pick up the changes. Another option would be to do a cherry-picked baseless merge between the hotfixes (which sounds worse than the original problem you described :-(  ).

What about testing integration of UIChange1 and UIChange2 periodically before releasing either to Production?
That sounds like larger change than should be done directly off Production. At this point I think these branches may belong parented to Trunk along with the longer running Project/feature branches.

Option #2: Nested UI branch for UIChange2 (if you know which change must release first)

If you know which UI change will release first then you could start work on UIChange2 in a child branch of your existing UI branch. After first change is merged to Production and confirmed good the changes for UIChange2 can be merged to UI (opening child for future changes that can't go until unblocked). This is effectively creating a NEXT branch.
Example: Change1 started in UI branch. Change2 requires changing some of the same files as Change1 but can't wait for Change1 to finish, so Change2 starts work in UI-Next branch

PRO: Change2 doesn't get partially released with Change1.
CON: Assumes you know which UI change will release first. Different from "whoever's ready next" pattern used for Trunk. Another nested level to regularly merge (merge tax).
Note these changes are not going through trunk so require more merges to get these UI changes RI'd to Trunk then FI merged to all your feature dev branches.

Option #3: Move UIChange* branches to be children of Main (Trunk) branch

If you have parallel UI development that requires integration testing AND your trunk branch does not get locked or have pre-release artifacts hanging around then you can develop UI changes just like other features.

What if another project/feature is merged in Trunk but not yet merged to Production branch for release?
Time to look back at Option #1 or Option #2.  If you let pre-release bits sit in Trunk (for UAT or other testing) then Trunk is effectively locked for hotfixes until the pending release is pushed to Production. Otherwise you might pick up unintended changes while merging through Trunk. A locked Trunk branch is not as critical if you have hotfix branch(es) off Production. One mitigation is to always have Trunk in a shipable state. This is an ideal, but often less realistic unless you run builds and test pass on bits built in Trunk in addition to source built in Production (double-testing).

You might need a combination of (#1 or #2) and #3.

In all cases you need to be very careful that all RI merges to Trunk happen AFTER an RI merge has been done from Production to Trunk plus a FI merge from Trunk to feature/product branch. Otherwise you risk regressions caused by overwriting or testing without the changes already in Production. Option #3 is only pattern where the Trunk branch is really the single release vehicle. Merging any changes from current UI branch (or any "UI#" branch) to Production requires RI-only

Good luck Rick. I Hope you or others provide some answers that do better than what I've listed above.

Cheers! -Zephan