Tuesday, January 6, 2015

My favorite books: The Phoenix Project

Have you ever read a computer book that was a real page turner? Well, although I love to read, reading technical books is mostly not a real page turner event.
But the Phoenix Project is different! While working with the ALM Rangers on a new project, Sam Guckenheimer mentioned the Phoenix Project as a great book so I ordered my copy and couldn’t stop reading.

The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win

The Phoenix Project is a novel about a company that, if you work in the IT or in a company that uses IT (so practically everyone) will recognize.

It shows the struggles of Parts Unlimited to keep up with competition in a world that keeps changing. In the end, Parts Unlimited adopts a DevOps way of working and learns quite while implementing it.

What I found surprising about this book was how much I can relate to everything that’s happening. You almost feel the pain of another deployment going wrong or a manager trying to push his little side project to the top of the line.

And then you see how they turn things around and you start feeling better.

If you work in an IT company that resembles the way Parts Unlimited works, this book is a must read. Not only for yourself but for your manager and every other person in your company.

Now the interesting question is, how can you apply all the principles from this book while working with products like Visual Studio, Team Foundation Server and Azure but that’s for another blog Winking smile

Have you read the Phoenix Project? Did you like it? Or do you know of similar books that are worth reading? Please leave a comment!

Tuesday, November 25, 2014

What I learned at the Microsoft ALM Summit

Last week I had the privilege of visiting Barcelona. A beautiful city with some great places to visit, nice weather and really nice food. But that wasn’t the goal of my visit.
Microsoft organized the West European ALM Summit in Barcelona last week. The ALM Summit is a partner event where all ALM partners have a change to get the latest ALM news and meet there fellow colleagues.

So from breakfast with Sam Guckenheimer to drinks with Craig Kitterman, the ALM summit is a great place to meet people and learn directly from Microsoft.


The summit took two days. One day was focused on Cloud, the other day on Mobile. Which isn’t strange since Microsoft is a ‘Mobile first, Cloud first’ company.

Microsoft is really targeting the Cloud with its ALM innovations. From the new Cloud Deployment Projects to Dev/Test, ALM and Cloud go hand in hand as far as Microsoft is concerned.
Although I understand their goals, this sometimes means that new features have no value to customers who are solely on-premises. Of course Microsoft hopes that ALM Partners help customers move to Azure.

This is why Microsoft is pushing on using Azure for Dev/Test scenarios. And to be clear, Dev/Test on Azure is a great scenario. It’s cheap and flexible and with the Virtual Network support Azure offers, it’s also secure.

One particular area that was discussed are the MSDN benefits for Azure. Did you know that as a developer with an MSDN subscription you have free monthly credits for Azure? Depending on if you have a Professional, Premium or Ultimate subscription you get between $50 and $150 Azure credits a month that you can use to experiment with Azure and run small workloads.

In addition to Dev/Test scenarios, Visual Studio Online is also complimentary to your on-premises TFS environment. For example, Application Insights and Cloud Load Testing are features that you can use without storing any code in VSO but that really complement your on-premises TFS.


The second day was all about Mobile. What I found particularly interesting was the comparison between Cordova and Xamarin. For example, Microsoft said they used Cordova for the Connect() event app. This app is only used for one or two days and is mostly about displaying data. Cordova is a great solution for apps that don’t require a great performance or deep interaction with the device.

Especially with the new Cordova support for Visual Studio, Cordova is something you should
consider for ‘quick and dirty’ apps. Xamarin is on a whole other level. This is if you want to invest in a quality app that runs natively on a device.


What is totally clear is that MIcrosoft is focusing on DevOps with their ALM platform. This means that Microsoft invests in tooling that makes DevOps practices easier.

If you understand this focus it’s easier to understand why tooling like Smart Unit Tests is being released instead of investments in Coded UI.

Forrester gave a presentation where they showed that DevOps is required for succeeding in developing modern applications. For example, the most popular apps in the different app stores are all updated on a daily to weekly basis. This can’t be done without a good DevOps implementation.

So Microsofts vision is clear. They are really pushing on Mobile and Cloud and are underpinning this with their ALM tooling. Visual Studio 2015, Visual Studio Online and Azure will be the future of their investments and if we want to stay up to date as ALM consultants that’s where we should focus.

All in all, it where a nice couple of days and I’m looking forward to the summit next year! Are you looking into Cloud, Mobile and DevOps on the Microsoft stack? Please let me know!

Tuesday, November 4, 2014

My favorite books: Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation

Lately I often get the question: “Which books do you suggest?”. So since I have only have a finite number of key strokes left, I’m starting a list of books on my blog that I recommend.

The focus of the books I read are on ALM/DevOps and Microsoft technology in general. For me, reading is one of the best ways to learn. I use Twitter and blogs to stay up to date with new developments but if I really want to learn something, I love a good book.

So without further ado, here is a book I definitely recommend:

Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation

The fact that this book is in the Martin Fowler series should be a trigger for reading it. The Fowler series has some great books and this is no exception.

The book is all about moving a project to adopt Continuous Delivery. Focusing on the concept of a delivery pipeline it discusses the different phases in your pipeline and shows ways how to implement them.

The book does not specifically focus on Microsoft technology but the concepts discussed are technology agnostic so you can apply in whatever environment you’re working.

The book is not perfect. Some content is repeated quite often and I found the later chapters a little less interesting. However, if you are serious about ALM and DevOps, go get it now!

Did you read this book? Did you like it? Or do you have any other books that you recommend? Please leave a comment!

Tuesday, October 7, 2014

Optimizing Team Foundation Server Build Time

Fast feedback is important. Knowing you broke something a month ago or just a few minutes ago can make a huge difference.

More and more teams use a build server to compile the code and run a variety of checks on each check in. This is often referred to as the commit stage. One nice feature of TFS to make sure the quality of your code base is guarded, is a Gated Check-in. A Gated Check-means the whole commit stage is executed on some code a developer wants to check in. Only when the commit stage succeeds, the check in is allowed.

This way, you can be sure that the code on your server is always in a good state. An important characteristic of successful gated check-ins is that they are fast. If developers have to wait a long time, they will start checking in less regularly and find ways around your build server to share code with other developers.

How can you optimize the build time of your Gated check-in?

What’s acceptable?

An acceptable build time depends on your team. However, from experience you should aim for a Gated Check-in build of less then five minutes. Somewhere around one to two minutes would be perfect but that’s often hard to realize. It’s important to regularly check your build time and make sure you constantly optimize whenever necessary.

1. Check your hardware

The Team Foundation Server build architecture consists of two important parts:
  • Build Controllers
  • Build Agents
A Build Controller connect to Team Foundation Server. They monitor your build queue and contain the logic of executing the build workflow. The actual build runs on a Build Agent. This means that  Build Controllers and Build Agents have different resource needs.

A Controller is memory intensive. An Agent is both CPU and Disk intensive. When you do a default installation of a TFS Build Server, both the Agent and the Controller are installed on the same machine. This is an easy setup and requires less servers but if build speed becomes an issue, an easy solution is just to scale out your build hardware. Moving your agents to a different server then your controller is a first and easy step.

Instead of scaling out you can also scale up by using a machine with more CPU cores, more memory and faster disks. I was at a customer once where their Build took 90 minutes to run. Because they where constrained on their on-premises hardware, we moved the build server to Azure. By using a larger VM size on Azure the build time dropped to 15 minutes. That’s a very quick win.

2. What are you building?

When you create a new standard Build Definition in Visual Studio, you get a build that monitors all the code in your team project. This is easy to get started since you probably only have a single source tree with one solution file in it that contains your code.

This is defined in your Source Settings in your Build Definition file as you can see in the following image:


Here you see there are two work folders defined. One points to your source code and is marked as Active. The Cloaked working folder makes sure that your Drops folder (which is created by a build that copies the resulting binaries to the TFS server) is not monitored.

When the Build server starts a new Build it will get a fresh copy (in the following section you will see if this is incremental or from scratch) of all the source files beneath your work folder mapping.
If you start getting multiple branches or other folders that contain documentation or other non-essential stuff for your build, the build server will download all those files every time a build runs.

Modifying your Source Settings to only include the files you really need can speed up your build time quite substantially.

3. How are you getting your sources?

If you create a new Build Definition you will see the following setting in the Process tab:


What’s important to note is that the Clean workspace property is set to true. This means that every time a build runs, all the source files are first removed from the Build Server and then the whole source tree is downloaded.

By setting this value to false, you will do an incremental get, just as you would in Visual Studio. The Build Server will only download the files that have changed since the last build.

Now of course, this is a nice default. Having a clean slate every time you run a build makes sure you won’t have any artifacts left from previous builds.

But you know your code and your team best. You can experiment with setting this value to false and checking what it does to your build time.

4. Where are you getting your sources from?

Normally you connect your Build Server directly to Team Foundation Server. This means that when the Build Server needs to download a newer version of a file, it will go directly to the TFS server and download it from their.

By using a Team Foundation Server Proxy you can use an in-between cache that sits between your Build Server and the TFS Server. The proxy will cache source files and optimizes getting files from TFS. Especially when your Build Server and TFS Server are not co-located, a proxy can save you huge amounts of time by installing it nearby your Build Server (on a controller machine or on a separate server in the same location as your Build infrastructure).

See the following MSDN article for configuration details: Configure Team Foundation Build Service to Use Team Foundation Server Proxy

5. How are you building?

There is a huge change your Build Agent is running on a multi-core machine. MSBuild, which is used in the Build Template to do the actual compiling of your code, has an option to start using multiple cores and parallelizing your build process.

If you look at MSDN you will see the following documentation:
/m[:number] Specifies the maximum number of concurrent processes to use when building. If you don't include this switch, the default value is 1. If you include this switch without specifying a value, MSBuild will use up to the number of processors in the computer. For more information, see Building Multiple Projects in Parallel with MSBuild.
The following example instructs MSBuild to build using three MSBuild processes, which allows three projects to build at the same time:
msbuild myproject.proj /maxcpucount:3

By adding the /m option to your TFS Build you will start utilizing multiple cores to execute your build. You can add this option in your Process tab for the MSBuild arguments property. This is also a setting you have to test and make sure it works for you. Sometimes you will get Access Denied errors because multiple processes are trying to write to the same folder.

6. Building less

When you are dealing with a huge application that maybe exists of several sub applications, you can start splitting your application in distinct parts that can be build separately. By using NuGet and your own NuGet repository you can then have those subparts publish a precompiled and ready to go NuGet package. This package can then be used by other applications that depend on it.

This means that you won’t have to build all your source code every time you run a build. Instead you only build those parts that have changed and reuse the resulting NuGet packages in other parts of your build.

If you have a look at the NuGet documentation you will find some easy steps that allow you to create a package and setup your own NuGet server.

And these are my favorite steps for optimizing a TFS build. What are your favorite steps? Anything that I missed? Please leave a comment!