Wednesday, September 19, 2012

BASTA! 2012 Slides on Agile Architecture

This week I had the great honor and pleasure to speak at the german BASTA! 2012 conference on .NET, Visual Studio and more ( I gave a talk about software architecture in an agile environment or to be more precise the integration of the agile architecture lifecycle into Scrum by defining a separate role to manage the systems architecture.
With the link below you may download the session slides (most of the slides are in german).
For detailed information about the additional role called Architecture Keeper you may also read my previous blog post.

Software Architecture in an agile Context

In agile process models like Scrum there is no role defined for an architect. There’s just the team (besides the Scrum Master and the Product Owner of course) and every team member shall be able to fulfill the different roles as needed. That means that even the system or software architecture is done by the whole team. This works great as long as the team is able to find a consensus on all problems arising from the requirements and concerning the software architecture. But what if the team cannot find a solution or someone disagrees on proposals made by other team members? This blog post introduces the concept of an architecture keeper and tries to define that role with all its rights and responsibilities.

In general it is a very good idea to let the software architecture emerge from the team itself. One of the twelve principles behind the agile manifesto states: “the best architectures, requirements and designs emerge from self-organizing teams”. But why is this a good idea? Think of your first little program you implemented completely for yourself. Weren’t you proud of it? Are you not still proud of ideas you have and solutions you build within complex environments? When everyone in a team is able to contribute to the overall software architecture they have a more positive attitude towards it. Chances are much greater the architecture gets accepted by all developers of the team and they will have more fun using it during development. Furthermore they will more likely stick to the initial architecture and do not want to break boundaries. They will also defend the architecture against others because it’s kind of their baby. So, in general it’s a good idea to have a “team architecture”. But what about the problem mentioned above? What if the team cannot reach a consensus on the architecture? For this case there needs to be an additional role that was not originally defined by Scrum. There are other authors who call that role Architecture Owner. I will not use this term because I think it implies that there is someone who owns the architecture. But as with agile methods were the code is owned by the team rather than a single developer the architecture also belongs to the whole team. So there is just no owner of the architecture. To better reflect the role I’d like to propose the term “Architecture Keeper”. The Keeper takes care of the software architecture and manages all the processes and meetings concerning the architecture. In order to describe the role there are three questions that need to be answered:
1.       Which skills constitute a good Keeper?
2.       How to become an Architecture Keeper?
3.       What are the rights and responsibilities of a Keeper?
Let’s start with the first question. What are the skills you need to bring along to be a good Architecture Keeper. The following listing shows an excerpt of the skills that are crucial for every software architect. Not only the ones who are acting in an agile environment.
Make decisions, document and defend them
It’s important to make decisions but it’s also not easy. As an architect you have to make decisions without knowing all the environmental influences to your system. So some of your decisions may not lead to the optimal solution but that’s okay. Philippe Kruchten said: “The life of a software architect is a long and rapid succession of suboptimal design decisions taken partly in the dark.”  Document and defend your decisions. Architectural decisions mostly have a great impact on a system. These are decisions that cannot easily be undone or reverted. So you shouldn’t change your mind every day. Stick to your decisions, substantiate and explain them.
Review and present your architecture
A good architect will always review his own architectural decisions. A decision that well suited the requirements today may not be valid tomorrow when other requirements come into play. So always take a step back and reflect your architecture and the decisions you made.
Present your architecture. Explain it to the team, to stakeholders and to whoever is interested in it or just affected by it. Choose the correct level of detail and complexity for your presentations according to the audience.
Know the technologies and best practices
As an architect you need to know the technologies, libraries, frameworks and best practices you are working with. It’s not enough to be able to draw nice diagrams with a sophisticated architecture modeling tool. You need to make decisions for the use of certain technologies in your project. So you need to know their advantages and disadvantages. You need to know how to use the technologies and their pitfalls. It helps you in making better or at least not so bad decisions.
Talk to the team, to experts and stakeholders
Communicate! Don’t sit in your ivory tower drawing diagrams and providing them to the masses without comment. Explain your design to the team and demand feedback. Talk to experts if you’re not sure about a certain aspect of your design or a certain technology. Ask stakeholders about their expectations, their requirements and even their hopes and wishes for the new system. Take care of them and listen carefully to their needs.
Keep an eye on the project environment and its influences
Turn your focus not only to the project and the system itself. Look out for environmental changes, for changing influences and of course changing requirements. Try to anticipate them and plan for possible changings of the systems architecture.
For architects acting in an agile environment there are a few more requirements that need to be met:
Communicate even more
Nothing to explain here. Agile is all about people and interactions, so communication is the basis for a successful project.
Be part of the team
I think it is very important for a Scrum team that the Architecture Keeper is part of the team. This increases the feeling that the architecture itself belongs to the team. In addition it facilitates communication and it becomes easier for the keeper to actually take care of the architecture because he can see how it is used and if everybody is still developing inside the defined boundaries.
Design for change
This one is not new. It’s the battle cry of all modern architects around the world. For agile environments sticking to that statement is fundamental. It doesn’t mean that you should design your system to handle any possible speculative requirement. This will lead to complex and over-engineered architectures. Instead it requires the careful handling of dependencies by appropriate separation and modularization of the systems components. So choose an agile architecture and appropriate patterns to achieve the needed level of isolation and try to build your system as flexible and extendable as possible.
Make decisions as late as possible
In an agile process model it is crucial for the success of the project that you are able to defer architecture decisions as much as possible. The more degrees of freedom remain the better you are able to respond to changing requirements or environmental influences. In order to defer decisions you need to find the last responsible moment (LRM). That is the moment where you finally have to make a certain decision because it can no longer be deferred. Communicate these moments to others and explain them why you don’t make the decision right now.
Let’s have a look at the second question: How to become an Architecture Keeper? Scrum Masters can do a certification and can then take a corresponding position in the Scrum team. But what is someone supposed to do to become an Architecture Keeper. Well, I think there are two possibilities besides that you need to bring along the skills mentioned beforehand. First the Keeper may be elected by the team. This is the better approach because it increases the acceptance of the Keeper and her decisions later in the project. This approach works only if the team knows each other or is at least able to estimate who may be suited for that role. When this is not the case the Keeper needs to be appointed by the responsible authority which is the second possibility. This holds the danger that the Keeper and her decisions will not be accepted by the team. A careful selection process is needed in order to select the best person for the job.
The third question is an interesting one. What are the rights and responsibilities of a Keeper? The Keeper may act within the team or even outside of the team.
Within the team the keeper makes decisions concerning the systems architecture if the team is not able to find a common solution to a given problem. This is also a possibility to prevent unnecessary discussions about possible scenarios or solutions that may otherwise take too long. The Keeper plans and conducts meetings that are relevant to architecture, for example modeling meetings at the beginning of a sprint or architecture review meetings at the end of a sprint. He acts as a moderator and supports the team with his technical expertise. He is also responsible for the selection and use of appropriate modeling tools and he takes care about documenting architecture decisions and the architecture itself.
In addition to the Scrum Master the Architecture Keeper is also allowed to act outside of the team. This applies especially for the presentation and defense of the architecture against stakeholders. The Keeper is the first contact for the Product Owner or Scrum Master concerning the feasibility of a certain feature regarding the systems architecture.
The following picture shows all the aspects that were mentioned in the post above concerning the architecture keeper. It shows the different roles of the keeper, her rights and responsibilities and what it takes to be an architecture keeper.

Thursday, August 9, 2012

C# Operator Overloading for Improved Code Readability

In this blog post I’m going to show you some basic stuff concerning operator overloading in C#. Operators can be very useful in situations where you want to write elegant and easy to understand code by using standard C# syntax. For this post I will use a simple API that creates Crystal Reports selection formulas out of a given DataSet and some operators.

Introduction: Crystal Syntax
In order to understand why operators perfectly fit this usage scenario we need to have a short glimpse at the Crystal syntax for selection formulas. Selection formulas enable you to filter the displayed data inside a Crystal Reports report according to the underlying data. Often you use a dataset and bind it to a report which in turn gets shown in the Crystal Reports Viewer. Formulas are given as pure strings, so if you want to use the programmatically you have to learn Crystal Syntax which is not really fun. Let’s have a look at the basic concepts:

  • Column References
References to Columns of the underlying data source are written as {TableName.ColumnName}. So if you have a dataset with a table called Customer and a Column called name, you would write {Customer.Name}.

  • Comparison Operators
You can use comparison operators in order to compare a column value to any given value. Beneath the standard comparison operators like ‘less than’ or ‘greater than’ or ‘equals’ you also have some string operators like ‘starts with’ or ‘like’. So you could write something like the following: {Customer.Name} startswith ‘A’.

  • Boolean Operators
Simple Expressions like the one shown above can be concatenated by using Boolean operators. Crystal Reports supports Not, And, Or, Xor, Eqv and Imp. Here’s an example: {Customer.Name} startsWith ‘A’ and {Customer.Age} < 18.

I think that’s enough for the moment to get started. If you want to see more operators or want to play around with selection formulas use the formula editor of the Crystal Reports designer in Visual Studio.

Crystal Syntax Converter
I don’t feel comfortable writing magic strings in code without getting them checked by some instance. Selection formulas can become very complex and error prone. So I searched for a solution to write down C# code and get the string generated out of it. My first approach was some kind of fluent API, so I could write something like the following:


The result of this expression is equivalent to the Boolean Operators example above. But it’s much longer and more difficult to read. So I dropped that approach and turned to operator overloading. As stated at the beginning, operators are very powerful when used in appropriate situations. For our example we need to overload binary operators in order to get boolean operators and of course the comparison operators.

Let’s start with the very simplest expression. We want to use a column of our data source and compare it to any other value (e.g. a string or number). I start by writing a simple extension method, because I don’t want to write down DataSet.Table.Column every time and I need a custom object where I can define the overloaded operators. The custom class will be called CrystalColumn and the extension method looks like this:

public static CrystalColumn ToCrystalColumn(this DataColumn dc)
    var sb = new StringBuilder("{");

    return new CrystalColumn(sb.ToString());

The method simply takes in a DataColumn and creates a string representation of it according to the Crystal Syntax. So now we a are able to write:

var firstname = dataSet.Customer

The next step would be to overload the needed operators on the CrystalColumn class. Here’s the example of the ‘less than’ operator:

public static SingleExpression operator <(CrystalColumn column, int value)
    return new SingleExpression(column.ToString() + " < " + value);

The method needs to be static and the first parameter depicts the object on which the operator is defined. The provided objects are converted to the according string representation and passed to a new instance of the SingleExpression class. This class encapsulates a simple expression. With the code so far we can write the following:

var age = dataSet.Customer.AgeColumn.ToCrystalColumn();
var singleExpression = age < 18;

That feels much more natural and even looks better, doesn’t it? Here’s another example of the ‘starts with’ operator, which is of course no operator in the sense of C# operators but we need it for our expression:

public SingleExpression StartsWith(params string[] s)
    var startsWithExpression = "'" + s[0] + "'";

    if (s.Length > 1)
        startsWithExpression = "[";
        for (int i = 0; i < s.Length; i++)
            startsWithExpression += "'" + s[i] + "'";
            if (i < s.Length - 1)
                startsWithExpression += ",";
        startsWithExpression += "]";

    var expression = columnExpression + " startswith " + startsWithExpression;
    return new SingleExpression(expression);

The ’starts with’ operator in Crystal Reports acceppts an array of strings. That’s why the method acceppts a string array as parameter. Again a string represenation of the provided objects is created and passed to a new instance of the SingleExpression class. This method allows us to write:

var firstname = dataSet.Customer.FirstNameColumn.ToCrystalColumn();
var singleExpression = firstname.StartsWith("Mike", "John");

What’s missing now is the ability to combine two or more simple expressions by employing boolean operators to form complex expressions. In order to achieve this, we can now overload the needed binary operators on the SingleExpression class. For example the ‘And’ operator can be overloaded as follows:

public static Expression operator &(SingleExpression ex1, SingleExpression ex2)
    return ex1.And(ex2);

private Expression And(SingleExpression singleExpression)
    return new Expression(this,
singleExpression, LogicalOperator.and);

The operator overload takes a second SingleExpression instance and calls the private ‘And’ method. This method in turn creates a new Expression instance that encapsulates two or more simple expressions and finally generates the needed string representation. As another example I will show you the implementation of the ‘Not’ operator:

public static SingleExpression operator !(SingleExpression ex)
    return new SingleExpression(
"not (" + ex.ToString() + ")");

The method takes only one argument, because the ’Not’ operator is an unary operator acting on a single argument. According to this only a simple expression is returned.

Putting it all together
With the classes and method we have so far, we can now write our example expression using operators:

var firstname = dataSet.Customer.FirstNameColumn.ToCrystalColumn();
var age = dataSet.Customer.AgeColumn.ToCrystalColumn();

var expression = firstname.StartsWith("A", "B") & age < 18;

When you compare that statement to the first approach, I think it’s easy to see that the readability improved a lot. We can also compare it to the resulting formula in Crystal Syntax and see that it is as nearly as short as the formula:

{Customer.FirstName} startswith ['A','B'] and {Customer.Age} < 18

The Expression class overwrites its ToString method in order to write out the complete selection formula, so you can now easily pass it to the report viewer:

crystalViewer.SelectionFormula = expression.ToString();

This was only a short example of operator overloading in C# but I hope you noticed the great benefit of using them. If you need more information about C# operators just search the internet. It’s not a new concept. Some words of wisdom: Operators are not reasonable in every situation. Additionally when you use them be sure to use them the correct way and implement only operations that act the way a user would expect them to do. If you for example implement the ‘+’ operator for a custom number class, don’t return the difference. People are anticipating the results of common operations and get confused if something strange happens.

kick it on

Monday, June 25, 2012

Update: Unity with ASP.NET Web API 4 RC

Lately the Release Candidate of ASP.NET MVC 4 was released together with an updated version of the ASP.NET Web API. This blog post is an update of the previous post about a custom controller factory with Unity in Web API 4 Beta.

Changes from ASP.NET MVC 4 Beta to RC
“Independently control IHttpController selection and activation: Implement the IHttpControllerSelector to control IHttpController selection. Implement IHttpControllerActivator to control IHttpController activation. The IHttpControllerFactory abstraction has been removed.” – This is the official statement from the ASP.NET MVC site. That means that you no longer may use the IHttpControllerFactory. Instead you get fine grained control over controller selection and activation by using to separate interfaces.
The custom controller factory
For our controller factory we need to implement the IHttpControllerActivator. The interface has just a create method, used to create a certain controller:

public class UnityControllerFactory : IHttpControllerActivator
    private readonly IUnityContainer container;
    private readonly DefaultHttpControllerActivator defaultActivator; 

    public UnityControllerFactory(IUnityContainer container)
        this.container = container;
        defaultActivator = new DefaultHttpControllerActivator();

    public IHttpController Create(
        System.Net.Http.HttpRequestMessage request,
        HttpControllerDescriptor controllerDescriptor,
        Type controllerType)
            return container.Resolve(controllerType) as IHttpController;

        return defaultActivator.Create(request,
controllerDescriptor, controllerType);

As you can see there’s a reference to the Unity container. Inside the Create method we first check, if the controller was registered with Unity or not. If it is not registered we will use a default activator to create the controller, otherwise it will be resolved using Unity.
Initializing the controller factory
The custom Http controller factory needs to be registered with the Web API pipeline in order to use it for controller creation. The HttpConfiguration in ASP.NET Web API 4 RC does not contain a service resolver any more. Instead it contains a collection of services. The collection is a specialized one and is of type DefaultServices. This collection provides methods to get services or manipulate the collection. For example it has a replace method that can be used to replace a registered type in the collection with an own implementation. We can use this method to replace the standard controller activator with our own:
var container = new UnityContainer();
container.RegisterType(typeof (Controllers.CustomerController));
var factory = new UnityControllerFactory(container); 

                typeof (IHttpControllerActivator), factory);

That’s it. After registering your own factory this way it will be used for all requests to the Web API. There’s one disadvantage using this solution. When you create controllers with the Unity container you lose caching mechanisms that are built in the default Http controller activator. That means no controller objects are reused. You get a new instance on each API request. This disadvantage may be overcome by either configuring Unity to use Singleton objects or by implementing own caching machanisms.

ASP.NET Web API 4 Beta – Custom Unity Http Controller Factory

The ASP.NET Web API provides support for Restful services that make use of JSON and the concepts of ASP.NET MVC. Instead of the Controller base class in MVC the Web API uses the ApiController class. In order to use a dependency injection container like Unity for controller creation you need to create your own controller factory. This blog post shows how to build a custom Http controller factory together with the Unity application block.
A custom controller factory
Each controller factory in ASP.NET Web API must implement the IHttpControllerFactory interface. This again is different from MVC where you have to implement IControllerFactory. The IHttpControllerFactory provides just two methods – CreateController and ReleaseController. I think by reading the names it’s obvious what both methods should do. Together with Unity we will use these methods to create and release controllers with the help of the Unity container:
public class UnityControllerFactory : IHttpControllerFactory
    private readonly IUnityContainer container;
    private readonly IHttpControllerFactory defaultFactory;
    private readonly HttpConfiguration configuration;
    public UnityControllerFactory(IUnityContainer container,
HttpConfiguration configuration)
        this.configuration = configuration;
        this.container = container;
        this.defaultFactory = new DefaultHttpControllerFactory(configuration);

    public IHttpController CreateController(
        HttpControllerContext controllerContext, string controllerName)
        if (container.IsRegistered<IHttpController>(controllerName))
            var controller = container.Resolve<IHttpController>(controllerName);
            controllerContext.ControllerDescriptor =
                new HttpControllerDescriptor(
            controllerContext.Controller = controller;
            return controller;
        return defaultFactory.CreateController(controllerContext, controllerName);

    public void ReleaseController(IHttpController controller)
        if (container.IsRegistered(controller.GetType()))
Let’s have a look at the code. We hold a reference to the Unity container in order to resolve controller dependencies. We also define a default factory in case a controller is not registered with unity. The framework provides the DefaultHttpControllerFactory which is normally used to create API controller. Another reference we need is to the current Http configuration. The configuration is needed to create a proper descriptor for the created controller.
The CreateController method tries to find the controller with the given name inside the Unity container. If the controller is not registered with Unity the default factory is used to create an instance of the controller. If the controller is registered it is resolved by Unity and a proper controller descriptor is generated. The descriptor together with the Http configuration is needed to build up a correct Http context in which the controller is executed.
The ReleaseController method again checks if the controller is registered with Unity or not and calls the appropriate release method.
Initializing the controller factory
If you have the custom Http controller factory there’s only one last step to use it. You need to instantiate the Unity container inside the global.asax and pass it to the factory. Additionally you need to set the factory as the default factory for creating API controllers. Here’s the short code snippet:
protected void Application_Start()
    var container = new UnityContainer();
    //register the customer controller with Unity
    //Add additional controllers here 
    var factory = new UnityControllerFactory(container,
        typeof(IHttpControllerFactory), factory);
I think the code is more or less self-explanatory. By using the static Configuration property of the GlobalConfiguration  class you get access to the current Http configuration object, which is needed for the factory to build the controller descriptor. Pay attention to the controller name passed to the RegisterType method. This name should resemble the name of the controller used by API calls because that name is passed to the controller factory.

Monday, May 28, 2012

Resetting local Ports and Devices from .NET

Currently, I am working on C# applications that communicate with several external devices connected via USB ports. In rare cases the ports just stop working correctly, so we needed a programmatic approach to reset them. Doing this by using C# is not trivial. The solution we implemented uses the following components:
-          C#’s SerialPort class
-          WMI (Windows Management Instrumentation)
-          P/Invoke with calls to the Windows Setup API

Accessing a port using C#
Using a certain port with C# is rather simple. The .NET framework provides the SerialPort class as a wrapper to the underlying Win32 port API. You can simply create a new instance of the class by providing the name of an existing port. On the instance object you have methods to open or close the port or to read and write data from its underlying stream. It also provides events to notify any listeners when data or errors were received. Here is a small example on how to open a port by first checking if the given port name exists in the operating system:

public SerialPort OpenPort(string portName)
    if (!IsPortAvailable(portName))
        return null;

    var port = new SerialPort(portName);

    catch (UnauthorizedAccessException) { ... }
    catch (IOException) { ... }
    catch (ArgumentException) { ... }

private bool IsPortAvailable(string portName)
    // Retrieve the list of ports currently mounted by
    // the operating system (sorted by name)
    string[] ports = SerialPort.GetPortNames();
    if (ports != null && ports.Length > 0)
        return ports.Where(new Func<string, bool>((s) =>
            return s.Equals(portName,
        })).Count() == 1;
    return false;

In rare cases the Open method of the port threw an IO Exception in our applications. This situation was not reproducible but also not acceptable. We noticed that after deactivating the port in the device manager and reactivating it, everything was working fine again. So we searched for a solution to do exactly the same thing from code.

Enable/Disable Devices
First of all a function or set of functions was needed to disable and enable a certain port. This can not directly be done from C# and needs some P/Invoke calls to the Win32 API. Luckily others had similar problems and so I found a very good solution at It uses the device installation functions from the Win32 Setup API. All you need is to retrieve the class GUID for the device set and the instance Id for the specific device you want to disable or enable. Just have a look at the link for the code or download the accompanying code for this blog post.

Retrieving the Ports Instance Id
Last thing to do is to acquire the correct instance Id for our port. We need a method that takes in the port name and retrieves the corresponding instance Id. For the exact definition of an instance Id in windows terms have a look at In our case we’d like to use the Plug ‘n’ Play device Id that can also be seen in the properties window of a device inside the device manager. For this purpose we are going to use WMI. If you need information about WMI have a look at WMI provides the Win32_SerialPort class that can be used to iterate over all mounted ports of the operating system. Two properties of the Win32_SerialPort class are important for us: The DeviceID property which represents the port name and the PNPDeviceID which gives the Plug ‘n’ Play instance Id. Notice while this works perfectly for Plug 'n' Play devices it may not work for other kinds of devices.

ManagementObjectSearcher searcher =
    new ManagementObjectSearcher("select * from Win32_SerialPort");
foreach (ManagementObject port in searcher.Get())
    if (port["DeviceID"].ToString().Equals(portName))
        instanceId = port["PNPDeviceID"].ToString();

If we found the appropriate instance Id we can use it together with the Win32 Setup API to retrieve a device info set and the corresponding device info data. If a device info for the instance Id was found we can use its class GUID and the instance Id to disable and enable the device. The following method is used to reset a port with a given instance Id:

public static bool TryResetPortByInstanceId(string instanceId)
    SafeDeviceInfoSetHandle diSetHandle = null;
    if (!String.IsNullOrEmpty(instanceId))
            Guid[] guidArray = GetGuidFromName("Ports");

            //Get the handle to a device information set for all
            //devices matching classGuid that are present on the
            diSetHandle = NativeMethods.SetupDiGetClassDevs(
                ref guidArray[0],

            //Get the device information data for each matching device.
            DeviceInfoData[] diData = GetDeviceInfoData(diSetHandle);

            //Try to find the object with the same instance Id.
            foreach (var infoData in diData)
                var instanceIds =
                foreach (var id in instanceIds)
                    if (id.Equals(instanceId))
                        //disable port
                        SetDeviceEnabled(infoData.ClassGuid, id, false);
                        //wait some milliseconds
                        //enable port
                        SetDeviceEnabled(infoData.ClassGuid, id, true);
                        return true;
        catch (Exception)
            return false;
            if (diSetHandle != null)
                if (diSetHandle.IsClosed == false)
    return false;

With the code set up so far we can now easily reset a port in case we’re getting an IO Exception while trying to open the port. We just have to call the method inside our exception handler and try to open the port again. That solved our initial problem. Be aware that it may take some time to disable and re-enable the port. It may be a good idea to do it inside a separate thread if you’re working inside an application using a GUI.

Download the sample code file: PortHelper.cs