Using Java 8 Streams API with a C# mind

I did a coding dojo with my team the other day. It went quite well until the pair at the keyboard wanted to sum a list of integers with Java 8 streams. This should only take a few seconds, right? The pair got stuck for almost 5 minutes. The whole dojo group tried to help, even though we have the rule that the other dojo participants should be quiet. But everybody got nervous, especially the poor coworker at the keyboard. "Let's do it with a for loop", he said multiple times.

Most of us where experienced Java developers. How could we struggle with such a simple problem? I have to say that we all where not much experienced with Java 8 streams as we cannot use it much in our daily work. In fact our codebase uses Java 8 but the legacy code we work with makes heavy use of checked exceptions. This makes it hard to work with lambdas. As streams depend on lambdas, there is not much streams for us.

But back to our integer summing problem. Coming from .NET where I can write ...

 // C# version  
 products  
 .Select(p -> p.UnitsInStock)  
 .Sum();   

... I would have expected something similar in Java. Maybe something like this:

 products.stream()  
 .map(p -> p.unitsInStock)  
 .sum();   

But the code above doesn't compile, right? There is no sum method here.

It turns out that the reason why Java doesn't meet my (C# inluenced) intensions here are two fold. The first problem is that Java doesn't support generic lists for primitive types. As the streams API works only with generic lists we only can stream on boxed primitives. To be able to sum the integers we need to map from Integer to int first:

 products.stream()  
 .map(p -> p.unitsInStock)  
 .mapToInt(i -> i.intValue())  
 .sum();  

or shorter:

  products.stream()   
  .map(p -> p.unitsInStock)   
  .mapToInt(i -> i)   
  .sum();   

It's not too bad. Better than a for loop for sure. But I prefer the C# version shown above, as it is more compact.

The second thing that struggles me is that I often have to use a helper class like Collectors or Comparator when working with streams. I need to know that these classes exist, otherwise I'm stuck. In C# everything is accessible via the LINQ operators on the IEnumerable interface. This makes the API very easy to use.

But even though the stream API could be more intentional, I like using it. When I moved from .NET to Java two years ago I had to use Java 7. To have no lambdas and no LINQ alternative was terrible. Now that I'm on Java 8 with lambdas and streams I think I'm as productive as I was with C#. I'm sure I will soon get addicted to streams. 

GOTO Zurich 2013

Last week I attended the GOTO software conference in Zurich. It was intensive 2 days with a lot of good presentations about Agile, Polyglot Programming, Multi-Core Programming, Continuous Delivery and more. There was a good mix of international and local speakers. My favourite speakers where Dan North, Bruce Tate and Greg Young.

In his presentation "8 Lines of Code" Greg Young talked about simplicity in software programming. He showed us a code example which used an AOP style approach to implement cross-cutting concerns. He showed us the depenendencies and complexity of these few amount of code. Multiple times he raised the question: "Can you explain this to a junior?". He then showed how the code can be restructured to remove the dependencies. He ended up with a simple design with no dependencies to external code.
Simplicity was also the topic of Greg's keynote the day after. The keynote had the provocative title "Developers have a mental disorder". He went after our sacred cows like ORM, relational databases, design patterns, DRY and unit testing. He didn't say these patterns and practices are bad. His point was that they are a lot overused or used without thinking. Regarding DRY he mentioned the disorder that programmers try to apply DRY to the extreme also in unit test code. I can confirm this observation. I too often saw completly unreadable unit tests where every common code was factored out in helper classes or methods.

Bruce Tate gave two talks with insides out of his journey Seven Languages in Seven Weeks. It was a very fascinating tour of programming languages like Clojure, Haskell, Io, Prolog, Scala and Erlang. Eventhough this was all exciting I don't know how this big choice of languages can help me as a developer who develops most of the time business applications. In my opinion the "Right Tool for the Job" is a myth. In the last 10 years I cannot remember a single project where I could choose the language. It was always given by the organization I worked with or by the customer.

Besides the talks I very enjoyed to get in touch with other attendees and speakers. My boss and me had a quick chat with Greg Young which motivated us to dig deeper into the implementation details of the event sourcing pattern which we already use for persistence in a product we are developing. These special moments are at least as valuable as the content of the presentations itself.

Automating the creation of your installation packages

Windows Installer XML (WiX) is a great technology for creating installation packages for your .NET based applications. The tools from the WiX toolset do a great job if you have to create frequent releases of your application. Tools like heat.exe help you to create Wix fragments but you have to maintain those files manually by hand. If you practice continuous deployment this is not an option.

With Visual Studio and some small chuncks of msbuild it's still possible to automate the creation of your installation package. The solution to automate your setup is based on the following tools:

• Wix Setup project for Visual Studio 2010 (part of Wix 3.5)
• HeatDirectory MsBuild task (part of Wix 3.5)
• for web applications: Visual Studio Web Deployment 2010

I'm showing you how you do this with Visual Studio 2010 but it's also possible with Visual Studio 2008 (with the use of Visual Studio Web Deployment Projects). Even if the solution is based on Visual Studio there is no need to install Visual Studio on your build server.

First you have the add a new setup project to your Visual Studio solution. You could generate wix fragment files for every component with heat.exe and include them in your setup project. But that's not what we want here. Our goal is that the fragment files will be generated on every build. If we add new output files to a project in our solution, these files will then automaticly show up in our setup.

Fortunatly Wix comes with an msbuild task called HeatDirectory that helps us in accomplishing our goal. We can hook into the build process of the setup project and call this task for every output directory of our solution in the before build target. As an example if your solution contains a windows service project in a folder WindowsService you had to modify the setup project file as follows:

 <Target Name="BeforeBuild">
  <ItemGroup>
    <CrtHeatDir Include="..\WindowsService\bin\$(Configuration)" />
  </ItemGroup>
  <HeatDirectory OutputFile="$(ProjectDir)\ProductCrtDirRef.wxs" Directory="@(CrtHeatDir)" ComponentGroupName="CrtComponentGroup" DirectoryRefId="INSTALLDIR_CRT" AutogenerateGuids="true" PreprocessorVariable="var.CrtDirPath" SuppressRegistry="true" SuppressRootDirectory="true" ToolPath="$(WixToolPath)" NoLogo="true" />
</Target>

Notice that we pass a preprocessor variable CrtDirPath to the HeatDirectory task. This serves as the source path of the wix component files in our generated fragment file. We have to define this as a constant in our setup project file:

 <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
  <DefineConstants>Release;CtDirPath=$(SolutionDir)WinodwsService\bin\$(Configuration)</DefineConstants>
</PropertyGroup>

Creating output files for Web Application Projects

In our example we could create our component files based on the output directory of our windows service project. If you have a web application in your solution you have to make a little detour as there is no such output directory. With the help of Visual Studio 2010 Web Deployment we can create the output of your web project and use is it as input for heat. If you right click on a web application project in Visual Studio 2010 there is a command called "Build Deployment Package". This is the command that we want to integrate in our setup project. The msbuild target behind this command is called Package. We can call this target of the web application project within our setup project with a call to the task msbuild:

<Target Name="BeforeBuild">
  <ItemGroup>
    <WebProjects Include="..\WebServices\WebServices.csproj" />
    <SwsHeatDir Include="..\WebServices\obj\$(Configuration)\Package\PackageTmp" />
  </ItemGroup>
  <Msbuild Projects="@(WebProjects)" Targets="Package" Properties="Configuration=$(Configuration);Platform=AnyCPU">
  </Msbuild>
  <HeatDirectory OutputFile="$(ProjectDir)\ProductSwsDirRef.wxs" Directory="@(SwsHeatDir)" ComponentGroupName="SwsComponentGroup" DirectoryRefId="INSTALLDIR_SWS" AutogenerateGuids="true" PreprocessorVariable="var.SwsDirPath" SuppressRegistry="true" SuppressRootDirectory="true" ToolPath="$(WixToolPath)" NoLogo="true" />
</Target>

Package creates it's output in the web application's obj directory. The directory PackageTemp contains all the files that we have to include in our deployment.

The next step in automating our setup would be to include the build of our setup in a continuous integration or nightly build. During this build it is often practical to include the product version in the name of the msi and to copy it to a drop location where it can be downloaded by testers or even customers.

Even if WiX rather supports a manual creation strategy of deployment packages the toolset provides tools for automation. Thanks to MsBuild and Visual Studio you only have to do a little bit of additional work to automate the creation of your installation package.

Download sample source

MSF Agile goes SCRUM

My new company uses Microsoft Team Foundation 2008. The last time I had a look at Team Foundation and it's supporting processes from MSF was five years ago when I had to evaluate the product for my former company. It was 2005 and Microsoft created yet another agile software development process called MSF Agile. In an STSC CrossTalk article MSF Agile creator Randy Miller praised the "new innovative techniques" which the process introduced. I never undestood which new techniques he was talking about as it appeared to me that MSF Agile took advantage of practices from the agile community and gave them other names. Scenarios (aka user stories), scenario list (aka backlog), shadowing (aka incremental architecture), iteration based planning and time-boxing are all proven agile practices that even at that time existed since years.
Last week I was wondering what the new version of MSF Agile based on Team Foundation 2010 will look like. I was very surprised when I read the pre-release documentation of MSF for Agile Software Development v5.0 on msdn. MSF for Agile v5 is based on SCRUM and the engineering practices from XP. It now also uses the same terminology as our industry uses since years. So no more scenarios, scenario lists etc. I think this is a wise decision from Microsoft. We don't need another termonology for the same thing. It's sometimes hard enough that there is no common style of software development in our industry.

ASP.NET MVC - Thoughts and advices

It's almost one year ago when I got into web development with ASP.NET MVC. We prototyped with the betas, started using the release candidate to develop a new release of a product early 2009 and switched to final 1.0 when it was released. We went for MVC instead of web forms because we liked the simplicity of the web mvc concept. We also wanted increased testability of our code.

We never regretted our decision. Here are some thoughts, advices, pros and cons about ASP.NET MVC 1.0 after using it on a day by day basis for almost one year:

• The mvc pattern and the web is really a good duo. I was fascinated when I saw how Rails implemented the pattern with concepts like actions, filters and routes. Fortunately Microsoft didn't reinvent the wheel and implemented most of these proven concepts in ASP.NET MVC.
• ASP.NET MVC is a presentation layer framework for the web. It's not an application framework like RAILS. There is nothing like ActiveRecord, migrations or test fixtures in the framework. The positive thing is that you can choose whatever technologies you want to add. The negative thing is that you waste time in debates while choosing these technologies.
• ASP.NET MVC is not a very opinionated piece of software. It provides the hooks to add conventions and functionality, but doesn’t provide much in the way of structure itself. I couldn't find any advices or guidelines from Microsoft. More opinions are available from the guys from CodeBetter or in the Book ASP.NET MVC in Action
• Model binders and action filters are very helpful to keep controller actions clean. Use these guidelines for deciding when to use them:

Consider moving logic to an action filter when you start repeating yourself in your controller code.

Consider using built in model binding whenever you have to access the requests forms collection in a controller action.

Consider writing your own model binder if you can't bind your form data with the default model binder.


• Even thought you can do everything with HTML, CSS and JavaScript the cost of a nice and rich user interface in the browser are not to be sneezed at. In web forms you have a wide range of controls for almost every UI element. In ASP.NET MVC you have to build a lot of UI stuff (at least at time of this writing) by your own. The wonderful JQuery library and it's plugins provide basics like auto complete text boxes, tree views, calendars and popups. But you always have to write your own html helpers to integrate them into your views.
• The ASP.NET MVC team did a good job and built the framework with testability in mind. Calling controller actions from within a unit test and asserting the action's result is very easy - as long as you don't have real world code in your controller. Even if you do a good job in layering your application the presentation layer is the place where everything is put together. That means that you will mostly end up with many dependencies in your controllers and that's exactly the scenario where unit testing gets hard. Isolation frameworks like Typemock, Moq or Rhinomock can help you overcome these issues. But the use of these tools often lead to over specified tests.
• Lately we had to extend our product with an outlook addin with a reduced feature set in comparison of the main web client. We decided to implement the web services as RESTful services. ASP.NET MVC 1.0 has no built in RESTful web service support but with a little help from the REST For ASP.NET MVC SDK and the WCF REST Starter Kit it was very easy to build a web API with MVC.

In summary I can say developing web applications with ASP.NET MVC is fun. I'm looking forward for Version 2 which will ship with Visual Studio 2010.

Layered Architecture with LINQ to SQL (Part 2)

In my last post I mentioned two popular approaches for structuring your data access logic when choosing the domain model: the Active Record and the pure domain model way. In this post I want to explain how I would implement a data access Layer for LINQ to SQL for a pure domain model. There are two reasons why I prefer this to Active Record :

• I like the idea of persistence ignorance, clean ordinary classes where you focus on the business problem. I'm not too dogmatic about that. As an example I don't care about the LINQ To SQL mapping attributes I have to put in my domain classes.
• I don't want to run most of my unit tests against the database. The Repository pattern helps a lot in achieving this.

Repositories, Unit Of Work and Entities with LINQ to SQL

The central object of LINQ to SQL is the DataContext object. It tracks changes to all retrieved entities. It implements the Unit of Work and the Identity Map patterns and also provides query functionality on a per table basis. It's similar to NHibernate's Session object. Too bad that Microsoft didn't define an interface for this class (like the NHibernate team did it with the ISession interface). Such an interface is import to provide a stubbed implemenation during unit testing. So let's define our own interface and name it IDataContext:

public interface IDataContext: IDisposable
{
   void Commit();

   void DeleteOnSubmit<T>(T entity) where T: class;

   ChangeSet GetChanges();

   IQueryable<T> GetTable<T>() where T: class;

   IQueryable<T> GetTable<T>(Expression<Func<T, bool>> predicate) where T: class;

   void InsertOnSubmit<T>(T entity) where T: class;
}

The class that implements this interface is just an adapter for the DataContext class. The code is straight forward:

public class LinqToSqlDataContextAdapter: IDataContext
{
   private readonly DataContext _dataContext;
   private bool _disposed;

   public LinqToSqlDataContextAdapter(IDbConnectionConfiguration connectionConfiguration): this(new DataContext(connectionConfiguration.ConnectionString))
   {

   }

   protected LinqToSqlDataContextAdapter(DataContext dataContext)
   {
       _dataContext = dataContext;
   }

   public void Commit()
   {
       _dataContext.SubmitChanges();
   }

   public void DeleteOnSubmit<T>(T entity) where T: class
   {
       _dataContext.GetTable<T>().DeleteOnSubmit(entity);
   }
   //... more adapter code
}

Let's continue with the Repository Pattern. According to Fowler a Repository "provides a layer of abstraction over the mapping layer where query construction code is concentrated", to "minimize duplicate query logic". A Repository usually provides a set of query operations for an Entity. In addition to that objects can be added to and removed from the Repository. My interface for a generic Repository looks like this:

public interface IRepository<T> where T: IGuidIdentityPersistence
{
   void Add(T entity);

   long Count();

   long Count(Expression<Func<T, bool>> predicate);

   void Delete(T entity);

   bool Exists();

   bool Exists(Expression<Func<T, bool>> predicate);

   T FindFirst(Expression<Func<T, bool>> predicate);

   T Find(object id);

   IQueryable<T> FindAll();

   IQueryable<T> FindAll(Expression<Func<T, bool>> predicate);
}

I decided to use IQuerable instead of returning a collection as the return value of the FindAll methods. This makes the usage very flexible. IQuerable is a deferred query so clients can add filters as needed. For more specialized methods I think it's better to return a collection than a query.

The generic implementation of IRepository<T> goes here:

public class Repository<T>: IRepository<T> where T: class, IGuidIdentityPersistence
{
   private readonly IDataContext _dataContext;

   public Repository(IDataContext dataContext)
   {
       _dataContext = dataContext;
   }

   public Repository()
   {
       _dataContext = UnitOfWork.Current;
   }

   private IDataContext DataContext
   {
       get { return _dataContext; }
   }

   public void Add(T entity)
   {
       DataContext.InsertOnSubmit(entity);
   }

   public long Count()
   {
       return DataContext.GetTable<T>().Count();
   }

   public long Count(Expression<Func<T, bool>> predicate)
   {
       return DataContext.GetTable(predicate).Count();
   }

   public void Delete(T entity)
   {
       DataContext.DeleteOnSubmit(entity);
   }

   public bool Exists()
   {
       return DataContext.GetTable<T>().Count() > 0;
   }

   public bool Exists(Expression<Func<T, bool>> predicate)
   {
       return DataContext.GetTable(predicate).Count() > 0;
   }

   public T FindFirst(Expression<Func<T, bool>> predicate)
   {
       return FindAll(predicate).FirstOrDefault();
   }

   public T Find(object id)
   {
       return DataContext.GetTable<T>().Where(e => e.Id.Equals(id)).FirstOrDefault();
   }

   /// <summary>
   /// Returns a query for all object in the table for type T
   /// </summary>
   public IQueryable<T> FindAll()
   {
       return DataContext.GetTable<T>();
   }

   /// <summary>
   /// Returns a query for all object in the table for type T that macht the predicate
   /// </summary>
   public IQueryable<T> FindAll(Expression<Func<T, bool>> predicate)
   {
       return DataContext.GetTable<T>(predicate);
   }
}

I made the class concrete on propose. As the Repository class already defines a lot of helpful methods the class can be used in situations where you don't need a custom Repository. Below is an example where a generic Repository is used in a ASP.NET MVC Controller:

public class BookController: Controller
{
  private IRepository<Location> _locationRepository;

   public BookController(IRepository<Location> locationRepository)
   {
       _locationRepository = locationRepository;
   }

   //...
   [AcceptVerbs("Post")]
   public ActionResult Edit(Guid id, string title, string author, Guid locationId)
   {
       Book book = GetBook(id);
       UpdateModel(book, new string[] {"Title", "Author"});
       Location selectedLocation = _locationRepository.Find(locationId);
       book.AddLocation(selectedLocation, GetCurrentUserName());
       UnitOfWork.Current.Commit();
       return View("Show", book);
   }

A custom Repository could look like this:

    public class BookRepository: Repository<Book>, IBookRepository
 {
     public BookRepository(IDataContext dataContext)
         : base(dataContext)
     {}

     public BookRepository()
     {}

     public IEnumerable<Book> FindByTitle(string title)
     {
         return FindAll().Where(b => b.Title.Contains(title)).ToList();
     }
 }

The implemenation of the Repository pattern I showed in this post adds a layer of abstraction on top of LINQ to SQL and results in a more decoupled architecture. As a side effect the design simplifies database independent testing. The generic Repository provides easy and flexible usage for simpler situations.

Layered Architecture with LINQ to SQL (Part 1)

As with many other Microsoft technologies LINQ to SQL mainly supports a RAD (rapid application development) style of development. You can drag and drop tables into the LINQ to SQL Designer and Visual Studio will define a simple 1:1 mapping, generate domain classes and a typed datacontext for your database. All set, ready to hack! Let's create a win form, drag some controls in it, code some LINQ to SQL queries in a click event handler and bind it to a grid. Sounds easy right? Well it's easy and you can do it, but consider the following problems:

• Writing database queries in your presentation layer is not as bad as using SQL code in it, but it’s still a questionable practice. If you do it, you’re mixing data access code with business logic and presentation code. This means that you’ve decided to abandon the benefits of the separation of concerns.
• LINQ to SQL queries are scattered around in your business logic or presentation code. Sooner or later you run into a maintenance problem.
  
• There is no concept for code reuse. Queries that need to be used at different places have to be duplicated.
  

But that doesn't mean that LINQ to SQL is not suitable for building layered enterprise applications. There are a lot of resources available that can help you to structure your data access logic. Most of them are documented in Martin Fowler's Patterns of Enterprise Application Architecture (PEAA) book. If you choose the Domain Model approach there are mainly two options for your DAL:

• Active Record. Active Record puts data access logic directly in the domain object. In most implementations you can see a set of static finder methods on the domain objects and instance methods for operations like save, update and delete.
• Pure Domain Model with a Unit Of Work and Repositories. This approach separates the data access logic from the domain objects. With the combination of these patterns different objects exist for different concerns: the unit of work is responsible for change tracking, repositories for data access and domain object for domain logic.

In my next post I want to show you how I would apply the pure Domain Model patterns in the context of LINQ to SQL.

ThoughtWorks Cruise: First Impressions

We use CruiseControl.Net as our continuous integration server in our company. CruiseControl.NET is a great tool and helps us a lot in adopting our continuous integration practice. Over the years a lot of cruise control projects accumulated. Currently we have 8 build servers with all together over 100 projects. Most of the time some servers just do nothing while others are under heavy load. So we're looking for a solution to use our hardware more efficiently.

What we really need is a tool that can distribute builds to different machines. Fortunately most of the commercial continuous integration servers do support this kind of feature. Most of them use some sort of a server/agent based concept to distribute builds. Thoughtworks provides a good overview of several CI products that are available on the market. I decided to have a deeper look at Cruise, the enterprise version of Cruise Control from Thoughtworks.

So I downloaded the free edition of Cruise 1.1 and installed it on my laptop. Cruise consists of a server (Cruise Server) and several agents (Cruise agents), that receive work delegated from the server. Therefore a build can be processed in parallel by several cruise agents. What I really liked about Cruise is their implementation of the concept of the deployment pipeline. Cruise allows monitoring of changes to an application as they progress from initial check-in to functional testing, performance testing, user acceptance testing, staging, and release. A pipeline consists of one ore more stages which again consists of jobs. You can consider a pipeline configuration as a workflow of a particular release process. In that way Cruise goes beyond basic continuous integration that many products support these days.

What's interesting is that not every state transition has to be automatically triggered. Some stages may be manually approved before the release process can go further. For instance, in my current project most of the tests are accomplished manually by domain experts outside of the development team. During an iteration or at the end of an iteration we provide builds for manuall testing. In Cruise, this process could be implemented by a stage that needs manual approval.

When I hit Cruise's dashboard for the first time I was disappointed about the few features the product seemed to provide. The main page has just four tabs for viewing the current activity, managing pipelines and agents and a administration tab. Everything looked so minimalistic. But when I begun to use it, I was surprised that most of the things I wanted to do I could do.

But there are some negative points too. In the current version 1.1 they just support Subversion, Mercurial, Git and Perforce for source control integration. There are some other features that are missing in comparison to CruiseControl.NET. There is no support for visualizing reports from tools like FxCop or NCover, just NUnit is supported. They don't provide a plugin mechanism like Cruise Control. So there is currently no contribution or self extension possible.

To summarize, Cruise looks very promising for me. I really liked the concept of pipelines and the minimalistic user interface. From a CruiseControl.NET user perspective there are some key features missing in the current version 1.1. But I'm sure that the agile guys from Thoughtworks will soon come out with a new version that may include those missing features.

Are Aggregates Practical?

My coworker Jörg Jenni from GARAIO reflects in his recent post on how his team is implementing aggregates. I worked with him and his team for a few months so I know how their current implementation looks like. To be honest I was one of the developers that had problems to follow the rules that come with aggregates.

From Evans, the rules we need to enforce include:

• The root Entity has global identity and is responsible for checking invariants
• Root Entities have global identity. Entities inside the boundary have local identity, unique only within the Aggregate.
• Nothing outside the Aggregate boundary can hold a reference to anything inside, except to the root Entity.
• Objects within the Aggregate can hold references to other Aggregate roots.
• A delete operation must remove everything within the Aggregate boundary all at once
• When a change to any object within the Aggregate boundary is committed, all invariants of the whole Aggregate must be satisfied.
  

Especially the "do not hold a reference from the outside to an object inside the aggregate" rule is very strict. Jörg suggests in his post to implement aggregate internals as private classes in the root class. With this approach sure you will not violate the rule anymore as the compiler enforces it. The problem is that at some point you will need access to these objects even if they are identified as conceptually internal. For example as a client I need a reference to these internals to show some information on a screen, to test the logic of the internal object or to execute some business logic that needs knowledge of data and behavior of objects in the aggregate.

Don't understand me wrong, I really would love to apply this pattern. The essence of aggregates to identify a cluster of objects that can be conceptually thought of as one unit is very important for managing complexity. But is it practical in our today's programming model? On the domain driven design discussion board I found an interesting discussion about aggregate boundaries between DDD adopters and Eric Evans. From the discussion I see that we're not the only ones that have problems with the implementation of aggregates. And what's interesting is that in the same discussion Mr. Evans wrote that he'll try to add a concrete example and that he we'll come up with some useful new insight - he didn't return to the discussion.

Does anybody has a real world example of aggregates? One that enforces all the rules?

Let's Do Practices

This week Ivar Jacobson gave a talk at the Microsoft Regional Architect Forum in Zurich. Ivar Jacobson is one of the founders of UML, RUP and use cases. In his talk "Getting Good Software, Quickly and at Low Cost" he spoke about his practiced-based approach to software development. At the beginning he mentioned the main problems of software processes. According to him every process tries to be complete, process documentation is never read and the way how people live the process is soon out of sync with the original process definition. He then suggested a more practice oriented approach for finding the right process. He described what a practice is and how practices can be composed to build custom processes.

The ideas he talked about made a lot of sense to me. I was very surprised by the lightweightness of his approach. As an example he proposed to document a practice by a set of index cards on which you only describe the essentials of the practice.

He also talked about agile software development. For him Agile is "a box" of the good stuff that existed since many years with some social additions. He sees SCRUM as a practice to project management. I always saw SCRUM as a process. But I think his right. It's a good way to do project management in the field of software development. But with SCRUM alone you don't deliver good software. To be successful you'll need good engineering practices, testing practices and other practices for capturing requirements or releasing software. I focused very much on SCRUM for some time now. Together with my fellows we put a lot of effort in adopting SCRUM for the projects we where involved and also for the company wide adoption of SCRUM. Maybe its time for me, to address some other things. Back to good old engineering stuff. There are a lot of practices to explore over at Ivar Jacobson International and also from the Eclipse Process Framework.

Linq To Sql and Value Objects

Unfortunately Linq to SQL lacks support for Value Objects. This is a big limitation when you do Domain Driven Design as we do it on a project that I'm currently involved. We didn't use this pattern so far but last week there was no way around it. I had to implement a feature based on the stock of monetary units. I identified an object MonetaryUnitStock that represents the stock for a monetary unit like e.g. 10 Swiss francs. I didn't want to care about identity for that object and I wanted it to be immutable.

For my mentioned feature I had no requirements for querying. So the implementation with Linq to SQL was straight forward. Here is the code in C#.

public struct MonetaryUnitStock: IEquatable<MonetaryUnitStock>
    {
        private readonly int _numberOfUnit;
        private readonly MonetaryUnitType _type;

        public MonetaryUnitStock(MonetaryUnitType type, int numberOfUnit)
        {
            _type = type;
            _numberOfUnit = numberOfUnit;
        }

        public int NumberOfUnit
        {
            get { return _numberOfUnit; }
        }

        public MonetaryUnitType MonetaryUnitType
        {
            get { return _type; }
        }

        public bool Equals(MonetaryUnitStock other)
        {
            return MonetaryUnitType.Equals(other.MonetaryUnitType) && this.NumberOfUnit.Equals(other.NumberOfUnit);
        }

        public override bool Equals(object obj)
        {
            if(obj is MonetaryUnitStock)
            {
                return Equals((MonetaryUnitStock)obj);
            }
            return false;
        }


        public override int GetHashCode()
        {
            return MonetaryUnitType.GetHashCode() ^ NumberOfUnit.GetHashCode();
        }
    }

    partial class MoneyList
   {
       private MonetaryUnitStock? _monetaryUnitStockUnit100;

       public MonetaryUnitStock MonetaryUnitStockUnit100
       {
           get
           {
               if (_monetaryUnitStockUnit100 == null)
               {
                   _monetaryUnitStockUnit100 = new MonetaryUnitStock(MonetaryUnitType.Unit100, NumberOfUnit100);
               }
               return _monetaryUnitStockUnit100.Value;
           }
           set
           {
               if(value.MonetaryUnitType != MonetaryUnitType.Unit100)
               {
                   throw new ArgumentException("MonetaryUnitStockUnit100 expects MonetaryUnitType.Unit100");
               }
               _monetaryUnitStockUnit100 = value;
               _NumberOfUnit100 = _monetaryUnitStockUnit100.Value.NumberOfUnit;
           }
       }
   }

I used the DBML designer for creating this simplified example. In the designer I marked the NumberOfUnit100 attribute as private to just provide the Value Object to the outside. So client just can set and get MonetaryUnitStockUnit100 and have no access to the underlying simple type.

The reason that I implemented the MonetaryUnitStockUnit100 Property with lazy loading is, that Linq to SQL provides no interception point when reconstituting an object from persistence. The OnCreated() method generated by the designer is not suitable as it is called before the actual values are set.

Using LINQ with Text Files

I'm currently reading the book LINQ in Action which I can highly recommend to all who want to get started with LINQ. In one of the first chapters there is a nice example on how you can use LINQ with text files (originally posted by Eric White). Rather than reading all lines of the file into memory and then query it, the example uses deferred execution with an extension method on the class StreamReader.

Inspired by this example I wrote a simple file reader that can read structured non-XML data from text files. As you can see in the Process method in the code below the text file is queried and processed line by line. Objects are created on the fly, as you loop through the results. This technique allows you to work even with huge files.

public class RecordReader{
       private Dictionary<string, IReaderStrategy> _strategies = new Dictionary<string, IReaderStrategy>();
       private const char Comment = '#';
       private int _typeFieldLength;

       public RecordReader(int typeFieldLength)
       {
           _typeFieldLength = typeFieldLength;
       }

       public IList Process(StreamReader input)
       {
           var result =
               from line in input.Lines()
               where !IsBlank(line)
               where !IsComment(line)
               select GetStrategy(line).Process(line);
           return result.ToList();
       }

       private IReaderStrategy GetStrategy(string line)
       {
           string typeCode = GetTypeCode(line);
           if(!_strategies.ContainsKey(typeCode))
           {
               throw new NoStrategyDefinedException("No strategy defined for code " + typeCode);
           }
           return _strategies[typeCode];
       }

       private static bool IsComment(string line)
       {
           return line[0] == Comment;
       }

       private static bool IsBlank(string line)
       {
           return string.IsNullOrEmpty(line);
       }

       private string GetTypeCode(string line)
       {
           return line.Substring(0, _typeFieldLength);
       }

       public void AddStrategy(IReaderStrategy arg)
       {
           _strategies[arg.Code] = arg;
       }
   }

You can download the full source code from here

Unit Testing Legacy Code

After my short HTML/CSS intermezzo I'm back to the business logic programming. Even though it was instructive to do web design and presentation logic I'm happy that I can dive into the lower layers now. With this change I can also switch to my preferred way of developing software, which means TDD.

Everyone who ever wanted to do TDD in a code base that was not designed for testability knows how hard it is. The hardest part is certainly to break dependencies so you can unit test a class or method in isolation. In my last project I was involved we had quite a large code base before we started to write unit tests. With the support of Typemock we could work around the untestable pieces and TDD got possible.

Unfortunately I can't use Typemock on my current project. So I have to get back to other techniques that I learned from "Working Effectively with Legacy Code" by Michael Feathers. Here are the techniques I use a lot:

• Subclass and Override Method. Create a subclass of the class and override the method to break the dependency.
  
• Break out Method Object. Perhaps you can't get a method under test. Create a new class with the method under test.
• Sometimes it's not possible to create an instance of an object because there are too many dependencies. Expose the method under test as a static method so it's easier to test.
• Expose private Method. To access a private method from your fixture subclass the class and expose the method.
  

Even though its a little bit more work in comparison of using Typemock I think the use of these techniques results in more readable test code.

The Agile Programmer With The Cowboy Hat

I recently got involved in a new project. It's not a project from my company. We just provide a few developers (including me) on a body leasing basis. We have to graphically redesign a web shop with some small changes in functionality. The process that we follow you could describe as a mix between waterfall and cowboy coding. The good thing is that we have a close collaboration with the product owner (yes, we have one) and a small self organizing team. But when I compare it to the project that I was involved before it's a totally different functioning.

In my last project we did scrum. Our level of scrum adoption was not too bad (7 yes answers in the nokia test). We had a good self organizing team with motivated people. We improved from sprint to sprint and ended as a very effective team.

Now, I realized once more how important practices like time-boxing, prioritized backlog and developer estimating are.

So what can you do as a an agile programmer if these key practices are missing?

Well, the good thing is that agile is not an all or nothing approach. You don't have to adopt all the practices at once. If you do that, changes are high that you will fail, especially when you have no people with agile experience in the team or support from an agile coach. As a developer your possibilities to change something are limited as well. But I think if you are an experienced agile developer and everything in your body resists against heading blindly for disaster you have to change something. Even if it's just the advocating of approved developer techniques. As I'm more a web designer than a programmer right now I can't to the last thing I mentioned. So I targeted the organizational level and introduced the backlog today .

We already had some work packages identified that where bundled into a gant diagram. This diagram was out of date from the first day of development and wasn't adjusted from that day on. So I translated this information into a prioritized product backlog and presented it to the product owner, the project manager and the team. The reaction was positive. The project manager was surprised when he saw how much work is still not done.

It's not a revolutionary change. But it's a small step towards improvement.

NUnit Test Templates for ReSharper

Every time when I create a new unit test in visual studio I bother me that I have to type the same 4 to 5 lines of code . I was too lazy to create a visual studio or resharper template for that. Fortunately not everybody is as lazy as me:

MbUnit and NUnit Test Templates for ReSharper

Scrumbreakfast

I just came across this blog where jp anounced a scrum breakfast on November 7, 2007 at Namics in Zurich.

Great stuff. Could be of interest to our Scrum Master.

Old School Book Reading

In my last post I wrote about reading Code to improve your software development skills. Beside of that I read blogs, news and E-Books to hang in there. Althought they are great resources for learning, I prefer to read old school hard-bound books that I can cuddle up with in my bed.

Here is my list of recommended reading ...

Architecture, Design & Patterns

• Patterns of Enterprise Application Architecture by Martin Fowler
• Code Complete by Steve McConnell
• Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development by Craig Larman
• Head First Design Patterns by Elisabeth Freeman et al.
• Working Effectively with Legacy Code by Michael Feathers
• Applying Domain-Driven Design and Patterns: With Examples in C# and .NET by Jimmy Nilsson
  

Agile Software Development

• Agile Software Development, Principles, Patterns, and Practices by Robert C. Marin
  
• Extreme Programming Explained: Embrace Change (2nd Edition) by Kent Beck
• Crystal Clear: A Human-Powered Methodology for Small Teams (The Agile Software Development Series) by Alistair Cockburn
• Agile Estimating and Planning by Mike Cohn
• Lean Software Development: An Agile Toolkit for Software Development Managers by Marry Poppendieck

.NET Programming

• Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries by Krzysztof Cwalina, Brad Abrams
• Test-Driven Development in Microsoft .NET by James W. Newkirk

C# Code Reading

I usually read books to learn more about software programming, design and architecture. As Grady Booch suggested in his post some time ago, studying real code is a good way to learn too. In his handbook about software architecture he has a reading list (authentication required) with code that is worth to study.

Unfortunaly there is not as much open source code in c# as in other languages but nevertheless here is my own reading list with elegant c# code:

• CruiseControl.NET
• DasBlog
• Castle Project
• Storyteller
• XUnit.net
• Neuparts Shipping (no real code, but high quality demo code with Evans Entities, Repositories and Factories)

For those of you that rather read books I will post shortly a reading list with my favorite programming literature.