restless developer

Old dog new tricks?

You can teach an old dog new tricks.

I am on the process of going live with a rather interesting project. Yes in the process, the project will only be fully live in about a month. Don't ask!! Real life sometimes gets in the way of technology projects. Basically the project serves as an abstraction layer between two third party systems (later more external systems). Loads of integration which of course means loads of interesting problems. So I thought I would just make a note of a couple of things I (being the old dog) learned during the project. More for myself but it might prove usefull to someone else.

Unit tests will save you when you need it most

Yes I realise it is obvious, but once again this point becomes very real when you make a last minute change at the end of the day and suddenly tests start failing. So spend time writing those unit tests and do them properly. And keep on writing those tests even if you think the piece of functionality could never fail. It might never fail, but the functionality will change and without unit tests the change will become more difficult. Write unit tests for failure scenarios. Write a test to simulate a database failure. Write a test to test a network failure. Those scenarios will happen so be prepared.

Write end to end tests. Call it integration tests, call it acceptance test, call it whatever just write it and use it

You know those tests testing a scenario from start to finish in an environment as close as possible to the live environment in which the system under test will run. Your unit tests might all pass but it does not mean all the scenarios are working as expected once the system is running hot (I always wanted to say that!!). These tests will surface any integration issues between the modules in your system, integration with external systems and integration with the infrastructure around your system. These tests are also great to test failure scenarios. It goes without saying these tests need to be automated.

Distributed system fallacies are not fictitious

The distributed systems fallacies (http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing) we all have read and all agreed with is real! Shock! Horror! It is easy to read something and agree in principle to what you are reading. But when it happens in your system you are surprised. And then the light bulb goes on and all the reading you have done around distributed systems suddenly makes a lot of sense. So read those fallacies again and look at your code and decide and implement the functionality to negate the assumptions.

Stay close to someone with a good understanding of the business

This person is invaluable during the development process. They have the answer. They have the correct answer. They understand the business side of the project because they work in the business process every day. So use their knowledge to the advantage of the project, with their contribution the project has a huge chance of being a success.

Automate everything

Be the laziest developer you could possibly be or more politely put, automate those tedious jobs and spend your time focusing on the difficult (and fun) parts of the project. This includes automated builds, automated tests and automated deployment scripts. But it also includes simple things like scripts/macros/snippets to generate boiler plate code for you. If your unit tests all have the same structure (which I would imagine they do), why not use a script/macro/snippet to generate a new unit test class? It will take you about 10 minutes to set up and save you about 1 minute when writing the first unit test class. But when you have 500 unit test class that little script/macro/snippet will save you 500 minutes. That is 8 hours. That is a full day. The other great thing about automation scripts/macros/snippets are that they are not temperamental, they will always behave in the same manner. This property is great when the stress is on and you have sufficient trust (after you have spent time doing it properly) in your automation. The script/macro/snippet can be run and it will work and that leaves you to focus on other things.

Measure and monitor you system

How else will you know whether you system is stable and will survive the onslaught that will come in the live environment? How will you know if you system is surviving out there in the big bad live enviroment? Build operational functionality into your system from day one of the development. And keep adding monitoring information to the system as a whole through out the development process. It does not have to be complex but focus on operational information that will make the support of your system easier. It could even feed into your team's dashboard.

Log, log, log and just to be sure log a bit more

Use your logging framework to it's full potential and use the functionality to the benefit of the system, but more importantly to make the support of the system easier. This means logging useful information which will assist the support of the system. Don't log rubbish. Log answers to questions which the support developer will ask when troubleshooting the system. And don't be afraid to update and add more logging to the system. The extra logging is a feature of the system, maybe not a tangible feature, but if it makes the support of the system easier it's worth will become very clear.

If something can go wrong it will go wrong

This applies to both the system under your control and any system your system integrates with. So be prepared for it and take proactive steps to handle failures in your system and failures occuring in the external systems. This might sound obvious but how often does a system fail and no one is quite sure why the system failed? The inevitable hunt through log files can be a royal pain and often just proves the lack of useful logging. A way to be prepared for the failures is to make testing part of the development process and not leave the testing to the end of the project. So write those unit and integration tests and continue writing the tests. And test for failures in both your unit and integration tests.

As a whole the project has been hugely enjoyable and a lot of lessons were (re)learnt.

Automation on the web

Automate, automate and then automate some more. Any good developer will tell you automation is the key to a smooth development cycle. And it just lets you get on with more interesting things. But what about the web? Could some service automate certain tasks and make my life just a bit easier?

What is the big deal?

The big deal is the wasted time spent manually performing tasks which could be automated. Who likes to manually deploy a system constisting of 5 Windows services, a web application and 3 different database? On a Friday, after working a 70 hour week? If you like it, you should really get out more. So we automated the deployment process. But what about all the manual tasks we perform on the web just to keep up with current events and articles of interest to us.

What about automating the web?

This post is not about automation in the development process. This is more about automating the web, or more specificly using the web to work for us. Remember when you first discovered RSS feeds? Remember the sigh of relief, knowing you did not have to visit about 20 blogs and sites in the morning hoping for new content? You got yourself a RSS feed reader (e.g. Google Reader), you subscribed to your favourite sites, and magically every morning new content was delivered to a central place for you reading pleasure. I wonder if someone has measured the time saving brought about by RSS feed readers. I am sure it is huge. And most importantly an application/service helped us to automate something we used to do manually.

Then Twitter came along and we could drink from the fire hose that is Twitter, getting more information fed to us in a single place than ever before. I personally follow about 140 people on Twitter, and my feed is constantly full of new posts. Not all valuable, but that is Twitter. So Twitter helped to get information (valuable or not is beside the point) out to the general public really quickly and in succinct form.

So the web is working for us collecting information and making it available in a couple of central places in aggregated format. Or should we rather say the web enables people to make information available for other people's consumption?

This is all great but with the proliferation of hand held devices all theses services needs to stay in sync, and I need to remember what I have read where. Have I read this article from my RSS feed? Am I interested in this Twitter post? Where is that article about throwing Mentos in Coke bottles? Amber Case mentions some interesting points about our digital lives in her TED presentation called We are all cyborgs now. All this makes my brain hurt, and I don't want to remember all these things. So is there a service which will automate the collection of the information from your RSS Reader and Twitter?

Taking it to the next level

In short yes! I stumbled over the ifttt service a couple of weeks ago, and I was astonished at the simplicity and genius of the service. ifttt stands for If-This-Then-That. You can read more the service here. But in essence the service monitors certain events on the web for you, and then take defined actions based on the occurence. Aaahhh yes event driven development! An example would be; if I star (the this part) an article in Google Reader then post the article to InstaPaper (the then that part). Genius.

So what does that mean? I have configured the service to listen for events in my Google Reader and Twitter usage, and based on my actions the service will update InstaPaper with information that I have shown an interest in. So from all the sources I get information, it all ends up in Instapaper. I can then read the articles at my leasure on any device anywhere in the world. But the biggest benefit is that I don't need to remember to manually post articles or tweets to InstaPaper, I just need to star and favourite articles or tweets, and the articles end up in InstaPaper.

A long winded way to get to the ifttt service, but I can not recommend it enough.

ØMQ

Strange name, excellent guide, fast and it works! ØMQ has all these things and more.

What is ØMQ?

A socket library that acts as a concurrency framework, carrying messages across multiple transports allowing for various distribution patterns. Or in English; ØMQ allows a developer to send messages (strings) using sockets to other applications over TCP and other transports. The developer can connect the sockets in a N-to-N fashion using patterns like pub-sub, request-reply and others.
The guide written by Pieter Hintjens is probably one of the most interesting and engaging technical guides I have read. Right up there with Zed Shaw's Learn Python The Hard Way. The guide is very clear and the what, whys and wherefores of ØMQ are explained very well.

Simple pub-sub

Pub-sub is a messaging pattern where publishers publish messages without any prior knowledge of the interested subscribers (if any). Subscribers express interest in certain messages, and will only get messages satisfying the interest. Thus there is a decoupling of publishers and subscribers, which in turn allows for greater scalability. And it is just very cool. An analogy would be the giving of a donation to a charity. A donator (the publisher), donates (publishes) their donation (the message), the donation (the message) is distributed to the relevant beneficiaries (the subscribers). The beneficiaries don't have any idea where the donation originally came from, nor does the donator know to whom the donation was given.

Before we get started

• Get the latest version of ØMQ, and follow the instructions for your environment. I had to compile the C++ project specified to get the actual ØMQ implementation. Once the project was compiled I copied the clrzmq.dll file to the relevant bin folder.
• Get the latest version) of pyzmq. I used the installation from GitHub for Python 2.7
• The code can be downloaded from GitHub

Publisher

The first thing we are going to write is the publisher. Remember the publisher publishes messages. We are going to publish a message in the format "Message {n}", where n is a integer and indicates the message number. The publisher is not aware of any subscribers, it's purpose in life is to publish messages. Think mad man screaming on the corner of Oxford street.

The code above publishes the numbered messages over port 5556 on any host for the local machine via the TCP transport. Simple? I think so.

Subscriber

Now that we have messages being published, we need a subscriber to get the messages. Remember the subscriber subscribes to messages and receives messages in which it is interested.

As you can probably tell, the code above subscribes to any message on port 5556 (sound familiar?) for the host called localhost. The first 100 messages will be received, then the subscriber will stop. If you were wondering about the localhost in the address, the localhost is just the host name for the local machine. It seems you cannot subscribe to all the hosts on the machine, you have to specify a host.

Putting it all together

If you compile the solution, and start the two console applications together you should see the publisher starting to publish messages. This is done even if the subscriber is not running.
Once the subscriber is started, the messages are picked up by the subscriber. Once 100 messages are received, the subscriber stops and exits. The publisher will continue publishing messages.
If you start the subscriber first, the subscriber will wait for the first of the messages to arrive. When you start the publisher, the subscriber will start receiving messages. WAIT A MINUTE! The first message received by the subscriber is not "Message 0", but "Message 3" (on my machine at least). Why is this? Why not "Message 0"? From the guide it seems the initial communication between the sockets takes a bit of time, and during the delay in communication between the sockets, the publisher has already published messages. There are ways around this, but I have not read the guide that far!

Making it a bit more interesting

What if we add another subscriber? Or another 10? If you start another instance of the subscriber console application, you will see that both subscribers receive the same message.

Making things even more interesting

This set up is all fine and dandy, but what if we want to publish a message from one system to another system where the systems are implemented in different languages. Will it still work?! Of course, why else will I bring it up? So lets write a subscriber in Python.

If we start the Python subscriber, the two .NET subscribers and the .NET publisher, the messages will be received by all the subscriber. Now that is brilliant.
I realise the messages being sent between the publisher and subscribers are only small string messages, but by using something like Protocol Buffers, we could publish more interesting and complex messages.

As mentioned earlier, I have only started reading the guide, so this is just a Hello World application, with massive room for improvement, but it does show the potential for some very interesting uses of ØMQ.

Generic services with Castle Windsor and Topshelf

Would it not be brilliant if we could write Windows services without worrying about the infrastructure elements of the services? You know the boring bits that seems the same for every Windows service. Or even better; we can change the behaviour of a Windows service by just copying new assemblies into the service’s folder?

Some context

The current project I am working on requires quite a few Windows services running, mainly running scheduled jobs via Quartz.Net and reacting to messages on MSMQ queues via Masstransit. We use Topshelf to write the services, which gives us more time to focus on the actual functionality of the service, and not the infrastructure of controlling Windows services. In conjunction with Topshelf we use Castle Windsor as Inversion of Control container which makes the services flexible and easy to configure. So when we write a new service, we create a console application, put the (same) Topshelf configuration in each console application, change the Windsor Castle configuration to load the appropriate components we need. And there is the service.

The problem is that each service looks basically the same. Just with a different configuration. So what we need to do is abstract the differences between the services, and somehow auto magically change the behaviour of the service based on the abstraction.

What do we want to archive?

• We want a generic application that can be run as a console application or installed as a Windows Service.
• The generic application should be able to install components defining the behaviour of the application dynamically

Side step

Quite a few tools are mentioned and used in the post, so here is a short explanation of each of the tools

• Topshelf – A framework to help with the development of Windows Services, and serves as a host for Windows Services. Topshelf will help us write a generic application that can run as either a console application of Windows Service.
• Castle Windsor – Castle Windsor is best of breed, mature Inversion of Control container available for .NET and Silverlight. Castle Windsor will be used to dynamically inject functionality into the generic application.
• Quartz.NET – The tool is mentioned but not used in the post, but a very useful tool. Quartz.NET is a full-featured, open source job scheduling system that can be used from smallest apps to large scale enterprise systems.
• MassTransit – The tool is not used in the post, but it is a great tool to be aware of. MassTransit is a lean service bus implementation for building loosely coupled applications using the .NET framework.

First steps – Generic application

The application builds a Topshelf configuration, and asks Topshelf to host an instance of ServiceRunner. Topshelf takes care of the Windows Service infrastructure, while still allowing the application to run as a console application.

Next steps – Implement the behaviour of the application dynamically

Castle Windsor allows numerous ways of installing components into the container. We are going to use some XML configuration in conjunction with Windsor installer instances to configure the container. For more Castle Windsor goodness.

The XML configuration in the app.config will allow us to defined which installers to run when the container gets instantiated. This is done by defining the following in your app.config

For detailed configuration.

We now have a way of dynamically running Castle Windsor installers to change the behaviour of an application.

Action!

Implement a simple application to run inside the skeleton application

Create a class library project (not a console project!). This will be our application we want to run “inside” the skeleton application. Your application does not have to be limited to a single assembly.

The important part of the new application is to define what the skeleton application should install into the Castle Windsor container in order to get your application to work. For our example I want to start a Http Server (code from BitterCode) to serve web pages. To get the server to run I need to install:

• the startable facility. The facility is part of the Castle Windsor framework, allowing a developer to mark classes to be run automatically when the Windsor container gets instantiated. The marked classes will be stopped when the container is disposed of. This facility warrants a blog post of it’s own, but for this post just know the facility will start our LameHttpServer when the container is initiated, and stop the LameHttpServer when the application stops.
• install the Http Server

into the container. As the skeleton application is looking for instances of Windsor installers (IWindsorInstaller implementations), we write two installers to install the required components. The way these installers are written is irrelevant to the skeleton application. So use the very cool fluent registration from Castle Windsor.

Get the new application to work inside the skeleton application

This is a bit of a manual step, but it could be automated of course. Create a folder for the application, copy the binaries of the skeleton application and the new application to the folder. Update the Skeleton.Service.exe.config to tell the skeleton application which installers to run.

The following sequence of events occur to make the application run as expected

• The skeleton application instantiates a ServiceRunner instance
• ServiceRunner creates an instance of type IWindsorContainer using the skeleton.application.exe.config file
• The container reads the installers node
• Each installer defined is executed
• The application starts

Code

The code can be downloaded from https://github.com/rivethead/Public. To see the code in action, run Skeleton.Service.exe in the HttpServer folder, and go to http://localhost:8089/ in a browser.

Notes

• The Castle Windsor installer functionality is available from Windsor 2.5. I have used version 2.5.1.
• I am pretty sure you could use Binsor to achieve the same configuration, you would just need to change the way the container gets instantiated in the ServiceRunner.