Category Archives: Software

Blogging and Continuous Integration

I find I have a high barrier to writing – getting started is hard, if I write few things then I get obsessive about the quality of my writing.  I need to get back on the horse – and write a few short topics that can get me flowing.

I’m going to write about old topics – things that I talk about and teach on a daily basis, and get down in this journal the views that I hold right now.  I’m going to start with a favourite topic – Continuous Integration.

I’m a big advocate of Continuous Integration, but I regularly find myself having to explain the wider concept.  Lots of folk I find think of CI as the server itself (Hudson, CruiseControl etc.) which performs automated build processes and runs your automated test suite.  That’s a useful tool to achieve Continuous Integration, but it certainly doesn’t stop there.  I think of CI more as a holistic approach – one in which we seek accelerated feedback at any opportunity.

I’ll drill down into a few topics here, unfortunately all of these topics are linked together in some way, so hopefully this makes sense…

Disclaimer: I’m writing these entries to distill some of the conversations I have with customers on a regular basis – I hope it’ll help with my thinking and expression and provide some feedback.  None of this is original thought – it’s based on a few years of reading and listening to smart people.  I probably don’t remember where I heard a lot of this, so I won’t do a good job of attribution.

Setter injection sucks

I know it’s not trendy to re-hash old java programming discussions – I should be discussing some new amazing functional language, however I’m in a grumpy mood. I have a beef with what appears to be the dominant trend of using setter-based dependency injection. I’m an old-skool dependency inject-er – I started with picocontainer when Spring was unheard of (well at least by me). What I particularly like was the way it changed the way I designed my java code. (For the spring-kiddies, PicoContainer *only* supported constructor injection)

A common complaint I hear about constructor injection is “I don’t like having so many constructor arguments”. Yup this crossed my mind when I first came across it too – then it dawned on me: perhaps having too many dependencies injected into a class is a code smell? How many other classes should this class be collaborating with? Am I missing an abstraction in my model? Many times I’ve looked at a class and simply by looking at it’s constructor signature I’ve had a nagging doubt about it’s hygiene.

A big reason I prefer constructor injection is that I can ensure that when an object is constructed is ready to do it’s job. I do not have to concern myself with whether the class has been ‘wired’ correctly in configuration and that all dependencies have been met. If I introduce a new dependency, I can find all places in the code (and pretty quickly in configuration) where I’ve caused a breakage.

However it’s now 2009 and I’m still (occasionally) writing java, but pretty regularly I’m in a team that has settled on setter injection as a standard. It’s the default option in the Rod Johnson scriptures I guess. I also believe in most cases it’s better to be consistent than argue what some see as a fine point, so again I fall into line and am forced to use setter injection. But I do want to bitch about it.

How about this test setup code:

        WidgetGetter widgetGetter = new WidgetGetter();
        widgetGetter.setHttpClient(new HttpClient());
        WeeResourceFactory resourceFactory = new WeeResourceFactory();
        resourceFactory.setSomeParameter("the quick brown fox");
        FooRepository repo = new FooRepository();
        repo.setBarFactory(new BarFactory());

versus this?:

        WidgetGetter widgetGetter = new WidgetGetter(new HttpClient());
        WeeResourceFactory resourceFactory = 
            new WeeResourceFactory(widgetGetter, "the quick brown fox");
        FooRepository repo = new FooRepository(resourceFactory, new BarFactory());

I’m also into configuration in code rather than in XML, but I worry I’ll get burned at the stake if I bring THAT up again.

BarCamping (without the actual camping part)

I attended my first BarCamp today and I was really impressed. It was a very full day of interesting presentations by a lot of very smart people, the range of topics was really quite startling. I’m familiar with user groups (although I don’t go along to as many nights as I’d like) and they tend to be quite focussed, however today people talked about whatever their current interest is – from Perl 5.1 to hardware devices to MythTV. I was particularly impressed with the support and encouragement attendees gave to speakers. If you’ve never given a public presentation and you’re unsure of yourself today would have been an excellent opportunity.

Like at user group nights and conferences I’m awkwardly shy and awful at striking conversation so I missed talking to a lot of interesting people today. It’s a different mix of people than I’m used to (like the Ruby user group) with a slant towards freelancing and smaller projects which is so refreshing compared to the behemoth corporate beasts I’m more familiar with these days.

Favorite presentations today: Mark Ryall’s Intro to Scala (second time around), and Paul Fenwick on an Illustrated History of Failure (with sound effects).

I winged a short demo of the Rspec Story Framework which is my current shiny-toy-of-the-week. I might have been better placed to answer questions if I’d actually used it in the wild yet…

Don’t miss BarCamp Melbourne 2009.

Factory and Repository in the Domain

I’m a big fan of the book Domain Driven Design, and for some time I’ve been pushing the principles and patterns in my workplace. I’ve never managed to get ‘into’ the more theoretical parts of the book, and I chuckle each time one of my colleagues refers to the “contours of the domain”, however it’s up there with books that have caused a substantial shift in my thinking.

When I’m evangelising the use of Entities, Aggregates, Value Objects, and Repositories, one question that comes up regularly is “what’s the difference between a Repository and a plain old DAO?”. It’s a very good question with the right level of skepticism.

I really enjoyed reading Christian Bauer’s post “Repository Pattern vs. Transparent Persistence” – particularly the title which implies that there is a choice to be made between two approaches. In actual fact transparent persistence as implemented in Hibernate makes DDD and the use of Repositories extremely powerful. I’ll backtrack a little…

A lot of web application code that I find these days that was written with ‘best practice’ of just a few years ago follows a rough outline like this. DAO objects (or Data Mappers) provide find/save/update/delete operations methods for each object in your domain. The DAO is a basic abstraction around database persistence, often JDBC.

Business logic is usually either written directly within Transaction Scripts (recently I’ve become used to inheriting hundreds of lines of code in Struts actions… *sigh*) or within a basic Service Layer. The script or service is the only thing that can hold references to the DAOs, so of course you HAVE to put the business logic there – in most web apps there is a lot of creation and persistence of objects. No problem, we’ll just divide and conquer – from our scripts we’ll extract services, then we’ll divide our services into more services and delegate between them.

Because the application does a lot of operations requiring persistence, there’s a natural force preventing any business logic being moved to anywhere but scripts or service layers. So what are our domain object? Nothing but structures carrying data, with maybe some rudimentary logic that only affect local state. The client code asks the domain to provide it with data, and then the client code makes decisions based on that data.

The transaction script and service layer approach is simple to begin with, but as the application becomes more complex it leads to a bunch of problems, especially as developers try to avoid duplication (cut and paste is bad kiddies). Ever picked up a hierarchy of Struts actions seven layers deep with multiple Template Methods to allow overrides for different sub-classes? Bloody nightmare.

What I’ve achieved by applying the DDD patterns is the elimination of those transaction scripts and (most) services. The transaction script (e.g. struts action in a web application if you’re so inclined) is simply responsible for finding an appropriate entry-point into the domain, then telling that domain class to do some work.

The consequence is that business logic can be expressed more deeply in your domain, you are free to find the right abstractions that will allow you to make your code understandable, and to create small classes with single responsibilities.  This also means that the creation and persistence of objects will be deep in the bowels of the domain, and even better – without the calling client code or script HAVING to be aware of the object creation.  How can this be?  Domain classes aren’t allowed to hold references to DAOs – that would break our traditional view of layering.

But… when I apply DDD, Repository and Factory interfaces are part of the domain.  This is a fundamental change – that a domain object deep in an aggregate can be constructed with a reference to a Repository in order to look up persistent data.  That a domain object can also use a Factory to construct objects, hiding the detail of construction (and dependency injection).  None of this work has to be done in a transaction script or script – move that responsibility into a place in your domain where it makes sense.

Repositories can be used to perform lazy instantiation of relationships in a persistent graph of domain objects.  E.g. retrieve a PurchaseOrder from the PurchaseOrderRepository, when the PurchaseOrder requires it’s line items it asks the LineItemRepository for matching line items.  A consequence is that this will lead to a bunch of Repositories being created in a complex model.  It’s simpler (less code) if you can use a tool like Hibernate to manage persistence of the entire graph, and configure it to automatically instantiate collection relationships lazily.  Nothing about Repository prevents you from doing that – you would just eliminate the use of Repository at that point and use the collection directly.   Hibernate is no solution to fit all problems however, and we commonly find places where explicit use of a Repository is cleaner.

I also deal with a lot of ‘legacy’ code that needs to be renovated into submission – often a safe path with JDBC code is to refactor towards the use of Repositories and moving the use of those Repositories into the domain to make it more understandable.  Often the JDBC code inside the Repository implementation does not have to change much.  When it does later on, you have the opportunity to make a shift towards a technology like Hibernate.

Transparent persistence and the Hibernate Session is a really important tool for implementing a rich domain model – if we make our client code (transaction script, service) as thin as possible, it is no longer responsible for remembering which objects have been updated and saving them back to the database.  Instead we use the Hibernate Session as a Unit of Work – our client code tells the domain to go and do something and the domain can go ahead and perform calculations, validate data, and update fields as need be.  When the work is complete we tell the Session to commit and it works out what needs to be updated in the database.  Brilliant – now if we add more complexity to the domain we don’t have to add matching complexity to our client code, and not in multiple places.

Christian is very concerned with adding additional layers into our code – a Repository wrapping a DAO.  I personally do away with the DAO idea altogether – the Repository implementation is performing the same thing.  The important thing is that the Repository interface (e.g. PurchaseOrderRepository) lives in the domain, and the implementation (e.g. JdbcPurchaseOrderRepository or HibernatePurchaseOrderRepository) lives outside the domain in infrastructure.

Conclusion (otherwise I’ll go on all day):  let go of the idea that domain classes must not interact with a database (or a message queue or a file system) – free them to do their responsibility, just make sure they do so through an appropriate domain abstraction – Repository and Factory are good examples.

Oh – and stop using Struts (1.x anyway).

Australian Architecture Forum Done.

Last week I presented with some ThoughtWorks colleagues at the Australian Architecture Forum in Sydney and Melbourne. My topic was titled “No Nukes – don’t detonate your legacy software” covering approaches to incrementally replacing a legacy application. I was a bit shakey in Sydney – I’m not very comfortable with public speaking – but I felt much better about my delivery in Melbourne. All round a really great experience and I hope to do more of this kind of thing in the future. The topic definitely sparked some interesting conversations – maintaining and renovating legacy applications (sometimes not more than 18 months old…) is such a big part of what we do as an industry.

The forum itself was mostly interesting – I don’t really identify well with the title “Architect” although that is a large part of the work that I do. I find that most software architects seem to struggle to remain relevant as they get further divorced from project delivery. For this reason I found my colleague Gianny Damour’s discussion group on “The role of the architect on agile projects” most entertaining as he described architectural practices found to be questionable including “pretentious modelling”, “over technical design” and “ivory tower mentality”. It definitely sparked a healthy debate, especially the assertion that architects should code.

Much less amusing was a discussion group by the local chapter of the International Association of Software Architects that was preaching _certification_ as the answer to peer recognition of your mad architecting skillz. That and several hundred pages of an “IT architect skills library”. I think the secret architect handshake is detailed in UML 2.0 on page 259. Forgive my skepticism.

Overall though a successful forum – congratulations to the organisers!

Careful of ‘technical stories’

Just came across an old article by Jeremy Miller titled ‘Balancing Technical Improvements vs New Business Features‘ which hits on a topic close to my heart. I’ve been doing a lot of work in the last couple of years on legacy software, and there’s often a large amount of technical debt already in place when we come along. It sometimes feels like there’s too much to do – and the temptation would be to stop working on new features until we get our ‘house in order’.

I strongly believe this is the wrong thing to do – how can I tell a customer that we are deferring work that provides critical (and sometimes way overdue) business value? Smells like gold-plating to me. I’d rather have the conversation that shows we are working on business priorities, and we are taking the right amount of care and just enough technical improvement to make sure we can deliver more quickly in the future. I never want to see ‘refactoring stories’ – the team needs to agree on a technical vision and ensure each story being played takes the software a step in the right direction. It might take multiple releases to acheive a refactoring or renovation goal in it’s entirety, but that’s a worthwhile cost for staying on priority.

It’s more challenging of course. You need to maintain that vision across multiple releases, and you are carrying for a long time the baggage of the old and new ways of doing things. You’d better be sure you’ve got enough time to reach the destination, or you may have left the code in a worse state than it was before. Sometimes when the changes are in java code we can use deprecation to document areas that are ‘old’ and need to be cleaned up next time a business story takes a developer into that area.

There’s also some changes which fall into the ‘stop the line’ category. Some things slow the team down so badly that you are forced to play some technical stories. Usually these are related to automated build, automated test, or building or remediating servers and environments. These are things that are often ‘all or none’ in nature, and are very difficult to change incrementally while working on business priorities.

Our teams use a ‘story wall’ to track user stories (on filing cards) being worked on in the current iteration. We colour code the stories which have business value (blue and white on this project) and those which are just technical stories (yellow). Sometimes I look at the wall from a distance and see that there is just too much yellow, and we need to have a conversation about priorities again.

Canary Tests

Neal Ford writes about Coalmine Canary Tests. Canaries were carried by miners in the ‘olden days’ as an early warning system for a buildup of gas, giving them time to exit the mine before imminent death. Neal describes using a set of simple tests that verify basic execution of parts of your application. When changes are made the canary tests can be run first and give earlier feedback when something has gone horribly wrong.

Recently I started using the same name ‘canary tests’ for a slightly different type of test. The team I’ve been working with has inherited a large web application which is suffering badly from a couple of environmental dependencies. It has to be started up just right, with components started in the right order, and the timing just right. We’ve been doing archaeology on the system, learning it’s tricks and failure modes – however we’ve been limited in the amount of resource we can spend on finding root causes. Most of the time when the application has been deployed or started in a bad state we can find some corresponding symptom. One by one the team has collected these symptoms and built a set of ‘canary tests’ that are run prior to handing the deployed environment over to be tested. If one of those tests fails, we need to work out what had changed in the environment, and probably at least restart and check again before continuing.

The canary tests are not a solution, they just help us make sure we don’t continue working with the environment while it’s in a bad state. We are separately investigating each problem to find it’s root cause and put in place a permanent solution if possible. Some of these may take a long time to put in place, but at least the impact can be managed.

Melbourne Ruby User Group

I recently went along to a meeting of the Melbourne Ruby User Group, the first user group type of meeting I’ve been to for years. ThoughtWorks now hosts the meetings and supplies pizza and soft drinks, so I really have no excuse not to attend given my interest in Ruby and Rails. For some reason I didn’t expect to get much out of the experience, but I was wrong. Two chaps, including Mark from RedBubble gave a slick presentation of their experience at RailsConf 2007, which sounded like a blast – I gained a lot of context of the direction of the Rails community and industry from their presentation. After pizza Pete Yandell discussed using RSpec with rails to unit test controllers, and showed an interesting technique he has devised to separate the stub behaviour of faked collaborators from the assertions about stub methods were actually called.

It took me a moment to grok, but it’s a clever thought – with jmock in java the expectation e.g. .expects("blah") is always in the same place as the outcome of the call e.g. .will(returnValue(foo). Pete has implemented an RSpec modification that means you can create the stub behaviour of a collaborator first, call the real code under test, then assert after the fact that the calls to the collaborator were made correctly. It does make the test code a lot easier to read, although it still doesn’t quite feel right. Nifty I’ll have to have a think about that.

I guess I didn’t expect such a good technical talk at a user group like this, I hope it’s like this all the time. It was good to hear Steve Hayes piping in from the background with wise words also. I enjoyed going along to the group – I’ll try to go along again next month. Especially as we did go for beers afterwards…

Correction: the two chaps presenting on railsconf were not both from redbubble.  Sorry Marcus, thanks Steve.

Commando Ninjas

From Gavin’s latest rant:

It might be nice if the whole world used Java, but they don’t. And Java won’t last forever. Really it won’t. Heh, that’s fine by me, just as long as it’s not Ruby that replaces it (I feel safe to say stuff like this, since I am guarded around the clock by an elite team of five hundred crack female IDF commando ninjas armed with the big machine guns out of Alien II).

Well put. Wrong, but well put…