The Agile Architect

It seems to me that we all have an understanding of what Agile development and Continuous Delivery are, but we're generally less clear on how the role of Architecture fits into this picture. Most of the Agile frameworks fail to mention architects at all, and hence many companies consider architecture to be something that's done "before you get to the Agile bit".

Recently "Evolutionary Architecture" and "Continuous Architecture" have been talked about a lot - techniques which require buy in and alignment throughout the organization. Here's what I've learned about what agile architecture can be done at the coal face in companies where these ideas aren't fully embedded.

1. Individuals

We still have this idea in the commercial world that the software development process is something like a factory conveyor belt, and Kanban (excellent though it is) sadly reinforces this idea.

Software delivery shouldn't be a conveyor belt.
It encourages high WIP, and low collaboration.

Two decades of Agile still haven't broken the notion that first this person does this, then that other person does that. Those people have specific roles, with job descriptions.

But in the real world, teams are made up of people, not roles. People aren't designed and built to adhere to precise job descriptions - they're complex, nuanced organisms with a range of abilities, needs, experiences, and motivations.

For example, the skills and characteristics required to be a BA and those needed to be an engineer are not mutually exclusive. Similarly with architects or testers. People typically have a variety of experience and knowledge - why should we restrain them?

My point is that we shouldn't be prescriptive about what a team looks like. And in the particular case of architectural design, I'd argue that it's not something that only an architect can do. In fact, the designs that architects produce often come (and should come) from the subject matter experts - engineers, testers and so on - who know the details of how it all hangs together, and will be able to identify many of the complexities and edge cases from the outset. 

2. Working Software

The traditional idea of Big Design Up Front, and an Architecture Team in their ivory tower is much derided, and rightly so. However it's still a reality in many companies over a certain size.

As any software developer will tell you, you can't design code up front and expect it to look like that when you're done. You discover things as you go along; the exact design emerges as you proceed, finding your way along, sometimes going back and refactoring when you perceive a better way of doing something. I still shudder when I remember an architect I once worked with whose idea of architecture was to produce UML class diagrams with literally dozens of classes and interfaces, to throw over the wall to the development team.

Most people would agree that architects should work at a high level. However, if they're not involved in the actual implementation of the things they design, they'll never learn about the implications of their decisions at the coal face of development and operations, and will make the same kinds of mistakes over and over again, with other people picking up the pieces.

So architects should operate at a high level, but strive to understand the low level details - which in practice means being involved with the team right the way through implementation and beyond.

If your company thinks that they can be much more efficient having architects working solely on the "initial design phases" of projects then they're mistaken - they should take a long hard look at how smoothly "implementation phases" typically go, and the quality of life of their Ops and Support folk.

Another important aspect here is that if your company is project focused (which is sadly usually the case) rather than product focused, then there may be the temptation to have a High Level Design Phase "before you get to the Agile bit". There's probably a lot of variation in what people understand by the term "High Level Design", so let me just say:

- You do need to understand the bigger picture, so by all means produce an HLD. But...

- No design work should take more than a day or two. If it does, you're trying to bite off too much scope, or too much detail. 

- I prefer the terms "High Level Approach" for the bigger picture stuff, and "Design" or "Elaboration" for the specific design work needed for a ticket. 

- Aim to do all design work just in time. This is in order to avoid waste, and gain and maintain momentum.

3. Collaboration

The role of an architect is, in my view, the technical role that can least afford to operate in an insular way.

There are broadly three types of collaboration which are pretty essential to be effective as an architect:

1. Identify and build relationships with all the stakeholders for a particular change (Marketing, Ops, Product, Technical Support, Service Desk...) to make sure that as many aspects as possible have been considered, everyone's aligned, and there are no nasty surprises for anyone later on (including you, if an unexpected requirement or constraint pops up at the eleventh hour).
2. Brainstorm, refine, and choose a technical approach in collaboration with the team that will actually implement it.
3. Get your decisions and design ratified with all the stakeholders. This is the scary bit. As an architect, your work will be scrutinised and critiqued more widely than anyone else's (except perhaps UX and front end designers, poor bastards). If it's not, then I'd say you're not doing your job properly.

That last point is probably the most important. By communicating and justifying your designs, you're forced to discover exactly how well you understand them, and how well considered they really are. Basically: Talk to people.

4. Responding to Change

This is always contentious, because no one likes change. Especially when you've slaved away on something, only to find that you have to take 2 steps backwards and redo it now that someone's changed the requirements, or the priorities.

However, as we all know from bitter experience, change is absolutely inevitable. And the answer given to us by all Agile frameworks is: Keep it simple, and keep it small. That's because simple, small things are easier to change than complex or big things.

This poses an interesting challenge for an architect, whose responsibility is usually to produce some kind of design not piece by piece, but for the design as a whole. And that's why I'd suggest that architectural designs should have two important features:

1. Be simple
2. Be modular

So we're talking clear separation of concerns. Loose coupling. Avoiding over engineering - remembering what XP (my favourite of the Agile frameworks) taught us about YAGNI. 

Designs should be embarrassingly simple: People should look at them and instantly understand them. People should say: "What the hell do we pay this architect guy for? This is obvious!"
(I have a blog post brewing on how we should define and evaluate the effectiveness of people and teams, but here's a spoiler: It shouldn't be how complex and clever we make things).

Blindingly obvious things get overlooked all the time. An architect should be one of the key people making sure that doesn't happen.

5. What not to do

So I tried to structure this post around the four main tenets of the Agile Manifesto, but I'll close by just listing some common ways that architecture is often done which are inherently not Agile:

Separate Architecture Team

This is born of the misguided notion that design is a finite activity, done at the start of a project. Instead: Embed your architects in the teams where the work will be done. Otherwise:

• They'll be making decisions they don't understand
• The team won't understand or embrace the vision
• The team's unlikely to have any respect for the architect.

Formal architectural governance

I'm not saying you shouldn't do this - nobody wants to use an Internet Banking application where no one confirmed compliance with ISO:IEC 27001:2013.

But if technical decisions have to go through an Architecture Steering Board, made up of senior techies who, at a distance, make decisions about projects, grant dispensations to "divergence from the original designs", or document "corrective actions" to go on a "programme register" etc. etc., then people simply won't bother. People will work around it.

Better to trust the people we employed to actually know what they're doing. 

(Test things - of course. Everyone makes mistakes. But don't centralise control - in my experience that kills efficiency and motivation).

Accurate Architecture Models

By this I mean the kind of architecture topologies you create in tools like Enterprise Architect and others, where the boxes and lines aren't just boxes and lines - they have properties, and the modelling software understands the boxes as entities with behaviours and relationships with other entities.

While I applaud the aspiration to have an architecture repository, TOGAF style, experience has shown this to be a futile exercise.

Don't do it, for goodness' sake, especially if it's manually updated. You'll spend your life trying to keep it updated for very little value.

The main value of architecture diagrams is to communicate a concept. These tools, which strive for accuracy, impose limits on the kind of information you can convey, Typically you'll struggle to bring out the important features without showing the unimportant ones too. And with a restricted palette of visual indicators, you won't be able to bring out all the things that matter (high risk areas, data flow versus flow of control, areas that are changing in different phases, variations by environment, areas holding or transporting sensitive data, and so on).

And of course you'll probably diagram the same thing more than once for different stakeholders at different levels of abstraction - and no one likes looking at diagrams which are uniform and soulless. As humans we like things to be visually compelling, and tools like EA really, seriously, aren't.

In summary

Avoid hand offs. Get the people you need
working together on each small thing, all the
way through. That includes architects!

The responsibility of an architect is to be the person who has thought of things from all angles, and from the broadest possible perspective; they should take the high level strategic view.

But that doesn't mean that they shouldn't involve themselves in the detail of the implementation. Architecture should come from the people who know the detail, and be ratified by them (as well as all other stakeholders). It's the architect who makes all this happen - collating, rationalising, and aligning - driving technical decision making. Ensuring that the design is as simple as possible, and as modular as possible.

Finally, the architect should be a member of the implementation team, collaborating on design just in time, rolling their sleeves up when necessary, and paying attention to the build, deployment, and operational support of the stuff they build.

It's common sense really - which, at the end of the day, is what Agile is really all about.

Further reading

http://www.agilearchitect.org/ - a pragmatic and detailed guide to agile architecture

https://pgppgp.wordpress.com/2014/08/18/continuous-architecture-principles/ - the 6 principles of continuous architecture from the blog by Pierre Pureur (co-author of Continuous Architecture).