Architecture vs Agile

There are various technology "holy wars" that have been fought throughout the years.
PC vs Apple
Thin client vs Desktop computing
etc.

Some have been resolved and some still continue to this day.

And now, as more digital / business transformation projects are adopting an agile approach to software configuration and development... another battle, albeit one that has been raging for some while, is growing in its intensity and prominence.... That of the agile approach vs the need to define architecture for a solution in advance.

As both a supporter of agile to create products & get feedback quickly, and an architect of digital products and services (both large and small) I understand both sides of this war.

Agile:
Wants to develop minimum viable products as quickly as possible, then get this in front of users and rapidly iterate towards completion. The focus is on "good enough" not excellence and likes the flexibility to change & improve features.

Architecture:
Designs those things that do not move... and the resemblance to buildings IS intentional. Architecture is usually fixed for the long-term and gives the foundations for other more nimble practices around it. Technical Architecture (what I define as the combination of the Enterprise Architecture and Solution Architecture disciplines) should provide general directions (e.g. public cloud/hybrid/private, buy vs build, etc.) and standards. Plus architecture should also make  higher-level decisions on technology and principles which would be hard to change subsequently (e.g.coding language, integration methods, etc.).

Where things get messy is in the lower-level design of applications and their solutions. Here is where the agile versus architecture battle is waged. Where teams need to be empowered sufficiently to create products that meet the user stories... but also need enough guard-rails and check-points to operate within governance.

API granularity

Deciding how to design a digital architecture that provides optimum agility and scale-ability for your organisation is a challenge for many architects in the midst of a digital transformation. Deciding if you should use APIs in your integration layer is perhaps much less of a dilemma (hint: of course you must). But then working out how to design those APIs and the services behind them can be a new problem to solve.

I think this is an especially important decision, as it is quite hard to undo an architectural choice. Issues, can subsequently occur such as:

• performance bottlenecks (e.g. from too course a design)
• too much 'chatter' (e.g. from too granular a design) 
• the inclusion of too much (any?) business logic

All of which could have major implications in the future.

An agile mindset is not optional

Agile seems to be one of the most over-used words in business these days. For example:

Agile development 

Agile marketing

Agile architecture too ;-)

Done well, agile can produce prototypes and products very quickly and efficiently.

Done badly it can be an excuse for reducing the amount of time to create quality software or as a justification for only delivering partial functionality (for all of the budget).

Overall though... agility is a mindset, a way that organisations approach the task of creating value or improving the processes and products they have. And those that do not apply agility to their core will find they are not able to respond quickly or well enough to an increasingly swift competitive environment.

Or put another way.. which company wants to states as their vision or values that they want to be less reactive and more long-winded?

The Agile Architect

The technical or solution architect who has to align to an agile delivery methodology has a lot on their plate. In fact I have even recently written a post on The Agile Architect's Dilemma on Linkedin about the internal tug-of-war I am having between my agile and architectural approaches.

3 key areas they have to consider are:

Scale-ability / Scaleability:
To a certain extent this issue has become less of an issue with cloud-based services that have elastic capabilities to grown on-demand a volumes increase. However bottlenecks and latency can create problems, especially when using multiple different cloud providers or when API's are large & painfully slow. In short, not everything scales easily or linearly.

Micro-services / Microservices:
Rather than have monolithic points of integration (e.g. the above mentioned large APIs) the services that connect your different systems should be light and just based around specific functions. These micro-services allow incremental change and improvements to be made over time and better supports evolving front-end user experience requirements too.

Central data source:
Almost inevitably a digital architecture will need to store data in a single and centrally-hosted data source (e.g. a database). This repository of customer and operational data will need to be flexible enough to accommodate growing changes in storage and real-time data access requirements. However, no database (especially optimised relational database systems) can be expected to constantly change as the input and output demands of an agile development change too.

Digital Architecture fundamentals

As part of a recent project, I was asked to come up with some architecture fundamentals of a digital system (not just a one application system, but a more complex and integrated solution assumed to be for a larger organisation).

Here’s my list:

• The architecture of a system or integrated systems needs to align to (or fit into) the overall ‘As-Is’ and ‘To-Be’ enterprise architecture of your organisation.
  
• Those creating the architecture need to first understand the business goals (e.g. the customer experience and commercial aims) and the major applications that ‘the business’ are likely to need in the medium term or have signed longer-term contracts with.
  
• A loosely coupled architecture provides the flexibility to use best-of-breed solutions from a range of vendors. The opposite is that either through design or legacy system adoption, your architecture shackled to one key supplier who is then possibly also integrated directly to multiple other systems… this obviously creates a dependency upon this key supplier, making them harder to subsequent remove and replace. Adopting a loosely coupled architecture also means you are not tied to specific front-end software, allowing your organisation the flexibility to either buy or build (typically using agile product & developers) to create the best user experience possible.
  
• Where possible, create a single in-house database, that can be updated by any other integrated systems… ideally in real-time. You don’t have to store all data that exists in every other integrated systems in this central repository, just those that you may need to align other data to at some point in the future.
  Note: It is possible that this service could also be provided as another integrated system, but for overall speed, agility and ultimately data ownership you may find it best that this database sits within the boundary of your main architecture. However, this does come with additional considerations: there is an overhead in managing and administrating a significantly large database, plus the data model used needs to be sufficiently flexible and scale-able to handle a variety of requirements over time.

Failing to Architect with Agile is Architecting to Fail

Agile approaches to development and online product delivery are almost de-facto these days. Every private and public sector organisation wants to be transformed, nimble, lean and able to deploy digital services quicker and quicker to their ever-demanding customers. And it is not just in the IT coding department that there’s been a change in vocabulary. An agile approach can even be adopted by a company’s commercial team, marketing department and even some of their operational functions (plus don’t even get me started on how much senior managers, procurement and HR have to change too – but that’s something for another post).

But I’m going to stick my next out (once again) on a subject I’m passionate about… Technical Architecture and how it clashes with agile delivery.
Or more succinctly put: Agile Architecture Doesn’t Work

There, I’ve said it now and I’ve been wanting to say it for a while. It has been on my mind as I’ve heard the opposite mentioned in podcasts, or read about it in blogs and books.
I guess I was trying to get this off my chest when I wrote my recent blog post “Digital Transformations starts and ends with Digital Architecture”. As in my mind, the science (or is it art?) of crafting a robust yet flexible technical architecture that supports digital business aims is the one thing you can’t build as you go.

Creating the technical architecture for your new venture takes planning. You also really only need one Technical Architect, the person who owns the architecture and has the responsibility for its solution design and ensures re-use of common components. Not a bunch of developers who all want to create a part of the architecture they are responsible for.

It’s like wandering around on a gap year between school and university (or school and work, or university and work).  You may be able to make up your journey as you go, with just you or a travelling companion making the decisions… but the roads and the map are pretty much fixed.

So...  although some agile practitioners talk about how agile approaches can help architecture deliver quicker or better.  I firmly believe that it is architecture that facilitates faster and more robust agile delivery.