Showing posts with label architecture. Show all posts
Showing posts with label architecture. Show all posts

Tuesday, November 5, 2019

Why the cost of your cloud infrastructure could be too high

I've recently been been looking at some significantly increasing cloud hosting costs for a client.

There are a lot of reasons why this is happening... but in short, a lot of additional cloud costs come from these three causes:

1. Incorrect cloud architecture
This could be: not using “build & burn” practices, failing to automatically spin up (and especially spin down) components as needed, etc.

2. Processing more data than is needed
This could be: the creation of too many environments, badly sized test databases, etc.

3. Having unnecessarily stringent NFRs for non-production
In this could be: the high specification of development or test environments, running batch jobs too often and when not needed.

Wednesday, July 17, 2019

LIFESPAR cloud architecture principles to follow

With more and more organisations adopting cloud technologies for their applications, I've seen the tendency to just "life and shift" architectures. Physical servers are replicated as virtual machines and the same software applications as before are run "on somebody else's computer". 
  • Latency-aware
  • Instrumented
  • Failure-aware
  • Event-driven
  • Secure
  • Parallelizable
  • Automated
  • Resource-consumption-aware

But this approach doesn't leverage many of the benefits of software-as-a-service (SaaS) or the new cloud-only components available as platform-as-a-service (PaaS). It also means that architects creating cloud-native solutions need to have a different set of principle to before.
Some of the best guidance I have seen in this area comes from Gartner, who use the acronym LIFESPAR to explain those principles to follow for designing cloud-native architectures.

So what is LIFESPAR and what does it mean to an architect?

Latency-aware:
Understand that every application needs to be designed & implemented knowing that it may not get an instant response to each request. This latency could be milliseconds, or it could be seconds, so ensure each solution elegantly deals with delays and is tested to prove it works under these conditions.

Instrumented:
Every solution and every component must generate sufficient data about their usage and ideally send this information back to a central location, so that real-time & subsequent decisions about the architecture, cost, volumes, etc. can be made. In this way, as instrumented approach supports an elastic automatically scaling system (scaling both up AND down).

Failure-aware:
Remember that things fail (hardware, processes, etc.) and that software, created by humans, is never usually bug-free. So always design solutions that wait or fail & re- in the way you need them to. Failures must also be comprehensively tested - if necessary, writing code to force failures (e.g. the Chaos Monkey). Also bear in mind that in some scenarios... latency and failure are the same thing.

Event-driven:
Applications used to developed with synchronous actions (as the performance, etc. of the target system was known). But now in a decoupled architecture, messages need be sent as events. This also simplifies scaling and makes them more resilient to failure.

Secure:
Always assume your solution will be subject to some sort of malicious activity and try to prevent it. This means restrict events and users, minimising attack surfaces and following best practice data handing and security processes.

Parallelizable:
Many small systems are usually cheaper that one large one, even in the cloud. Therefore find ways to scale-out your solution and its processing & messaging, rather than scaling-up.

Automated:
Every cloud-based component and also your overall solutions should be able to be deployed, started, stopped & reset via scripts. Remember to test this from a command line, from the beginning of development through to your Operational Acceptance testing (and even as a means of testing Disaster Recovery processes).

Resource-consumption-aware:
So, you now have almost limitless processing and storage resources at your fingertips. But you also either have your credit card charged as you use the service or are going to get an invoice for what you use very soon... Therefore, always consider using the least amount of cloud resources possible. Simplify your solution, build & burn of components & environments, automate components to start & stop only when they are needed and don't store more than you need (e.g. by sharing test data across solutions & environments).

Thursday, April 18, 2019

Architecture vs Agile

There are various technology "holy wars" that have been fought throughout the years.
PC vs Apple
Thin client vs Desktop computing
etc.

Some have been resolved and some still continue to this day.

And now, as more digital / business transformation projects are adopting an agile approach to software configuration and development... another battle, albeit one that has been raging for some while, is growing in its intensity and prominence.... That of the agile approach vs the need to define architecture for a solution in advance.

As both a supporter of agile to create products & get feedback quickly, and an architect of digital products and services (both large and small) I understand both sides of this war.

Agile:
Wants to develop minimum viable products as quickly as possible, then get this in front of users and rapidly iterate towards completion. The focus is on "good enough" not excellence and likes the flexibility to change & improve features.

Architecture:
Designs those things that do not move... and the resemblance to buildings IS intentional. Architecture is usually fixed for the long-term and gives the foundations for other more nimble practices around it. Technical Architecture (what I define as the combination of the Enterprise Architecture and Solution Architecture disciplines) should provide general directions (e.g. public cloud/hybrid/private, buy vs build, etc.) and standards. Plus architecture should also make  higher-level decisions on technology and principles which would be hard to change subsequently (e.g.coding language, integration methods, etc.).

Where things get messy is in the lower-level design of applications and their solutions. Here is where the agile versus architecture battle is waged. Where teams need to be empowered sufficiently to create products that meet the user stories... but also need enough guard-rails and check-points to operate within governance.

Monday, August 20, 2018

API granularity

Deciding how to design a digital architecture that provides optimum agility and scale-ability for your organisation is a challenge for many architects in the midst of a digital transformation. Deciding if you should use APIs in your integration layer is perhaps much less of a dilemma (hint: of course you must). But then working out how to design those APIs and the services behind them can be a new problem to solve.

I think this is an especially important decision, as it is quite hard to undo an architectural choice. Issues, can subsequently occur such as:

  • performance bottlenecks (e.g. from too course a design)
  • too much 'chatter' (e.g. from too granular a design) 
  • the inclusion of too much (any?) business logic

All of which could have major implications in the future.

Thursday, August 2, 2018

The Agile Architect

The technical or solution architect who has to align to an agile delivery methodology has a lot on their plate. In fact I have even recently written a post on The Agile Architect's Dilemma on Linkedin about the internal tug-of-war I am having between my agile and architectural approaches.

3 key areas they have to consider are:

Scale-ability / Scaleability:
To a certain extent this issue has become less of an issue with cloud-based services that have elastic capabilities to grown on-demand a volumes increase. However bottlenecks and latency can create problems, especially when using multiple different cloud providers or when API's are large & painfully slow. In short, not everything scales easily or linearly.

Micro-services / Microservices:
Rather than have monolithic points of integration (e.g. the above mentioned large APIs) the services that connect your different systems should be light and just based around specific functions. These micro-services allow incremental change and improvements to be made over time and better supports evolving front-end user experience requirements too.

Central data source:
Almost inevitably a digital architecture will need to store data in a single and centrally-hosted data source (e.g. a database). This repository of customer and operational data will need to be flexible enough to accommodate growing changes in storage and real-time data access requirements. However, no database (especially optimised relational database systems) can be expected to constantly change as the input and output demands of an agile development change too.