Time to be open about Open Data

The concept of Open Data has been around for many years (the earliest I can find is around 2000, but there may well be much earlier usage). Yet some individuals still seem to be confused by the concept.

Here is the Open Data Handbook’s definition:
"Open data is data that can be freely used, re-used and redistributed by anyone - subject only, at most, to the requirement to attribute and sharealike."
https://opendatahandbook.org/guide/en/what-is-open-data/

This clear and understood approach to data provides increased transparency, improved decision making, fosters innovation and can increase civic engagement.

But its range of use can perhaps make it difficult for people to understand what Open Data is.

Here’s some examples:

Research:
Researchers & academics can use Open Data to study impacts and changes across economies, governments and populations

Business:
Organisations can Open Data to develop new products and services, such as a map app that shows the location and availability of parking spaces

Public Sector:
Governments can use Open Data to demonstrate and improve transparency & accountability, such as by publishing data on Public Sector spending.

Individuals:
Citizens and journalists can use Open Data to hold governments & authorities accountable, such as tracking the performance of local schools.

However, in my experience some people have deliberately misunderstood the term Open Data. Why?

Open Data can lead to change, and some people may be afraid of this. They may worry that Open Data will subsequently make it more difficult for them to control information, perhaps leading to job losses. Others may have vested interests in keeping data closed. For example, businesses may not want their competitors to have access to their data, and governments may not want citizens to have access to data that could be used to hold them accountable.

Ultimately Open Data is a powerful concept. It is therefore up to all of us to ensure that we all share the same definition and push for Open Data's continued societal benefit, not just the few.

Transport & mobility - data quality

Lately I have been looking at data quality in the transport & mobility sector.

Helpfully UK Gov has 6 "Core data quality dimensions":

1. Completeness
describes the degree to which records are present

2. Uniqueness
describes the degree to which there is no duplication in records

3. Consistency
describes the degree to which values in a data set do not contradict other values representing the same entity

4. Timeliness
describes the degree to which the data is an accurate reflection of the period that they represent (and that the data and its values are up to date)

5. Validity
describes the degree to which the data is in the range and format expected

6. Accuracy
describes the degree to which data matches reality

These terms are use across UK Gov and were created with the UK Data Management Association https://www.dama-uk.org/

More info here: https://www.gov.uk/government/publications/the-government-data-quality-framework/the-government-data-quality-framework

Developing Open Transport isn't about the the technology

I have been pretty involved in the creation of the Open Transport Initiative. A project, along with some very clever folk in the transport industry, to create a new interoperability open standard for transport / mobility accounts.

It has been personally rewarding (so far) and on 14 October we launched a near-finalised version of the customer account specification for peer review & feedback.

Several people have commented that this is a pretty significant technology, which (assuming it is adopted across the transport industry) could provide a similar level of integration and openness as Open Banking has provided to the Financial Services industry.

But the reason for creating this Open Standard and giving it away (eventually) isn't to show how technically proficient I and those around me are. It is to meet a growing need that has been identified... that a customer's transport data such as: purchases, usage and concessions is locked away in an account for each mode of transport.

Steve Jobs once famously remarked 'You've got to start with the customer experience and work backwards to the technology.' and this is what we have done. 

Why the cost of your cloud infrastructure could be too high

I've recently been been looking at some significantly increasing cloud hosting costs for a client.

There are a lot of reasons why this is happening... but in short, a lot of additional cloud costs come from these three causes:

1. Incorrect cloud architecture
This could be: not using “build & burn” practices, failing to automatically spin up (and especially spin down) components as needed, etc.

2. Processing more data than is needed
This could be: the creation of too many environments, badly sized test databases, etc.

3. Having unnecessarily stringent NFRs for non-production
In this could be: the high specification of development or test environments, running batch jobs too often and when not needed.

Is data driven marketing that simple

In a Linkedin.com post recently I commented about the increased adoption of data driven marketing ...

It's simple really:
1. Build insight with real data
2. Foster loyalty
3. Communicate creatively
4. Analyse to improve

But is it that simple?

Behave, Deliver and Grow Like A Digital Company

Delivering digital interfaces to your organisation's customers, partners and employees is no longer optional. It is now essential for long-term effectiveness (and survival).

But this means unlocking the data, systems and functionality your business operates with and exposing this both internally and externally to meet increasingly shifting needs. But it is not easy... hardly any sizeable company has an entirely blank slate to work from. Legacy applications, processes and thinking tie any business down so that it can work well. But it is these very constraints that often limit speed and agility, which are needed to succeed now.

Digitally enabling your business means changing the way you behave, deliver and grow.

Behaviour:
Being customer focused means creating a better customer experience that can win and maintain custom in the competitive digital landscape. It also means understating and controlling your data, so you to make informed decisions quickly based on what you are observing or being told.

Delivery:
Start by using new platforms, tools and methods to build products quickly, plus then to evolve them rapidly over time. If you think your quarterly website functionality is fast now, consider that over 7 years ago Amazon stated it makes changes to production every 11.6 seconds (it may even be faster now) and Facebook releases to production twice a day.

Growth:
Don't be afraid to unleash the creativity and innovation within your boundaries to help you build. Employees must be part of the Digital journey (not observers) and everyone, not just your test manager, must work towards the continuous improvement of products and services.

Wake up and smell the data

Does your organisation value data?

Does it know how much value there is in each customer account & interaction it records?

Does it really appreciate that the timeliness, accuracy and depth / richness of the data captured stored or created has competitive advantage?

No... I didn't think so, and I'm pretty sure most other organisations don't either.

That's a shame. 

Data Is Beautiful

Sometimes data can look visually appealing (albeit when coloured by the author) such as this graph of what people mean when they provide a statement around probability.

Source:

https://www.reddit.com/r/dataisbeautiful/comments/6vythg/i_redid_my_perceptions_of_probability_and_numbers/

Digital Transformation Starts and Ends with a Digital Architecture

The implementation of business transformations within organisations, and especially digital business transformations, is growing to a peak level right now. Chief Information Officers and Heads of Transformation are stepping in to: “digitally enable businesses”, “implement customer self-service channels”, “put the customer at the centre/focus” or just to simply “be more digital” (whatever that means).

However, when you ask these organisations what they are doing to change their internal systems and technical architecture design to facilitate this change, many either go quiet or simply utter something such as “it’s not about technology, it’s mainly about people”… Which I have worked out to actually mean “that technology stuff isn’t as interesting as building something nice & glossy I can show to the board”.

But let’s flip this around for a minute…

Digitally enabling your business usually means taking control of the data in your organisation and enabling it via online technologies. Yes, it does therefore mean the creation of some sort of new database or cloud-based big data lake that can then have modern web services integrated to it, so that some or all of this can be presented within a browser interface.

Implementing customer self-service channels, typically boils down to pretty much the same thing.  Web services and functionality are (securely - obviously) exposed to external customers via web and mobile App channels, so that contact centres or telesales operations can be scaled back or redeployed to different tasks. This also usually comes with a more onerous set of performance & availability criteria, so that a (near) 24/7 service can be offered to customers. However, presenting these services to real users also means that the systems behind-the-scenes need to be able to scale and adapt to changing user demands. Just plugging a rich user interface into a legacy system and hoping for the best is not digital architecture, it is digital anarchy.

Putting the customer at the centre of a business is an easy thing to say and a much harder objective to implement. Most organisations have been created to make money and therefore have lines-of-business designed to perpetuate this purpose. Consequently, technology systems are developed to support these structures and maintain the status-quo, rather than re-orientate things to make sense to the customer or help facilitate their engagement. It might be the ideal, but very few companies actually have end-to-end integrated systems that enable a single customer to be consistently tracked throughout their entire lifecycle. In short, creating technology to enable a customer to be in the middle of a business isn't always as easy as the sales PDF brochure states, especially if you don’t have a decent vision of how these systems need to work together.

So what can a decent technical architecture do for your company’s digital transformation?

It can provide a stable backbone that can support your technical & process change objectives. It can facilitate agile incremental delivery based upon re-usable components. It can help your business grow by supporting integration of other online services, API’s and data sources.

If you’re planning any of this, can you afford to NOT have the right digital architecture?

Why does Google Analytics show cities not postcodes?

As you may have seen in Google Analytics, it is possible to identify your digital visitors down to the city level. This is cleverly done by Google taking multiple data sources and then "guessing" where visitors are coming from.

Firstly there is something called reverse IP address lookup. This is when your Internet Service Provider gives information away about where you are connecting from. Then there's other big hints online users give away about their location, such as using a Chrome Browser on a mobile device or use Google Maps to look at their local vicinity. Each gives Google major indications as to where users actually are.

But this "guessing" does not go down to postcode level in Google Analytics.

My view is that some postcode area (e.g. PA = Paisley) are so broad they cover hundreds & hundreds of square miles and even postcode districts cover large amounts of land & people (e.g. PA1).

However postcodes themselves change over time can be so detailed as to cover as little as one house.... which would identify a person or household, which is against Google's data policy.

Customer Data Is Essential To A Digital Strategy

The cornerstone of any Digital Transformation Programme, to move an organisation to a place of improved digital maturity, is customer data

The collecting, storing, processing, analysing and presenting of data collected about your users needs to be a fundamental requirement. From data comes an understanding of who your customers are and what they are doing.

Who they are:
(Typically obtained from account / registration database)
Address
Age, sex & other demographics
Communication preferences
etc.

What they do:
(Typically obtained from your online analytics package)
Device(s) used
Time of day they visit
Browsing behaviour (e.g. pages visited)
Marketing campaigns they have interacted with
Goals completed
etc.

Or put a different way... structuring your digital transformation efforts around your customer is more important than you possibly realise.

You can't answer everything with analytics

I'm a big believer in the use of digital analytics to inform business decision making. And this isn't just the correct presentation of data in the right format, but in the delivery of insight from this data.

But statistician John Tukey had a different perspective. He once stated:
“Data may not contain the answer. The coordination of some data and an aching desire for an answer will not ensure that a reasonable answer can be extracted from a given body of data.”

In other words... sometime the data you have collected just isn't enough. Or more pointedly... if you have not originally set-out to collect the data you need to make the right decision, then no amount of analysis effort can generate the answer.

So how do you resolve this? Collect everything you possibly can, in the hope that you will one day find a use from  it?

Retailisation - what it means to be a modern digital retailer

"We want to be more like a retailer"
"We need to think more like a retailer"
"Our business needs to evolve into more of a retail model"

Sound familiar? Well I'm hearing these sorts of quotes more and more often these day, and  not just from the obvious brands you would think. But from established product manufacturers and service providers, who realize that they need to up their game and drive people to consideration and purchase/subscription/take-up.
It seems that despite a recent Global economic melt-down driven by over-spending and an economic reliance on spending... retailisation seems to be the way forward. Everyone apparently now wants to be the next Amazon, Zappos or Play.com

So what does it actually mean to think and be more like a digital retailer these days? (Especially the major online or multi-channel retailers, who seem to epitomize this ethos).

Well here's my thoughts:

1. A good online retailer never stops looking for ways to improve what they have. This constant & iterative approach to goal optimisation means sites need to constantly change to increase their conversion ratio, average order value and other KPI's. eCommerce giants like Amazon, Argos, Tesco, etc. no longer launch major re-developments once in a while, but have a tried & tested process of smaller changes planned based on analytics & insight. These changes are then implemented in an  optimisation road-map as quickly as they can, with the idea of building up a picture of what works and what doesn't.
   This is also not just something that done on the homepage of your site,  where every product/service wants to get visibility, but on every page / template, including: landing pages, product pages, etc.
   
2. Use every opportunity to maximise each individual transaction. From useful up-sell and cross-sell opportunities through to optimised abandoned basket messages or a clever eCRM communication that pulls in dynamic product suggestions based on browsing history... you have the data, use it to persuade and encourage.
   
3. Carry out regular user experience site reviews, but ensure they are done from the perspective of a prospect/customer.
   Examples could include:
1. A new customer looking for product information
2. An soon-to-be customer looking for product validation
3. An existing customer looking for support or returns information
4. A lapsed customer who has forgotten their password.
   
4. A PPC & display budget should focus on those campaigns that deliver conversions and not just visits or other vanity metrics. In other words, deliver a bought media strategy that targets goals using input from you site analytics.
   (And if I hear one more senior exec say "we have X number of hits on our site" - I think I'll scream)

Retailisation isn't for everyone. But as more & more sites move beyond just the basics, I'm sure it is an approach that will continue to increase in use.

Online success - 8 soundbites for digital improvement

I'm typically not in favour of using soundbites, without backing them up with facts or information. However I find myself using the following more and more... so thought I'd note them down for future reference.

Find and identify your audience
Turn monologues into dialogues via web and social networks
Match the style of message with the style of audience
Create, implement and analyse ideas which deliver results
Measure value and worth properly
Build engagement, loop it and exploit it
Developing a programme for continuous improvement
Evaluate, set-up, use and  tools that use the data you already own

Hopefully in another post or two I'll go into these in far more detail.

Building your corporate digital analytics capability

As an organisation develops its digital understanding, you see certain trends and processes emerging. One of those is the increasing usage of digital analytics and increasing business reliance on the figures produced.

Digital analytics is now taken seriously as a business tool. From what was once a mainly geek-ish domain has emerged a significant service that can empower the business to make more rational or efficient decisions.  But just as other digital resources have grown (and grown-up) over time, the analytics resource in your organisation may well have grown too. If fact, making the point a bit stronger, if your web analytics team has not grown in size or depth in their understanding as the rest of your online capability has matured, you are probably missing something.

However… creating, scaling and keeping your web insight team is not an easy task.
Firstly positioning the team as just another marketing service is not the right approach. Having them regarded in the same light as a search engine optimisation or pay-per-click resource misses the point. This is not to take anything away from the SEO or PPC staff you might employ but the online analytics function is not just there to inform and maintain the current activity… but can also be used to feed insight back into your organisation too.

Creating the right online customer analysis and insight team structure depends a lot on the size & scale of your company. Most small companies do not have someone dedicated to this role (unless they are a digitally-focused business such as an eCommerce site) and even a lot of bigger companies combine the work of a web analytics function with other disciplines, and in a lot of cases this is digital marketing. It is therefore typically only much larger enterprises that can usually afford or utilise a dedicated person or persons in a digital analysis capacity.

Structuring this multi-person team can then be a little different from the way you might structure another digital functions. Although a lot depends on the types and quality of the individuals you hire. Just having a bunch of people who can all do the same things might not provide enough specialisation or focus… and analytics can quickly get into specifics. Some larger teams can range in skill-sets from more technical-orientated people through to business-based modellers who can pull trends and opportunities from complex data structures.

Keeping the team (aside from the effects of your own management style) can be the hardest thing to achieve. From my own experience there is currently a lack of decent experience digital analytics professionals out in the market right now. Quite frankly, we need more digital analytics experts. People with the right skills and experience are given far more choice about who and where they work, with many choosing a more lucrative career as a freelancer or consultant. Therefore holding on to good digital insight staff is crucial if your organisation is to you want to grow your capability and retain best practice knowledge.

A data-driven digital strategy

Having a digital strategy for your organisation is a stepping stone to taking advantage of online channels to: make money, save money, maintain & improve service and deliver your brand promise.

Easy eh?

Well... no.

Having a digital strategy is better than not having one, but having the wrong one (or half of one) can be detrimental. And typically the wrong digital strategy is one that doesn't focus on the right things. For example just basing a digital plan on the deployment, use and support of online applications is giving yourself too narrow a scope.

If I could therefore give everyone just one  piece of advice, it is to create a data driven digital strategy; an approach that is:

1. Powered by information from your digital analytics

2. Informed by insight from analysts, who use different data sources to give your organisation a true picture of what your customers are doing and also want from you.

3. Continually updated based upon the most recent understanding of users, trends, their goals and the value this creates for your business.

4. Full of facts, rather than assumptions and guesses

Insight, the one digital metric we don’t measure

I hope that by now most large company execs by now will have heard of the existence of website analytics. Some (especially the more marketing and technology focused) may even have seen dashboard reports from their analytics suites.

These displays of graphs and numeric tables can help the senior team quickly get an idea of the value of their digital channels, with ‘Goals’ (online results such as sign-ups or purchases) and ‘CPA’ (Cost per acquisition) figures being obvious KPI’s to get regular updates on.

These hard and fast numbers can go a long way to dispelling half-truths, rumours and ‘gut feel’ that humans instinctively use when there are gaps in their knowledge.
Note: I still see and hear of execs citing ‘hits’ as a great indicator of online greatness, with no understanding of how this figure is derived nor the understanding that traffic without purpose just creates a burden on IT infrastructure more than anything else.

Receiving and reviewing these dashboards might be one way of checking the digital success of an organisation, but a few pie charts and year-on-year comparisons does not even start to show the key output that needs comes out of these figures… insight. Insight that your digital analytics team (perhaps only made up of one person or even just part of a role somewhere) should be craving to provide

Measuring facts from all your digital touch-points is now possible and relatively easy in the online world. You insert a tag or two into each page (or action) of your website and sit back and watch the numbers scroll before your very eyes. Real-time reporting is now a reality, with even the free packages such as Google Analytics telling you where, when and what your visitors are doing at every step of their connected customer journey. But gaining insight from these figures is a different matter and measuring the value of this insight is exponentially more difficult still. Perhaps that’s why to-date it isn’t measured

But actually, it is...kind of. Insight derived online reports can show up in all sorts of ways, usually when there is a feedback loop from this data back into the business, for example:

• Geographic data about where website or app visitors are coming from can inform business strategy. Imagine the mergers and acquisitions team knowing which counties the biggest increases in valuable business traffic have recently come from
• Significant differences in the search engine keywords entered by users to reach your sites can predict market or investor trends. Data that could be of potential use to many central functions, including finance, proposition and commercial teams
• Site bounce rates can not only inform your User Experience team of potential issues, but can reflect on product price, content quality or site speed (or a possible combination of all three plus other factors

It’s a shame however that this contribution to the organisation can’t be effectively measured. You can’t really put a price on the provision of internal data within a company, without coming up with some sort of mad model that will be more hypothesis than fact… the very thing thatdigital analytics tries to constantly minimise

CMO vs CIO : the battle for data

"In the future, wars will be fought over water" is a phrase you hear quite a lot these days. And it might well be true... As important global resources become more scarce and as rulers realise exactly what matters (in this case, the health of their population).

However, in the boardrooms across the world, there may well be a number of battles. But this time fought over something very important to both technologists and marketers. Data. More precisely, customer & user data, along with any associated information that helps the organisation function / sell / behave better. Consequently, there is a scenario that is looking increasingly likely, the showdown between the Chief Marketing Officer (AKA the marketing director) and the Chief Information Officer.

Its my belief that as data relating to the customer grows and becomes more and more useful (along with the ability to model and then predict future likely behaviour) these two company board members are the two most likely to both stand up and stake their claim to its ongoing ownership. Customer transactions in branches / stores / premises, their online browsing history, their engagement across social media channels and their contact preferences will be like water to these roles.

However, this doesn't necessarily have to be the case. And not just because I'm advocating peace and love in the exec suite, but for a different reason... It has been a prediction of mine for a while now, that the roles of the CMO and the CIO will evolve and in some cases merge, quite possibly around the need to have a single point of data ownership.

This could therefore give rise to a new role seated around the walnut table of major organisations, the CDO. But not the 'Chief Digital Officer' that I have already mentioned a couple of times on this blog, but that of the Chief Data Officer.

Are you wasting your digital analytics?

It's pretty clear to me that a lot of organisations still haven't moved their website analytics beyond the collecting and reporting stage (and some haven't even done that very well).

What's the point of collecting data and getting it turned into pretty graphs? Especially when all it does is:

1. Get ignored or misunderstood
Have you ever had a report sent to stakeholders who really don't do anything about it? ("so this 90% average bounce rate is good?")

2. Not get sent to those who actually need it
Has anyone considered that your content / editorial / CMS team might find key metrics on site engagement useful?

3. Not get actioned
"Yes we know that users aren't finding content down at the 4th level of navigation, but it would take too long to change things now"

If you find this happens in your company, then you're missing out on the contribution of a valuable business resource. I'm not saying you need to send analytics reports to everyone all the time or that every single observation needs to be acted on immediately... But there is a role in:

• Educating specific staff and partners about what these figures and graphics mean.
• Making sure that you have the correct reporting structure
• Prioritising and escalating the changes required that could deliver meaningful improvements.

Are you going to do this or waste an opportunity?

Are data silos holding your company back?

In a recent blog post I commented on the issues with collecting data across the organisation. But one it is collected, it's the storage that's the next problem.

You see, data in businesses is still pretty much silo based, with information retained within specific people, units and departments. Decision making is consequently restricted to this data and is therefore restricted in its scope, outlook and ultimately its potential. And typically where there is a lack of data, this leads to assumption and inherent inaccuracies. In short, silo-based thinking leads to silo-based data capture, which leads to silo-based-decision making.

This isn't a new phenomenon and there are articles that exist online since the beginning of the web that talk about this. But with the increase in the amount of data being captured about people, their activities, their multi-device habits and transactions they perform all the time ... the world is collecting and storing more information than ever before (often with little consideration of how it will be used subsequent) and this creates a growing problem. For example, I remember just a few years ago working on the implementation of a large eCommerce site, where a key requirement was to capture every possible piece of  web analytics data (including the complete user journey for every visitor) in another database not accessible by the rest of the business . When I asked why this was, the only rationale given at the time was "I don't know, but we might want to look at it later".

However this issue is something that 'big data' practitioners are now starting to get their teeth into. Products like Autonomy are now worth billions of dollars (although perhaps only to HP at the beginning of 2012 now) because they can make sense of huge amounts of structured and unstructured data to provide business insight.
So maybe there was a reason for storing all that individual website visitor data back then...