Better Data not Bigger Data

Big Data is Big News. Big consultancies and senior managers are all using the term "Big Data" these days. The words are the "Information Superhighway" of this decade*. Over-used, over-hyped and mis-understood.

In practice I tend to refer as something as "Big Data" if I can't inset it & work with it in an Excel spreadsheet... even though we've has databases for half a century that can deal with more than 1,048,576 rows (which is actually the maximum rows you can get in an Excel spreadsheet).

However there can be a lot of insight that can be gleaned from much smaller data sources. You don't need to have access to every single customer record in a database to analyse most trends about your users. You only need to examine a smaller accurate and representative data set. It can't be incomplete, out of date or incorrect.

So shouldn't we really use the term "Better Data"?

 

*BTW: What is this decade actually called? Sure we had "the Eighties", "the nineties" and even the "noughties"... but are these "the teenies"?

How do you segment financial services customers?

In an earlier post I made it clear that I thought a certain way of segmenting customers has had its day. The method of classifying customers just by their life stage is no longer relevant, with both lifestyles and finances having moved on.

So rather than dwelling on this, I thought it better to look at ways that banks in the future could segment their customers. But to be honest, I couldn’t come up with a single simple way of segmenting financial services customers that made sense in the modern world. Nothing really fitted nicely and any possible model had loads of exceptions to the rules.

And then it struck me... that perhaps there's no longer a few simple segments that fitted all financial services customers. Perhaps this subject is just too complex to put into simple terms... or in other words, perhaps now with the advert of clever data analysis and personalisation, there's no need to segment them into a handful of categories, everyone is in their own segment!

It's therefore my prediction that banks will eventually integrate big data analytics into their CRM systems. This will develop their understanding of who each individual customer is, alongside a record of what products and services they have already got do describe how best to meet their needs.

Some financial services companies may already be on their way to delivering this vision and this should be an interesting space to watch as the use of dig data analysis takes off.

Presenting attribution correctly

Today I was not only lucky enough to be invited along to the Strathclyde University Business School's workshop on Big Data and Social Media... I also presented on one of my favourite topics:

The Multi-Channel Consumer And Their Influences To Purchasing

One of the hardest things I wanted to show was the different paths that consumers take on their multi-channel paths to purchase.

This journey (albeit just for one customer) is quite difficult to depict in a linear way and in the end I resorted to showing all the channels as a set of swim-lanes. In the photo taken above, you will see me in the middle of my presentation trying to explain the concept.

I hope the audience understood what I was trying to say.

Are data silos holding your company back?

In a recent blog post I commented on the issues with collecting data across the organisation. But one it is collected, it's the storage that's the next problem.

You see, data in businesses is still pretty much silo based, with information retained within specific people, units and departments. Decision making is consequently restricted to this data and is therefore restricted in its scope, outlook and ultimately its potential. And typically where there is a lack of data, this leads to assumption and inherent inaccuracies. In short, silo-based thinking leads to silo-based data capture, which leads to silo-based-decision making.

This isn't a new phenomenon and there are articles that exist online since the beginning of the web that talk about this. But with the increase in the amount of data being captured about people, their activities, their multi-device habits and transactions they perform all the time ... the world is collecting and storing more information than ever before (often with little consideration of how it will be used subsequent) and this creates a growing problem. For example, I remember just a few years ago working on the implementation of a large eCommerce site, where a key requirement was to capture every possible piece of  web analytics data (including the complete user journey for every visitor) in another database not accessible by the rest of the business . When I asked why this was, the only rationale given at the time was "I don't know, but we might want to look at it later".

However this issue is something that 'big data' practitioners are now starting to get their teeth into. Products like Autonomy are now worth billions of dollars (although perhaps only to HP at the beginning of 2012 now) because they can make sense of huge amounts of structured and unstructured data to provide business insight.
So maybe there was a reason for storing all that individual website visitor data back then...

Big Data and The Republican Loss

This is more of a question than an observation, which is why I've posted it on a Sunday (I'm not expecting an answer and I'm assuming most professional users are out or not reading blogs & Twitter ).

I've been reading a fair bit this weekend about Orca, Mitt Romney's voter assessment software, built by an anonymous supplier on a Microsoft platform.

According to various articles, Orca suffered stability and accuracy issues in the run-up to the election last week. Something many people are now claiming cost The Republicans the Presidency of The USA.

Orca was apparently built pretty hastily 7 months ago to rival the effect of a similar piece of kit used successful by The Democrats in 2008 and wheeled-out and improved upon again this year. Both systems apparently crunch 'big data' to identify likely voters and contributors for their respective parties. Only Orca didn't deliver.... It apparently crashed, locked out users and couldn't handle the data entry required of it.

So my question is simply this... If Mitt Romney is from such a prestigious financial management background (Bain), where spreadsheets have been the backbone of accurate decision making for over a decade ... then how come it was this crucial element that was so weak in The Republican campaign?

Does big data mean big problems?

The subject of Big Data is currently reaching the level of hype previously reserved for Social Media, Web2.0 and the dotcom boom & bust before it. Just like these previous concepts... big data is large enough, complex enough and technical enough to bewilder a lot of 'normal' people. And like its predecessors, it is sufficiently hard enough to describe... however I like the explanation:

"Big Data is anything that doesn't fit into an Excel spread sheet"

However it is obvious that some companies are making progress with big data... whilst others, especially the consultancies, are keen to make out they have been involved with big data since before it was big news.

Of interest to me was this video posting by McKinsey, which explains how to deliver big data and the necessary analytics tools into the hands of  managers who need it.
http://www.mckinsey.com/features/advanced_analytics

However I think the challenge with all of this talk about big data is not only of defining it , but also getting started in the subject. You can talk about big data all you like, but how do you actually start making progress with analysing it and when does it actually become database administration (which has been dealing with gigabytes of data since before the dotcom bubble).

Who do you trust with your Big Data?

OK, I admit it..... I think Big Data is pretty sexy (But then I find anything remotely sexy if I don't fully understand it and want to. Its why I married a woman far more intelligent than me!). And if you're also a regular reader of this blog, you'll have noticed that I've recently been going on about the use of data in the motor insurer and aggregator market.

For me, the challenge with collecting vast amounts of data on each individual person's activity, such as driving speed and their late night policy breaking trip to the nearest supermarket, isn't about how an insurance company is going to calculate exactly how much extra to charge you for running out after midnight for those cigarettes, nappies, etc. but how you are going to get hold of that data when you decide to change suppliers. Because surely if you find it difficulty to move your own data away from one company who stores your data, they have an advantage over you?

So surely the answer is to not put your data with any one specific insurance company, as they could well have the advantage mentioned above. But to put your data in the hands of an intermediary service.

But again, you have the issue of who you trust to hold this data on your behalf. Perhaps a company with the infrastructure to easily hold your data securely, who can have access to a complex search service to be able to find trends in big data and who understands the online insurance aggregation market?

Which company could provide such a service? I must Google it......

Big data and telematics insurance challenge

In my research on the development of the telematics and the insurance industry  I have realised that my slide and previous post on the maturity of the aggregators needs to change. This is caused by the addition of data..... Big data!

As premiums get more specific to the individual driving behaviour and more data is collected on each policy holder, there are questions that I've got:

1. If I want to have access to all the data about my driving behaviour, can I?
2. How much data is this and how do I get it & store it?
3. How do I provide this data to another potential insurer to be able to compare their premium?
4. How would I then provide it to several insurers at the same time as part of an aggregate service and are they able to produce comparable policies?

So far there doesn't seem to be any easy way of taking one set of telematics driving data and moving it across to another company. Surely this will become of more and more concern to drivers and also to the people that set legal policy & regulations? So if the Data Commissioner's office and even the financial service regulation bodies are not taking an obvious interest, you can be pretty sure they will do soon.

In the end, telematics does not become an issue about creating a comparable policy from insurer to insurer, it is about the bigger issue of how user data is collected, where and who stores it, who has access to it (with and without the insurers permission) and what eventual use this is put to.

What it means to be Head of Digital

I've had senior online roles for way over a decade now and have met some of the best people in the UK digital industry along the way. This also includes individuals who have been given or earned the title of Head of Digital (as well as having done the role myself).

I therefore thought I would explain what I think the Head of Digital's role encompasses and perhaps give some food for thought on the subject:
Note: it should go without saying that this also refers to a significant extent to other similar titles such as head of online, vp of ecommerce or director of digital and multi-channel.

1. Leadership
First and foremost, any Digital head must possess leadership. Just because online is a relatively new route up the corporate ladder, it doesn't mean we have to ignore one of the key characteristics needed. I will also go as far as saying that they also need to have two specific leadership qualities:
a) To lead change wherever and whenever they can
b) To provide thought leadership (and not just by repeating and re-processing the ideas of others, I mean by having new and original ideas that are innovative and still add business value - and you though this stuff was easy?)

2. Strategic vision
Clarity of what online success looks like and how to get there is key. You don't necessary need an MBA from one of the top business schools, but you need to have the ability to understand where digital is going within your market and what the latest trends are. Also being able to communicate this vision is also really important (as there's no point having a vision if you don't share it).

3. User Experience insight
Its imperative to have an understanding of the key ways to optimise the online customer journey. From minimising bounce rates on the homepage, through to sharpening up the conversion processes across your site. Note: I'm not saying you need to be a leading information architect (and very few Head of Digital people I have met have actually come directly from this route), but an appreciation of the main concepts and having an understanding of when to focus on this is essential.

4. Advanced Digital Marketing skills
Yes, an understanding of the main digital marketing channels is vitally important for this role. However more and more I'm seeing the combination of data and marketing to optimise the customer contact opportunity (and therefore the revenue).

5. A passion for numbers
Sure, there's all the great innovative stuff to look at, but the data you can now pull from your web analytics and associated packages is immense, not including the challenge that comes from 'big data'. If you're a Head of Digital and you're not prepared to stick your nose into a cross-tabulated spreadsheet now and then.... you're probably in the wrong job.

6. Technical expertise
How to split your readership in one easy way..... by insisting that the most senior digitally-orientated person in an organisation has to have some technical nous. But its true!
Note: I'm not stating that the VP of Internet needs to have the same level of technical competence as your Development Manager, but I'm definitely suggesting they should at least be able to confer on a lot of technical issues.

Have I missed anything?

How can companies stay ahead in the 21st Century?

Business life is hard right now and even the large corporates are having to examine everything they do to stat ahead. Here’s some of my high-level thoughts on the trends andapproaches that are shaping the modern digital workplace and therefore how competitive edge can still be maintained:

1. Digital DNA

Online is not a 'bolt on' to the customer's life, it is howthey live their life now; digital has become part of the customer DNA. Theyadopt, use and integrate technology all the time, to either save time or wastetime. Businesses therefore need to put themselves in the mind of thedigitally-savvy customer or just think like the customer they actually are.This is especially true of the newer generation (Generation Y / Millenials /etc.) who are ‘Digital Natives’, for whom most cannot remember a time beforethe Internet and the use of multiple screens is an everyday occurrence.

The aim is therefore to create understanding, encourageinnovative & agile ‘web 2.0’ thinking within a company and learn how to apply this within the modern working environment.

2. Customer-centric 

The concept of ‘User Centred Design’ has been successfullyutilised for over a decade now to create online experiences that put the userat the heart of the process, rather than just being a passive node that has todeal with whatever interface the system creates at the end.

This turns some business processes on their heads, as theway some corporate departments and product catalogues are structured are notnecessarily the way that users want to browse, search, consume, etc.

Be prepared to turn things upside down if it means thedifference between doing what you’ve always done and what needs to be done tomove forwards.

3. The connected corporate ecosystem

As more systems become interconnected and as we all learnthat data does not have to be re-entered if it already exist online in someform…. This will provide the opportunity for some data (e.g. inferred from auser’s browsing history) to be integrated with other data (e.g. explicitknowledge about a user’s age, purchase habits, etc.) and to significantlyreduce human errors such as those from simple data re-keying. The semantic webis almost upon us, is your business ready?

4. Use of Data for insight

Thanks to the Internet, every 2 days the human race nowcreates as much information as it did in its entire history up to 2003. Thisgrowth in ‘big data’ creates its own challenges (e.g. the storage and abilityto quickly access the right data, re-posted mis-information, etc.) as well asits own opportunities, such as an unprecedented amount of useful data on userbehaviour.  This therefore facilitates afar greater level of personalisation and supports various real-time activities& incentives such as individual offers and rewards (perhaps the very reasonwhy mega supermarket Tesco used & subsequently purchased DunnHumby forcustomer insight and data driven marketing purposes) plus then the future predictivecapabilities that may come from this.