The perils of sharing different types of data at once

Organisations of all sizes have data that is of benefit to users both inside and outside their boundaries (both in the private and public sectors). Therefore having the tools and processes to find and share this data should make things run more efficiently and effectively... hopefully.

In a recent client conversation, we discussed the creation of a data portal / platform for the easier discovery and sharing of data. This inevitably led to the discussion about the types of data that the organisation wanted to share and who they wanted to share it with. Leading to the realisation that they actually had the need for sharing data that ranged from the very secure (restricted customer or commercially sensitive information) through to Open Data (information that they wanted to share for free outside of the organisation). They also had requirements for sharing data that sat somewhere in the murky area between those two extremes (information limited either by license or by access / user) including some they wanted to monetize.

Most data platform projects I have worked on previously have focused on the sharing of particular types of data (e.g. just Personally Identifiable user information or a mix of limited /restricted and Open data), but not the need for sharing different data sources from across The Data Spectrum:

https://www.theodi.org/about-the-odi/the-data-spectrum/

So creating a single data sharing technology platform that can share any sort of organisation data creates a number of issues:

Privacy and security:
Balancing accessibility and protection is tricky. You want your service to be easy to use, but with more sensitive data, strict access controls and strong security measures are vital.

Data standardisation:
Different users across an organizations often format their data differently. Such a service needs to be able to handle these inconsistencies or offer tools to convert data to a common format for sharing.

Traceability & reusability:
Data consumers need to understand how the data was collected and what it can be used for. Your service should provide clear audit trails and data provenance, as well as ways of licensing and charging for the data - especially if the data consumer is an external user.

World Passenger Festival 2023 - questions part 6

In advance of the World Passenger Festival 2023, I was asked to answer a few questions. These were published in a document for those attending. (Question 1 is here , Question 2 is here, Question 3 is here, Question 4 is here and Question 5 is here)

If you had one message for the public transport industry, what would it be?

Transport is no longer about regional applications, platforms or even modes.
It is about making data F.A.I.R. (Findable, Accessible, Interoperable & Reusable) across entire mobility ecosystems, to quickly take advantage of new technologies and achieve Net Zero.

World Passenger Festival 2023 - questions part 5

In advance of the World Passenger Festival 2023, I was asked to answer a few questions. These were published in a document for those attending. (Question 1 is here , Question 2 is here, Question 3 is here and Question 4 is here)

Here's the fifth question.

AI is the technology of the moment: do you think the public transport sector is taking the most of the opportunity with this technology? What do the possibilities look like?

It is pretty hard to avoid the mass of AI related articles and opinions right now. With Artificial Intelligence ‘experts’ seemingly appearing from every direction (especially on LinkedIn!).

But it is obvious that our sector is not currently making full use of this technology. Sure, some more innovative companies have been using Machine Learning (ML) processes for some while, such as to ‘watch’ video clips & automatically report safety issues or to cleverly merge disparate transport data sources to analyse & enrich them. But transport is already late to the AI party in many ways.

As an example, the current generation of Generative AI services (e.g. Chat GPT & Google Bard) learn by taking in huge amounts of content and processing it to create in-depth replies to a range of questions. But these services are limited at scraping documents & websites, so the best way to train them is to provide them with access to APIs of data sources. But in a sector where the zipped-up text file is often the most popular data exchange format … we are missing the technological opportunity to train these services properly and then learn how to use them for better public transport.

World Passenger Festival 2023 - questions part 4

In advance of the World Passenger Festival 2023, I was asked to answer a few questions. These were published in a document for those attending. (Question 1 is here , Question 2 is here and Question 3 is here)

Here's the fourth question.

You are a big proponent of open data, what is the benefit of this approach?

Yes, I am. But firstly, let’s be very clear on the definition of Open Data:

“Open data is data that can be freely used, re-used and redistributed by anyone - subject only, at most, to the requirement to attribute and share alike.”
https://opendatahandbook.org/guide/en/what-is-open-data/

Open Data is NOT data that is shared but limited in its subsequent use. Open Data is NOT data that a company charges some else for using. Open Data is NOT specific customer account data that is securely shared with a third party (that’s what the UK Government calls ‘Smart Data’).

Open Data is the type of data which sits on the far right of The Data Spectrum and has significant potential benefits for the transport & mobility sector and society.
https://opentransport.co.uk/the-data-spectrum-for-transport-mobility/

These benefits include:

- transparency & accountability of transport operations

- time saving from better informed journeys

- growth from data-powered innovations

- reduction in costs for providing information (e.g. contact centres)

- integration of other providers & modes (e.g. healthier / greener options)

With a study by Transport for London back in 2017 showing that the release of their Open Data is generating £130m annually.

The UK Department for Transport (DfT) has also fully embraced the use of Open Data. In its recent Transport Data Strategy “Innovation through data” it clearly states that:
“Data should be open by default and using open standards”

This ‘Open by default’ approach means that UK public transportation providers must now make their data openly available, unless it is controlled (e.g. it is sensitive), and DfT will even challenge data owners as to why they cannot make their mobility data open.

World Passenger Festival 2023 - questions part 3

In advance of the World Passenger Festival 2023, I was asked to answer a few questions. These were published in a document for those attending. (Question 1 is here and Question 2 is here)

Here's the third question.

What are the biggest opportunities you see for data applications in the transport space?

There’s a few of major transport data trends taking shape in different regions and countries right now, which should enable much better services for both passenger & freight services.

1 System standardisation
The sector is moving towards more standardised and interoperable data technologies that are going to make it much easier and cheaper to implement new digital mobility systems. This is demonstrated by the success of the Open Sales and Distribution Model (OSDM), a pan-European transport authority & supplier initiative that has developed & published an Open Standard API specification for rail retailing.
Hint: If your new online retailing service RFP does not specify the use of OSDM, you may find yourself with more technical complexity & cost in the longer-term.

2 Multi-modal and multi-sector data collaboration
Each mode of transport is gradually realising that it needs to standardise its data and provide it in a more consistent and accessible way. But this work is still happening in relative isolation from each other mode and also from other sectors. For example… buses don’t just share the roads with private cars and motorbikes, they also share them with taxis, delivery lorries, refuse collection trucks, sightseeing coaches, and emergency vehicles. But there’s little to no discussion, let alone collaboration effort, between the various standards bodies to ensure that one sector’s definition of something (e.g. a road, a pavement, etc.) is the same as another’s.
By working with these different sector standards bodies (e.g. smart cities, utilities, tourism, etc.) now, we can hopefully save a lot of data focused effort and re-work in the future.

3 The development of Data Sharing Platforms
RDG has made a great step forward to making UK train data more discoverable and usable, with the recent introduction of the Rail Data Marketplace. And other countries are doing the same, by launching their own transport data sharing platforms.
The obvious next step in their evolution is to extend the scope of these platforms to provide new, innovative, and trusted data sources for an entire country-wide mobility ecosystem, as well as supporting data-driven innovations such as Digital Twins.

4 Moving from historic to predictive data
We are now very good at collecting & analysing transport data about things that have happened in the past (e.g. tickets purchased, journeys made, delays incurred, etc.). However, we are far less accomplished at working out what is going to happen, even in the near future, so that we can understand demand and create better experiences for passengers.

The annoying thing is a lot of relevant data for making better transport predictions is already available, it’s just not aligned. For example, systems already know when planes and ferries are going to be delayed in the next few hours and others know when large festivals & concerts are going to take place months in advance.

World Passenger Festival 2023 - questions part 2

In advance of the World Passenger Festival 2023, I was asked to answer a few questions. These were published in a document for those attending. (Question 1 is here)

Here's the second question.

What is your vision for the future of public transport and where are the biggest challenges we face today?

We are all unfortunately doing a great job of destroying our environment. We are doing this by pumping so much new Carbon Dioxide and other gases into our atmosphere that it reflects heat back at us and cooks the planet. Second on the list of the biggest Greenhouse Gas emitters (after Energy production) is our own sector, which includes all passenger & freight transportation via land, sea and air. We therefore have a responsibility to stop this and urgently move to modes that can convey people and goods for the least amount of carbon.

Public transport therefore has a huge role to play in our shift to Net Zero, especially as part of a more joined-up mobility ecosystem. Or put more clearly… all multi-modal mobility services & platforms must now have sustainable public transport at their core.

One key data related challenge, that most people across the sector are not yet aware of, is customer data lock-in. The GDPR Right to Data Portability is very clear and enforceable. It allows individuals to obtain and reuse their personal data for their own purposes across different services. Meaning they have the right to move, copy or transfer personal data easily from transport platform to another in a safe and secure way, without affecting its usability. But when you look at the functionality of most transport accounts, there’s no way for a customer to move their data between providers and platforms. It therefore cannot be exported, shared, or integrated with data from other modes and providers to create a more holistic view of a passenger’s tickets & journeys. Also, with many regions or city authorities either having recently implemented or now looking to introduce different Account Based Ticketing and Mobility-as-a-Service (MaaS) apps, the issue of locked-in transport data is going to get worse. Then, once this GDPR Right starts to be enforced by the relevant governments, the sector will have a lot of work on its hands to change systems and processes quickly.

What does data portability have to do with Net Zero aims? Well, I believe that it is only by having a completely joined up view of all mobility data, can we then hope to properly change customer behaviour and move them to more sustainable public transport modes. Or stated more simply… no data, no decarbonisation!

World Passenger Festival 2023 - questions part 1

In advance of the World Passenger Festival 2023, I was asked to answer a few questions. These were published in a document for those attending.

In the next few posts, I will publish those questions and my replies. Here's the first...

Can you share your background work with us?

I’ve been working across the Digital sector for 30 years. This includes being the Tech Director for a start-up in the dotcom era and now running Ideal Interface, a technology & online marketing consultancy based near Glasgow.

My love of transport started back in 2005, when I joined P&O Ferries as their Head of eBusiness. Since then, we’ve provided digital and technology assistance to a range of public and private organisations across the transport and mobility sector.

Until recently I was the Architect & Technology Lead for the Rail Delivery Group’s [RDG] Rail Data Marketplace, a great project that provides a single data sharing platform for rail related information.  And currently I am back supporting several start-ups in the race to make sense of data and integration technologies across both the mobility and Net Zero markets.

I’m also the founder of The Open Transport Initiative (the Open Standard that enables customer account data integration & interoperability across the sector – best explained as the ‘Open Banking for Transport’). Plus I’ve just been invited to join the Steering Board of MaaS Scotland, where I regularly contribute to their Data & Governance Special Interest Group.

World Passenger Festival 2023

 Last week I not only attended the World Passenger Festival in Vienna (moved this year after several in Amsterdam), I was the moderator for the Digital Tech / Transformation Summit... a large presentation room dedicated to the increasing amount of online technologies being applied to passenger transit globally.

This was a packed 2 days of presentations and Q&A hosted my yours truly, where representatives from across the transport & mobility sector got to hear about the latest innovations and projects.

The highlights for me were:

1. Being judge (and 'Dragon') at an event held the day before at the OBB Innovation Factory

2. Moderating a panel oon the topic of:
"Driving Innovation in transport: overcoming the barriers to success in developing a digitally savvy mindset in risk averse organisations. "

With European innovation leaders:

• Peter Schindlecker, Head of Open Innovation, ÖBB Open Innovation  
• Robert Lynn, Head of Innovation, Dublin Bus  
• Carolin Kodde, Lead, Long Distance Train Start Ups, DB Mindbox 
• Benoit Muller, Innovation Lab Director, SNCF Connect & Tech 
• Severine Mastikian, Rail and Transit, Europe Lead, Accenture

3. Moderating the keynote panel with senior leaders