Challenges of governing a Data Sharing Infrastructure

Summary

The Government response to the energy system ‘digital spine’ feasibility study articulated a compelling vision for a digitalised and net-zero energy system based on widespread adoption of Data Sharing Infrastructure (DSI). DSI is defined as a minimum set of responsibilities, governance, functions and processes for secure, interoperable data sharing across the energy system.

Ofgem’s consultation on the governance structure of a DSI marks a crucial first step in developing a pathway towards delivery. However, as our response sets out, we are concerned that Ofgem’s minded-to position will hamper the development of DSI including risks of vendor lock-in and associated costs.

This blog highlights three key challenges with the pathway indicated in the consultation and three recommendations on how the System Operator can enable an alternative, industry-inclusive (‘Hybrid’) model to development. It concludes with a call to action, urging policymakers to act decisively now to avoid misalignment with the key European data space initiatives and lock-in a costly and cumbersome NESO-centric governance model until 2028.

Challenges:

• The need for a clearer technical definition to DSI components.
• The need for a funding mechanism, timeline and governance model which doesn’t fully rely on the System Operator until 2028.
• The need to be realistic about the System Operator’s ability to fulfil key roles and responsibilities, and to better leverage industry engineering expertise.

Recommendations:

• Empower industry co-design and delivery through a ‘Hybrid’ System Operator-led governance model.
• Ensure data valorisation in the B2B context is proven early on.
• Develop policy and regulatory measures now to support ‘steady-state’ operation.

Introduction

Arenko appreciates the opportunity to contribute to Ofgem’s consultation on the governance of Data Sharing Infrastructure (DSI). With over a decade of extensive experience in energy storage and enterprise software development, we recognise that widespread adoption of DSI would catalyse the delivery of a sustainable energy system, enabling the industry to rapidly digitalise and manage future complexity and volatility.

The potential of DSI lies in empowering energy sector participants to seamlessly explore and combine data from different owners, as well as exchange new models and knowledge extracted from shared data. Data standardisation is also crucial to systemically reduce the costs associated with the common layer of digital communications infrastructure required for integrating low-carbon technologies—such as PV arrays, standalone batteries, charge points, and electric heating—into the energy system.

As highlighted in a plethora of European literature on Data Spaces, exemplary use-cases of DSI being actively trialled include, but are not limited to:

• Renewables O&M Optimisation and Grid Integration
• TSO-DSO Coordination for Flexibility
• Residential Home Energy Management Integrating Distributed Energy Resources Flexibility Aggregation
• Electromobility: Services Roaming, Load Forecasting, and Schedule Planning
• Collective Self-Consumption and Optimised Sharing for Energy Communities

Ofgem hints at wider use-cases such as regulatory reporting (e.g. submission of RIGs), strategic planning (e.g. CSNP work), and regional planning (e.g. RESP work) in the October 2023 FSNR publication. This underscores the promise of standardised data sharing and pooling that DSI enables. One could also envision broader applications, such as enhancing NESO-DSO connections queue management and improving the exchange of network data for the development of joint System Operator flexibility needs assessments—pertinent use-cases for Great Britain.

However, we believe that the governance model supported by Ofgem in this consultation will hamper the development of DSI. It is rigid, exclusive, and not directly aligned with industry needs, and thus is destined to underdeliver on its promises.

This blog highlights three key challenges with the pathway indicated by Ofgem, before providing three recommendations on how we can enable an alternative, industry-inclusive (‘Hybrid’) model. We encourage all readers to read our full consultation response, which offers greater detail and includes an annex mapping the latest version (v2) of the Common European Energy Data Space blueprint to Ofgem’s articulation of DSI.

Challenge #1: The need for a clearer technical definition to DSI components

The first challenge is simply the lack of technical definition to the constituent components and building blocks of the DSI - beyond Trust, Prepare and Share - which hampers deliberative public consultation on governance at this stage. We fear this will lead to the pursuit of non-interoperable solutions with unclear outputs and expectations about how they all fit together. It also may serve to obfuscate potential alignment opportunities with the European Data Space initiative which is the world’s most advanced initiative towards development of common industry-wide DSI right on our doorstep.

The latest version of the CEEDS architecture (published July 2024) developed by int:net side-by-side with the functional components of a DSI offered by ARUP, ESC and University of Bath (published August 2024).

Challenge #2: The need for a funding mechanism, timeline, and governance model which doesn’t fully rely on the System Operator until 2028

The second challenge concerns Ofgem’s considerations regarding funding, timeline, and structure of the short-term governance model, which currently justify the System Operator being responsible for development until 2028. We believe that relying solely on the System Operator in this manner overlooks successful stakeholder-driven governance models already deployed across European initiatives.

The proposed high-level roadmap for Pilot and MVP delivery (2024 - 2036)

By progressing only the outage planning pilot project—involving a consortium of networks with a history of failed digitalisation projects, exchanging datasets using standards and mechanisms bespoke to network companies—Ofgem ignores broader opportunities.

The MVP candidates for DSI seem to be a list of problems the NESO needs to solve, as echoed in DESNZ’s recent letter stating that “initial work should be focussed on areas that contribute to strategic priorities, particularly as ESO transitions into the NESO.”

While we recognise the potential of DSI to offer solutions here, this does not imply that the NESO should exclusively deliver more innovation projects or develop another ‘product’ it will ‘own’.

Instead, DSI should emerge from a living set of agreements and interactions among parties driving new decentralised data management strategies using shared open-source containerised solutions. The framing should be reset to position the System Operator as an independent sponsor of core interoperability mechanisms, shared by a federation of industry DSIs. DSI fundamentally needs to be actively maintained by stakeholders and delivered through real software development as part of industry deals and partnerships. Meanwhile, core interoperability mechanisms between DSIs need to be procured ‘once and for all’ to facilitate specific use-case agnostic, not-for-profit and cross-sectoral DSI building blocks.

This approach involves incentivising industry consortia to explore a wider range of use-cases, particularly those better served by existing harmonised data models, ontologies, and IT architectures, leading to shorter development timelines. We should leverage the terminologies, reference implementations, and already-existing open-source DSI components (e.g. OneNet connector is TRL 8) developed in Europe, to enable the rapid and industry-inclusive development of a GB energy data space. The final section of this blog outlines in more detail an alternative way forward to enable timely, industry-inclusive, System Operator-orchestrated DSI development.

Challenge #3: The need to be realistic about the System Operator’s ability to fulfil key roles and responsibilities, and to better leverage industry engineering expertise

We are apprehensive about Ofgem’s assessment that critical functions (such as privilege management, data standards, cybersecurity, and cross-sector integration) are best managed by establishing a new central expert entity reliant on procurement and external consultants, nevermind if that entity is the System Operator.

Given Ofgem’s minded-to position that this central entity should be the System Operator, evidence and our experience show that NESO IT projects have been almost fully reliant on consultants - from project scoping to testing. Additionally, we find that the NESO has difficulty judging whether it has done a good job, and has a poor track record in basic technology delivery: user requirement gathering, product delivery management, doing user testing, basic definitions of ‘done’ for internal projects, let alone on external facing projects/interfaces. Independent reviews by Zühlke and Coforge document the significant overspending and poor practices in IT delivery. Alarmingly, the RIIO-2 Zühlke overview reveals that £517 million (93%) out of £556 million of IT investment raised concerns, with 55% (£307 million) raising serious concerns.

It’s troubling therefore that each supposed DSI component in Ofgem’s framing aligns so neatly with a solution architecture that the interim DSI coordinator can simply ‘shop for’ via procurement. The role of Architecture is conceived as: “Will procure the right solutions of Trust, Prepare, Share, evolve the architecture to the needs of its users, seek feedback regularly, work with providers to ensure solutions are still appropriate for use-cases.”

Such an approach would mirror classic mismanaged digitalisation projects we’ve seen before—like the costly Energy Balancing System (EBS), conceived in 2013, which never delivered tangible value and is set to be decommissioned in 2025. Fundamental issues stemmed from ‘big bang’ implementations instead of agile delivery, and vendor lock-in (and associated costs) with Hitachi. Ironically, the NESO acknowledges these issues with EBS delivery, yet seems poised to repeat them with its new network control and management system provided by GE, as highlighted by Coforge. This appears to be the same path proposed for DSI MVP development.

An Alternative Way Forward?

Having documented the challenges associated with the current SO-centric, industry-exclusive development model, our response proposes a more inclusive and effective approach. This section outlines three key recommendations to enable industry-inclusive DSI development.

Recommendation #1: Empower industry co-design and delivery through a ‘Hybrid’ System Operator-led governance model

In our proposed alternative model, the interim System Operator DSI coordinator would primarily manage, coordinate, and provide specifications to use-case-oriented industry working groups. These working groups would comprise actual product and engineering representatives from ongoing industry digitalisation partnerships. Their goal would be to deliver standard software artefacts, ensuring that development is driven by practical needs and real-world applications.

A rough sketch of what Arenko’s proposed industry-inclusive hybrid model would look like

By adopting a Hybrid model, we refine the responsibilities of the System Operator to effectively harness insights from industry-consortia-led initiatives. This facilitates a more collaborative and efficient development process. Key activities of the SO in this model would include:

• Promoting and incentivising priority use-cases: Identify and advocate for use-cases that offer significant industry benefits, sharing best practices and success stories to encourage wider adoption; and provide the seed investments to encourage industry development of DSI.
• Developing architecture for common building blocks: Competitively procure shared DSI building blocks such as certain container platform features (e.g. logging systems and health monitoring tools), as well as sponsoring cross-sector capabilities that require independent development beyond specific industry projects.
• Acting as an authority on standards implementation: Serve as a central body for enforcing standards, contributing to international standardisation efforts, and promoting domestic adoption to ensure global interoperability.
• Ensuring consistency in Trust Framework components: Maintain uniformity in trust elements such as access and usage policies, and identity management across different use-cases, while providing guidance on use-case specific legal and cybersecurity issues.
• Overseeing a knowledge base and support centre: Manage a repository of standard artefacts and reference implementations, supporting industry consortia in their development efforts.

Beyond these activities, the System Operator should employ competitive tender exercises to advance its own IT development work. This would enable the SO to trial common DSI components relevant to its specific priority use-cases. By doing so, the SO addresses its immediate operational needs while contributing to the broader ecosystem of the DSI.

Recommendation #2: Ensure data valorisation in the B2B context is proven early on

A significant advantage of a ‘hybrid’ approach is that it tackles data valorisation concerns in the B2B context—an area not addressed in the consultation and which we do not believe is achievable through an SO-exclusive funding approach. While obtaining consumer consent can be relatively straightforward, securing business consent requires clear incentives and value propositions.

Open data initiatives that implement the ‘presumed open’ principle are valuable for data of common interest. However, sustainable business models for data exchange platforms need to assess data value based on specific use-cases. This requires exploring models where data providers are fairly compensated for their data, encouraging more businesses to participate in data sharing. In the B2B context, it’s crucial to trial the following incentives:

• Data-by-money (monetary incentive): Data owners accept to share data because they are financially compensated if their data is relevant for solving analytics/optimisation tasks and pay in case data from others is relevant to their own tasks. This is more suitable for the B2C or B2B2C trials (such as for flexibility service providers).
• Data-by-data (non-monetary incentive): Barter trading, i.e. data-by-data exchange schemes for non-monetary compensation. Data owners agree to share and receive data with approximately the same value, exchanged data includes services as, for example, alarm signals, fault notifications, and indications of maintenance actions.

Our proposed approach allows data valorisation to be addressed early on through direct engagement with industry stakeholders.

Recommendation #3: Develop policy and regulatory measures now to support ‘steady-state’ operation

To transition from an interim model and ensure long-term viability, new policy and regulatory measures are essential. Developing a light-touch DSI steady-state regulatory framework—which sets the policy and social parameters within which constituent components operate—is likely to be needed. Measures for ensuring cybersecurity is not compromised and data sovereignty is respected are two obvious starting points. Other key areas it might include based on eminent European literature are:

• Avoidance of vendor and initiative lock-ins. The accompanying DSI regulatory framework should support competition and disruptive innovation, avoiding vendor lock-ins. It should facilitate the reuse of open-source libraries to ensure the fastest innovation to market and minimise development duplications.
• Foster harmonisation of mechanisms and standards. Most of the energy and transport use cases in Europe deploy data models, ontologies and functionalities that are already covered by existing standards, with a predominance of CIM, SAREF4ENER and OCPP. Effective and long-term harmonisation mechanisms are necessary, particularly with regional ecosystems, both in the energy domain and on a cross-sectoral basis.
• Quality assurance standards. High data quality assurance is the bedrock of usable DSI. It is imperative to develop frameworks and guidelines that specifically highlight data quality considerations. Organisations will need to build capabilities to detect anomalies, outliers, and data discrepancies, and proactively identify and resolve data inaccuracies.

What is at risk? The implications of misalignment and the urgency of timely development

The current high-level roadmap suggests DSI will only be made ‘generally available’ beyond regulated networks after 2029. Yet DSI is inherently stakeholder-driven and needs to include industry from the outset, not as an ‘end-user’. These timelines will result in the UK falling behind global advancements, especially in this decade of rapid power system transformation, making it imperative to adopt a more collaborative and industry-inclusive approach now. Otherwise we risk vendor lock-in and mounting associated costs from lack of data standardisation and/or dependence on large multinational tech vendors who develop closed industry integrations, limiting flexibility and innovation.

Immediate Actions to Foster Industry Engagement and Technical Consistency with Europe

To avoid costly delays and ensure that DSI development keeps pace with both UK and European digitalisation goals, we propose the following actions to policymakers:

• Alignment with European Standards: Ofgem or the interim DSI coordinator (IDC) should oversee a technical alignment exercise with outputs to date of GB DSI components and the int:net CEEDS blueprint components, and adopt their terminology (or map how GB terminology aligns).
• Publication of VES DSI Components: Ofgem should support the VES programme in publishing DSI components and building blocks on the EU Data Spaces Support Centre to encourage openness and transparency.
• Issuance of a Request for Proposal (RfP): Ofgem should commit to publishing a Request for Proposal for industry-consortia-inclusive DSI development using common DSI components and requiring standard artefacts, as soon as possible.
• Development of grant-funding approaches: Ofgem or the IDC should support industry consortia in seed funding in-motion projects that focus on delivering participation in Proof of Concept (POC) initiatives and developing DSI components that engage with European standards. A Hybrid governance model should be explored with industry stakeholders.

We once more welcome all stakeholders to read the full response which includes an extensive documentation of European progress in this area and an elaboration of our challenges and recommendations above. We welcome any and all feedback from industry peers on the proposed Hybrid model, our proposed actions or any other aspects of our response.

Arenko’s Interests in This Area:

In line with our commitment to data-driven energy solutions, Arenko is already actively collaborating with global energy companies, asset owners, OEMs, and battery health management companies to explore standardised data sharing infrastructure for asset optimisation and maintenance use-cases. This could involve a standard, open data management approach for sharing asset-level and market-level data that utilises common DSI, and is compatible with broader flexibility services use-cases. However, we are currently missing the incentive to drive forward GB-specific learnings through involvement in interim DSI development. We would support providing valuable insights and feedback on the real-world efficacy of proposed DSI components and building blocks and gladly contribute to a shared knowledge repository that supports ongoing improvements and the scaling of DSI across the industry.