Data Assurance Service
Accurate and reliable data is critical to the effective operation of the non-household (NHH) market and is key to operational efficiency, evidence-based improvement, and positive customer and environmental outcomes. However, data quality remains a principal market friction that is driving significant cost for trading parties and impacting the market’s ability to deliver better outcomes.
The logic and cost of trading parties working independently to fix core data fields has been questioned, with the current rate of improvement being too slow for benefits to be realised.
As part of the 2023-26 Business Plan, trading parties have approved a cleanse of non-eligible supply points and a proof of concept on a centrally managed cleanse service of premises and address data.
As the programme moves into year two, it has been been renamed from Data Cleanse Service to Data Assurance Service, to more accurately reflect the deliverables of the programme.
On 5 October MOSL presented a webinar on the progress of the Market Eligibility Assurance (previously referred to as 'high-confidence non-eligible') phase of the programme.
In partnership with a data services provider, MOSL will deliver a centrally supported data assessment and cleanse service to enable trading parties to improve and maintain the data held in CMOS.
This service will complement the work that is already underway within trading parties. Trading parties will remain the data owners and will be responsible for making the appropriate changes.
MOSL will be responsible for implementing, maintaining, and monitoring the service, incentivising data quality improvement, and ensuring data quality targets are being met.
We will work collaboratively with trading parties during the detailed design phase to agree the specific data items that are provided and the mechanism by which feedback is received and addressed. The data provided to trading parties will include:
- Ineligible premises (residential, demolished, and duplicate)
- Unmatched supply points (i.e., address data insufficient to identify a unique premises with confidence)
- Missing Unique Property Reference Number (UPRN) and Valuation Office Agency (VOA) reference data
- Full address data non-deliverable and non-addressable premises.
We will adopt a gated approach which will set out a number of ‘proof points’ to evidence the value in continuing the next phase of work. This approach mitigates the potential impact on retailers to implement changes and reduces the costs of the service in year one to the lower range proposed in our consultation at £550k (covering third party costs and MOSL resource).
In parallel, we will review the impact of key market processes on the enduring data quality, for example, the new connections process, and recommend improvements.
The key deliverables for 2023/24 can be found below. Please note these dates are subject to change:
For further information on these deliverables, please read the Central Cleanse Service section of MOSL's 2023-26 Business Plan.
Project TIDE and the Data Quality Assessment
Project TIDE (Transformation in Data Enrichment) was established in April 2022 by MOSL and Sagacity to assess the quality of premises, address, and customer data within CMOS.
Sagacity, undertook a Data Quality Assessment, which reinforced the extent of the issues in core market data. Key findings were shared with trading parties and the wider market in August 2022, with further detail on the impact of poor data quality provided in the Data Cleanse Consultation document issued in November.
To review the full findings, read the report below.
Market Eligibility Assurance Review
From 22 November, the non-household market will begin the Market Eligibility Assurance Review phase of the Data Assurance Service.
It involves trading parties reviewing and assuring or deregistering supply points that are considered ineligible for the market.
From 22 November, trading parties will have access to CMOS data which has been grouped into assurance levels 1, 2 or 3. Whilst trading parties have access to the entirety of this data, they are only required to review the ‘Level 1’ supply points within 12 months. Level 1 supply points are defined as those that have the highest match against multiple external data source indicators, suggesting that it holds a household status and is therefore ineglible for the non-household market.
The full process is outlined within the specification document, which is available via the Key documents page. It also includes further information on file formats, roles and responsibilities, and reporting.
The assurance files will be available in My Files, which can be accessed via the My MOSL homepage. Access to My MOSL has already been given to Contract Managers. If you require additional access, please contact IT Support via My MOSL.
If you have any questions on the activities required within the Market Eligibility Assurance Review phase, please contact us via IT Support on My MOSL.
To review both market-wide and trading party-specific progress on this assurance activity, please visit the dashboard on My Charts.
An interview with former MOSL CIO John Davies on the scope of the programme and the impact on trading parties.
The assurance levels beng used to segment the data required to review as part of the Market Eligibility Assurance phase is as follows:
Level one: high probability to be ineligible due to matches against multiple external data source indicators suggesting household status.
Level two: medium probability due to lesser confident matches against multiple external data source indicators suggesting household status.
Level three: low probability due to single matches to external data source indicators suggesting household status.
As a reminder, whilst trading parties will receive all levels of data, you are only required to review your 'level one' suppy points within 12 months.
MOSL is responsible for implementing, maintaining, and monitoring the service, incentivising data quality improvement and ensuring data quality targets are being met.
As part of this, MOSL is available to providing ongoing support to trading parties.
If you have a query, please raise a query to IT Support via My MOSL.
The Data Assurance Service is a multi-phase assurance programme which will take place across a number of years. The timeline of each phase is dependent on its complexity and will be reviewed before implementation to ensure the commitment from trading parties is reasonable and achievable. The indicative timeline for 2023/24 can be found here.
The first data drop for the Market Eligibility Assurance phase of the programme will take place on 22 November 2023, with data provided to trading parties monthly following this date. The full process is outlined in the specification document, which is available in the Key Documents section.
From 22 November, trading parties will have 12 months to review and assure or de-register their 'level 1' supply points provided by MOSL. Level 1 supply points are defined as those that have the highest match against multiple external data source indicators, suggesting that it holds a household status and is therefore ineglible for the non-household market.
The first and second year of the Data Assurance Service (2023-25) is being funded through Market Operator (MO) charges to wholesalers only – this includes third party costs and MOSL research.
Cost for future years’ will be determined in collaboration with trading parties – it is important to note that this method of funding is by no means creating a precedent for other MOSL activities to be funded solely by wholesalers.
The goals for year one of the service are:
- To develop and implement an assurance service for eligibility, address, and premises (including a small-scale pilot for Customer and Occupancy management)
- For trading parties to review and cleanse high-confidence ineligible premises (expected to be circa 100,000 supply points)
- For trading parties to review and cleanse a sample of high-confidence address/premises issues across a number of use cases (expected to be circa 6,000 supply points in total)
- To assess the veracity and value of a central assessment and cleanse service for each use case, refining the business case for consideration in 2024/25 (as part of business planning)
- To develop a data quality scorecard and define data quality performance metrics that will inform a reformed Market Performance Framework (MPF).
In collaboration with a data services provider and market-led working group, MOSL will deliver a centrally supported data assurance service to enable trading parties to improve and maintain their data held in CMOS.
MOSL will be responsible for implementing, maintaining, and monitoring the service, incentivising data quality improvement and ensuring data quality targets are being met.
Trading parties will remain the data owners and will be responsible for making the appropriate changes to their data in CMOS.
Through our position at the centre of the non-household market, MOSL has a unique opportunity to drive centralised change to improve the quality of data it holds in CMOS. Accurate and reliable data is critical to an effective operation of the market.
Responding to this, as well as member feedback, MOSL committed to delivering a data cleanse plan for core market items, and issued a Request for Information (RFI) in October 2020 to seek input from trading parties on the costs and challenges of managing poor quality data. In April 2022, Project TIDE (Transformation in Data Enrichment) was established to look at how data quality could be improved. Our data services partner, Sagacity, undertook a Data Quality Assessment to investigate the extent of the issues in the market. The results of this were shared with the market, and were used as part of the consultation for a Central Data Cleanse consultation in November 2022.