Case Study · MSc Dissertation · Time-Series Forecasting

GB Balancing Mechanism Cost Forecasting

Forecasting the total cost of balancing Great Britain's electricity grid, half hour by half hour. Elexon BMRS data, statistical and machine learning models, and an open reproducible pipeline.

Python LEAR XGBoost LSTM Elexon BMRS API Time-series Pandas Scikit-learn

131k+

Half-hourly rows

Models compared

Significance testing

Background

The GB electricity balancing mechanism (BM) is the final tool the system operator uses to match supply and demand in real time. Renewable intermittency, unexpected demand and plant trips all push balancing costs up, and those costs ultimately land on consumer bills.

The dissertation question: can machine learning forecast the total aggregate cost of the balancing mechanism per settlement period, using only publicly available half-hourly data? Published GB work targets the imbalance price; no paper has targeted aggregate BM cost directly or released an open pipeline for it on the BMRS v2 API. That gap is the contribution.

Data Pipeline

⬇

Elexon BMRS API ingestion

Half-hourly settlement data: system prices, imbalance volume and cost (131k+ rows), demand outturn, and generation mix (wind, solar, gas, nuclear shares) via the Carbon Intensity API.

⚙

Feature engineering

Lag features (1, 2, 48 periods), 24-hour rolling statistics, calendar features (hour, weekday, month, weekend), wind generation derived from mix share and demand.

🎯

Target definition

Regression on total balancing cost in GBP per half-hour settlement period. Aggregate cost, not the imbalance price that existing papers target.

🌲

Model training

LEAR (regularised linear baseline), XGBoost and LSTM, trained and compared under an identical rolling-window scheme so no model sees the future.

🔍

Explainability

MAE and RMSE out of sample, with Diebold-Mariano tests to check whether model differences are statistically significant, following the Lago et al. (2021) evaluation standard.

Models Under Evaluation

Model	Approach	Key strength	Status
LEAR	LASSO-estimated auto-regressive linear model	The statistical benchmark that deep learning must beat (Lago et al. 2021)	Next up
XGBoost	Gradient boosted trees, L1/L2 regularisation	Strong on tabular features, robust to outliers	Planned
LSTM	Recurrent neural network for sequences	Can learn temporal patterns lag features miss	Planned
Naive baseline	Persistence (lag-1 and lag-48 prediction)	Sets the minimum bar to beat	Done

Novel Contribution: Aggregate Cost as the Target

Existing GB forecasting papers target the imbalance price (System Buy/Sell Price): what one MWh of imbalance costs. This project targets something different and arguably more decision-relevant: the total aggregate cost of the balancing mechanism per settlement period, the number that flows through to consumer bills and that NESO reports months in arrears.

No published ML paper has forecast aggregate GB BM cost directly, and none provides an open, reproducible pipeline on Elexon's current BMRS v2 API. The dissertation delivers both, plus a like-for-like comparison of a statistical baseline (LEAR) against machine learning (XGBoost) and deep learning (LSTM).

Key Challenges

→ Live-only API endpoints. Several BMRS v2 forecast endpoints silently ignore historical date parameters and only return live data. Documenting which endpoints are usable for backtesting is itself a useful output for anyone building on this API.
→ Temporal leakage. Standard k-fold cross-validation is invalid for time series because future data leaks into training. A rolling-window scheme preserves temporal ordering throughout.
→ Non-stationarity. GB energy market conditions shift significantly across years. Testing whether models trained on 2022-2023 (post-Ukraine energy crisis) generalise to 2024-2025 is a key validation question.

Timeline

Proposal submitted June 2026. Data ingestion and feature engineering complete: three raw datasets merged into a 131k-row half-hourly feature table. Model training runs through autumn 2026, with the final dissertation due January 2027. This page will be updated with results and a link to the public GitHub repo as the work progresses.

← back to portfolio ← reception system