Case Study · MSc Dissertation · Time-Series Forecasting

GB Balancing Mechanism Cost Forecasting

Early warning system for cost spikes in Great Britain's electricity balancing mechanism. Half-hourly Elexon BMRS data, gradient boosting models, and SHAP explainability.

Python XGBoost LightGBM SHAP Elexon BMRS API Time-series Pandas Scikit-learn

48k+

Half-hourly rows

SHAP

Full explainability

EWH

Novel metric

Background

The GB electricity balancing mechanism (BM) is the final tool National Grid uses to balance supply and demand in real time. When it goes wrong — renewable intermittency, unexpected demand, plant trips — balancing costs spike sharply. These spikes are expensive and, in theory, partially predictable.

The dissertation question: can we build an early warning system that flags high-cost periods hours before they happen, using only publicly available half-hourly data from Elexon's BMRS API?

Data Pipeline

⬇

Elexon BMRS API ingestion

Half-hourly balancing settlement data. Wind generation, system price, imbalance volume, BM unit bids/offers. Multiple endpoints, async fetching.

⚙

Feature engineering

Lag features (1, 2, 4, 8, 48 periods), rolling statistics, time-of-day / day-of-week cyclical encodings, weather proxy features (wind penetration ratio).

🎯

Target definition

Binary spike label: top decile of system price in rolling 7-day window. Also testing regression on raw imbalance cost for secondary analysis.

🌲

Model training

XGBoost and LightGBM with time-series cross-validation (no data leakage). Optuna for hyperparameter optimisation.

🔍

Explainability

TreeExplainer SHAP values. Global feature importance + local explanations per spike event. Novel Early Warning Horizon (EWH) metric.

Models Under Evaluation

Model	Approach	Key strength	Status
XGBoost	Gradient boosted trees, L1/L2 reg	Handles missing data, robust to outliers	Training
LightGBM	Leaf-wise tree growth, histogram binning	Faster on large feature sets, lower memory	Training
Baseline	Persistence model (lag-1 prediction)	Sets minimum bar to beat	Done

Novel Contribution: Early Warning Horizon

Standard classification metrics (AUROC, F1) don't capture the most operationally useful property of an early warning system: how far in advance can it reliably flag an event?

The Early Warning Horizon (EWH) metric measures, for correctly-flagged spike events, the median number of half-hour periods before the event at which the model first exceeds a confidence threshold. A model with AUROC 0.85 but EWH of 1 period is far less useful than one with AUROC 0.82 and EWH of 6 periods (3 hours).

This metric is the primary contribution of the dissertation alongside the forecasting models themselves.

Key Challenges

→ Class imbalance. Spike events are by definition rare (top decile). SMOTE and class-weight tuning both tested — SMOTE tends to degrade time-series cross-validation performance due to synthetic sample leakage.
→ Temporal leakage. Standard k-fold CV is invalid for time-series — future data leaks into training. Using expanding-window cross-validation throughout to preserve temporal ordering.
→ Non-stationarity. GB energy market conditions shift significantly across years. Testing whether models trained on 2022-2023 (post-Ukraine energy crisis) generalise to 2024-2025 is a key validation question.

Timeline

Data pipeline complete. Feature engineering in progress. Model training begins June 2026, with full results and the final dissertation due July 2026. This page will be updated with results, SHAP plots, and a link to the public GitHub repo once the work is submitted.

← back to portfolio ← reception system