Tech stack is a mix of things, but the big-ticket items:
- Dagster for data pipeline
- Parquet files for data format
- Cloudflare R2 for data storage
- DuckDB for data processing
- NodeJS for API
- NextJS + React for the app
- MapLibre for the base map functionality
- Deck.gl for the complex map layer (eg. wind particles)
- WeatherLayers (+ GFS) for the wind particles
- Cloudflare as a caching layer
The main difficulty was learning about the underlying data and industry nuances. I've been working on this map the past few years and I'm still learning new things that force me to change my approach. It's an incredibly complex domain and it's part of the reason I made this, to try and make this complex data a bit more digestible.
Nice job dodging the awkwardness of whether to include Ireland, Northern Ireland, etc (IE and NI operate as a single, all-island wholesale electricity market). :)
I actually have plans to include Ireland and Northern Ireland once I get the GB side nailed down. The data seems to be mostly available though I'd have to really think about how to make it work well on the app as it'd be a pretty fundamental change to how things currently work (assumes a single market).
Hey Robin, big fan of this map, congrats on getting front page on HN :).
Two suggestions/questions if you may:
1) Would be good to see how many MW each boundary can handle, not just %? Btw, I can't see the number for the south east england boundary.
2) Great job on the battery info. I'm seeing some battery storage is curtailed. How is that possible? Please don't tell me that we are paying batteries to _not_ export :/?
1) Absolutely agree. The current approach for the boundaries is a quick hack until I can implement something more sophisticated. Safe to say an update is already in the works that adds a MW value and more insight into the state of each boundary (and is also more accurate in general)
2) "Please don't tell me that we are paying batteries to _not_ export" – it's actually the opposite, the batteries paid to not export (at least today). You can dig into this yourself via the Detailed System Prices dataset [0] and looking at one of the batteries on the sell stack (eg. KILSB-5)
No worries. The Detailed System Prices dataset is lagged by a couple hours so try going back in time.
The simplest answer I can give is that assets place bids and a volume of energy that they are willing to turn down if the system operator needs to. Those bids are either positive or negative in value and this depends a lot on the type of asset, for example wind assets usually bid negative (ie. we pay them to turn down) while gas assets usually bid positive (ie. they pay us to turn down). The reason for that is a lot to do with complexities of the market and also the cost of running that assets, the cost of fuels, etc.
This has just had a major update to include other renewables (eg hydro, solar) as well as non-renewable energy sources (gas).
It’s also added grid boundaries and cabling recently - and I think there’s also wind roses planned?
My one UI wish - the energy sources filters are currently displayed as options on the map - rather than controls for the entire UI - which meant I missed them entirely at first. Would be great if they could be given more prominence.
Definitely wind roses planned, I posted about them on my LinkedIn page if anyone is curious (linked on the main map).
That's a great point about the controls. Originally the map only showed wind farms and as I've expanded the features the control system (and really the entire UX) hasn't kept up. I need to sit down and have a proper think about how best to design the controls and documentation.
> New wind power records are set regularly, and between 5:30pm and 6:00pm on 5th December 2025 British wind farms averaged a record 23.94GW of generation.
Love this! Would be super interested in any details the author could share on the data engineering needed to make this work. The vis is super impressive but I suspect the data is the harder thing to get working.
The most time and energy has been getting my head around the source data [0] and industry-specific nuances.
In terms of stack I have a self-hosted Dagster [1] data pipeline that periodically dumps the data onto Cloudflare R2 as parquet files. I then have a self-hosted NodeJS API that uses DuckDB to crunch the raw data and output everything you see on the map.
What are the black dots around the map when you zoom in? For instance South-East Wales / Bristol, I see a few black dots that look like they cover shaded areas or something. They also show up at the sites for wind farms, possibly they're the turbines themselves?
Yup, that's exactly it! When you zoom in you get to see the wind farms and wind turbines using data from the amazing Open Infrastructure Map [0]. I also show the cables for the offshore wind farms.
Some solar power assets are being tracked though unfortunately (for now) I'm only able to display data for assets that report on the balancing system in Great Britain. This means most solar has no data that I can easily access.
The squiggly ones across the country? Those are the National Energy System Operator (NESO) transmission boundaries for Great Britain. They represent areas of the country that are tracked for the purposes of managing constraints, which is when the transmission network over the boundary can't handle the amount of power flowing over it. When they are overloaded (or rather, before that happens) NESO will step in and "balance" the various zones by turning up or down generators.
The lines are sort of arbitrary in a geographic sense as in reality they are defined based on the intersection with the schematic version of the transmission network [0]. Though yeah, that specific example is an odd one and I'd be interested to understand more about why it was placed in one boundary and not the other.
I'll make sure to get the about page updated soon, it's been a bit neglected with being so focussed on the recent upgrades. Definitely a lot I can explain better, and also ways that I can solve that with better UX in general.
Yup! I have two ways to do it right now, the first is to click the plug-looking button on the map and this'll switch into curtailment mode to filter the assets that are currently being turned down due to transmission constraints, as well as the assets being turned-up elsewhere to balance things out. Also if you click on a curtailed assets you'll see the curtailment as a yellow area on the charts.
The other method is to visit my dedicated page [0] for tracking aggregate curtailment and turn-up, plus the costs for this.
Thanks for answering my curtailment question! I removed it from my comment as I commented before I found the data in your UX, but the extra context is useful. So much curtailment, much more transmission and battery storage required.
Oh wow, I made this! I didn't expect to see my map on here
Let me know if you have any questions or feedback and I'll do my best to respond to them.
Tech Stack used? Any specific non-technical difficulty getting this done ?
Tech stack is a mix of things, but the big-ticket items:
- Dagster for data pipeline
- Parquet files for data format
- Cloudflare R2 for data storage
- DuckDB for data processing
- NodeJS for API
- NextJS + React for the app
- MapLibre for the base map functionality
- Deck.gl for the complex map layer (eg. wind particles)
- WeatherLayers (+ GFS) for the wind particles
- Cloudflare as a caching layer
The main difficulty was learning about the underlying data and industry nuances. I've been working on this map the past few years and I'm still learning new things that force me to change my approach. It's an incredibly complex domain and it's part of the reason I made this, to try and make this complex data a bit more digestible.
Nice job dodging the awkwardness of whether to include Ireland, Northern Ireland, etc (IE and NI operate as a single, all-island wholesale electricity market). :)
I actually have plans to include Ireland and Northern Ireland once I get the GB side nailed down. The data seems to be mostly available though I'd have to really think about how to make it work well on the app as it'd be a pretty fundamental change to how things currently work (assumes a single market).
Hey Robin, big fan of this map, congrats on getting front page on HN :).
Two suggestions/questions if you may:
1) Would be good to see how many MW each boundary can handle, not just %? Btw, I can't see the number for the south east england boundary.
2) Great job on the battery info. I'm seeing some battery storage is curtailed. How is that possible? Please don't tell me that we are paying batteries to _not_ export :/?
Hey! And thank you
1) Absolutely agree. The current approach for the boundaries is a quick hack until I can implement something more sophisticated. Safe to say an update is already in the works that adds a MW value and more insight into the state of each boundary (and is also more accurate in general)
2) "Please don't tell me that we are paying batteries to _not_ export" – it's actually the opposite, the batteries paid to not export (at least today). You can dig into this yourself via the Detailed System Prices dataset [0] and looking at one of the batteries on the sell stack (eg. KILSB-5)
[0] https://bmrs.elexon.co.uk/detailed-system-prices
> "Please don't tell me that we are paying batteries to _not_ export" – it's actually the opposite, the batteries paid to not export (at least today).
Unfortunately you've got at least one negative wrong in this sentence and I'm still confused, and the linked dataset is currently blank?
Sorry for complaining, this is a great website!
No worries. The Detailed System Prices dataset is lagged by a couple hours so try going back in time.
The simplest answer I can give is that assets place bids and a volume of energy that they are willing to turn down if the system operator needs to. Those bids are either positive or negative in value and this depends a lot on the type of asset, for example wind assets usually bid negative (ie. we pay them to turn down) while gas assets usually bid positive (ie. they pay us to turn down). The reason for that is a lot to do with complexities of the market and also the cost of running that assets, the cost of fuels, etc.
This has just had a major update to include other renewables (eg hydro, solar) as well as non-renewable energy sources (gas).
It’s also added grid boundaries and cabling recently - and I think there’s also wind roses planned?
My one UI wish - the energy sources filters are currently displayed as options on the map - rather than controls for the entire UI - which meant I missed them entirely at first. Would be great if they could be given more prominence.
Definitely wind roses planned, I posted about them on my LinkedIn page if anyone is curious (linked on the main map).
That's a great point about the controls. Originally the map only showed wind farms and as I've expanded the features the control system (and really the entire UX) hasn't kept up. I need to sit down and have a proper think about how best to design the controls and documentation.
It's pretty windy here today! Around half of the UK's generation is currently coming from wind[1]
[1] https://grid.iamkate.com/
> New wind power records are set regularly, and between 5:30pm and 6:00pm on 5th December 2025 British wind farms averaged a record 23.94GW of generation.
Cool!
Semi-related: I also have a page where you can track the wind records as they happen
https://renewables-map.robinhawkes.com/records
Love this! Would be super interested in any details the author could share on the data engineering needed to make this work. The vis is super impressive but I suspect the data is the harder thing to get working.
The most time and energy has been getting my head around the source data [0] and industry-specific nuances.
In terms of stack I have a self-hosted Dagster [1] data pipeline that periodically dumps the data onto Cloudflare R2 as parquet files. I then have a self-hosted NodeJS API that uses DuckDB to crunch the raw data and output everything you see on the map.
[0] Mostly from https://bmrs.elexon.co.uk/ [1] https://dagster.io/
What are the black dots around the map when you zoom in? For instance South-East Wales / Bristol, I see a few black dots that look like they cover shaded areas or something. They also show up at the sites for wind farms, possibly they're the turbines themselves?
Yup, that's exactly it! When you zoom in you get to see the wind farms and wind turbines using data from the amazing Open Infrastructure Map [0]. I also show the cables for the offshore wind farms.
[0] https://openinframap.org/
Very cool. Thanks
Great visualisation...but doesn't seem to have any display of the amount of Solar (though won't be much on such a cloudy day!)
Some solar power assets are being tracked though unfortunately (for now) I'm only able to display data for assets that report on the balancing system in Great Britain. This means most solar has no data that I can easily access.
I hope to work around this soon.
Can someone explain what the lines mean?
The squiggly ones across the country? Those are the National Energy System Operator (NESO) transmission boundaries for Great Britain. They represent areas of the country that are tracked for the purposes of managing constraints, which is when the transmission network over the boundary can't handle the amount of power flowing over it. When they are overloaded (or rather, before that happens) NESO will step in and "balance" the various zones by turning up or down generators.
I note that one of those lines seems to take a big detour up and round Cruachan?
https://www.visitcruachan.co.uk/
The lines are sort of arbitrary in a geographic sense as in reality they are defined based on the intersection with the schematic version of the transmission network [0]. Though yeah, that specific example is an odd one and I'd be interested to understand more about why it was placed in one boundary and not the other.
[0] https://www.neso.energy/data-portal/day-ahead-constraint-flo...
Lovely site. As a wind energy ignoramus, it would be great to add descriptions to your FAQ for the red lines, and curtailment.
I'll make sure to get the about page updated soon, it's been a bit neglected with being so focussed on the recent upgrades. Definitely a lot I can explain better, and also ways that I can solve that with better UX in general.
Great work, this is so cool to see!
Thank you
Very well done, beautiful data communication.
Yup! I have two ways to do it right now, the first is to click the plug-looking button on the map and this'll switch into curtailment mode to filter the assets that are currently being turned down due to transmission constraints, as well as the assets being turned-up elsewhere to balance things out. Also if you click on a curtailed assets you'll see the curtailment as a yellow area on the charts.
The other method is to visit my dedicated page [0] for tracking aggregate curtailment and turn-up, plus the costs for this.
[0] https://renewables-map.robinhawkes.com/curtailment
Thanks for answering my curtailment question! I removed it from my comment as I commented before I found the data in your UX, but the extra context is useful. So much curtailment, much more transmission and battery storage required.