I got frustrated that every fuel price app just shows you what's cheap nearby. I wanted to know how stations actually behave: do prices go up faster than they come down, do supermarkets really save you much, how bad are motorway prices really?
So I built a scraper that hits the UK government's mandatory Fuel Finder API every 10 minutes and stores every price change. 90k records across 7,700 stations since January.
Some things I found that surprised me:
The rocket and feather effect is real and measurable. When stations raise prices the average move is 2.35p/litre. When they cut, it's 1.85p. There are also more up moves than down moves. I queried the raw history to check this rather than eyeballing a chart.
Motorway fuel is 28.4p/litre more expensive than everywhere else right now. That's about £14 extra on a 50L fill. Everyone knows motorways are expensive but I didn't expect the gap to be that wide.
The supermarket discount is only about 1.7p. I assumed it would be bigger.
Stack is Azure Functions, TimescaleDB, PostGIS, Next.js. The interesting thing about this project is the history. No public site shows how an individual station has priced over time or how a local cluster of stations react to each other. That's what I'm building towards.
Another insightful way to look at this is to include gasoline spot market data as a comparison.
I kept hearing about the vast profits of gas stations, so one day I started a spreadsheet of my gas purchases and kept it going over 10 years. When I tried lining up the graph of what I have actually paid per litre with a spot market graph, after converting for currency, units, taxes etc, they were almost identical, indicating extremely slim margins, if any. Yes there were differences, places in the graph where stations had likely made money on my purchase, but there were just as many where they likely lost money, unless I also stepped inside to but a snack.
You are correct. Non-chain gas stations often make only as little as one or two cents per liter, and that's before you look at pump maintenance, inspections, periodical tank replacements/upgrades/liners and other costs.
Manned stations really need that shop otherwise they'd go bankrupt.
Chains make a bit more money but mostly because they can play longer games with stock and options on much larger volume buys.
And yet I see in earnings that companies like BP and Shell make record profits over increased gas prices. How come that they do profit but the station holders not? Are shell/bp increasing the margin harder and eating the station’s lunch?
The majors that do upstream (taking it out of the ground) as well as downstream (refining and selling it on the forecourt or wholesale) make their profits on the oil markets.
When crude is high, it's upstream that earns the income. When it's low, it's selling it to the customer.
Fun fact: when I worked at BP, the product with the highest margin on the forecourts was the Wild Bean coffee.
It's funny to think that BP is a coffee shop with a side-business in petrol.
McDonald's makes its largest margins from the soft drinks, according to an ex's brother who worked in their HQ. From a $1 drink, the cup, straw and lid is 5 cents, and the liquid is 3 cents.
Opening a gas station is a lot easier than acquiring mineral rights, drilling an oil well, refining that oil, and getting it to market. Oh, and your customers can't just drive across the street to your competitor because they are 1c/litre cheaper.
There's naturally going to be a lot more friction and a lot less pop-up competition and therefore a lot more margin on the supply side of things.
The station has no power to raise margin - they are in tight competition with every other low-margin station around them. The suppliers, on the other hand... If they invested into wells that aren't affected by the war 10 years ago, and their competitors haven't (or have, but can't supply all the world's oil needs), and there's a global supply shortage - they have lots of room to raise prices.
I think majority of the profits come from extracting the oil from the ground, much less from refining (more competetive - everyone can just build more refineries if margins become high enough), and the least from retail (gas stations).
The people who drill a hole in the ground and then a valuable product literally flows out of it, make more profits when prices of that product go up, compared to the refiners and resellers of the valuable product. I don't think this is mysterious?
Jokes aside, the upstream side has costs (both capex and opex) which are predominantly independent of the actual oil price, but income which is tied to it.
It's like with NVIDIA: their cost of producing a B200 is not directly related to the cost of (and demand for) Claude Code, but what they can charge for a B200 is.
> How come that they do profit but the station holders not? Are shell/bp increasing the margin harder and eating the station’s lunch?
Here in Germany, many stations aren't even involved in selling the gas. That's what that magical line "Verkauf von Kraft- und Schmierstoffen im Namen der <Firma>" on the receipt says - the fuel and oil are on paper/for accounting purposes sold by the oil company whose brand is on the flag.
Independent gas stations (e.g. in Bavaria, the Allguth chain) exist, and they buy, store, distribute and sell their fuel on their own, but in the end everyone is bound to the same few refineries - virtually all (!) of Eastern Germany, Berlin, the Berlin airport and Western Poland for example depend on the PCK refinery complex in Schwedt [1], in Bavaria 2/3rd of the market is supplied by the two Bayernoil refineries in Vohburg and Neustadt [2], the rest by Gunvor (ex-Esso/Exxonmobil) [3].
No matter if you are an independent or brand-owned gas station... there is about zero competition on the supply side. It's all the same gas and diesel, the only practical difference is the additives for the ultra-high-octane fuel. And that in turn means very little competition at the pump, and owners of independent gas stations being hit the hardest.
There's a reason why Allguth stations more resemble a 24/7 supermarket, restaurant and beer hall than a gas station.
Oh and the supply side competition is pretty bad even for refineries. Refineries are fitted to refine only a specific composition of oil - that's the distinctions sweet/sour and light/heavy. For that reason, the US can't process a bunch of its own oil [4], which means the US is actually dependent on Canada and Mexico [5] to meet its oil product demand. A refinery which, like almost all are, is tuned to a specific country's (or, worse, a specific oil field in a specific country) composition is in a real bind, should the supply chain ever get screwed up. The Russian invasion of Ukraine was bad enough, the Iran war was what could very well be the final, fatal blow to many a refinery, especially in Asia.
Retrofits are possible to allow a refinery for light sweet oil to process heavy sour oil (i.e. add a sulphur removal stage to deal with that, and a cracking stage to deal with heavier molecules), and it is possible for a refinery that is tooled for heavy sour oil to run on light sweet oil without modification - but it is seriously throwing wrenches into the financials, and that's a blocking issue in our money driven world [6].
Good shout. DESNZ publishes weekly wholesale rack prices and they are OGL, so there is no barrier there. The interesting bit isn't just showing the gap; it's the propagation lag. Wholesale spikes and pump prices follow within days. Wholesale drops and pump prices take their time. That asymmetry is basically what I built this dataset to measure. Adding the wholesale series as a reference line is on the list.
>The rocket and feather effect is real and measurable. When stations raise prices the average move is 2.35p/litre. When they cut, it's 1.85p. There are also more up moves than down moves. I queried the raw history to check this rather than eyeballing a chart.
Comparing the absolute size of price rises vs drops doesn't make sense, because it could very well be an issue with the underlying price (eg. crude oil or whatever). It seems hardly fair to blame gas stations for being slow to lower prices, when refineries are still also slow to lower prices. Same for blaming refineries when the global market is slow to lower prices.
The US government publishes data on this (eg. https://www.eia.gov/petroleum/gasdiesel/). The UK government might have something similar. Barring that, you can use Brent crude as a proxy.
For the "supermarket saving", did you include Asda in the supermarket pool, or as a general pool? They seem to be rather less price competitive than other supermarkets, I'd presumably because of the recentish private equity takeover involving petrol station operator Euro Garages meaning they've kinda opted out of the petrol price war (they're hardly likely to want to undercut their existing forecourts).
Although the other recent private equity takeover of Morrisons led to some sort of deal with Motor Fuels Group to operate their petrol stations (but no ownership stuff in this case?), but they're seemingly still being competitive with Sainsbury's and Tesco's.
Yeah Asda is in there. I match on brand name directly rather than the API's is_supermarket flag because that flag is all over the place (loads of Asda stations don't have it set). So it explicitly checks for Asda, Tesco, Morrisons, Sainsbury's, Co-op, Costco.
Your point about post-PE Asda is interesting, I've noticed it too. If you want to see how they compare individually you can check the brands page on the site, shows each supermarket chain as its own line. Pretty easy to split the supermarket aggregate out per brand too, would probably show Asda creeping back towards the independents since the takeover. Might add that.
It is well established that Costco sells Top Tier brand fuel at zero margins, so you could have just tracked Costco prices. In the US, Walmart, Kroger, and Albertsons also sell fuel at near zero margins, but it’s not Top Tier branded.
I wonder if your API queries get capped / paginated or if those stations really do not report any changes for so long. It would be interesting to know if there is a pattern with those low frequency updates.
> I got frustrated that every fuel price app just shows you what's cheap nearby.
Yes it gets a bit tiring. Considering the difference is usually 0.01 penny per litre, so it doesnt make a big difference. But it does mean in a panic everyone runs to that one station and it runs out
I never understood the logic of driving 20 miles to save 0.01p/litre (but then I live in the countryside, maybe its different in cities-- though cities also have huge traffic so you are wasting the same amount of fuel)
My company network wont let me see newly registered domains, so I couldn't check out your build (I'll wait until I get home!), but our local state government set up a tool called Fuel Watch that does a pretty good job of showing the today/tomorrow pricing based on post code.
Yeah aware of some data quality issues in the feed, bad coordinates, missing brands, occasional obviously wrong prices. I filter out anything outside 100-300p and exclude stations flagged as closed but there's definitely still noise in there. check the methodology page for the full detail on how i clean it.
On the colours, esso and shell are both red brands so they ended up close on the chart. will fix that.
> So I built a scraper that hits the UK government's mandatory Fuel Finder API every 10 minutes and stores every price change. 90k records across 7,700 stations since January.
Only 1 change per station per week on average? Fewer than I expected. Not sure I'd call it a scraper, myself.
Europeans seem to be far more comfortable with Pigovian taxes than Americans. Or does it even come up for discussion when people are referencing the retail price at the pump?
It definitely does come up. A common complaint in the UK is that there are two types of tax added to fuel prices; fuel duty and VAT. So you end up paying VAT on the fuel duty you’re paying.
fair point on scraper, it's more of an ingest pipeline really. the 90k is total price records across 7,700 stations since january, stations update a few times a week on average. the 10 min polling catches every move when it happens. the value is in the per-station, history over time, not the raw volume.
Hi, economist focused on monetary policy/inflation here! That data would make for a cool empirical paper that could help central banks and other economists better understand price dynamics around a significant shock. Are you interested in discussing it?
I would love too. But I have not found an API that surfaces this data. This data is only public as the CMA determined the market was fundamentally broken.
It is, up to the point where payment limits hit, or things are not set to auto-scale for that reason, or the project just isn't designed in a way to actually take advantage of the scaling (in which case you might be better off on your own/rented servers which will be much cheaper than one of the big clouds, unless you don't want the faf of managing backups & other admin and freedom from that is worth the extra cost).
In Germany fuel processes must be reported to the anti trust authority (Bundeskartellamt) the data is than published to providers of apps and websites. Unfortunately there isn't a free public data stream from them.
I don't think there is a large intersection between these two groups at least in Europe. Price sensitive people didn't buy SUV. Better-off folks who buy SUV unlikely to be significantly affected by this price increase. In the US it may be different because most cars on the market are SUV/tracks and it's harder to find a small car.
I'd say that a 2022 Subaru Outback is not a "big SUV" - the EPA classified it as a small SUV (medium starting 2026), and it had a crossover station wagon body style.
Now I do wonder about those people who buy a Chevy Tahoe, Ford Expedition, or Toyota Sequoia and then complain about gas consumption...
I'll just mention that the site is very hard to read for me (to the degree it's unusable). I know, dark mode is cool, but for me it causes immediate eye strain.
I have read that people with astigmatism will often have an easier time reading light mode. Something like 30% of adults have that issue.
Just wanted to provide the feedback so you're aware.
If you can find a way to combine this with local population to end up at pence per litre per thousand population, I bet you’d uncover some fun trends. Bet it’d also get interesting if combined with population within an X min drive too.
Tho really need some car population per road segment stats to drive the most out of it IMO.
My main takeaway is you guys have a database (even if imperfect) for this. Here in the US we only have GasBuddy which is purely a crowdsourced best-effort thing. Problem is it’s tough to tell at a glance if a bunch of prices are equally old or if only some have been updated today. If some are stale and others fresh, pretty tough to even use the data.
US does not really need it as the government does not try to screw people. as it’s £2 / gallon on a good day and £3/gallon on a bad day. Uk tax is wild making it like £10 a gallon.
Pricing a product in an attempt to capture the cost of the negative externalities is probably not a bad idea. Also, the UK tax is supposed to raise every year to incentivise a move to EVs but they famously haven't increased it for years.
sounds plausible, but also at the same time that'll have some pretty bad side effects since the only people who can get by without driving a car are the completely destitute who don't have anyplace to be (but are a massive burden on the taxpayer in general) and the very wealthy who can afford to live in places like NYC, or like the 5% of addresses in SF that have good transit that could take you to your work in a reasonable time.
So if we made our fuel prices go from £2.50 to £8 what we are really doing is administering an ongoing massive punishment to everybody but those two groups. The middle 80% or so. (And also, hurting transit agencies too since buses run on diesel).
I've had this stubborn idea in my head, for a long time, that the petrol station up on Knight's Hill in South London is placed at the absolute nexus of London traffic to collect the best petrol rates across the entire city at any given time.
But yet to explore how I'd validate this idea. Your site helps remind me to have a dig at it.
The interactive map seems to be a bit broken (lots of grey dots vs brand colours, lots of broken mouseovers where many stations don't show price on hover)
Dashboard Price Comparison, you really need to re-think the colours, perhaps especially Shell Red vs Esso Red
Interesting that the prices were at their highest level a month ago and not today. Here in Calgary the prices seemed to be C$1.829/L (US$5.09/gal or £0.99/L) this weekend, and I don't think I've ever seen them that high before in Alberta.
checked the db, tesco e5 is 163.8p average across 292 stations in the last 21 days. Worth spot checking a few specific stations if it still looks off to you.
try CF. tried a version of this in feb, and it handles quite fine even under ai bot surges of 1.5m hits per day. and it’s still within the free limit btw ;)
Update for anyone who hit a slow site earlier: b1ms postgres wasn't having a great time with 140 concurrent users. scaled the db up, bumped the instance, and shipped in-memory output caching while the thread was live so repeated requests stop hammering the db. Site is fully up and fast now. appreciate the interest, genuinely didn't expect to front page today.
I got frustrated that every fuel price app just shows you what's cheap nearby. I wanted to know how stations actually behave: do prices go up faster than they come down, do supermarkets really save you much, how bad are motorway prices really?
So I built a scraper that hits the UK government's mandatory Fuel Finder API every 10 minutes and stores every price change. 90k records across 7,700 stations since January.
Some things I found that surprised me:
The rocket and feather effect is real and measurable. When stations raise prices the average move is 2.35p/litre. When they cut, it's 1.85p. There are also more up moves than down moves. I queried the raw history to check this rather than eyeballing a chart.
Motorway fuel is 28.4p/litre more expensive than everywhere else right now. That's about £14 extra on a 50L fill. Everyone knows motorways are expensive but I didn't expect the gap to be that wide.
The supermarket discount is only about 1.7p. I assumed it would be bigger.
Stack is Azure Functions, TimescaleDB, PostGIS, Next.js. The interesting thing about this project is the history. No public site shows how an individual station has priced over time or how a local cluster of stations react to each other. That's what I'm building towards.
Site: https://fuelinsight.co.uk
Happy to talk through the architecture or the data if anyone's interested.
Another insightful way to look at this is to include gasoline spot market data as a comparison.
I kept hearing about the vast profits of gas stations, so one day I started a spreadsheet of my gas purchases and kept it going over 10 years. When I tried lining up the graph of what I have actually paid per litre with a spot market graph, after converting for currency, units, taxes etc, they were almost identical, indicating extremely slim margins, if any. Yes there were differences, places in the graph where stations had likely made money on my purchase, but there were just as many where they likely lost money, unless I also stepped inside to but a snack.
You are correct. Non-chain gas stations often make only as little as one or two cents per liter, and that's before you look at pump maintenance, inspections, periodical tank replacements/upgrades/liners and other costs.
Manned stations really need that shop otherwise they'd go bankrupt.
Chains make a bit more money but mostly because they can play longer games with stock and options on much larger volume buys.
Source: former gas station owner.
And yet I see in earnings that companies like BP and Shell make record profits over increased gas prices. How come that they do profit but the station holders not? Are shell/bp increasing the margin harder and eating the station’s lunch?
The majors that do upstream (taking it out of the ground) as well as downstream (refining and selling it on the forecourt or wholesale) make their profits on the oil markets.
When crude is high, it's upstream that earns the income. When it's low, it's selling it to the customer.
Fun fact: when I worked at BP, the product with the highest margin on the forecourts was the Wild Bean coffee.
It's funny to think that BP is a coffee shop with a side-business in petrol.
McDonald's makes its largest margins from the soft drinks, according to an ex's brother who worked in their HQ. From a $1 drink, the cup, straw and lid is 5 cents, and the liquid is 3 cents.
Opening a gas station is a lot easier than acquiring mineral rights, drilling an oil well, refining that oil, and getting it to market. Oh, and your customers can't just drive across the street to your competitor because they are 1c/litre cheaper.
There's naturally going to be a lot more friction and a lot less pop-up competition and therefore a lot more margin on the supply side of things.
The station has no power to raise margin - they are in tight competition with every other low-margin station around them. The suppliers, on the other hand... If they invested into wells that aren't affected by the war 10 years ago, and their competitors haven't (or have, but can't supply all the world's oil needs), and there's a global supply shortage - they have lots of room to raise prices.
I think majority of the profits come from extracting the oil from the ground, much less from refining (more competetive - everyone can just build more refineries if margins become high enough), and the least from retail (gas stations).
Yep. In short, this is an oil shortage, not a gas station shortage.
The people who drill a hole in the ground and then a valuable product literally flows out of it, make more profits when prices of that product go up, compared to the refiners and resellers of the valuable product. I don't think this is mysterious?
Jokes aside, the upstream side has costs (both capex and opex) which are predominantly independent of the actual oil price, but income which is tied to it.
It's like with NVIDIA: their cost of producing a B200 is not directly related to the cost of (and demand for) Claude Code, but what they can charge for a B200 is.
Reuters had a material on the topic. The big producers have trading desks that can optimise based on spot prices and redirect tankers as needed.
A lot of their record earnings come from trading revenues as well, as they all effectively have in-house trading desks to hedge risk.
Not nearly as much profit as Vitol made but they make large sums too.
> How come that they do profit but the station holders not? Are shell/bp increasing the margin harder and eating the station’s lunch?
Here in Germany, many stations aren't even involved in selling the gas. That's what that magical line "Verkauf von Kraft- und Schmierstoffen im Namen der <Firma>" on the receipt says - the fuel and oil are on paper/for accounting purposes sold by the oil company whose brand is on the flag.
Independent gas stations (e.g. in Bavaria, the Allguth chain) exist, and they buy, store, distribute and sell their fuel on their own, but in the end everyone is bound to the same few refineries - virtually all (!) of Eastern Germany, Berlin, the Berlin airport and Western Poland for example depend on the PCK refinery complex in Schwedt [1], in Bavaria 2/3rd of the market is supplied by the two Bayernoil refineries in Vohburg and Neustadt [2], the rest by Gunvor (ex-Esso/Exxonmobil) [3].
No matter if you are an independent or brand-owned gas station... there is about zero competition on the supply side. It's all the same gas and diesel, the only practical difference is the additives for the ultra-high-octane fuel. And that in turn means very little competition at the pump, and owners of independent gas stations being hit the hardest.
There's a reason why Allguth stations more resemble a 24/7 supermarket, restaurant and beer hall than a gas station.
Oh and the supply side competition is pretty bad even for refineries. Refineries are fitted to refine only a specific composition of oil - that's the distinctions sweet/sour and light/heavy. For that reason, the US can't process a bunch of its own oil [4], which means the US is actually dependent on Canada and Mexico [5] to meet its oil product demand. A refinery which, like almost all are, is tuned to a specific country's (or, worse, a specific oil field in a specific country) composition is in a real bind, should the supply chain ever get screwed up. The Russian invasion of Ukraine was bad enough, the Iran war was what could very well be the final, fatal blow to many a refinery, especially in Asia.
Retrofits are possible to allow a refinery for light sweet oil to process heavy sour oil (i.e. add a sulphur removal stage to deal with that, and a cracking stage to deal with heavier molecules), and it is possible for a refinery that is tooled for heavy sour oil to run on light sweet oil without modification - but it is seriously throwing wrenches into the financials, and that's a blocking issue in our money driven world [6].
[1] https://www.faz.net/aktuell/wirtschaft/unternehmen/pck-raffi... / https://archive.ph/qtvd8
[2] https://de.wikipedia.org/wiki/Bayernoil
[3] https://de.wikipedia.org/wiki/Gunvor_Raffinerie_Ingolstadt
[4] https://www.afpm.org/newsroom/blog/how-much-oil-does-united-...
[5] https://atlasinstitute.org/heavy-oil-heavy-dependence-how-us...
[6] https://www.forbes.com/sites/rrapier/2026/04/05/debunking-a-...
Good shout. DESNZ publishes weekly wholesale rack prices and they are OGL, so there is no barrier there. The interesting bit isn't just showing the gap; it's the propagation lag. Wholesale spikes and pump prices follow within days. Wholesale drops and pump prices take their time. That asymmetry is basically what I built this dataset to measure. Adding the wholesale series as a reference line is on the list.
You’re absolutely right!
>The rocket and feather effect is real and measurable. When stations raise prices the average move is 2.35p/litre. When they cut, it's 1.85p. There are also more up moves than down moves. I queried the raw history to check this rather than eyeballing a chart.
Comparing the absolute size of price rises vs drops doesn't make sense, because it could very well be an issue with the underlying price (eg. crude oil or whatever). It seems hardly fair to blame gas stations for being slow to lower prices, when refineries are still also slow to lower prices. Same for blaming refineries when the global market is slow to lower prices.
yeah that's fair, without wholesale rack prices as a baseline you can't really isolate station behaviour from what's happening upstream.
>without wholesale rack prices as a baseline
The US government publishes data on this (eg. https://www.eia.gov/petroleum/gasdiesel/). The UK government might have something similar. Barring that, you can use Brent crude as a proxy.
For the "supermarket saving", did you include Asda in the supermarket pool, or as a general pool? They seem to be rather less price competitive than other supermarkets, I'd presumably because of the recentish private equity takeover involving petrol station operator Euro Garages meaning they've kinda opted out of the petrol price war (they're hardly likely to want to undercut their existing forecourts).
Although the other recent private equity takeover of Morrisons led to some sort of deal with Motor Fuels Group to operate their petrol stations (but no ownership stuff in this case?), but they're seemingly still being competitive with Sainsbury's and Tesco's.
Yeah Asda is in there. I match on brand name directly rather than the API's is_supermarket flag because that flag is all over the place (loads of Asda stations don't have it set). So it explicitly checks for Asda, Tesco, Morrisons, Sainsbury's, Co-op, Costco.
Your point about post-PE Asda is interesting, I've noticed it too. If you want to see how they compare individually you can check the brands page on the site, shows each supermarket chain as its own line. Pretty easy to split the supermarket aggregate out per brand too, would probably show Asda creeping back towards the independents since the takeover. Might add that.
It is well established that Costco sells Top Tier brand fuel at zero margins, so you could have just tracked Costco prices. In the US, Walmart, Kroger, and Albertsons also sell fuel at near zero margins, but it’s not Top Tier branded.
Quite interesting that some service stations did not report a change for weeks.
Like this one: https://www.fuelinsight.co.uk/stations/7fd603ba12430695595b2...
I wonder if your API queries get capped / paginated or if those stations really do not report any changes for so long. It would be interesting to know if there is a pattern with those low frequency updates.
> I got frustrated that every fuel price app just shows you what's cheap nearby.
Yes it gets a bit tiring. Considering the difference is usually 0.01 penny per litre, so it doesnt make a big difference. But it does mean in a panic everyone runs to that one station and it runs out
I never understood the logic of driving 20 miles to save 0.01p/litre (but then I live in the countryside, maybe its different in cities-- though cities also have huge traffic so you are wasting the same amount of fuel)
My company network wont let me see newly registered domains, so I couldn't check out your build (I'll wait until I get home!), but our local state government set up a tool called Fuel Watch that does a pretty good job of showing the today/tomorrow pricing based on post code.
Probably not doing exactly what your tool does, but you might be interested: https://www.fuelwatch.wa.gov.au/
Are you aware the some of the Fuel Finder data is total garbage?
https://successfulsoftware.net/2026/03/29/stop-publishing-ga...
Why did you make 'Esso' and 'Shell' the same colour in your brand chart?
Yeah aware of some data quality issues in the feed, bad coordinates, missing brands, occasional obviously wrong prices. I filter out anything outside 100-300p and exclude stations flagged as closed but there's definitely still noise in there. check the methodology page for the full detail on how i clean it.
On the colours, esso and shell are both red brands so they ended up close on the chart. will fix that.
The Shell logo is mostly yellow, isn't it?
shell's logo does have a lot of yellow but morrisons has already claimed yellow on the chart.
I built the very first prototype of petrolprices.com almost exactly 20 years ago I think.
One man job, 2 weeks I think and we launched.
Nice project :)
petrolprices.com is still going strong. nice to meet one of the OGs.
> So I built a scraper that hits the UK government's mandatory Fuel Finder API every 10 minutes and stores every price change. 90k records across 7,700 stations since January.
Only 1 change per station per week on average? Fewer than I expected. Not sure I'd call it a scraper, myself.
157p/L national average is about 8 USD/G.
> Only 1 change per station per week on average? Fewer than I expected.
My uneducated guess is it correlates to the weekly delivery of fuel. ;-)
> 157p/L national average is about 8 USD/G.
Europeans seem to be far more comfortable with Pigovian taxes than Americans. Or does it even come up for discussion when people are referencing the retail price at the pump?
It definitely does come up. A common complaint in the UK is that there are two types of tax added to fuel prices; fuel duty and VAT. So you end up paying VAT on the fuel duty you’re paying.
We have had fuel price protests too, sometimes involving blockades by hauliers. https://en.wikipedia.org/wiki/Fuel_protests_in_the_United_Ki...
fair point on scraper, it's more of an ingest pipeline really. the 90k is total price records across 7,700 stations since january, stations update a few times a week on average. the 10 min polling catches every move when it happens. the value is in the per-station, history over time, not the raw volume.
Hi, economist focused on monetary policy/inflation here! That data would make for a cool empirical paper that could help central banks and other economists better understand price dynamics around a significant shock. Are you interested in discussing it?
Absolutely, feel free to reach out. contact details are on the site. would love to see how I can help!
I made something similar for Thailand in the peak of the crisis .. all on GitHub, regional concerns and such..
https://www.fuelthai.com/
Since you are happy to talk on this topic, can I mention your privacy policy page?
If a new user clicks on Accept, he has no (normie) mechanism to revoke that as the choice is stored in local storage and it doesn't expire.
Ahhh thanks for that. I have deployed a change to fix this ASAP.
What did you use for the UI widgets, graphs, etc? Looks like tailwind is being used but is the design bespoke or a CSS/dashboard lib?
bespoke design but used tailwind for styling. charts are recharts. no dashboard lib.
Baffled that there is now a fuel price API and yet there is no such electric charger price API
Any appetite in adding data for electricity rates for EVs?
I would love too. But I have not found an API that surfaces this data. This data is only public as the CMA determined the market was fundamentally broken.
I can only go back to March 31?
https://petrol.now/trends has data since feb
yeah that's as far back as my data goes, started ingesting in march. no history existed before that
I see-- no problem, thank you. I thought it was just user error or a UI bug because of this comment:
> 90k records across 7,700 stations since January.
what is "since January" about then?
data goes back to january but coverage is patchy early on as we were doing full fetches rather than continuous polling. march onwards is complete.
Is at least part of this comment LLM-generated?
...
Just curious, not intending this as an attack on your project.
some of my replies yeah, been a bit busy keeping the site up while the thread was live. helps with the typos too.
If only this wasn't hosted on azure, we'd be able to actually look at the data
I thought cloud was supposed to be an answer to sudden surges in traffic. /s
It is, up to the point where payment limits hit, or things are not set to auto-scale for that reason, or the project just isn't designed in a way to actually take advantage of the scaling (in which case you might be better off on your own/rented servers which will be much cheaper than one of the big clouds, unless you don't want the faf of managing backups & other admin and freedom from that is worth the extra cost).
Microslopinator strikes again
In Germany fuel processes must be reported to the anti trust authority (Bundeskartellamt) the data is than published to providers of apps and websites. Unfortunately there isn't a free public data stream from them.
List of authorized places using the data: https://www.bundeskartellamt.de/DE/Aufgaben/Markttransparenz...
One of those vendors publishes it as creative commons data set, though. Including historic data. https://creativecommons.tankerkoenig.de/
It keeps surprising me how people obsess over a 20% fuel price increase, but take a 20% fuel consumption increase for granted when they buy a big SUV.
Dito driving at 80 mph instead of a (legal) 70 mph on the motorway.
That extra 10 mph typically uses 25% more fuel.
(Guilty as charged yr'Honour)
I don't think there is a large intersection between these two groups at least in Europe. Price sensitive people didn't buy SUV. Better-off folks who buy SUV unlikely to be significantly affected by this price increase. In the US it may be different because most cars on the market are SUV/tracks and it's harder to find a small car.
People obsess and complain, but the interstate drivers around me aren't driving like gas is expensive yet (currently $4-4.20/gal)
> but take a 20% fuel consumption increase for granted when they buy a big SUV
My '22 Subaru Outback burns less per mile than my '03 Volkswagen Jetta did, for what it's worth.
I'd say that a 2022 Subaru Outback is not a "big SUV" - the EPA classified it as a small SUV (medium starting 2026), and it had a crossover station wagon body style.
Now I do wonder about those people who buy a Chevy Tahoe, Ford Expedition, or Toyota Sequoia and then complain about gas consumption...
In Québec every station must report price change within 5 minutes and we have access to a map https://regieessencequebec.ca/
For now data can only be exported as xlsx but with the open data orientation of Québec's government, I guess it will be available soon
that 5 minute reporting requirement is way stricter than the uk.
I'll just mention that the site is very hard to read for me (to the degree it's unusable). I know, dark mode is cool, but for me it causes immediate eye strain.
I have read that people with astigmatism will often have an easier time reading light mode. Something like 30% of adults have that issue.
Just wanted to provide the feedback so you're aware.
really appreciate the feedback, that's a valid point. I will review and make it more accessible.
If you can find a way to combine this with local population to end up at pence per litre per thousand population, I bet you’d uncover some fun trends. Bet it’d also get interesting if combined with population within an X min drive too.
Tho really need some car population per road segment stats to drive the most out of it IMO.
Might have a play with that thank you.
My main takeaway is you guys have a database (even if imperfect) for this. Here in the US we only have GasBuddy which is purely a crowdsourced best-effort thing. Problem is it’s tough to tell at a glance if a bunch of prices are equally old or if only some have been updated today. If some are stale and others fresh, pretty tough to even use the data.
It’s a relatively new GOV.UK requirement - this year IIRC.
US does not really need it as the government does not try to screw people. as it’s £2 / gallon on a good day and £3/gallon on a bad day. Uk tax is wild making it like £10 a gallon.
Pricing a product in an attempt to capture the cost of the negative externalities is probably not a bad idea. Also, the UK tax is supposed to raise every year to incentivise a move to EVs but they famously haven't increased it for years.
> capture the cost of the negative externalities
sounds plausible, but also at the same time that'll have some pretty bad side effects since the only people who can get by without driving a car are the completely destitute who don't have anyplace to be (but are a massive burden on the taxpayer in general) and the very wealthy who can afford to live in places like NYC, or like the 5% of addresses in SF that have good transit that could take you to your work in a reasonable time.
So if we made our fuel prices go from £2.50 to £8 what we are really doing is administering an ongoing massive punishment to everybody but those two groups. The middle 80% or so. (And also, hurting transit agencies too since buses run on diesel).
And they’re introducing an EV tax that’ll be charged per mile.
https://www.bbc.co.uk/news/articles/c8jw9l7gx92o
I've had this stubborn idea in my head, for a long time, that the petrol station up on Knight's Hill in South London is placed at the absolute nexus of London traffic to collect the best petrol rates across the entire city at any given time.
But yet to explore how I'd validate this idea. Your site helps remind me to have a dig at it.
Nice work, couple of observations ...
The interactive map seems to be a bit broken (lots of grey dots vs brand colours, lots of broken mouseovers where many stations don't show price on hover)
Dashboard Price Comparison, you really need to re-think the colours, perhaps especially Shell Red vs Esso Red
really appreciate you taking the time to check it properly. I will get them both fixed some time this week!
Interesting that the prices were at their highest level a month ago and not today. Here in Calgary the prices seemed to be C$1.829/L (US$5.09/gal or £0.99/L) this weekend, and I don't think I've ever seen them that high before in Alberta.
Love the site. I do wonder why waze hasn't adopted the UK govt new API to give better fuel info as you drive.
I prefer Google maps but they never had petrol prices....
Any Waze Devs here? Tried to raise a feature request but it was such a pain
Good work, that. The area data is very interesting as well, but what surprises me is the relatively low spread excluding motorways.
I would rather push the car then pay motorway prices.
Very interesting! Inspired me to make the same thing for the IT market: prezziallapompa.it
The Tesco premium/super e5 is so much cheaper than everyone else I'm actually pretty sceptical of it.
checked the db, tesco e5 is 163.8p average across 292 stations in the last 21 days. Worth spot checking a few specific stations if it still looks off to you.
Awe, yes sorry I'm sceptical of Tesco not your data.
Cheapest stations look very off, to me.
Something went wrong
Could not load data from the server. Please try again.
this is one of the coolest most random things ive seen. idk why this resonated with me so much ahahah
bros server crashed
Well they did call themselves the azure guy
haha not quite, db was struggling under the load for a few minutes. b1ms postgres doesn't love 6k requests in 30 mins.
A 30 second memcached/redis/etc goes a long way when sharing on HN. You go from 6k db hits to 60 hits in 30 minutes. Worked for me since 2013 [0].
[0]: https://idiallo.com/blog/handling-1-million-web-request
already on it, just shipped in-memory output caching while the thread was live. next step redis if it needs it
Those tiny b1ms VMs are absolutely pitiful (long time Azure sufferer here). It's crazy how little compute and memory Azure give you for so much money.
yeah scaling it up now, was hoping to keep costs down but HN had other ideas
Using azure managed PG is certainly an interesting choice if you were hoping to keep costs down
try CF. tried a version of this in feb, and it handles quite fine even under ai bot surges of 1.5m hits per day. and it’s still within the free limit btw ;)
stationwatch.co.uk
That’s… 3.3 r/s?
Update for anyone who hit a slow site earlier: b1ms postgres wasn't having a great time with 140 concurrent users. scaled the db up, bumped the instance, and shipped in-memory output caching while the thread was live so repeated requests stop hammering the db. Site is fully up and fast now. appreciate the interest, genuinely didn't expect to front page today.
[dead]
[dead]