IDK lot of noise in report, I need more signal to noise ratio, already reading so much AI output as it is.
But the example PR... It said code generate with Claude, is it really still this bad? Last I was using SQLAlchemy with Claude was Opus 4.5, and it took some heavy coaching and skill gen not to generate code like this--but why even use SQLAlchemy at all if you're just going to use string interpolation jeez
owner = "" if current_user.is_superuser else f"WHERE owner_id = '{current_user.id}'"
query = f"SELECT status, count(*) FROM item {owner} GROUP BY status"
rows = session.execute(text(query)).all()
Part of me thinks maybe because it's not popular or well documented, but SQLAlchemy has an eDSL of SQL, so you can skip the whole ORM/table mapping layer, and just use it to programmatically construct SQL syntax
> It said code generate with Claude, is it really still this bad?
Anecdata, datapoint of 1 personal experience: I had been using Opus 4.6 with no problems until yesterday, then it seemed like it was missing things constantly, introducing random bugs, forgetting basic stuff that was in its memory and skills and should never miss
Same thing happened on 2 separate sessions I had with Opus 4.6
Ended up switching to 5.5 and Fable, things improved instantly
Out of curiosity, was this on a codebase where other folks started using Opus 5.5? I’ve been curious if each generation of model has its own biases that make it generate output that’s more difficult for other models/previous models to work with. That’s been my personal suspicion when I’ve hit situations like yours
It was on 2 separate private repos, personal projects. No one else working on those
But now that you asked, I recently had a similar experience with Codex, but on a shared repo. I also was using a slightly older model, and the all of a sudden it kinda stopped running tasks. I would ask it to do something and the it would just kind of do an analysis, then give me a recommendation of what to do. I had to tell it to execute the task, and then it would “be lazy”, like run one part then give me a recap and recommendation again instead of just going
Switched to a newer model, and the issue was instantly gone
this is very needed - I don't think we'll see a world where agents approve all our PRs but there can definitely be better tooling for focusing human attention during review
There was a post here recently, "Using LLM for confidence scoring is a bad idea", my mind went immediately to using LLMs for scoring risk. Seems like a bad idea. If there's anything we still need a human for, it's assessing risk.
An engineer where I work built something like this so that when QA is about to deploy, they have a better indication of who to call and what is potentially going to be impacted. Our implementation is done prior to production deploy (which requires human approval) and compares currrebt deployed build code to new build changes diff and then analyses. I think there is a lot of room for tools like this in the future. Scoping/homing/etc
I think tools like this are needed and will become standard in the near future.
However, the backend demo report seems longer and more complex than just reading the code. I’m trying to build something similar as well, and no one has quite nailed this yet.
The risk map / assisted review needs to be succinct and information dense, to guide your attention. My current philosophy is that we need to be annotating code, not replacing the entire code review interface with a bunch of text.
It suggested based on the files most used (through graph dependency), for this reason it appears as a important file (as if it breaks, it can break several places in your codebase), but that being said, you can add some files to be ignored (if you have special files, like consts or i18n)
why is the only comment by the OP flagged and dead? Is it because they were supposed to include that data in the description text rather than a comment, was it flagged as AI generated, or..?
Trying to understand why the poster's own comment was flagged and killed but the original article remains up.
AI-edited or -generated comments are disallowed by HN guidelines: https://news.ycombinator.com/newsguidelines.html#generated. I don't think any such restriction exists for submissions themselves, so that's my guess as to why the comment but not the submission has been flagged to death.
No restriction for the submission itself, unless it’s something like an Ask/Tell/Show HN, in which the content goes directly on the post (and will get flagged if AI-generated/assisted)
You can vouch for it, if you view the comment by clicking the time since posted a “vouch” option appears. I did it. I’ve noticed this happening more and more on here for show HN posts especially. I’m not sure if it’s like some sort of troll gang or what…
In my personal experience, it’s very rare to catch a comment/story while vouchable. Not sure how many flags does it take for something to be killed, but it seems like almost as soon as something gets flagged, it becomes a signal for others to flag it too immediately without thought (similar thing kinda happens too when you post a comment and the first vote is a downvote, it almost never recovers, even if it was a good comment, as the initial drop/greying biases anyone that sees it before they even read it)
I like the concept, a risk map could be useful in some circumstances but I absolutely hate the UI design; it reeks of AI with no human thought or care put into it.
Hi HN, I built farol at Winnin, where we open-sourced it (MIT).
AI agents now write a lot of our code, and AI reviewers catch a lot of
the line-level problems. But someone still has to press "merge", and
that person has the same question every time: how risky is this, and
where should I spend my attention? A 40-file PR where 38 files are
isolated is very different from a 3-file PR that changes a model used
by 23 endpoints.
farol is a CLI (and a GitHub Action) that answers that statically:
- classifies the PR (global / touches-existing / isolated / unknown)
and gives a 0-100 merge risk score
- ranks the files by risk and says why: what each one reaches
(routes, endpoints, jobs), contract breaks in API responses,
dependency changes, security patterns
- diff coverage from your existing coverage report, so "38% of the
changed lines are tested" sits next to the risky files
- an HTML report where you mark files as reviewed and the score and
test plan follow; incremental mode for "what changed since my last
review"; suggested reviewers from git blame
How: it uses the TypeScript compiler API for TS/React (React Router,
Next.js, TanStack Router) and TS backends, a small go/types helper for
Go, and Python's ast for FastAPI/Flask/Django and friends. No LLM, no
network during analysis, and it never executes the analyzed repo's
code. It's deterministic and informs only; it never blocks a merge.
Limitations: only TS, Go and Python for now, and the reachability is
static, so dynamic dispatch and reflection make it fail closed
("unknown") rather than guess. I'd love feedback on the risk score
and on false positives in your codebase.
IDK lot of noise in report, I need more signal to noise ratio, already reading so much AI output as it is.
But the example PR... It said code generate with Claude, is it really still this bad? Last I was using SQLAlchemy with Claude was Opus 4.5, and it took some heavy coaching and skill gen not to generate code like this--but why even use SQLAlchemy at all if you're just going to use string interpolation jeez
https://github.com/erich-oliveira-winnin/full-stack-fastapi-...
Part of me thinks maybe because it's not popular or well documented, but SQLAlchemy has an eDSL of SQL, so you can skip the whole ORM/table mapping layer, and just use it to programmatically construct SQL syntax> SQLAlchemy has an eDSL of SQL, so you can skip the whole ORM/table mapping layer, and just use it to programmatically construct SQL syntax
Yes, this is called SQLAlchemy Core: http://docs.sqlalchemy.org/en/latest/core/
You can also just use psycopg (the Postgres driver layer) directly, it’s surprisingly ergonomic in my (limited) experience.
> It said code generate with Claude, is it really still this bad?
Anecdata, datapoint of 1 personal experience: I had been using Opus 4.6 with no problems until yesterday, then it seemed like it was missing things constantly, introducing random bugs, forgetting basic stuff that was in its memory and skills and should never miss
Same thing happened on 2 separate sessions I had with Opus 4.6
Ended up switching to 5.5 and Fable, things improved instantly
This left me with a weird taste
Out of curiosity, was this on a codebase where other folks started using Opus 5.5? I’ve been curious if each generation of model has its own biases that make it generate output that’s more difficult for other models/previous models to work with. That’s been my personal suspicion when I’ve hit situations like yours
It was on 2 separate private repos, personal projects. No one else working on those
But now that you asked, I recently had a similar experience with Codex, but on a shared repo. I also was using a slightly older model, and the all of a sudden it kinda stopped running tasks. I would ask it to do something and the it would just kind of do an analysis, then give me a recommendation of what to do. I had to tell it to execute the task, and then it would “be lazy”, like run one part then give me a recap and recommendation again instead of just going
Switched to a newer model, and the issue was instantly gone
That’s my favorite part of sqlalchemy. I don’t even use the orm anymore.
this is very needed - I don't think we'll see a world where agents approve all our PRs but there can definitely be better tooling for focusing human attention during review
it's hard for me to imagine something made with so little taste can be any good...
This looks nice, but I would use it with caution.
There was a post here recently, "Using LLM for confidence scoring is a bad idea", my mind went immediately to using LLMs for scoring risk. Seems like a bad idea. If there's anything we still need a human for, it's assessing risk.
>we still need a human for
A trained human mind that is.
Your average human in any given topic will give confidently incorrect risk assessments just like an LLM.
Good idea. Do you take bribes to exclude certain lines from the risk map? Asking for my good friend Jia Tan.
What, are you doubting in the integrity of the large language binary blob?
An engineer where I work built something like this so that when QA is about to deploy, they have a better indication of who to call and what is potentially going to be impacted. Our implementation is done prior to production deploy (which requires human approval) and compares currrebt deployed build code to new build changes diff and then analyses. I think there is a lot of room for tools like this in the future. Scoping/homing/etc
I do agree those kinds of tools are needed... great to know youve been working on something similar
>> Claude Code lives in its own repository, separate from the monorepo
I thought mono meant one.
If there couldn't be multiple monorepos, there would be no such thing as a monorepo.
Well there's one at Google, one at Facebook...
Monorepo -> microservices -> monoreposervices
Birepo
I think tools like this are needed and will become standard in the near future.
However, the backend demo report seems longer and more complex than just reading the code. I’m trying to build something similar as well, and no one has quite nailed this yet.
The risk map / assisted review needs to be succinct and information dense, to guide your attention. My current philosophy is that we need to be annotating code, not replacing the entire code review interface with a bunch of text.
Thanks... it surely is possible to try to reduce the amount of information
Looked at the backend demo, and the first lines suggested to look at were importing things. Exactly what I really don't need to be reviewing
It suggested based on the files most used (through graph dependency), for this reason it appears as a important file (as if it breaks, it can break several places in your codebase), but that being said, you can add some files to be ignored (if you have special files, like consts or i18n)
I want off this train. Please, make it stop…
why is the only comment by the OP flagged and dead? Is it because they were supposed to include that data in the description text rather than a comment, was it flagged as AI generated, or..?
Trying to understand why the poster's own comment was flagged and killed but the original article remains up.
AI-edited or -generated comments are disallowed by HN guidelines: https://news.ycombinator.com/newsguidelines.html#generated. I don't think any such restriction exists for submissions themselves, so that's my guess as to why the comment but not the submission has been flagged to death.
No restriction for the submission itself, unless it’s something like an Ask/Tell/Show HN, in which the content goes directly on the post (and will get flagged if AI-generated/assisted)
It was classified by our software as genai, which is explicitly against the guidelines: https://news.ycombinator.com/newsguidelines.html#genai
I'm not sure either. The comments don't look wrong to me.
You can vouch for it, if you view the comment by clicking the time since posted a “vouch” option appears. I did it. I’ve noticed this happening more and more on here for show HN posts especially. I’m not sure if it’s like some sort of troll gang or what…
In my personal experience, it’s very rare to catch a comment/story while vouchable. Not sure how many flags does it take for something to be killed, but it seems like almost as soon as something gets flagged, it becomes a signal for others to flag it too immediately without thought (similar thing kinda happens too when you post a comment and the first vote is a downvote, it almost never recovers, even if it was a good comment, as the initial drop/greying biases anyone that sees it before they even read it)
I like the concept, a risk map could be useful in some circumstances but I absolutely hate the UI design; it reeks of AI with no human thought or care put into it.
Thats is surely true... i did it with AI and i do lack design skills
Or you could, you know, read the changes your approving
Sire, wouldst thou perchance elaborate upon this ancient, forgotten technique, long buried in the mists of antiquity and shrouded in mysterious lore?
Assuming it works this could focus your attention.
Hi HN, I built farol at Winnin, where we open-sourced it (MIT).
AI agents now write a lot of our code, and AI reviewers catch a lot of the line-level problems. But someone still has to press "merge", and that person has the same question every time: how risky is this, and where should I spend my attention? A 40-file PR where 38 files are isolated is very different from a 3-file PR that changes a model used by 23 endpoints.
farol is a CLI (and a GitHub Action) that answers that statically:
- classifies the PR (global / touches-existing / isolated / unknown) and gives a 0-100 merge risk score - ranks the files by risk and says why: what each one reaches (routes, endpoints, jobs), contract breaks in API responses, dependency changes, security patterns - diff coverage from your existing coverage report, so "38% of the changed lines are tested" sits next to the risky files - an HTML report where you mark files as reviewed and the score and test plan follow; incremental mode for "what changed since my last review"; suggested reviewers from git blame
How: it uses the TypeScript compiler API for TS/React (React Router, Next.js, TanStack Router) and TS backends, a small go/types helper for Go, and Python's ast for FastAPI/Flask/Django and friends. No LLM, no network during analysis, and it never executes the analyzed repo's code. It's deterministic and informs only; it never blocks a merge.
Live demo (two reports from a real PR, nothing to install): https://winnin.github.io/farol/demo/
Try it on a branch: npx @winninjs/farol --help
Limitations: only TS, Go and Python for now, and the reachability is static, so dynamic dispatch and reflection make it fail closed ("unknown") rather than guess. I'd love feedback on the risk score and on false positives in your codebase.