I'm not sure what this means for AI startups if their innovations can be copied by OSS so quickly (what, like 2 weeks?). There's "consumer surplus" for everyone, to borrow an economic concept. But we do ideally want some of the surplus to flow to the innovator, too. I know there were precursors, but that's fine - it's hard to have a totally novel idea in such a popular field. I don't know what the end game is for TypeSafe - they'd need to demonstrate perpetually better results, or compete in another axis: UX, support, custom solutions, etc. So much of the time, someone proving a concept, or it simply getting enough publicity, is enough for a "Cambrian explosion" of follow-ups and copies. Famously, that was true for "Attention is All You Need", and the general idea of "next-token prediction" being so powerful.
We've stumbled into general differentiable models..
Because what they did is kinda trivial. Its basically like the Dropbox comment really[0], except here you don't need petabytes of storage and infinite VC pockets.
After chatgpt everything in AI mostly became LLMs and building wrappers around them. It's like people forgot how to do ML.
To those of us who actually trained models back in the day, its kind of cute to see people wowed by a classifier. Yes, this is 0 shot and doesn't need training (most people wanting this would've used structured output, this is cool because it's cheaper and faster). But anyone with basic ML knowledge could've built this in a few hours.
The question is mostly why wasn't this productized. And it's interesting indeed that it took this long to become a finished product.
Presumably the training recipe and training dataset itself cannot be easily copied in a week or two. So if they want to shut down these competitor models they need to make it obvious how they are better than them.
> difference between an instruct based re-ranker and laya/jev I just don't see it
Main difference is that laya/jev/et-al give you a zero-shot classifier that requires no training. You can prompt engineer your way to a quick fairly reliable cheap enough decision engine that you can use to iterate quickly (by prompt engineering).
Right now a lot of people are doing this with LLMs and it's too slow and expensive.
Imo the right iterative approach to productionizing these systems is something like:
1. Build it with an LLM. Iterate on the prompt
2. Start building a real-world dataset
3. When the prompt works, turn it into a clear rubric for Jev or similar
4. Keep iterating until desired accuracy achieved
5. Use the real-world evals you've built to train a custom classifier fine-tuned to your needs
You now have a system that has produced useful results in production from the very beginning and by the end it's a reliable super cheap classifier that can make thousands of decisions per second.
I don’t think that’s it. I sincerely doubt most developers are doing side by side comparisons of calibration quality.
OpenAI has a section on their embeddings model api page for zero shot classification. Of course you can choose an open weights embedding too if you’d like.
I think Jev wins on marketing and convenience. Most SWEs don’t want to talk about embeddings, cosine similarity, or precision/recall tradeoffs. They want something which plausibly works and is easy to use.
Yes it turns all that work of building a classifier into an api call. This is hugely valuable for prototyping and while you iterate on what the product should even do.
Calibrated probability across multi task with zero shot I guess. A reranker is single task and tuning it make it even more narrow. And I guess some piping to make multiclass efficient since you cannot mask logprob for independent questions in the same output space without throwing calibration away.
Has anyone actually seen better or the same results with Laya compared to Jev? From my experience, Laya performs significantly worse. It's less confident and often makes wrong decisions with more complex queries.
Yes. JEV generalizes better because they probably have an enormous corpus and trained on it for a long time. Laya's out of the box model is much weaker. However, in the age of LLM's it's incredibly easy and cheap to generate large datasets to fine tune laya for your task, and the training loop is pretty quick and cheap too.
It's so easy that I question why I would ever pay for JEV when eventually I'll have done enough random things that I will also have a large corpus and likely a general model as well.
Isn't the point of Jev that it generalises better?
It's a fast classifier you can use out-the-box, ~1.5bn tokens is about $40 (I've been hammering it)
It just works ... a whole bunch of low-level/low-importance workflow stuff that was getting farmed out to small/fast LLM models now has a competitive alternative ... and bits that hadn't even been considered to go into some external descision/classifier service can be tested/deployed at ~$0.00003/req
I don't get this wall of negativity on it, it's genuinely innovative/useful tech ... would expect HN to be more positive, regardless of whether it's the absolute best execution
I think your point is valid but many are annoyed that it is presented as groundbreaking, revolutionary, novel frontier tech when it is a known classification system. It’s the hype that feels undeserved. Honestly it was one of the best marketing campaigns I’ve seen.
It really does just work. And it works so well I already integrated it into my product. Saves me about 75% of costs for the section its working in, which isn't a small amount. I see a lot of negativity and I don't really get it either. Its so cheap and so fast, why not give it a try?
I didn't see any negativity in the post you replied to.
I think the point being made is that Jev is great but it has no competitive moat, and open source versions will very soon catch up if their secret sauce is just synthetic data.
Developer here. You're right, Laya is a lot weaker than Jev, especially on harder queries. It's a small model, so it's fast, but that's the trade-off. The open models that get close to Jev are much bigger, and running those is what I'm working on next.
Probably Kev and/or the decider models. Kev is trained on one of the 4B qwen models, similar for decider but it ranges from 0.8B through to the 35B-A3B model so far I believe.
one day, perhaps people will click through to the laya author's arxiv paper content and the why may become clearer, you won't have to read it, a skim will suffice
Nothing yet. Unfortunately it sometimes feels like our industry has been overrun by grifters and chancers.
I’m sure this has been a gradual and long decline. Maybe it even started with the dot com boom and accelerated with crypto. With AI it seems to have got worse.
I installed it, I tried the examples, it works.... But forgive my lack of imagination... what is this useful for?
Like, their example is of classification for a support interface.... `refund_requested`. Pretty convenient bool given the example is about a refund- what if 99% of submissions don't ask about a refund? Also, is that user not a `churn_risk`? What could possibly qualify as a churn risk if not a user asking for a refund?
https://ollaya.dev/library/laya The examples suffer the same problem of why I'd prefer to use a string column vs an enum. Changing an enum means you need to update the db, using a string you can do whatever.
I'm not trying to be negative, I genuinely want to know about some practical examples (that don't require tons of backwards maintenance).
I have a lot of semi-practical examples of how you can use this model wrapped in unix-ish tools - https://github.com/aurorainfra/grev (readme links to docs of each tool with some more or less practical examples)
Really I think "smart grep" is a pretty good one ('look for an error looking vaguely like this'). Also I think sql-based shell history + decision model is quite good to make the last 'which one of those choices is best fit given users past few commands' etc.
Ok, those are pretty decent examples, and clears up the utility a bit: speed and tokens. Some of it's still a bit iffy (e.g. `cutv 'email address' 'phone number' < examples/users.csv`, csv is already in columns), but I can see using it for some niche queries. Neat tool.
I very much appreciate your to-the-point, non-vibed README as well, ty for that.
I still think that `churn_risk` above is incorrect and unacceptable (perhaps there are sensible fixes, but saying "no churn risk" about a refund, in a leading example on their homepage, flabbergasting).
But if that were solved, I could see giving ollaya/grev to LLMs themselves, giving LLMs their own massive token-saver.
Yeah, speed is the one, I believe the default TypeSafe API quota is 1.5-2k queries per second (batched in bigger requests).
On the readme I'm so sorry to tell you that, but it's 100% written by Opus 5.5 with zero "pretty please don't write slop" prompting, it's just how slop is going to look like from now on. I've been writing code for 15 years or sth like that and the code is also what I'd call pretty reasonable..
It has some jargon hallmarks, which I noticed, but vibed or not, it's a massive improvement on other repos. Maybe it's because it's only a few commits so far... perhaps if you were to vibe 100 more commits it would devolve. Or maybe 5.5 really did improve (doubt it, still sounds like an asshole for me). But idk.
But with Jev you're just paying for input (prefill) which is really fast, and in case of Jev specifically costs 50% of Deepseek V4.1 Flash (which has famously really cheap input token pricing).
I put 250MB / 1M lines of logs through Grev and it cost ~$10USD, DSv4.1 would be at least 10x that and much, much, much slower. With Jev/Grev that 1M requests took 10 mins
Edit: completely misread your question - yeah you could finetune specialized models to do that, probably based on some decent pretrained llm base, that is true for roughly any Jev-shaped problem. Do you want to bother doing that, also having to deal with having to host a zoo of specialized models?
Is for when you want an AI to make a decision. If you have been using gpt or claude or open source models for that, than it’s a way cheaper alternative.
And if you have not been, it’s for when you have to extract the context from text. When you have numbers or fixed options, it’s just a matter of code.
So if you find yourself having to decide if a given user comment is a refund_request, that’s for that.
It’s not perfect, you still have to fine-tune (or calibrate) using examples you have (and keep those examples updated over time). But it’s way better than trying to parse text with regexes.
that laya is even a thing is further evidence, people took that author at face value, the paper contents are incomplete and describe something that does not sound like Jev at all
this was the period of arxiv history that led to the new vouching system, laya author contributed to that imo
My understanding is that Laya (or whatever it was called in 2025) was yet another fine-tuned classifier, not a general purpose one.
That said, Typesafe false marketing caused Laya to fit perfectly into pretty much every advantage that they are claiming: "system one decision model", cheap, fast, no hallucinations, structured, confidence output, parallel, calibrated. Their BS is their own demise.
I think Laya's author genuinely bought their BS and thinks he built the same thing. Unlike Typesafe, I don't think he's intentionally misleading people.
The only unique thing about Jev is that it's a general purpose classifier. Funny enough, they were so busy spreading marketing bullshit that they forgot to mention the only real thing that makes Jev unique.
Laya author is spitting more BS than Typesafe, the (incomplete) papers are nothing like Jev, they use RAG and azure hosted services for calculating embeddings, with an orchestrator. Jev is just a model, Laya was put together after Jev, almost certainly based on what the author learned from Typesafe, and then backported "his" idea
I suspect most people only read the blog post, and thought it was great how a VC company "stole" an idea and was "outdone" by a rando... without actually checking the facts. Confirmational reading bias, we live in a post-truth world with dysfunction media ecosystem
To be fair, he's just asking how to get customers. And the post is 2 days before Jev's launch date? I don't think he's trying to sell Laya there (though he probably will at this point).
Their marketing language is misleading. They must still use some transformer language model backbone to encode the text input (BERT or decoder-only LLM). The biggest difference is the output, instead of auto-regressively generating tokens, they produce probabilities over a bounded set of decisions (more flexible classification).
It would be good to list 1) zero-shot accuracy and 2) latency on the models page . The LLM-based models' latency is probably much higher than the BERT approaches I would assume.
Also curious, it seems from looking at the accuracy scores you gave that it seems to be NLI > Gliclass > Laya (for Bert types)? Why do you seem to feature/recommend Laya more - is Laya better in some way?
Laya is pretty easy to set up on its own without ollaya. I just did that and replaced my current jev API usage to laya running on a GTX 970 with 4GB of vram.
Very small context window, but for some existing small llm work I was doing, it was a drop-in replacement and it makes me happy I can get use out of old hardware I have running.
Depends on the model. The small ones I support today are well below Jev on harder queries, but fine for simple, well-defined questions. The open models that get close to Jev are bigger, and I'm adding support for those next.
The link rgbrgb posted is a good overview. The best open ones are close to Jev now, but they're big models. And I agree, if you have an eval set for a fixed task, a trained classifier is the better choice.
I am fairly confident if Jev-style decision models are seen as prominent (which, they seem to be), Ollama will support them. Surprised the team hasn't implemented this already.
I'm not sure what this means for AI startups if their innovations can be copied by OSS so quickly (what, like 2 weeks?). There's "consumer surplus" for everyone, to borrow an economic concept. But we do ideally want some of the surplus to flow to the innovator, too. I know there were precursors, but that's fine - it's hard to have a totally novel idea in such a popular field. I don't know what the end game is for TypeSafe - they'd need to demonstrate perpetually better results, or compete in another axis: UX, support, custom solutions, etc. So much of the time, someone proving a concept, or it simply getting enough publicity, is enough for a "Cambrian explosion" of follow-ups and copies. Famously, that was true for "Attention is All You Need", and the general idea of "next-token prediction" being so powerful.
We've stumbled into general differentiable models..
Because what they did is kinda trivial. Its basically like the Dropbox comment really[0], except here you don't need petabytes of storage and infinite VC pockets.
After chatgpt everything in AI mostly became LLMs and building wrappers around them. It's like people forgot how to do ML.
To those of us who actually trained models back in the day, its kind of cute to see people wowed by a classifier. Yes, this is 0 shot and doesn't need training (most people wanting this would've used structured output, this is cool because it's cheaper and faster). But anyone with basic ML knowledge could've built this in a few hours.
The question is mostly why wasn't this productized. And it's interesting indeed that it took this long to become a finished product.
[0] https://news.ycombinator.com/item?id=9224
Are you saying laya copied from jev, and released in two weeks? If so I don’t thinks it’s quite as simple a story as that. https://xtxinversexty.com/layas-prior-art-claim-is-absurd/
Presumably the training recipe and training dataset itself cannot be easily copied in a week or two. So if they want to shut down these competitor models they need to make it obvious how they are better than them.
Guys I have a real q, what is the difference between an instruct based re-ranker and laya/jev I just don't see it.
Edit: One is that jev/laya are tuned to have better probabilities, but a reranker can be fine tuned to do that as well. And jev/laya use RLCD?
> difference between an instruct based re-ranker and laya/jev I just don't see it
Main difference is that laya/jev/et-al give you a zero-shot classifier that requires no training. You can prompt engineer your way to a quick fairly reliable cheap enough decision engine that you can use to iterate quickly (by prompt engineering).
Right now a lot of people are doing this with LLMs and it's too slow and expensive.
Imo the right iterative approach to productionizing these systems is something like:
You now have a system that has produced useful results in production from the very beginning and by the end it's a reliable super cheap classifier that can make thousands of decisions per second.I don’t think that’s it. I sincerely doubt most developers are doing side by side comparisons of calibration quality.
OpenAI has a section on their embeddings model api page for zero shot classification. Of course you can choose an open weights embedding too if you’d like.
https://developers.openai.com/cookbook/examples/zero-shot_cl...
I think Jev wins on marketing and convenience. Most SWEs don’t want to talk about embeddings, cosine similarity, or precision/recall tradeoffs. They want something which plausibly works and is easy to use.
Yes it turns all that work of building a classifier into an api call. This is hugely valuable for prototyping and while you iterate on what the product should even do.
Jev's value becomes more apparent when the task is a moving target. eg an auto-mode classifier.
Calibrated probability across multi task with zero shot I guess. A reranker is single task and tuning it make it even more narrow. And I guess some piping to make multiclass efficient since you cannot mask logprob for independent questions in the same output space without throwing calibration away.
Has anyone actually seen better or the same results with Laya compared to Jev? From my experience, Laya performs significantly worse. It's less confident and often makes wrong decisions with more complex queries.
I've been following jevbench twice a day for the past week and that's been a lot of fun. Latest update:
Rank System Score Public / sealed accuracy Evidence
1 decider-4b v2 64.13 83.5% / 34.7% Evaluator-run, offline
2 Jev 1.13 63.29 86.6% / 36.7% Evaluator-run API
3 JevK5 v0.2 62.04 85.3% / 33.1% Evaluator-run
4 Cygnet 12B 61.76 87.9% / 33.8% Evaluator-run, offline
5 Hopper 59.43 82.3% / 34.1% Evaluator-run
28 Kev 4B 36.14 66.2% / 22.4% Evaluator-run
41 Laya 421M 30.25 58.4% / 30.8% Evaluator-run
https://benchmarkheaven.com/jev-models
Amazing - was looking for some benchmarks around this earlier
What the best way to see how a homegrown version compares?
Yes. JEV generalizes better because they probably have an enormous corpus and trained on it for a long time. Laya's out of the box model is much weaker. However, in the age of LLM's it's incredibly easy and cheap to generate large datasets to fine tune laya for your task, and the training loop is pretty quick and cheap too.
It's so easy that I question why I would ever pay for JEV when eventually I'll have done enough random things that I will also have a large corpus and likely a general model as well.
Isn't the point of Jev that it generalises better?
It's a fast classifier you can use out-the-box, ~1.5bn tokens is about $40 (I've been hammering it)
It just works ... a whole bunch of low-level/low-importance workflow stuff that was getting farmed out to small/fast LLM models now has a competitive alternative ... and bits that hadn't even been considered to go into some external descision/classifier service can be tested/deployed at ~$0.00003/req
I don't get this wall of negativity on it, it's genuinely innovative/useful tech ... would expect HN to be more positive, regardless of whether it's the absolute best execution
I think your point is valid but many are annoyed that it is presented as groundbreaking, revolutionary, novel frontier tech when it is a known classification system. It’s the hype that feels undeserved. Honestly it was one of the best marketing campaigns I’ve seen.
It really does just work. And it works so well I already integrated it into my product. Saves me about 75% of costs for the section its working in, which isn't a small amount. I see a lot of negativity and I don't really get it either. Its so cheap and so fast, why not give it a try?
I think it’s the infamous Dropbox reaction - anyone can wrap an FTP server, where the innovation?
Starting from a business POV one should inflate terminology, hack together an MVP, and see if the market demands it before doing hardcore R&D.
But starting from technical/craftsman POV all you see is a hack and a lot of big words, so it’s easy to become jaded.
If you don't mind me asking, what are you using it for?
I've been unable to find a good use case for now.
I didn't see any negativity in the post you replied to.
I think the point being made is that Jev is great but it has no competitive moat, and open source versions will very soon catch up if their secret sauce is just synthetic data.
(Whether or not that is true, I don't know.)
Developer here. You're right, Laya is a lot weaker than Jev, especially on harder queries. It's a small model, so it's fast, but that's the trade-off. The open models that get close to Jev are much bigger, and running those is what I'm working on next.
It doesn’t to be a ton bigger, 16k and reliable 8k would be a godsend. (I run at 2k)
What are the models? I am super curious in these as well
Probably Kev and/or the decider models. Kev is trained on one of the 4B qwen models, similar for decider but it ranges from 0.8B through to the 35B-A3B model so far I believe.
In my experience it's not close and the benchmarks I've seen don't reflect my experience at all.
But I'm guessing people will find the right training regime and data mix soon to close the gap.
But big things I see are instability and inaccuracy - like pick a random problem.
one day, perhaps people will click through to the laya author's arxiv paper content and the why may become clearer, you won't have to read it, a skim will suffice
Nothing yet. Unfortunately it sometimes feels like our industry has been overrun by grifters and chancers.
I’m sure this has been a gradual and long decline. Maybe it even started with the dot com boom and accelerated with crypto. With AI it seems to have got worse.
I installed it, I tried the examples, it works.... But forgive my lack of imagination... what is this useful for?
Like, their example is of classification for a support interface.... `refund_requested`. Pretty convenient bool given the example is about a refund- what if 99% of submissions don't ask about a refund? Also, is that user not a `churn_risk`? What could possibly qualify as a churn risk if not a user asking for a refund?
https://ollaya.dev/library/laya The examples suffer the same problem of why I'd prefer to use a string column vs an enum. Changing an enum means you need to update the db, using a string you can do whatever.
I'm not trying to be negative, I genuinely want to know about some practical examples (that don't require tons of backwards maintenance).
I have a lot of semi-practical examples of how you can use this model wrapped in unix-ish tools - https://github.com/aurorainfra/grev (readme links to docs of each tool with some more or less practical examples)
Really I think "smart grep" is a pretty good one ('look for an error looking vaguely like this'). Also I think sql-based shell history + decision model is quite good to make the last 'which one of those choices is best fit given users past few commands' etc.
Ok, those are pretty decent examples, and clears up the utility a bit: speed and tokens. Some of it's still a bit iffy (e.g. `cutv 'email address' 'phone number' < examples/users.csv`, csv is already in columns), but I can see using it for some niche queries. Neat tool.
I very much appreciate your to-the-point, non-vibed README as well, ty for that.
I still think that `churn_risk` above is incorrect and unacceptable (perhaps there are sensible fixes, but saying "no churn risk" about a refund, in a leading example on their homepage, flabbergasting).
But if that were solved, I could see giving ollaya/grev to LLMs themselves, giving LLMs their own massive token-saver.
Yeah, speed is the one, I believe the default TypeSafe API quota is 1.5-2k queries per second (batched in bigger requests).
On the readme I'm so sorry to tell you that, but it's 100% written by Opus 5.5 with zero "pretty please don't write slop" prompting, it's just how slop is going to look like from now on. I've been writing code for 15 years or sth like that and the code is also what I'd call pretty reasonable..
It has some jargon hallmarks, which I noticed, but vibed or not, it's a massive improvement on other repos. Maybe it's because it's only a few commits so far... perhaps if you were to vibe 100 more commits it would devolve. Or maybe 5.5 really did improve (doubt it, still sounds like an asshole for me). But idk.
Isn't it better to use an LLM to train modernbert or xgboost et al?
It is /possible/ to use an LLM.
But with Jev you're just paying for input (prefill) which is really fast, and in case of Jev specifically costs 50% of Deepseek V4.1 Flash (which has famously really cheap input token pricing).
I put 250MB / 1M lines of logs through Grev and it cost ~$10USD, DSv4.1 would be at least 10x that and much, much, much slower. With Jev/Grev that 1M requests took 10 mins
Edit: completely misread your question - yeah you could finetune specialized models to do that, probably based on some decent pretrained llm base, that is true for roughly any Jev-shaped problem. Do you want to bother doing that, also having to deal with having to host a zoo of specialized models?
Is for when you want an AI to make a decision. If you have been using gpt or claude or open source models for that, than it’s a way cheaper alternative.
And if you have not been, it’s for when you have to extract the context from text. When you have numbers or fixed options, it’s just a matter of code.
So if you find yourself having to decide if a given user comment is a refund_request, that’s for that.
It’s not perfect, you still have to fine-tune (or calibrate) using examples you have (and keep those examples updated over time). But it’s way better than trying to parse text with regexes.
>Run decision models locally.
>example is a text classification task instead of a decision
"Decision model" is just marketing jargon.
decision model = classifier
system one model = small non-reasoning LLM
noul = boolean
confidence = f(probabilities)
It's sad to see how gullible engineers are today.
> how gullible ... today
that laya is even a thing is further evidence, people took that author at face value, the paper contents are incomplete and describe something that does not sound like Jev at all
this was the period of arxiv history that led to the new vouching system, laya author contributed to that imo
My understanding is that Laya (or whatever it was called in 2025) was yet another fine-tuned classifier, not a general purpose one.
That said, Typesafe false marketing caused Laya to fit perfectly into pretty much every advantage that they are claiming: "system one decision model", cheap, fast, no hallucinations, structured, confidence output, parallel, calibrated. Their BS is their own demise.
I think Laya's author genuinely bought their BS and thinks he built the same thing. Unlike Typesafe, I don't think he's intentionally misleading people.
The only unique thing about Jev is that it's a general purpose classifier. Funny enough, they were so busy spreading marketing bullshit that they forgot to mention the only real thing that makes Jev unique.
Laya author is spitting more BS than Typesafe, the (incomplete) papers are nothing like Jev, they use RAG and azure hosted services for calculating embeddings, with an orchestrator. Jev is just a model, Laya was put together after Jev, almost certainly based on what the author learned from Typesafe, and then backported "his" idea
I suspect most people only read the blog post, and thought it was great how a VC company "stole" an idea and was "outdone" by a rando... without actually checking the facts. Confirmational reading bias, we live in a post-truth world with dysfunction media ecosystem
I think you're right about Laya (and confirmation bias).
But like you said, at the end of the day he's just a rando.
He's not asking for $40m, not saying "I made ChatGPT, but i hate it, so I built the next big thing". Not claiming to co-invent RLHF.
Laya is just noise. Jev's bullshit affects me today - I see people injecting it into the codebases where it has no place.
> He's not asking for $40m
Just how to "make fkn $500k ARR fast?"
https://news.ycombinator.com/item?id=49674396
too much LI/Xitter influencer consumption
That's really funny, nice find.
To be fair, he's just asking how to get customers. And the post is 2 days before Jev's launch date? I don't think he's trying to sell Laya there (though he probably will at this point).
There's something to be gleaned from the sum of their output across GitHub, arxiv, reddit, and HN (didn't delve LI, I hear it's a hot mess)
text classification is equivalente to decision. This is exactly the same thing Jev does.
If it has four legs, a tail and barks why not call it a dog?
Because this specific dog only barks in structured text
This dog only barks when given biscuits
It is not. In a benchmark with actual decisions - navigation, traffic, waypoints - laya does only slightly better than a small classifier.
Jev does it more efficiently because it doesn't use an LLM https://typesafe.ai/blog/introducing-system-one-models-and-j...
Their marketing language is misleading. They must still use some transformer language model backbone to encode the text input (BERT or decoder-only LLM). The biggest difference is the output, instead of auto-regressively generating tokens, they produce probabilities over a bounded set of decisions (more flexible classification).
Fair point, that example is basically classification. I'll change it to something that looks more like a real decision.
Ollama is for large language models, so this is for large language... yodels?
It would be good to list 1) zero-shot accuracy and 2) latency on the models page . The LLM-based models' latency is probably much higher than the BERT approaches I would assume.
Also curious, it seems from looking at the accuracy scores you gave that it seems to be NLI > Gliclass > Laya (for Bert types)? Why do you seem to feature/recommend Laya more - is Laya better in some way?
Laya is pretty easy to set up on its own without ollaya. I just did that and replaced my current jev API usage to laya running on a GTX 970 with 4GB of vram.
Very small context window, but for some existing small llm work I was doing, it was a drop-in replacement and it makes me happy I can get use out of old hardware I have running.
It would be really cool to have LLMs and System One in a single tool - in this case, if Ollama implemented it.
next vLLM release will have this
if you use gateways, GoModel support the S1 endpoints, my favorite feature is the virtual models, stable name, I can swap out the backing model(s)
https://gomodel.enterpilot.io/docs/getting-started/quickstar...
(the "kev" in the docs is my fault, I should have said Jev / System1 in my feature request)
Sounds good on latency but how is its actual decision quality vs. Jev?
Depends on the model. The small ones I support today are well below Jev on harder queries, but fine for simple, well-defined questions. The open models that get close to Jev are bigger, and I'm adding support for those next.
FAQ[1] says:
> It is an independent project, not affiliated with Ollama.
[1]: https://ollaya.dev/docs/faq
Are there many models that are comparable to Jev for generic decision making?
Smarter move if you have an eval set is to just train a classifier and call it a day.
there's this thing with a bunch of similar models https://huggingface.co/spaces/multimodalart/jev-decision-ind...
top open one is trained by perplexity cto for $3k, kinda cool https://x.com/denisyarats/status/2102252088067850507
<<<"i was curious to see if i could train a competitive Jev-like model completely autonomously with a swarm of agents using our internal system."
Bro is writing off the H200 lol
On a sidenote I really can't stand the term "swarm" and definately plays into AI doomerism.
The link rgbrgb posted is a good overview. The best open ones are close to Jev now, but they're big models. And I agree, if you have an eval set for a fixed task, a trained classifier is the better choice.
Would be great if you supported CUDA 12; I don't feel like paying $15K to upgrade my GPU right now
wait another week or so for vLLM's next release
Cool... but this does seem undermined by the fact that Ollama can add support for decision models at any time.
Fair, and I'd be happy if they did. Ollaya uses the same API as Jev, so your code isn't tied to it either way
and that ollama is go-llama and not rust, so it's not really the ollama of anything
Does anyone know what laya multi lang is faster than laya en? I would have thought focusing on a single language would be faster.
Nice work! Making open models easier to run locally is valuable on its own. Keeping the API compatible with Jev is a thoughtful touch, too.
I am fairly confident if Jev-style decision models are seen as prominent (which, they seem to be), Ollama will support them. Surprised the team hasn't implemented this already.
I have also tried this and its really awesome
great project for empowering open-source alternatives.
open-source is the only way for safe AI development. whoever doesn’t share the weights/code will lag behind.
Thanks!
Why do you need another model-type specific Ollama? Can't Ollama be made to support these models?
Hey Claude, make ollama for Jev like models. Make no mistakes /s
hey Claude, download and run vllm nightly for me
(already merged)
GoModel (gateway) already supports Jev like endpoints too
https://gomodel.enterpilot.io/docs/providers/jev
This inference engine is soooo much faster btw: https://github.com/tamnd/kime