valid question. I thought a lot about this. The prefix before every fold stays intact (cache hit). But for the messages after the fold there is always a cache miss. Folds don't happen too often, so in practice this turned out to be worth it, because the cache miss comes with a big token reduction in general. I don't have a solid cost analysis for this yet, though.
To those of us who can barely keep up with the proliferation of homonyms (words with the same name and different meanings):
> Pi is a minimal agent harness.
Not 3.14159 and so on
Not a computer.
I just switch to pi from OpenCode (another polynomer), and have no intentions of going back
Isn't this going to completely slaughter cache hit rates?
valid question. I thought a lot about this. The prefix before every fold stays intact (cache hit). But for the messages after the fold there is always a cache miss. Folds don't happen too often, so in practice this turned out to be worth it, because the cache miss comes with a big token reduction in general. I don't have a solid cost analysis for this yet, though.
I won't touch anything that touches my context without very compelling evals. The readme is too minimal, imo
Which evals would you be interested in specifically? And what are you missing in the readme?