> remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job
I don't really think that this is true, in the way that it's written.
I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machine code isn't the part that requires unsafe. The language's runtime is a more likely site to find unsafe.
> I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machine code isn't the part that requires unsafe. The language's runtime is a more likely site to find unsafe.
Agreed! Emitting machine code is not unsafe, since it's just writing bytes down - it's only once you execute that machine code that there's potentially unsafety. The reason I said "a big part of the job" is that in practice a lot of compilers both emit machine code and execute it - but you're totally right that it's not a requirement that a compiler do both.
In addition to the examples you gave (hot binary patching/code reloading, language runtime, etc.), others would be things like evaluating userspace code at compile time (e.g. const fn in Rust, or in Roc any expression that could be hoisted to the top level), running tests and inspecting their output to decide what to display to the user, etc.
Those are the types of things I had in mind when I wrote that.
I am disappointed you're downvoted, Richard. This is a fine reply, and I hope you know that a minor quibble with a single line in the post doesn't mean that I think it's a bad one overall. (EDIT: a few minutes later, the parent comment is no longer grey.)
I also think it's a good thing that you wrote the post in general, when I saw it pop up I was like "oh, of course, this post should exist!" I'm surprised I didn't think about it earlier.
> evaluating userspace code at compile time
Usually this would be done via an interpreter, so I'm not sure that it really requires unsafe either. If you are literally executing machine code, sure, but const fn in Rust and constexpr in C++ and many other languages do not do that, as it causes a number of problems (for example, cross-compilation).
Also a good point! TIL that Rust and C++ use interpreters for const, although of course that wouldn't work for running tests. Then again, in the specific case of Rust I believe rustc only compiles the tests and then something else like Cargo executes them. Of course, as I noted elsewhere, if rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupted (due to miscompilation) as if rustc and cargo shared a code base.
By the way, I thought your question was totally reasonable - my first thought reading it was "Oh yeah I wasn't trying to say that writing bytes is unsafe, I definitely should have worded that differently."
Cool, I'm not sure that people know that we know each other and have some deeper mutual understanding. :)
> although of course that wouldn't work for running tests.
Why not? Unless you mean in the cross-compilation case, in which yeah, to run the compiled tests you'd need an emulator.
> in the specific case of Rust I believe rustc only compiles the tests and then something else like Cargo executes them.
It doesn't have to be Cargo, but yes, rustc produces executables for the tests, and you have to then run them.
> there's the same opportunity for end user memory being corrupted (due to miscompilation)
I agree for sure that the safety of the outputted binary is completely distinct from the safety of the compiler itself.
I think the reason that this framing specifically (in the post and in this comment) strikes me as odd is that "requires unsafe code" sort of implies that you need to use unsafe to fix the unsafety of the outputted binary. That just isn't the case. Of course, this is a serious bug that needs to be fixed, but there's just something about "doing memory unsafe things" in this area that like, I think can be a little mis-leading, even if that's not intentional. But I am going to sit with this and think about it, regardless, because I am not sure that my gut reaction here is completely accurate.
(And, hilariously, looking over some work my agents did on my compiler last night, they fixed some mis-compilations that occurred, entirely in safe code. I bet that's also part of why I'm in this headspace at the moment, it's not like those fixes required dropping down into unsafe to fix either!)
> if rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupted (due to miscompilation) as if rustc and cargo shared a code base
Your tests run in an entirely separate process from the compiler (and from cargo). This makes it very different from memory corruption in the compiler:
- The test process can only corrupt its own memory.
- You don't need "unsafe" to run tests. Just the ability to start another process.
- If you're cross-compiling, you wouldn't even be able to run the tests on the same machine (without emulation/compatibility layers)
Does roc run tests in the same process as the compiler?
> Does roc run tests in the same process as the compiler?
We do for tests of pure functions, yes.
> Your tests run in an entirely separate process from the compiler (and from cargo).
That's a great point and a relevant distinction, although Rust tests can run arbitrary I/O, so it's not like having them be in a separate process means memory corruption is harmless! :)
True, but not sure how it's relevant - you don't need memory corruption for the test to perform arbitrary I/O. And even if you trust the tests to not do anything bad, you don't need memory corruption for this to be an issue - any bug in the code emitted by the compiler could cause bad things to happen.
But all of this is ultimately completely unrelated to the concept of "unsafe". You can delete your home directory just fine in safe Rust/Go/Python/etc. You can write a compiler that emits broken code in the same languages; even in a 100% pure functional language with 0 side effects and a perfect bug-free implementation.
> rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupted (due to miscompilation) as if rustc and cargo shared a code base.
Cause this hasn't been true for me or for anyone maybe your definition of memory being corrupted is the not same as mine.
I am not even sure what you are trying to prove with this.
I appreciate the time and effort in building stuff like Roc I don't use it but this comment and the article feel like...
Oh some guy said Zig not nice because memory safety so here, a post why memory safety doesn't exist because we have to do memory unsafe things sometimes and so everything is memory unsafe already, so maybe it doesn't matter.
I get the energy that we are going for seeing useless claims and wanting to push back but I think the article deserves a clearer part 2 where you elaborate on your thoughts about stuff maybe even get it peer reviewed a bit before posting or maybe don't I guess we could use more raw thoughts in the post AI age.
Either way I appreciate someone trying to put forward their own thoughts and explain problems with a different perspective.
Scenario A: A program has a memory vulnerability. That's bad, and the reason it's bad: attacker-controlled data can potentially trigger this vulnerability, which could even lead to privilege escalation.
Scenario B: A program writes machine code in an executable region of memory, and the code has a memory vulnerability. That's bad, and the reason it's bad: attacker-controlled data can potentially trigger this vulnerability, which could even lead to privilege escalation.
Scenario C: A program write machine code to disk, which is then read into memory and executed. That's bad, and the reason it's bad: attacker-controlled data can potentially trigger this vulnerability, which could even lead to privilege escalation.
How is that a memory safety issue in Rust compiler? What point are we making here. Am I just too dense to understand, is how the hell is that a need unsafe issue? Or anything to do with compilers???
> Usually this would be done via an interpreter, so I'm not sure that it really requires unsafe either.
Well, I personally have written a const-expression evaluator that actually reuses the rest of the compiler: it compiles the expression in the current environment with some specific adjustments to the codegen settings, launches the temporary executable and gathers its output... frankly, it's more hassle than it's worth compared to writing a separate const-expression interpreter. Plus, of course, it also runs slower since most constant expressions are usually pretty trivial.
Side concerne, but reading the post initially I thought it was about Rocq (formerly Coq). Maybe both team could coordinate and find a way that everybody agree on to avoid any confusion?
Because they view "unsafe" as an escape hatch instead of a feature. It's a way to encapsulate dangerous behavior, tightly, with clear postcondiitions. Sometimes it's the only way to do things like interact with inherently unsafe FFI code, or hardware.
I adore unsafe, appreciate it as a feature... but it is an escape hatch. One that is sometimes necessary, one that is sometimes not necessary but might still be (ab)used for performance, or initial 1:1 porting of C/C++ code. There are a lot of cases where that escape hatch should probably welded shut though. Fortunately, the Rust ecosystem has tools like `cargo geiger`, and straight out of the box I can also write:
I feel like this perpetuates a bad mental model. Unsafe is not an escape hatch. Code within unsafe blocks must uphold the same semantics as code outside it but the compiler cannot guarantee that those semantics are upheld.
If we're using analogies, `unsafe` is like a "hard hat required" sign. There's nothing intrinsically different about the space inside or outside of it, other than that you can't be sure a brick isn't going to fall on your head once you cross over. So it's on you to wear a hard hat. And to not drop any bricks and trust other people to do their best not to drop any bricks.
Yeah, I'd have more sympathy for the "It's an escape hatch" model of this feature if the C++ approaches to this problem didn't keep adding actual escape hatches and waving away concerns as "Rust does this too so it's fine".
Memory safe languages, where usage of Miri and Valgrind (tools to for instance debug memory unsafety) are common and integrated into CI for some of the projects in the language. Even some Rust guides encourages running Miri in CI https://microsoft.github.io/RustTraining/engineering-book/ch... . Searching on GitHub yields a lot of projects that run Miri in CI. And there have been a lot of CVEs for Rust projects caused by memory unsafety.
I think it's a reaction to many simpler advocates of Rust saying "it's unsafe and irresponsible of you to not use Rust, you can't write software in C or C++ or Zig (or Go?) and have memory unsafety and put your users at risk, it's not safe, you have to use Rust to be safe".
Freakin' safety. I like doing lots of unsafe things in life: rock climbing without gear (short bouldering sections on hiking trails), bicycling and skateboarding without a helmet, etc. I developed pretty good skills in all these things, and programming C too. What kind of life would be worth living with enforced perfect safety? Without developing skill in life's many un-safe activities?
So people want to emphasize that, if they shouldn't use non-Rust languages because their safety can't be guaranteed, well your safety using Rust isn't absolute, either. So you can't tell me I must use Rust, or the Rust rewrite of my favorite tools, to "be safe".
Do you recall the early rust web framework, that used a lot of "unsafe" "inappropriately" and had some vulnerabilities ... actix web? ... reminds me of the bun-in-zig situation, kinda makes the language's PR situation more complicated and tricky.
I don't think using an example that includes licensing, plenty of regulations governing drivers, the cars themselves and the built environment, enforcement, and plenty of research on the impact of passive infrastructure to make things safer (bollards, daylighting, raised pedestrian intersections, curbs, traffic lights, etc.) makes the case you seem to be trying to make.
Yep: there's a difference between driving a car in traffic and driving a car in a rally or in an endurance race. There will always be some rules and regulations unless you're on your own privately owned race track with no one else on there, but the regulations can differ greatly depending on the situation.
I agree that it’s not inherent to emitting machine code but I do think it reflects a different set of priorities.
In extremely high performance code you use different data structures and algorithms and change your approach to memory allocation. TigerBeetle famously does all memory allocation once on startup.
Roc is attempting to make a similar set of trade-offs in their compiler as Zig, so it makes sense that the author finds many shared patterns.
I do think it reflects different priorities, but one of those differences is that from my perspective, safety and performance are not inherently at odds. Yes, sometimes it is needed, but not as much as some people seem to think. Sometimes, it also means writing code in ways that communicate things to the compiler that you may not think of if you're not used to thinking in this manner.
A lot of the ways in which the zig compiler works doesn't use pointers, it uses indices. This stuff is easier to write as safe code, not less easy.
> Roc is attempting to make a similar set of trade-offs in their compiler as Zig, so it makes sense that the author finds many shared patterns.
I do think that that makes sense, but it also doesn't mean that they have to. I am doing a compiler project that takes a lot of inspiration from Zig (as my language currently inherits some major things from Zig, and I also care a lot about compiler performance) and it's written in Rust, and does not use much unsafe code (outside of the usual suspects of FFI in the runtime, etc).
> A lot of the ways in which the zig compiler works doesn't use pointers, it uses indices. This stuff is easier to write as safe code, not less easy.
I respectfully disagree. This is only true if you view malloc as qualitatively different from a piece of code that gives you an index for a free object in an object pool.
Provided you don't ask/give back memory from/to the OS, what malloc is doing is giving you an index (pointer) into a pool of bytes, while manipulating an internal bookkeeping structure.
Use after free is just you using an index after said bookeeping structure has marked that piece of memory as available for something else (and perhaps claimed already).
If you have an array of Node structs to represent a graph (like the AST in Zig), and use indices to represent references, you have essentially zero protection in Rust that helps you with finding Node-s that have been used for something else etc.
The 'asking memory from the OS' aspect for malloc doesn't really change the safety of your language compared to this where it matters - if you do use-after-free on a page claimed by the OS, you get a segfault, which immediately tells you there's a problem, which is much better than silent corruption.
At least with malloc, you get debug allocators, or other features that can help you in this case. If you are careless with indices in an object pool, and overwrite stuff, essentially, it's up to you to figure out what went wrong and you have no tools to help you.
I fully agree that there are similarities here. But we disagree about some of the details.
> you have essentially zero protection in Rust that helps you with finding Node-s that have been used for something else etc.
The most obvious technique is generations. You can of course do that in Zig as well.
> if you do use-after-free on a page claimed by the OS,
This assumes that you're working in the context where there is an OS. That isn't always the case. Also, there are other cases than just use-after-free: for example, compilers will optimize around null pointers being UB, which can cause other problems, whereas an index of zero does not get the same treatment.
But also, again: Zig does not use malloc for its ASTs, as far as I know. It uses lists and indices. I haven't literally read the code lately myself, but I would be surprised if they went back to malloc'ing individual nodes.
> In extremely high performance code you use different data structures and algorithms and change your approach to memory allocation.
It's worth noting that the reason Rust doesn't include support for custom memory allocation patterns like Zig does has nothing to do with memory safety. It's more of a historical accident that it just wasn't something that was prioritised early in the projects history and is now hard to change.
It's also important to note that Rust as a language does not know about the heap at all, it is purely a library concern. This means that "doesn't include support for custom memory allocation patterns" is purely talking about standard library data structures, and if you need a ton of performance, you're probably going to be writing your own anyway.
It will be nice when the Allocator trait stabilizes so that the ecosystem can coordinate on making this stuff pluggable, but that's not a direct blocker for getting things done if you need to do things today.
The interfaces to custom allocators are more idiomatic, standardized, and normalized in Zig, but there's nothing systematically unsafer about the Rust equivalent. You can use custom allocators all you want in Rust, as long as you're okay using third-party crates like Bumpalo.
Yeah that is definitely 1000% wrong. A compiler can do its job with totally abstract data structures. If anything would need to do unsafe stuff in memory, it would probably be a linker.
I think it's an understandable prior. Historically, "low level stuff" was near-exclusively (see my comment below about OCaml...) written in unsafe languages. Even if that wasn't always literally required, it sometimes was, and so thinking this is the case was a reasonable thing to think.
It is only relatively recently that we have gained more realistic options in these spaces, and so not fully understanding the implications, or preferring the historically normal choices, is understandable.
I'll play devil's advocate. I think emitting machine code intended to run is unsafe because you could emit unsafe machine code, which could run. It's the whole system that is either safe or not, not the individual components. If your system gets hacked by a buffer overflow in the end, nobody cares whether it was the linker that overflowed or the code emitted by the linker.
"Safe" has a very specific definition in Rust. It's not identical to the broader definition used in technical English. You can easily have safe rust code with behaviors any reasonable layperson would call unsafe, like crashing a plane. The original article, comment, and replies were using the word in the Rust sense from my reading, not the English meaning.
2. Has an objective definition, when some other forms of safety are either subjective or inter-subjective.
That said, I don't understand why your parent brought this up to you, you are talking about memory safety in your original comment here, so that's what Rust's safety is about.
I feel that the buzz phrase "memory safety" has been defined by Rust to mean "the safety Rust gives you". Obviously memory usage can be more safe or less safe, and Rust is decidedly on the safe end of the spectrum, but it also has the gaping type system holes demonstrated in cve-rs which completely shatter any claim that safe code is safe, and there are other bugs which occur in Rust while the programmer is distracted by trying to prove their code is memory-safe.
The part of the system that it involves was in the process of being re-written already. The re-write fixes the bug. Because it is essentially a theoretical issue, and not an actual problem in any real code, it is not a five alarm fire. Waiting for that re-write to land makes the most sense, instead of putting in a ton of work that will be thrown away.
This one was impacting actual users, and did not require re-writing entire subsystems to fix properly. So the engineering and product tradeoffs are different.
If cve-rs exists, we still say that safe Rust is safe, in the same way that we say Python is safe despite potential bugs in its interpreter or native libraries, and that Java is safe despite potential bugs in the JVM or JNI libraries, and so on.
This is impossible. General words like "safe" and "good" are subjective, and useless in a technical context unless you ground the discussion by giving them specific definitions. Otherwise everyone ends up talking past each other.
Good for what? A hammer is good for driving a nail, but not good for driving a screw.
Safe for what? My house is safe for humans, but not safe for tropical birds.
Clean enough for what? Our water is clean enough to wash my ass, but not clean enough to wash a telescope mirror.
Sorry but life is not a Disney movie where some things are unequivocally good/safe and other things are unequivocally bad/unsafe. There are gradients and conditions, and communication requires a shared language between participating parties to navigate them.
What nail? A hammer is good for driving a nail from the hardware store, but not good for driving a finger nail.
See? I can play stupid word games too.
How tropical are the birds? I'm afraid life isn't a Disney movie where some things are unequivocally tropical/not tropical. How shared is the language? Congratulations on using only two adjectives in your comment besides the ones you're complaining about, but two is greater than zero.
How much your is the house? Do you own it? Without any mortgage or lien?
> It's the whole system that is either safe or not, not the individual components.
This is a core perspective disagreement. While this is true:
> If your system gets hacked by a buffer overflow in the end, nobody cares whether it was the linker that overflowed or the code emitted by the linker.
That does not mean that increasing the amount of safety in the individual components isn't helpful, because it helps minimize the above outcome, even if it will never be zero.
Safety is a feature of a system - yes. It's also a property of what it's against. A computer could be safe against being hacked but still be dangerously easy to drop on someones toe and break it.
Safety [against something] is also a feature of components - a system made up of only safe components [against a thing] is safe [against the same thing... I'm going to stop this qualification now for brevity]. A system containing unsafe components may or may not be safe but at least you know what components usage you need to look at carefully.
If your linker is safe, linking code will never result in the thing it is safe against. Ever. This is a useful property even if running the linked thing is not safe because it means:
1. When things go wrong in strange ways, you have strict bounds guiding you in figuring out what went wrong.
2. You can build reliable systems that do part of the job, and only have to sandbox the other half of the job. Compiling in a CI system will (if the compiler was entirely safe) be safe. You can do it with secrets present against malicious code. Running tests will have to be sandboxed (assuming running tests isn't safe). This could for instance enable safely sharing significantly more artifacts for incremental builds in CI.
Unfortunately very few compilers are really safe against anything (though I do wonder how I could break my toe on one). Rustc for instance has a giant C++ half called llvm that isn't really hardened at all. We get away with this by just not trusting the compiler when run against potentially malicious code.
It wouldn't be the linker that has to be unsafe, it'd be the "and now execute!" jump. And that could be abstracted as memfd+execveat, which are fairly normal operations.
OP's argument is roughly "doings things with computers has to be unsafe to be useful", which is.. uninteresting.
They are saying that running the compiled code is memory-unsafe when there is a compiler bug, and that’s what developers do next. The memory corruption happens in a different process.
In this respect, effectively all the compiler should be treated sort of like an unsafe region because it requires extra care to avoid memory corruption bugs.
The article is a bit confusing because they also write:
> Regardless of which process had the bug—the compiler or compiled program—in both cases the processor only did the bad thing because the compiler told it to. And in both cases the fix is the same: the compiler's code must change, since that code was what caused the memory corruption.
But yeah, I wonder what those 1,200 unsafe uses actually did?
I think if you interpret it charitably it means that any bug in the emitted machine code is already a likely memory-unsafe miscompilation if it is ran.
The compiler itself might be perfectly "memory safe" but the generated binary fundamentally is always at risk (besides WebAssembly I suppose).
I'm fully aware of the separation of compiler and binary, and being able to compile untrusted code safely is nice, but a perfectly safe compiler that generates vulnerable binaries isn't that much better.
In context that's clearly not what he's saying, the next sentence is this:
> Zig has more features than Rust for making memory-unsafe code work correctly, and that was the area where we wanted the most help.
Zig definitely does not have more features for successfully emitting memory-unsafe machine code than Rust does. I can emit memory-unsafe machine code from typescript if I really want to and nothing at all in the language will get in my way. So the sentence quoted above must refer to the idea that the compiler itself needs to be unsafe, which Steve is right is simply untrue.
I do think that is a good point, it's just not what the line actually says. But that's why I wasn't saying "zomg this is WRONG!!!!" but instead, trying to point out that there are subtleties here. For people who aren't as deep in the weeds in this subject, I think the details matter. But again, as I said, I like the post, this is just one thing.
I am also probably in a more pedantic mindset because, well, I'm writing a compiler in Rust, and the words as written do not resonate with me at all.
> a perfectly safe compiler that generates vulnerable binaries isn't that much better.
I do think it's much better. Eliminating classes of bugs in one component is a good thing, even if it's not every component. This is a core lesson of Rust! unsafe still exists, but going from "I don't know what is unsafe" to "only this part is unsafe" is a major improvement.
>ReleaseSafe catches use-after-free errors through runtime checks which panic if the program tries to use freed memory.
I don't know Zig so maybe they know something I don't, but I have seen no evidence that it catches any type of use-after-free including double-free?
While writing a blog post (below) I went through the documentation to figure out the possible runtime memory safety checks Zig can insert. The term "use-after-free" or "UaF" never occurs on that documentation page. Searching for "safety-checked" doesn't yield any related hits either.
Unless maybe they're using the DebugAllocator in release builds? Even that does not reliably surface UaF.
I as someone with writing Zig a bunch, can safely say if it does it hasn't even worked for me.
I am talking from experience from a pre-ai human mitts writing code perspective maybe Zig + LLMs do some magic.
The more I read the article the more I feel like this is just bad not sure if I should be giving it as much latitude as I have been in my prior comments.
There are other claims as well that are weirdly phrased at least.
Reads like an article written to justify some arguments they had rather than a genuine take at this point.
But I will give the benefit of doubt I enjoy weird articles, languages and share a dislike for aggressive AI-ness of all things.
The DebugAllocator catches use-after-free (at least on page-level), but at the cost of never recycling memory addresses (e.g. it eats through the virtual address space).
For higher level code, "generation-counted index handles" might be the better solution to provide temporal runtime memory safety, not part of Zig the stdlib though.
Or even better: never use dynamic memory allocation and make all lifetimes 'static' :)
as a reminder its a construct in the zig stdlib to take an arbitrary chunk of memory (could be stack, could be in the programs static space) and wrap it in an allocator interface and give that to any data structure that needs an allocator and use it as if it were just malloc/free, with a fixed memory limit and the correct memory errors.
I don't know enough about Zig to explain it, but there is more to ReleaseSafe than checks and panics. ReleaseSafe also clears memory that no longer has an owner (I might be describing that wrong, that is just how I understand it). I found this out with a rendering issue recently.
The bug was around passing a slice to OpenGL which referenced memory outside of its lifetime. Since the memory location had no owner, vertices would still exist in Dev builds and everything would work fine, but in ReleaseSafe the application would run and just have nothing to render.
Since OpenGL was trying to read the memory, there was no panic from Zig, but it was a cool look into how the different build modes handle memory.
> The Zig standard library also has a general-purpose debug allocator. This is a safe allocator that can prevent double-free, use-after-free and can detect leaks.
>Never reuses memory addresses, making it easier for Zig to detect branch on undefined values in case of dangling pointers. This relies on the backing allocator to also not reuse addresses.
But it's not really clear what this means. "branch on undefined values" would I think indicate that maybe they're doing a fill pattern that the compiler can detect at runtime when dereferenced? But I don't see it in the `free` path. It's not clear if this is deterministic or not either.
I'm not sure what "branch on undefined values" means there, yeah, but never reusing memory addresses is enough to prevent use-after-free.
Or, rather, you can use a value after freeing it, but it will not be exploitable, because it will contain valid data of the right type. This is the same idea as Type-After-Type,
Interesting that OCaml was flexible and expressive enough to be used as a prototype testbed but not chosen as the implementation language, especially given the maturity of both. I would be surprised if Zigs incremental builds could be meaningfully faster than dune's.
Cross compilation is great, but not mentioned in the "why Zig" section. Is memory control that crucial for a compiler?
Rust itself was originally written in OCaml, same with WASM. I'm curious about what milestone gets reached where the maintainers collectively decide to transition away.
Rust moved away from OCaml when it decided to be re-written in Rust. The post alludes to this as being a usual time for a wholesale re-write, and I'd agree.
I appreciate the insight, and on closer reading the post clearly states that realistically only Zig and Rust were ever considered anyway.
Since you're here, could you comment on the approach Rust took in their rewrite? Was it more of a straight translation like Go did when they self hosted -- similar to the recent Bun transliteration? Or were there architectural changes made along the way like this article describes with Roc?
The Rust re-write happened before I got involved. If pcwalton is around and sees this comment, maybe he can provide a more first-class account.
> Was it more of a straight translation like Go did when they self hosted -- similar to the recent Bun transliteration? Or were there architectural changes made along the way like this article describes with Roc?
From what I remember, it was a whole-sale re-write from scratch, not a transliteration. While Rust took a lot of inspiration from OCaml, especially in those days, it was different enough that I'm not sure that a more direct transliteration would have been particularly possible, though again, see above, I wasn't there, so I don't know for sure.
OCaml compiler is incredibly fast. I wonder how it'd fare with Jane Street's extensions for the borrow checker etc in OxCaml, if it's good enough for their HFT I'm sure it's good enough for a new language.
I suspect this "not a systems language" alludes only to OCaml's rather steeper learning curve and until-recently difficulty with multiple threads. I am sure it could roll just fine as a single-threaded compiler language written by a small team, which indeed, it was.
I wrote a toy Scheme implementation in OCaml by using the Camplp4 preprocessor. In benchmarks, it was faster than Gambit Scheme, which compiles through C.
OCaml has often historically been considered a language that's been appropriate to write systems tooling like compilers, runtimes, and unikernels in, even though GC'd languages were/are not often considered for such projects.
I don’t think there’s too many of us on the ‘GC did nothing wrong’ hill.
Reading the average HN opinion, it seems everybody is writing high-performance latency-sensitive systems that would implode if a response would take 1 ms longer than normal.
Sampling bias. Most of the people responding are probably those with a strong opinion because of what they work on. Everyone else is likely relatively indifferent to it.
It is a misconception that GCs only affect latency-sensitive systems. High-performance throughput-optimized systems are also sensitive at ~1µs granularity for different reasons, so GCs are not used there either.
That a GC is adverse to the performance both latency-oriented and throughput-oriented workloads doesn't leave many use cases in "high-performance" systems. Maybe systems that are severely I/O bound but is barely a thing these days.
> Maybe systems that are severely I/O bound but is barely a thing these days.
Any kind of web service is barely a thing today? Which is what 99% of HN posters are working on, hence my comment.
> High-performance throughput-optimized systems are also sensitive at ~1µs granularity for different reasons, so GCs are not used there either
Games are high-performance throughput-optimized systems that have adopted GC languages for 15+ years now, and again a type of application which is much more latency sensitive than most people deal in their day to day.
Nobody is claiming GC is a panacea, but it’s good enough for a lot more use cases people give it credit for.
If you are severely I/O bound it isn't intrinsic, it means your server is badly under-provisioned in the I/O department. Linux on a modern server can push 200 GB/s of I/O. Even if web services were engineered to a standard that could consume that much I/O, which they are not, you would have to be astonishingly wasteful to burn it all.
It is rare to be severely I/O bound because software engineered for I/O performance tends to run out of memory bandwidth first.
Games are not throughput-optimized systems in any conventional sense. They are a canonical example of latency-optimized systems.
I have nothing against GCs, I use them regularly even in performance-sensitive contexts. But too many people understate the adverse impact of GCs on performance contrary to evidence and theory.
I/O bound means waiting on I/O, which isn’t necessarily because it is slow, but because it is simply waiting on data to arrive, like a web service most of its idle time. If your client is dozens of milliseconds away, a GC pause is pretty much invisible, unless you are trying to squeeze every last request/second from a machine (instead of simply scaling horizontally)
That said, from your profile, you seem to work on a very sensitive niche that might colour your opinion, with good reason. What I am claiming is most of us are not building such strict a system.
Even in my toy hobby of OS development a GC isn’t the end of the world unless your goal is to compete with, say, Linux in a some kind of performance challenge, where in that case memory allocation might be the least of your bottlenecks.
It is also a misconception that all GC are born alike, and that don't exit languages with support for value types, stack allocation and GC free memory regions with C like pointer fun if so desired, while being mostly GC enabled.
Could you elaborate on "GCs are not used there [high-performance throughput-optimized systems]"? Are you referring to the cascading effects of tail latency on systems with high fanout?
Sophisticated throughput-optimized systems rely on deep latency-hiding. Schedulers see millions of atomic operations into the future, continuously rewriting the schedule globally to maximize locality and minimize resource contention based on real-time changes to workload, resource availability, and system behaviors.
In short, for each of the millions of in-flight operations (which might only map to a handful of user operations), it is trying to precisely optimize the concurrency, timing, and dependency sequencing such that when operations are executed every resource required is hot, uncontended, and available with high probability. When this works well it dramatically reduces the number of hidden stalls in execution. The schedule is constrained by tail latency requirements; a theoretically throughput-optimal schedule can defer execution indefinitely.
For an analytical database engine, an "atomic operation" is typically a query operation on a database page. A modern server can retire 100M ops/sec. While I am oversimplifying a bit, a 1 millisecond GC pause can blindly wreck the schedule for 100,000 operations in an unpredictable way. In these architectures we try to eliminate all context switches for the same reason which are 100x cheaper.
Practically, 1µs stall is a good heuristic for a noise floor. The schedulers have pretty wide concurrency on big systems, so the implied 100 operations are unlikely to have a dependency. Many stalls that are difficult to precisely control like cache line fills fit in here too.
If there was a GC that had a worst-case stall of 1µs then you could probably use it for these cases. Unfortunately, "low-latency" GCs tend to be more like 1000x that. I don't think there is any way of closing that gap short of putting a GC in hardware.
Real time GC as used by the military in weapons control systems, and on factory automation robots, keeping PTC and Aicas in business, people pay to use them.
TBH, physics limits how latency-sensitive weapons systems need to be and you can largely just disable the GC in these contexts. They use CPUs from the 1990s to do hypersonic terminal guidance. You don’t have to do any performance engineering for many latency-sensitive weapon systems. Could probably write it in Javascript.
For throughput-optimized systems, some of which are real-time, you never see a GC. That loss in performance is simply too large such that the computation becomes intractable. A lot of really poor systems admittedly exist but no one considers them “good”.
> Most of the people responding are probably those with a strong opinion because of what they work on.
I can assure you that's not the case on here. The people working in truly low latency environments are not commenting on GC threads to begin with because it's a non-starter for them. For whatever reason, there is just a chunk of people that eat a lot of FUD around GCs who are working in the exact domains they thrive in.
Zig's incremental builds are DEFINITELY a killer feature. In the short term, I could see why you'd make a switch to get it. But, in the medium term, can we really not expect to see this in Rust in the somewhat near future?
I want to go fast, but I don't want to go fast just to shoot my foot off.
If only somehow we could get Rust's safety with all of Zig's features and Go's runtime without GC...
This is cool and will likely enable some cool tooling.
I don't think a borrow checker is likely to be in that tooling. Borrow checking requires shaping the code, and all the dependencies, into easily analyzable (and at least in rust's version annotated) patterns. You can't borrow check arbitrary code not designed for it without false positives.
That's not sufficient - consider the following pseudocode
x = malloc();
if (opaque_cond()) free(x);
if (other_opaque_cond()) use(x);
Conditions can be opaque and non-analyzable due to rices theorem - in any turing complete language. This code is correct (or at least not memory unsound) if opaque_cond and other_opaque_cond are never both true. Otherwise it isn't.
And functionally compiler analyses of whether conditions hold have to be trivial because using some form of theorem prover to decide of code is correct or not leads to code that is brittle against compiler version changes, and slow compile times. Thus opaque_cond could be as simple as `len == 0` and `other_opaque_cond` could be `len > 0` and it's unlikely you'd want the compiler to realize those are mutually exclusive (at the stage where it accepts programs, obviously during optimization it is very likely to take advantage of this).
Rust solves this by simply rejecting the pattern. Very roughly forcing you to write if opaque_cond() { free(x) } else if other_opaque_cond() { use_x } (or something else where the program structure and not just the logic in the conditions guarantees correctness). Zig simply allows it and leaves it up to the programmer not to make a mistake.
And as onlyrealcuzzo suggests aliases are where this type of analysis (accepting enough programs to be useful but still imposing enough structure you can prove correctness) is really tricky.
It's possible to detect and ban the version of the problematic pattern that I made as short and simple as possible to illustrate the point, sure.
I'm general though, I don't believe it is practical to do so. Not without every library being designed with the checker in mind and annotated to more precisely describe their APIs. Which is why I'm not surprised to see the limitations.md that seems to exclude all the hard cases (aliasing, pointers used as first class values, cross function analysis): https://github.com/ityonemo/clr/blob/main/LIMITATIONS.md#mem...
Obviously if you rewrite the zig world to obey rust like rules and include rust like annotations you can implement a rust like borrow checker, but I don't think that it would still be meaningfully zig. It might be an interesting language worth exploring.
i have in mind a strategy to address all of them. this is a side project, a proof of concept, i have other things going on in my life. i dont chip away at it every week.
you make, without any evidence ("Obviously"), an assertion that "it would look like another language". so far if anything applying zig clr would push a user to write more idiomatically ziggy code, away from idiomatically c-ish code. i dont see why continuing with clr wouldn't go further along that trend. so consider what is "obvious" to you might just be flat out wrong.
> It might be an interesting language worth exploring.
worth how much? youre welcome to sponsor my exploration and put your money where your mouth is:
I wish them the best of luck, but I don't expect them to succeed, and unspecified plans don't constitute a demonstration of feasibility. Plenty of people have made plans to solve unsolvable problems in the past, me included. I strongly suspect that that is the case here (or that they're willing to iterate away from zig).
The evidence is the amount that rust had to iterate on the underlying language to make the borrow checker work well. Something it had the freedom to do since it was co-designing the language and linter.
Edit: Didn't realize this was your project (responded before you added the donation link) - I would have worded my response slightly differently but my opinion is unchanged. Seriously mean it with the best of luck getting this to work.
i dont understand the downvotes here. the point of any safety checker is to flag and ban potentially unsafe code, and force the author to rewrite with existing language patterns that guarantee the desired safety parameters.
in this case, zig has a first class nullable syntax that the checker can use ti guarantee correctness for, so a checker can deterministically sidestep this turing completeness issue, by squeezing indeterminate code into the knowably safer language idiom.
1. rust didnt invent affine ownership. 2. It's possible to bolt on to other languages (see ada). zig in particular is easy (disclaimer: i think, i haven't implemented it yet)
intermediate representation. attempting to analyze zig code directly would be too hard (especially with comptime). on the way to the compiler backend, the compiler builds a simplified representation that only has "actually existing functions" and is very straightforward, e.g.
function 10112:
0: argument 0
1: argument 1
2: argument 2
3: add 0, 1
4: store 2
5: call function 1342, (2, 4)
6: return 5
you can see how building a data dependency graph from this would be easy.
Are compile times that big of a deal? I haven't used Rust a ton, but the few times I have it seemed like the bulk of the compile time was a one off compiling the crates, and then compiling your own code was super fast.
I feel like I'd massively prefer to end up with a binary free of memory exploits than shaving some time off compile.
Instead of waiting for faster compiler in Rust, how about from the other direction, adding some kind of borrow checker to Zig? That sounds more within reach and practically achievable, possibly even in userland.
It's impossible to add a borrow checker to any existing language.
The reason Rust has a working borrow checker is because every part of the language from structs, enum, traits, generics and all the way to the syntax itself has been designed to support lifetimes and borrow checking.
It's is not something you can just tack on to an existing language without fundamentally changing it.
> We focus on three mode axes: affinity, uniqueness and locality. Modes are fully backwards compatible with existing OCaml code and can be completely inferred. Our work makes manual memory management in OCaml safe and convenient and charts a path towards bringing the benefits of Rust to OCaml.
I wouldn't say it's impossible, rather un-ergonomic. TypeScript can add type information to ordinary JavaScript code via JSDoc comments; the result can both be executed as ordinary JavaScript as-is and type-checked with TypeScript. But it's a huge pain to try to write (and maintain) everything that way, it was supported as a hack to help migrate legacy codebases. You could probably take a similar "the lifetimes are embedded in comments" approach with other languages, and the result would be similarly un-ergonomic.
You may have missed the point here. You could add a comment to the struct field that marks the field as private, and build a TypeScript/JSDoc analogue that analyzes all accesses to the field and fails if it finds accesses from functions that aren't part of the struct that owns the field. You don't even need a comment on the field - you could copy Go's convention, add a comment to the struct definition marking it as "follows Go convention", and then fail any access from outside the struct to a field that starts with a lower-case character.
It doesn't prevent you from ignoring that tool and writing Zig code that imports the struct and accesses the field. It is, of course, not part of the Zig language itself. But if you adopted a tool like that, it would be your responsibility to run it across-the-board and pay attention to the results - same as how it is your responsibility to pay attention to the results if you added those JSDoc comments.
I have picked private fields as an example of feature that is needed because it is very simple. You're right that you can build an analyzer (with additional code annotations) to support that, but it's only one example.
Take another example: unsafe traits. They are fundamental to some safety encapsulations, most famously concurrency (`Send`/`Sync`). Here you cannot just build an analyzer to mark something unsafe, because Zig has no traits, its generics are duck-typed.
You can, of course, add traits. But at this point you're essentially creating your own language that compiles to Zig, with all problems this entails (e.g. bad ecosystem support). It's also hard to claim that Zig can be memory safe then.
> You can, of course, add traits. But at this point you're essentially creating your own language that compiles to Zig
I think herein lies the rub. What's the difference between a static analysis tool and an actual separate language that transpiles to the original? Hypothetically - again, very un-ergonomically - you could add traits to Zig code in comments, or in example-traits.typezig files that would be skipped by the Zig compiler (like how *.d.ts files are skipped). How much of a language is writing code in a particular syntax, versus how much of a language is writing code that will pass a tool "building" it, versus how much of a language is about the final compiled output that you get from the tool? All static analysis tools that support line-level exceptions are, essentially, programmed by comments, with their own language (typically highly simplified compared to a "full" programming language), that affect whether or not the "language" passes or not. What Typescript/JSDoc shows is that, actually, much more complicated tooling can be built with this programming-by-comments model than had been done before (to my knowledge), and thus even more powerful still tooling could be built with that model.
Of course there's a difference between static analysis and a language that transpiles. But perhaps it's more a question of degree than a simple binary classification.
no. You don't need private fields. All you have to do is analyze the code, harness the compiler to generate a time-dependent data dependency graph, and map allocation/frees/uses, if you can 'color' branches where data are shared you can also track and check to see there isn't an aliasing violation too.
it is easy to patch the zig compiler to enable this this (export the code graph; about 50 LOC). The analysis is much much harder to get right.
it is possible to do in-code annotations in zig, if you're clever. you can get pretty far without them too.
as an example, you can check for double free without ownership tagging, by being agnostic about who should free, and flagging if two nondisjoint code paths attempt to free the same allocation.
It is only feasible to do this if the whole of the codebase idea designed to allow it, and it's still going to blow up in odd ways of you don't have a way to describe lifetimes in your interfaces. The magic of rust's design is that it turns this memory tracking into a local problem, such that you can design an interface and be sure that every use case is safe and verifiably so.
empirically untrue. several projects exist that bolt on extra safety to unsafe languages or unsafe parts of language. SeL4 for C, MIRI for rust unsafe. i guess ada/spark for ada too, is the OG, spark being added to ada 4 years after its first release
Hardening is definitely possible, we've had sanitizers in C/C++ for a long time. It's not full memory safety though. Miri is the same.
SeL4C is formal verification, and while it can prove memory safety (and much more) it is much more difficult, to the point that you're basically programming in a different language.
Ada/SPARK is your best example, and also the example I know the least of, so I won't comment on.
SPARK omits some features of Ada, so it would only reinforce the sentiment that bolting on verifiability after-the-fact is difficult. Expressivity is generally the antithesis of static analysis, and it's very easy and tempting to make a language that is accidentally too expressive to support a given analysis without being required to make breaking changes to reduce expressivity.
A language is more expressive when it allows more programs and less expressive when it allows fewer programs. I don't know zig-clr, but if it rejects programs that Zig accepts (for example, by rejecting the aforementioned ambiguous pointers), then it is less expressive, not more (keeping in mind that being less expressive is not a pejorative).
no thats not the definition of more expressive. more expressive means the language can encode more programmer intent without making a dog's breakfast of the code.
Now we're just having a semantic argument over the word "expressiveness", which is not especially interesting; see https://chrispenner.ca/posts/expressiveness-spectrum . My argument above remains true regardless of the terms you choose to use.
> It's impossible to add a borrow checker to any existing language.
Why do you say that. Have you tried and failed? It seems to be possible to add a borrow checker to zig, just as you can add MIRI to rust to get extra safety in unsafe blocks.
> how about from the other direction, adding some kind of borrow checker to Zig? That sounds more within reach and practically achievable, possibly even in userland.
It's doable, and as static analysis. see sibling comment.
the architecture doesn't make sense. MIRI doesn't perform static analysis on MIR. It is, as the name says, an interpreter. The borrow checker is entirely different from miri.
Rust's borrow checker requires lifetime annotations. Zig code doesn't contain any such annotations. How does your design handle this?
1. it is possible to do code annotations in zig even though i havent implemented it in this iteration of clr (the first poc demonstrated this). i want to see how far i can get without them.
2. let's take double free (easiest to explain).
you dont have to tag ownership, you can be agnostic about who should free, and merely report if two nondisjoint code paths attempt to free the same memory.
I'm not sure it would ever make sense to be. That makes the assumption tons of allocations get made that don't live long, which was(maybe is still?) more common in some languages. Go is more aggressive about not heap allocating, and has tools to help you avoid them.
You seem to be really hostile for no apparent reason. There are plenty of reasons to be generational, there are lots of workloads that the current implementation might fall flat if it wasn't.
> n practice, Go can typically outperform Rust in throughput (using more memory), despite having a mountain of disadvantages against it in theory
This is a huge claim that disagrees with both my real-world experience and everything I've seen from artificial comparisons.
Every high performance Go system I've worked on has quickly reached the point where we're optimizing memory management and doing things that would have been explicit in a non-GC language like Rust anyway.
The Go runtime is amazingly optimized, but it comes with overhead over doing the same work directly in a lower level language.
Go has few issues with performance (lack of in-line union types, interface overuse, inefficient idioms reg. collections, some missed optimizations) but its seems plausible for a idiomatic Go program to outperform an idiomatic rust program in some situations.
> It's literally the most sophisticated scheduling engine in the world.
That seems unlikely regardless of how good it is. This is a domain where state-of-the-art research is not in the public literature. Scheduling is an AI-complete problem.
I think this is interesting and warrants explanation. There are cases where a GC can be faster (sort of, Arenas get you most of the gains) but "the most sophisticated scheduling engine in the world" should be easy to at least partially support.
Erlang's scheduler is not sophisticated, which is what makes it AWESOME.
but yeah. i would be surprised if the JVM's scheduler is not more sophisticated than go's if for no other reason than it has way more knobs you can tune. you know they put that knob in there because someone (probably Google cough cough) asked for it
I am not sure, but there might be a bug in their pattern matching example.
What happens if 'verb' is "GET" and 'path' is "/users/1234/posts/1234/extra_path/and/more/"? Will 'post_id' become "extra_path/and/more/"?
I tried running it in the sandbox, and it does indeed seem to buggily result in:
"Post ID: 1234/extra_path/and/more"
I suspect that the reason it is behaving like it is, is due to how it handles characters in the string literal. The example program exploits that only the slashes present in the string literal pattern are matched, to enable matching on 'page' having slashes. But then in the nested 'match', it forgot to account for any possible extra slashes.
Nitpicking end.
I have not read the whole post yet, but the pattern matching not requiring any allocations, seems very nice. The string literal patterns also seem interesting, though I am not completely sold on them, also as per the above possible bug. It seems really clean in some ways, but the specific semantics, I am not fully sure about. Maybe it is excellent, and is so clean and concise that it is overall less bug-prone than alternatives in other programming languages. I do not know.
Tangentially relates, but if any Roc devs are around I'm curious about the use cases for Roc.
It's supposed to be a scripting language right you embed into your C ABI right?
Do you see it competing with WASM for the plugin use case (i.e. a really large Roc platform)? Why would an app author prefer to expose a Roc layer to their app rather than a WASM layer? With a WASM layer, plugin devs can write in any language.
Another use case I've heard from it is as a more app-level language (i.e. a really small Roc platform). Do you see it competing with Gleam for server side http code? Do you see it competing with Elm for client side code?
Same question. I always like learning about languages I had not heard of, especially functional languages, so I was immediately curious what sorts of applications this might have. But after looking over the roc-lang.org website and the FAQ, I still don't know.
This piece would have been a lot more compelling if they had actually done science on selecting a language for compiler development. From what I can tell, they had an untested hypothesis that a low level systems language is necessary for a high performance compiler https://www.roc-lang.org/faq#self-hosted-compiler and from that concluded that their only
choice besides rust was zig.
I know from experience that this initial assumption is wrong. Compiler performance is dominated by algorithms. The fastes managed languages tend to be at worst within a factor of two for wall time on any given algorithm. Algorithmic differences can be unbounded in their performance gaps. Zig itself is a perfect counterexample to the theory that writing a compiler in a low level systems language will lead to a fast compiler. Roc seems to compile at around 15k lines per second. That is not fast. There were evidently compilers written in ml that did 3k likes per second in 1998 https://flint.cs.yale.edu/cs421/case-for-ml.html
The zig rewrite of roc looks like the author's second compiler. Compiler and language design is a skill like any other and from my vantage point, they appear to have overcommitted to an initial design at the expense of developing their higher level design skills. In my opinion, the best thing they could do for the future of roc is stop working on their current compiler and use it to write a self hosting compiler for a much smaller subset of roc. They should be able to do that in less than 10k lines of code. They might even find that their self hosting compiler is faster than their zig based bootstrap compiler for the self hosted subset of roc. If the self hosting compiler is inadequate. Now they at least have identified a smaller useful subset of roc and can experiment with different compiler implementations in 10k likes of code rather than 300k lines of code. Then they could actually test the theory of whether or not a low level language is necessary to meet whatever arbitrary compiler performance goals they have.
By self hosting, they would also discover what roc features actually matter and they would spend much more time actually writing roc code. The features that are needed to write a self hosted compiler are all features that are generally useful. By improving the self hosted compiler, they also improve downstream programs.
Your comment is very assertive, but also doesn't offer much in the way of science.
Being able to compile ML quickly in the 90s tells you little about being able to compile Roc or some other language today because the language design enforces hard constraints on the algorithms necessary to compile it and the hardware today is much more complex. It's not hard to write a fast Pascal compiler that targets a 1980s chip with shallow pipelines. But that's not the problem being solved here.
I don't know much about Roc but it looks like it's got some amount of overloading and the linked article alludes to sophisticated algorithms to avoid heap allocating closures. Those can enforce algorithmic complexity in the compiler that is essential and can't be eliminated.
Once you're at the limits of algorithmic optimization, all that's left is reducing constant factors. I've written code in many languages in different performance regimes over the years and it's certainly the case that higher level languages, especially managed memory ones, put a hard floor in terms of how low you can go when optimizing to improve those constant factors.
I have seen in real-world code where explicit control over memory layout improved performance by more than an order of magnitude. I have friends in the game industry where much of their career is this kind of work. Those people would love to live in the luxurious world you describe where all they need to do is find a sufficiently clever algorithm and all of their performance problems will disappear.
Zig itself is an incredibly fast compiler. And the language and standard library is designed to support writing that compilers. And yeah, that’s all about algorithm and data structures. A large part of why struct-of-arrays is easy to do in Zig is because Andrew wanted it for making the compiler fast. The article also points out that it’s reusing code from the Zig compiler source as well.
Roc may not be super fast now, but that doesn’t mean they haven’t seen ahead and set themselves up for success.
Yeah, I agree there is value in self-hosting as you say. Zig itself is an example of that. But zig is a systems language. If Roc isn’t aiming for that nice it might not be the best choice for writing a compiler in. As an example in the other extreme end of language flavours: Python is still mainly CPython and efforts to make a python interpreter in some variant of python hasn’t been particularly successful.
But you can always use the best algorithm no matter what your implementation language is, so it still makes sense to prefer a language that makes it easy to write fast code.
One thing I wish Rust would improve over time is the builds. Its one of the biggest sources of wasted storage space on all my computers, builds a ton of libraries can take tens of gigs, it adds up very quickly. Not sure what the best solution is, one I found is to set the global build folder so dependencies get reused across projects, but imho it should be an OOTB default behavior whatever the real solution should be.
We are trading away disk space for faster builds. We could make them faster in some cases by using even more...
On the other hand, it would be good to garbage collect those caches. We are wrapping up work on a new layout for intermediate build artifacts that will make it easier to GC them.
Sounds like what I was thinking, that or for third party deps to go into the same temp folder, and anything not accessed in over a week, or so just gets auto wiped by rustc / cargo build process?
Rust isn't great, and it shouldn't be a surprised since it's designed after npm. However one metric where nodes_modules is still worse for me is the sheer number of small files in it.
Having nearly one million files in nodes_modules isn't that unusual. The problem is that on most common file systems the minimum allocation is usually at least 4KB. So even if the actual data is less than 500MB, you end up with 4GB disk space used/wasted.
I wish ext4 had a feature to mark a file as "atomic" where it would allocate all atomic files in a long run, without room for expansion, and I suppose with very inefficient compaction upon deletion, but without any padding bytes.
Zig is a pre-1.0 language while Rust is post-1.0. This alone is settles which one to pick for may developers. The library support is probably favours Rust too. Rust build times are much slower than Zig, I get that, but I rarely optimize software for build times.
That’s totally reasonable. But while each minor version has breaking change, a minor version release itself is generally fairly stable and useable. There are solid production software which is already using Zig, like TigerBeetle and Ghostty/libghostty.
I am waiting for (near) 1.0 myself before doing anything major with it. But that’s mainly because I want at least the async IO stuff to be settled for my use cases.
Zig is not pre-1.0 because it’s not ready for production (bugs or missing features), it’s pre-1.0 because they want to be able to make breaking language changes.
Nowadays when you can just point an agent at release notes and have it update everything, I actually prefer not having to wait through rare major releases to get new language features.
> Nowadays when you can just point an agent at release notes and have it update everything
Except that means that not only you lose compiler bugfixes, you also pretty much has no access to the ecosystem. For most production codebases, this is a deal breaker.
I invite you to read the release notes and see for yourself the types of breaking changes we’re talking about.
To me it is not much different from Lua, which despite being on 5.x for decades, makes breaking changes on minor releases (because it predates SemVer).
I also don’t see it being much different from any other language or language runtime that has a major release every year.
If by production-ready you mean you can forever avoid changing code you wrote 8 months ago, sure, pick something else.
To me production-ready means it can be trusted to power production workloads, has all tooling I need, and has a consistent long-term vision. Zig ticks all the boxes.
> Zig is not pre-1.0 because it’s not ready for production (bugs or missing features), it’s pre-1.0 because they want to be able to make breaking language changes.
This is a solved problem in other projects. Either use the version numbers as intended and bump the major version number on breaking changes, or use Rust-style editions to opt in to the newer versions of the changes.
Calling a project production-ready but keeping the version number below 1.0 and saying breaking changes are expected is a tired game. We've seen it backfire across a number of language projects like Elm, where the exact same claim was used to both encourage people to use it and then blame them when it backfired.
If it's production ready, go to 1.0 and then follow semver for breaking changes. I don't care if we get to Zig v73.2.0 as a result. At least we can see from a glance which versions need to be checked for breaking changes.
languages ideally should not have breaking changes ever.
on the other hand, a language with frequent breaking changes should not be considered production ready.
people are of course free to live on the edge, and if someone decided that zig is good enough and they are not bothered by breaking changes then they are free to use it for their production system, but that doesn't mean it's ready for everyone. so i prefer the zig approach.
> languages ideally should not have breaking changes ever.
I disagree MIGHTILY. This is how you wind up with C++ and Java.
Languages need to be able to remove features to stay coherent. Occasionally, you get things wrong, it takes time to figure that out, and that's just the way life is.
different preferences i guess. i much prefer language stability. the idea that i should have to test my code with every python version out there for example is disturbing, but there are tools to do exactly that. they should not be needed.
The solution to this is editions/epochs, like Rust. You can set your project to the 2025 edition and be sure that, even if future versions introduce breaking changes, they will not affect your project; it will continue to compile as before.
pike has that. or had it. for two decades. inline in the code, in each file you could specify the version. they decided to drop it because of the maintenance overhead.
Counter Argument - with Zig it's perfectly viable to make programs using just the standard library. You have json serde, zon (zig object notation) serde, system functions (ie libc/rustix equivalent), random number generators... all without a single dependency.
Zig team member here---obviously I can't say for sure yet, but I'm pretty confident the number will be basically identical. In the Zig compiler, incremental updates (rebuilds) have a small amount of overhead which is roughly proportional to the total size of the codebase (rather than just the amount of code which was changed). This comes from a) detecting which source files changed, and b) traversing a graph to figure out which declarations are referenced (necessary due to Zig's "lazy analysis" feature). But performance analysis reveals that for small updates, this overhead actually dominates the update time, by a lot. Of the 35ms, I would guess that under 5ms are actually spent rebuilding the function(s) that changed. Of that 5ms, code generation---the only thing which would really be different on AArch64---is an even smaller slice of the pie (it often doesn't even impact the overall time, since it runs in parallel with other parts of the pipeline, and those other parts are usually the bottleneck there). So even if the AArch64 backend was significantly slower (which, right now, is the opposite of what we expect---instruction selection and encoding for x86_64 is unusually complicated!), I wouldn't expect the number to change from 35ms.
The compiler is one of the most significant trust boundaries we have. Its decisions can intentionally or unintentionally create vulnerabilities in programs compiled by the compiler, which means that if you can compromise a compiler you can compromise everything downstream.
Unsafe memory access in a compiler can be exploited in order to hijack the compiler itself (this is reported regularly in production compilers), allowing the attacker to then insert arbitrary code into compiled binaries. Not everything that a compiler absorbs from its environment is meant to be treated as source to be compiled, and in a memory unsafe compiler any of that input can silently turn into machine code in the compiled binary if an attacker is able to exploit the memory safety bug and hijack the compiler.
Most compilers do not consider themselves a security boundary, and will not try to limit malicious code from hijacking the compiler. E.g. LLVM is explicitly not designed to handled malicious code (https://llvm.org/docs/Security.html#what-is-considered-a-sec...).
For this to be useful you would need to modify the compiler binary to make the exploit persistent. Otherwise why put an exploit for the compiler in the source to so that the compiler can put some malware in a binary, when you can put the malware in the source directly?
1. There are lots of things that compilers load into their memory that aren't actually source code. A memory exploit turns non-source data into executing-in-the-compiler code.
2. Depending on the language semantics, a memory exploit can allow substantially higher privilege than just being loaded as library code. Latent malicious code that never gets called into never becomes active, but if you can exploit a weakness in the compiler you can make your code execute at any time you'd like instead of relying on the main application calling in to your malicious library.
1. What would this be?
2. Good point, but for most purposes I think exploiting some aspect of the build system or the language that moves the exploit code or causes it to be executed before main would be easier than exploiting a memory issue in the compiler (I agree though in principle).
Considering that you often run the code after you compile it, it might not matter. Anyway, like it or not, most compilers don't consider themselves security sensitive and will not consider malicious code that is able to hijack the compiler a security vulnerability.
Totally unrelated; The trusting trust attack was about downloading a malicious compiler because malicious code was injected into it in some early version.
(And if you ask me, it was indeed overrated, but that's unrelated).
Does this mean every time you find yourself using lots of “unsafe” Rust blocks, it’s not the right tool for the job? I suspect it’s not that simple, but what are people’s experience?
It really depends. For example, it might mean that you do not know the way to do the same thing, but in a safe manner. It might mean that you could refactor your code to do things more safely.
Of course, reasonable people may also believe that it is easier to use an unsafe language directly rather than change the ways that you code.
In my experience doing embedded, operating systems work, compiler work, and others, you never need a large amount of unsafe code. 1%/4% is really about it.
I currently have a problem with Rust that I use Rust-Analyzer for syntax and autocomplete in VS Code and it has to run the compiler when you save a file to "refresh" things.
As you may imagine, this is insanely slow.
So slow that when I switch from a .rs file to a .ts file I feel like I switched computers.
> As discussed earlier, having full control over allocations and deallocations is what I want in our compiler's implementation. And in tests, I also appreciate the testing allocators detecting leaks—it can even detect leaks in compiled Roc code! Unfortunately, to get that benefit requires a lot of "init this, defer deinit" code in tests that has to be correct or else the test fails on a memory leak. None of that is necessary in Rust. I care more about the compiler's implementation being the way I want it than the tests looking nicer, but in a perfect world I could somehow have both.
haha, just for fun... do you want a C++ in your perfect world :D?
> I enjoy Rust, I've taught a course on it, and I happily use it daily for my work at Zed. Despite what Internet comments might have us believe, it's extremely normal for one language to be the best fit for one project, while a different language turns out to be the best fit for a different project. One size does not actually fit all!
Amen. I like both Zig and Rust, and if I praise/criticise one of them, people act like I'm "switching", as if they were two exclusionary religions (though I think some people may indeed view them that way).
The stuff on memory safety written in the article is well worth reading, because in a lot of programmer discourse it's talked of as if it's some binary, that Rust Is Memory Safe, and Zig Is Not Memory safe. That's simply not the case.
I've sort of thought this too. Rust adoption feels a lot like Haskell to me in that it's centered around ideological things that don't really improve the final product and arguably slow development. If your goal is to write a bunch of LLM code there are better languages than rust, and rust development is often regarded as too annoy for anyone not enthusiastic about the ideological improvements.
There are cool ideas in the language but I don't see it enduring over time.
I like Rust and I'm an advocate of Rust, but this really isn't true (at least it hasn't been and I doubt much has significantly changed).
The syntax complexity and the ecosystem haven't been ideal for LLM development. And there have been publications on findings of LLM efficacy with different languages. Rust is most often towards the lower end of efficiency/correctness when benchmarked.
This paper uses models that are over a year old at this point. Many people didn't believe that LLMs were worth using for programming until 6 months after that, and now again this week we've had another huge leap in abilities.
This is beyond the other issues with the methodology of this study. For example, their Rust code was created by asking Deepseek to port their C++ code, not having it try and write Rust itself.
From what I am seeing in big corp, with low code/no code tooling, coupled with agentic orchestration, for many scenarios the actual programming language will become irrelevant.
Sure the programming platform still needs to be programming in something, but everything else on top will migrate to such tools.
This might not come to all corners of programming, but in the domain of orchestrating SaaS products, with MCP tools replacing classical microservices, it is getting there already in 2026.
While I'm a rust enthusiast, I do agree that certain languages lend themselves well to particular domains. So a rewrite from Rust to something better suited is fine by me. In fact, while I do work on a rust project, I would not have and still would not recommend it as the choice for that particular project.
That being said, I had to do some double takes while reading this.
I feel that it's a bit weird to compare a rather well tested 7 (?) year old rust implementation with a brand new not yet released less than a year old Zig implementation. Without that context, this looks like a bad comparison for rust, when it is in fact the complete opposite.
The swiftness of the Zig compilere here is insane, and would would very much shift my recommendation of Rust if it got to similar speeds.
That being said, I do find it funny that currently, the compilation speed is actually worse on Zig than Rust, despite Zig (anonymous commenters at least tbf) claiming the opposite for years.
How did you eventually discover the 35 ms figure for Roc? Did you have to temporarily update the codebase to 0.17?
Nothing negative here. I did play around with implementing a scripting language in this DOD-ish, index-based paradigm and yeah, it is neat.
I was thinking that it might be possible to do resumable computation across the network like this (in the context of frontend frameworks "resuming" UIs), but ultimately I have no use for this so just the experience itself was enough.
One note here is that it does tend to break completely if non-pointer-free data is introduced. It seems like it's either all or nothing.
> In fact, while I do work on a rust project, I would not have and still would not recommend it as the choice for that particular project.
wondering what type of project is that? I think besides some very embedded projects with very little memory where you need C/assembly, rust is good enough for all kind of projects..
I work both on a pretty much bog-standard web (GraphQL) backend and the frontend that uses it. We switched over from Apollo on node to async-graphql on Rust.
The runtime performance is much better, but the compiler time performance is terrible. To be fair, this is mostly the fault of async-graphql, but that doesn't really matter all that much. For example, it's not uncommon for a single character SQL query change to trigger over a minute long incremental rebuild.
The rust compiler is just choking on the number of generics and codegenned functions.
I've personally looked at how to improve this, but short of breaking up the type graph using federation, nothing can help. Not even cranelift makes a noticeable dent.
Additionally, the team started off composed by a bunch of TypeScript/React/Node developers, so mistakes were made along the way.
Honestly, I would have recommended to just use C#.
That's not to say that I don't think Rust can work for web development. We have some (GraphQL-less) services where Rust is a great fit. Just maybe shouldn't have been the default. That or give up graphql ...
For whatever it's worth, I share some of your async-graphql woes, though I haven't investigated things deeply enough to have strong opinions about what to do instead.
> though (more on this later) and we ended up with about 1,200 uses of unsafe (out of our 300K lines of Rust code
Vs
>Rust code has a different source of memory-safety gaps: the unsafe sections that nearly every Rust program has somewhere in its dependencies. Unsafe Rust has all the memory unsafety risk of ReleaseFast Zig code, but none of the runtime checks to catch issues during development
Well if ReleaseFast would work for you then just write that in rust. The unsafe keyword can be used to write a shitshow, or it can be used to write small pieces of functionality, such as ArcUnion, that are safe to use. I don’t agree that every application has to have any unsafe code. If you find yourself needing it, then you go build the abstraction you need in another crate, miri the fuck out of that, and then just consume it in your application.
If you’re fine with the shitshow, then use zig, because that’ll improve rusts stats.
If you think you’re in the magical “I know what I’m doing and I need the extra performance” then you probably don’t know what you’re doing. You’re young. Knock yourself out. I’ve written assembly language hackery for video games you wouldn’t believe. Would I use any of that for a language others are going to use? You’re not that smart. And if you are, someone else on your project isn’t. Just a matter of time.
"Claude Code uses Bun" is the reason that's given. But even though Anthropic tells us that "coding is solved", instead of rewriting Claude Code in something other than JS, they bought a company that made a JS runtime and then did a mass rewrite of that JS runtime.
Bun was a startup. Startups can go out of business, and then you are now scrambling to move your code to something else. They could also be bought by someone who has different priorities regarding future development than you, and that's also a risk.
The simplest solution to these problems, if you have the capital, is to buy them.
Irrespective to the technical merits of both language, moving from a stable language to a pre-1.0 one that just lost his most popular open source project is a wild move.
> that just lost his most popular open source project
As they state in the article, they started the migration a year and a half ago, something that happened a few weeks back would never come into the decision making process.
I don't even know what Zig is but I've seen this topic come up so many times on this site that I'm starting to think the people who are actually doing this are unsure themselves whether it's a good idea or not.
I think this is a fine post. But one comment:
> remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job
I don't really think that this is true, in the way that it's written.
I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machine code isn't the part that requires unsafe. The language's runtime is a more likely site to find unsafe.
> I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machine code isn't the part that requires unsafe. The language's runtime is a more likely site to find unsafe.
Agreed! Emitting machine code is not unsafe, since it's just writing bytes down - it's only once you execute that machine code that there's potentially unsafety. The reason I said "a big part of the job" is that in practice a lot of compilers both emit machine code and execute it - but you're totally right that it's not a requirement that a compiler do both.
In addition to the examples you gave (hot binary patching/code reloading, language runtime, etc.), others would be things like evaluating userspace code at compile time (e.g. const fn in Rust, or in Roc any expression that could be hoisted to the top level), running tests and inspecting their output to decide what to display to the user, etc.
Those are the types of things I had in mind when I wrote that.
I am disappointed you're downvoted, Richard. This is a fine reply, and I hope you know that a minor quibble with a single line in the post doesn't mean that I think it's a bad one overall. (EDIT: a few minutes later, the parent comment is no longer grey.)
I also think it's a good thing that you wrote the post in general, when I saw it pop up I was like "oh, of course, this post should exist!" I'm surprised I didn't think about it earlier.
> evaluating userspace code at compile time
Usually this would be done via an interpreter, so I'm not sure that it really requires unsafe either. If you are literally executing machine code, sure, but const fn in Rust and constexpr in C++ and many other languages do not do that, as it causes a number of problems (for example, cross-compilation).
Also a good point! TIL that Rust and C++ use interpreters for const, although of course that wouldn't work for running tests. Then again, in the specific case of Rust I believe rustc only compiles the tests and then something else like Cargo executes them. Of course, as I noted elsewhere, if rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupted (due to miscompilation) as if rustc and cargo shared a code base.
By the way, I thought your question was totally reasonable - my first thought reading it was "Oh yeah I wasn't trying to say that writing bytes is unsafe, I definitely should have worded that differently."
Cool, I'm not sure that people know that we know each other and have some deeper mutual understanding. :)
> although of course that wouldn't work for running tests.
Why not? Unless you mean in the cross-compilation case, in which yeah, to run the compiled tests you'd need an emulator.
> in the specific case of Rust I believe rustc only compiles the tests and then something else like Cargo executes them.
It doesn't have to be Cargo, but yes, rustc produces executables for the tests, and you have to then run them.
> there's the same opportunity for end user memory being corrupted (due to miscompilation)
I agree for sure that the safety of the outputted binary is completely distinct from the safety of the compiler itself.
I think the reason that this framing specifically (in the post and in this comment) strikes me as odd is that "requires unsafe code" sort of implies that you need to use unsafe to fix the unsafety of the outputted binary. That just isn't the case. Of course, this is a serious bug that needs to be fixed, but there's just something about "doing memory unsafe things" in this area that like, I think can be a little mis-leading, even if that's not intentional. But I am going to sit with this and think about it, regardless, because I am not sure that my gut reaction here is completely accurate.
(And, hilariously, looking over some work my agents did on my compiler last night, they fixed some mis-compilations that occurred, entirely in safe code. I bet that's also part of why I'm in this headspace at the moment, it's not like those fixes required dropping down into unsafe to fix either!)
> if rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupted (due to miscompilation) as if rustc and cargo shared a code base
Your tests run in an entirely separate process from the compiler (and from cargo). This makes it very different from memory corruption in the compiler:
- The test process can only corrupt its own memory.
- You don't need "unsafe" to run tests. Just the ability to start another process.
- If you're cross-compiling, you wouldn't even be able to run the tests on the same machine (without emulation/compatibility layers)
Does roc run tests in the same process as the compiler?
> Does roc run tests in the same process as the compiler?
We do for tests of pure functions, yes.
> Your tests run in an entirely separate process from the compiler (and from cargo).
That's a great point and a relevant distinction, although Rust tests can run arbitrary I/O, so it's not like having them be in a separate process means memory corruption is harmless! :)
True, but not sure how it's relevant - you don't need memory corruption for the test to perform arbitrary I/O. And even if you trust the tests to not do anything bad, you don't need memory corruption for this to be an issue - any bug in the code emitted by the compiler could cause bad things to happen.
But all of this is ultimately completely unrelated to the concept of "unsafe". You can delete your home directory just fine in safe Rust/Go/Python/etc. You can write a compiler that emits broken code in the same languages; even in a 100% pure functional language with 0 side effects and a perfect bug-free implementation.
I would like to understand this more,
> rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupted (due to miscompilation) as if rustc and cargo shared a code base.
Cause this hasn't been true for me or for anyone maybe your definition of memory being corrupted is the not same as mine.
I am not even sure what you are trying to prove with this.
I appreciate the time and effort in building stuff like Roc I don't use it but this comment and the article feel like...
Oh some guy said Zig not nice because memory safety so here, a post why memory safety doesn't exist because we have to do memory unsafe things sometimes and so everything is memory unsafe already, so maybe it doesn't matter.
I get the energy that we are going for seeing useless claims and wanting to push back but I think the article deserves a clearer part 2 where you elaborate on your thoughts about stuff maybe even get it peer reviewed a bit before posting or maybe don't I guess we could use more raw thoughts in the post AI age.
Either way I appreciate someone trying to put forward their own thoughts and explain problems with a different perspective.
Scenario A: A program has a memory vulnerability. That's bad, and the reason it's bad: attacker-controlled data can potentially trigger this vulnerability, which could even lead to privilege escalation.
Scenario B: A program writes machine code in an executable region of memory, and the code has a memory vulnerability. That's bad, and the reason it's bad: attacker-controlled data can potentially trigger this vulnerability, which could even lead to privilege escalation.
Scenario C: A program write machine code to disk, which is then read into memory and executed. That's bad, and the reason it's bad: attacker-controlled data can potentially trigger this vulnerability, which could even lead to privilege escalation.
Or scenario D:
A text editor stores a URL in a location on disk, and another component then fetches and executes that with its current privileges.
Or in this day and age, that component is an AI agent, and it executes with the things stored in its memory, and is abused into exfiltrating data.
How is that a memory safety issue in Rust compiler? What point are we making here. Am I just too dense to understand, is how the hell is that a need unsafe issue? Or anything to do with compilers???
> Usually this would be done via an interpreter, so I'm not sure that it really requires unsafe either.
Well, I personally have written a const-expression evaluator that actually reuses the rest of the compiler: it compiles the expression in the current environment with some specific adjustments to the codegen settings, launches the temporary executable and gathers its output... frankly, it's more hassle than it's worth compared to writing a separate const-expression interpreter. Plus, of course, it also runs slower since most constant expressions are usually pretty trivial.
Side concerne, but reading the post initially I thought it was about Rocq (formerly Coq). Maybe both team could coordinate and find a way that everybody agree on to avoid any confusion?
https://en.wikipedia.org/wiki/Rocq
Many people try to twist the fact memory safe languages have unsafe code blocks to make the pivot that why bother.
It is like someone arguing that since they always bump the head somehow while wearing seatbelts, then they are only a nuisance and should not be used.
Because they view "unsafe" as an escape hatch instead of a feature. It's a way to encapsulate dangerous behavior, tightly, with clear postcondiitions. Sometimes it's the only way to do things like interact with inherently unsafe FFI code, or hardware.
I adore unsafe, appreciate it as a feature... but it is an escape hatch. One that is sometimes necessary, one that is sometimes not necessary but might still be (ab)used for performance, or initial 1:1 porting of C/C++ code. There are a lot of cases where that escape hatch should probably welded shut though. Fortunately, the Rust ecosystem has tools like `cargo geiger`, and straight out of the box I can also write:
I feel like this perpetuates a bad mental model. Unsafe is not an escape hatch. Code within unsafe blocks must uphold the same semantics as code outside it but the compiler cannot guarantee that those semantics are upheld.
If we're using analogies, `unsafe` is like a "hard hat required" sign. There's nothing intrinsically different about the space inside or outside of it, other than that you can't be sure a brick isn't going to fall on your head once you cross over. So it's on you to wear a hard hat. And to not drop any bricks and trust other people to do their best not to drop any bricks.
You wouldn't call that an escape hatch.
Yeah, I'd have more sympathy for the "It's an escape hatch" model of this feature if the C++ approaches to this problem didn't keep adding actual escape hatches and waving away concerns as "Rust does this too so it's fine".
It can be abused as an escape hatch, but it really shouldn't be.
Memory safe languages, where usage of Miri and Valgrind (tools to for instance debug memory unsafety) are common and integrated into CI for some of the projects in the language. Even some Rust guides encourages running Miri in CI https://microsoft.github.io/RustTraining/engineering-book/ch... . Searching on GitHub yields a lot of projects that run Miri in CI. And there have been a lot of CVEs for Rust projects caused by memory unsafety.
Interesting perspective that you have.
People still die while wearing helmets and seatbelts.
Yes, those unsafe blocks should be cross checked, but lets not pretend it is the same as writing C or C++ where each line of code is a possible CVE.
I think it's a reaction to many simpler advocates of Rust saying "it's unsafe and irresponsible of you to not use Rust, you can't write software in C or C++ or Zig (or Go?) and have memory unsafety and put your users at risk, it's not safe, you have to use Rust to be safe".
Freakin' safety. I like doing lots of unsafe things in life: rock climbing without gear (short bouldering sections on hiking trails), bicycling and skateboarding without a helmet, etc. I developed pretty good skills in all these things, and programming C too. What kind of life would be worth living with enforced perfect safety? Without developing skill in life's many un-safe activities?
So people want to emphasize that, if they shouldn't use non-Rust languages because their safety can't be guaranteed, well your safety using Rust isn't absolute, either. So you can't tell me I must use Rust, or the Rust rewrite of my favorite tools, to "be safe".
Do you recall the early rust web framework, that used a lot of "unsafe" "inappropriately" and had some vulnerabilities ... actix web? ... reminds me of the bun-in-zig situation, kinda makes the language's PR situation more complicated and tricky.
There's a material difference on one taking risks for oneself and one taking risks where the brunt of the consequences fall on others.
You mean like driving a car?
I don't think using an example that includes licensing, plenty of regulations governing drivers, the cars themselves and the built environment, enforcement, and plenty of research on the impact of passive infrastructure to make things safer (bollards, daylighting, raised pedestrian intersections, curbs, traffic lights, etc.) makes the case you seem to be trying to make.
Yep: there's a difference between driving a car in traffic and driving a car in a rally or in an endurance race. There will always be some rules and regulations unless you're on your own privately owned race track with no one else on there, but the regulations can differ greatly depending on the situation.
I agree that it’s not inherent to emitting machine code but I do think it reflects a different set of priorities.
In extremely high performance code you use different data structures and algorithms and change your approach to memory allocation. TigerBeetle famously does all memory allocation once on startup.
Roc is attempting to make a similar set of trade-offs in their compiler as Zig, so it makes sense that the author finds many shared patterns.
I do think it reflects different priorities, but one of those differences is that from my perspective, safety and performance are not inherently at odds. Yes, sometimes it is needed, but not as much as some people seem to think. Sometimes, it also means writing code in ways that communicate things to the compiler that you may not think of if you're not used to thinking in this manner.
A lot of the ways in which the zig compiler works doesn't use pointers, it uses indices. This stuff is easier to write as safe code, not less easy.
> Roc is attempting to make a similar set of trade-offs in their compiler as Zig, so it makes sense that the author finds many shared patterns.
I do think that that makes sense, but it also doesn't mean that they have to. I am doing a compiler project that takes a lot of inspiration from Zig (as my language currently inherits some major things from Zig, and I also care a lot about compiler performance) and it's written in Rust, and does not use much unsafe code (outside of the usual suspects of FFI in the runtime, etc).
> A lot of the ways in which the zig compiler works doesn't use pointers, it uses indices. This stuff is easier to write as safe code, not less easy.
I respectfully disagree. This is only true if you view malloc as qualitatively different from a piece of code that gives you an index for a free object in an object pool.
Provided you don't ask/give back memory from/to the OS, what malloc is doing is giving you an index (pointer) into a pool of bytes, while manipulating an internal bookkeeping structure.
Use after free is just you using an index after said bookeeping structure has marked that piece of memory as available for something else (and perhaps claimed already).
If you have an array of Node structs to represent a graph (like the AST in Zig), and use indices to represent references, you have essentially zero protection in Rust that helps you with finding Node-s that have been used for something else etc.
The 'asking memory from the OS' aspect for malloc doesn't really change the safety of your language compared to this where it matters - if you do use-after-free on a page claimed by the OS, you get a segfault, which immediately tells you there's a problem, which is much better than silent corruption.
At least with malloc, you get debug allocators, or other features that can help you in this case. If you are careless with indices in an object pool, and overwrite stuff, essentially, it's up to you to figure out what went wrong and you have no tools to help you.
I fully agree that there are similarities here. But we disagree about some of the details.
> you have essentially zero protection in Rust that helps you with finding Node-s that have been used for something else etc.
The most obvious technique is generations. You can of course do that in Zig as well.
> if you do use-after-free on a page claimed by the OS,
This assumes that you're working in the context where there is an OS. That isn't always the case. Also, there are other cases than just use-after-free: for example, compilers will optimize around null pointers being UB, which can cause other problems, whereas an index of zero does not get the same treatment.
But also, again: Zig does not use malloc for its ASTs, as far as I know. It uses lists and indices. I haven't literally read the code lately myself, but I would be surprised if they went back to malloc'ing individual nodes.
> In extremely high performance code you use different data structures and algorithms and change your approach to memory allocation.
It's worth noting that the reason Rust doesn't include support for custom memory allocation patterns like Zig does has nothing to do with memory safety. It's more of a historical accident that it just wasn't something that was prioritised early in the projects history and is now hard to change.
It's also important to note that Rust as a language does not know about the heap at all, it is purely a library concern. This means that "doesn't include support for custom memory allocation patterns" is purely talking about standard library data structures, and if you need a ton of performance, you're probably going to be writing your own anyway.
It will be nice when the Allocator trait stabilizes so that the ecosystem can coordinate on making this stuff pluggable, but that's not a direct blocker for getting things done if you need to do things today.
No disagreement in principle; in practice taking that approach in Zig is safer than in Rust, because in Rust it’s “all or nothing.”
The interfaces to custom allocators are more idiomatic, standardized, and normalized in Zig, but there's nothing systematically unsafer about the Rust equivalent. You can use custom allocators all you want in Rust, as long as you're okay using third-party crates like Bumpalo.
On that topic, worth mentioning that Rust's long-awaited `Allocator` trait is perilously close to stabilizing; watch for https://github.com/rust-lang/rust/pull/157428 to be merged, then the stabilization PR to progress here: https://github.com/rust-lang/rust/pull/156882
Yeah that is definitely 1000% wrong. A compiler can do its job with totally abstract data structures. If anything would need to do unsafe stuff in memory, it would probably be a linker.
> probably be a linker
I don't think that's any different either. The core job of linking isn't particularly unsafe.
(Unless, similarly, you're doing the hot reloading stuff)
I've noticed that people equate "low level stuff" with unsafe, regardless of whether it's contextually justified.
I think it's an understandable prior. Historically, "low level stuff" was near-exclusively (see my comment below about OCaml...) written in unsafe languages. Even if that wasn't always literally required, it sometimes was, and so thinking this is the case was a reasonable thing to think.
It is only relatively recently that we have gained more realistic options in these spaces, and so not fully understanding the implications, or preferring the historically normal choices, is understandable.
I'll play devil's advocate. I think emitting machine code intended to run is unsafe because you could emit unsafe machine code, which could run. It's the whole system that is either safe or not, not the individual components. If your system gets hacked by a buffer overflow in the end, nobody cares whether it was the linker that overflowed or the code emitted by the linker.
"Safe" has a very specific definition in Rust. It's not identical to the broader definition used in technical English. You can easily have safe rust code with behaviors any reasonable layperson would call unsafe, like crashing a plane. The original article, comment, and replies were using the word in the Rust sense from my reading, not the English meaning.
Then that's equivocation. Why do we want a very specific form of safety instead of wanting safety in general?
Memory safety is:
1. Foundational for other forms of safety
2. Has an objective definition, when some other forms of safety are either subjective or inter-subjective.
That said, I don't understand why your parent brought this up to you, you are talking about memory safety in your original comment here, so that's what Rust's safety is about.
I feel that the buzz phrase "memory safety" has been defined by Rust to mean "the safety Rust gives you". Obviously memory usage can be more safe or less safe, and Rust is decidedly on the safe end of the spectrum, but it also has the gaping type system holes demonstrated in cve-rs which completely shatter any claim that safe code is safe, and there are other bugs which occur in Rust while the programmer is distracted by trying to prove their code is memory-safe.
> the buzz phrase "memory safety" has been defined by Rust to mean "the safety Rust gives you".
It's more that Rust's safety guarantee is memory safety. No more, no less. It's not about buzz, this term was used long before Rust existed.
> it also has the gaping type system holes demonstrated in cve-rs
This is not a "gaping hole". It is a compiler bug, which has never been found in the wild.
> there are other bugs which occur in Rust
This is true! Every language can have bugs in it, and Rust does not claim to solve all bugs.
Does cve-rs break any type system rules? If so, why hasn't it been fixed yet?
> Does cve-rs break any type system rules?
Yes.
> If so, why hasn't it been fixed yet?
Pretty classic software engineering reasons.
The part of the system that it involves was in the process of being re-written already. The re-write fixes the bug. Because it is essentially a theoretical issue, and not an actual problem in any real code, it is not a five alarm fire. Waiting for that re-write to land makes the most sense, instead of putting in a ton of work that will be thrown away.
Other, more serious miscompilations get fixed faster. In fact, a version of the Rust compiler was released today to fix one, even https://blog.rust-lang.org/2026/07/16/Rust-1.97.1/
This one was impacting actual users, and did not require re-writing entire subsystems to fix properly. So the engineering and product tradeoffs are different.
If cve-rs exists, is safe rust safe? Can one prove that Rust code is safe only by auditing the unsafe blocks?
Every compiler has bugs.
Not what I asked
If cve-rs exists, we still say that safe Rust is safe, in the same way that we say Python is safe despite potential bugs in its interpreter or native libraries, and that Java is safe despite potential bugs in the JVM or JNI libraries, and so on.
> Why do we want a very specific form of safety instead of wanting safety in general?
Because a “very specific form of safety” is a useful tool in achieving “safety in general”
Because a “very specific form of safety” is tractable for a compiler and language runtime to achieve, “safety in general” isn’t
> safety in general
This is impossible. General words like "safe" and "good" are subjective, and useless in a technical context unless you ground the discussion by giving them specific definitions. Otherwise everyone ends up talking past each other.
Okay, thanks for debunking all good products, safe houses and clean water. I guess they are just products, houses, and water.
Good for what? A hammer is good for driving a nail, but not good for driving a screw.
Safe for what? My house is safe for humans, but not safe for tropical birds.
Clean enough for what? Our water is clean enough to wash my ass, but not clean enough to wash a telescope mirror.
Sorry but life is not a Disney movie where some things are unequivocally good/safe and other things are unequivocally bad/unsafe. There are gradients and conditions, and communication requires a shared language between participating parties to navigate them.
What nail? A hammer is good for driving a nail from the hardware store, but not good for driving a finger nail.
See? I can play stupid word games too.
How tropical are the birds? I'm afraid life isn't a Disney movie where some things are unequivocally tropical/not tropical. How shared is the language? Congratulations on using only two adjectives in your comment besides the ones you're complaining about, but two is greater than zero.
How much your is the house? Do you own it? Without any mortgage or lien?
You're just doing the same thing and further proving my point. Thanks, I guess?
That would mean no language can ever be considered safe, because any language can emit bytes to a file that will later be executed.
Correct. Safety is a system property, not a language property. Calling a language safe is about as sensible as calling a metal alloy unsinkable.
The only safe program by this measure is the one that's never ran.
> It's the whole system that is either safe or not, not the individual components.
This is a core perspective disagreement. While this is true:
> If your system gets hacked by a buffer overflow in the end, nobody cares whether it was the linker that overflowed or the code emitted by the linker.
That does not mean that increasing the amount of safety in the individual components isn't helpful, because it helps minimize the above outcome, even if it will never be zero.
Safety is a feature of a system - yes. It's also a property of what it's against. A computer could be safe against being hacked but still be dangerously easy to drop on someones toe and break it.
Safety [against something] is also a feature of components - a system made up of only safe components [against a thing] is safe [against the same thing... I'm going to stop this qualification now for brevity]. A system containing unsafe components may or may not be safe but at least you know what components usage you need to look at carefully.
If your linker is safe, linking code will never result in the thing it is safe against. Ever. This is a useful property even if running the linked thing is not safe because it means:
1. When things go wrong in strange ways, you have strict bounds guiding you in figuring out what went wrong.
2. You can build reliable systems that do part of the job, and only have to sandbox the other half of the job. Compiling in a CI system will (if the compiler was entirely safe) be safe. You can do it with secrets present against malicious code. Running tests will have to be sandboxed (assuming running tests isn't safe). This could for instance enable safely sharing significantly more artifacts for incremental builds in CI.
Unfortunately very few compilers are really safe against anything (though I do wonder how I could break my toe on one). Rustc for instance has a giant C++ half called llvm that isn't really hardened at all. We get away with this by just not trusting the compiler when run against potentially malicious code.
Perhaps the parent meant dynamic linker.
It wouldn't be the linker that has to be unsafe, it'd be the "and now execute!" jump. And that could be abstracted as memfd+execveat, which are fairly normal operations.
OP's argument is roughly "doings things with computers has to be unsafe to be useful", which is.. uninteresting.
> I don't really think that this is true, in the way that it's written.
Well, it's true when it's written badly (eg. the same way as when somebody writes in-line "shellcode" by doing `*(T*)ptr = (T)val;` or similar.
I think there's some history of some compilers that were basically non-trivial to port to other architectures/bit-sizes b.c. of this piggybacking.
That line confused me, too. What parts of their compiler require memory-unsafe operations to produce machine code?
They are saying that running the compiled code is memory-unsafe when there is a compiler bug, and that’s what developers do next. The memory corruption happens in a different process.
In this respect, effectively all the compiler should be treated sort of like an unsafe region because it requires extra care to avoid memory corruption bugs.
That's not what it says at all. The section we're talking about is for the compiler and emitting machine code
> we ended up with about 1,200 uses of unsafe
> remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job
Anywhere talking about the `unsafe` keyword is within the Rust code.
The article is a bit confusing because they also write:
> Regardless of which process had the bug—the compiler or compiled program—in both cases the processor only did the bad thing because the compiler told it to. And in both cases the fix is the same: the compiler's code must change, since that code was what caused the memory corruption.
But yeah, I wonder what those 1,200 unsafe uses actually did?
I think if you interpret it charitably it means that any bug in the emitted machine code is already a likely memory-unsafe miscompilation if it is ran.
The compiler itself might be perfectly "memory safe" but the generated binary fundamentally is always at risk (besides WebAssembly I suppose).
I'm fully aware of the separation of compiler and binary, and being able to compile untrusted code safely is nice, but a perfectly safe compiler that generates vulnerable binaries isn't that much better.
In context that's clearly not what he's saying, the next sentence is this:
> Zig has more features than Rust for making memory-unsafe code work correctly, and that was the area where we wanted the most help.
Zig definitely does not have more features for successfully emitting memory-unsafe machine code than Rust does. I can emit memory-unsafe machine code from typescript if I really want to and nothing at all in the language will get in my way. So the sentence quoted above must refer to the idea that the compiler itself needs to be unsafe, which Steve is right is simply untrue.
I do think that is a good point, it's just not what the line actually says. But that's why I wasn't saying "zomg this is WRONG!!!!" but instead, trying to point out that there are subtleties here. For people who aren't as deep in the weeds in this subject, I think the details matter. But again, as I said, I like the post, this is just one thing.
I am also probably in a more pedantic mindset because, well, I'm writing a compiler in Rust, and the words as written do not resonate with me at all.
> a perfectly safe compiler that generates vulnerable binaries isn't that much better.
I do think it's much better. Eliminating classes of bugs in one component is a good thing, even if it's not every component. This is a core lesson of Rust! unsafe still exists, but going from "I don't know what is unsafe" to "only this part is unsafe" is a major improvement.
The section this came from was talking specifically about usages of `unsafe` in the compiler code.
It's not about the memory safety of the resulting binary.
Agreed, that’s disturbingly incorrect.
If anything, compilers are perfect models of trees and well formed programs.
Maybe in theory. In practise you have thing like super optimisers. You have side effects that the compiler needs to understand etc.
That said I'm struggling to think of something that would need to be unsafe.
>ReleaseSafe catches use-after-free errors through runtime checks which panic if the program tries to use freed memory.
I don't know Zig so maybe they know something I don't, but I have seen no evidence that it catches any type of use-after-free including double-free?
While writing a blog post (below) I went through the documentation to figure out the possible runtime memory safety checks Zig can insert. The term "use-after-free" or "UaF" never occurs on that documentation page. Searching for "safety-checked" doesn't yield any related hits either.
Unless maybe they're using the DebugAllocator in release builds? Even that does not reliably surface UaF.
https://landaire.net/memory-safety-by-default-is-non-negotia...
I as someone with writing Zig a bunch, can safely say if it does it hasn't even worked for me.
I am talking from experience from a pre-ai human mitts writing code perspective maybe Zig + LLMs do some magic.
The more I read the article the more I feel like this is just bad not sure if I should be giving it as much latitude as I have been in my prior comments.
There are other claims as well that are weirdly phrased at least.
Reads like an article written to justify some arguments they had rather than a genuine take at this point.
But I will give the benefit of doubt I enjoy weird articles, languages and share a dislike for aggressive AI-ness of all things.
I believe you are correct.
I think ReleaseSafe just adds bound checking and panics on unreachable code.
I don't think Zig offers any temporal memory safety.
The DebugAllocator catches use-after-free (at least on page-level), but at the cost of never recycling memory addresses (e.g. it eats through the virtual address space).
https://ziglang.org/documentation/master/std/#src/std/heap/d...
For higher level code, "generation-counted index handles" might be the better solution to provide temporal runtime memory safety, not part of Zig the stdlib though.
Or even better: never use dynamic memory allocation and make all lifetimes 'static' :)
>The DebugAllocator catches use-after-free (at least on page-level)
To clarify, is that to say that you have to use the `std.heap.page_allocator` as its backing allocator?
as a reminder its a construct in the zig stdlib to take an arbitrary chunk of memory (could be stack, could be in the programs static space) and wrap it in an allocator interface and give that to any data structure that needs an allocator and use it as if it were just malloc/free, with a fixed memory limit and the correct memory errors.
I don't know enough about Zig to explain it, but there is more to ReleaseSafe than checks and panics. ReleaseSafe also clears memory that no longer has an owner (I might be describing that wrong, that is just how I understand it). I found this out with a rendering issue recently.
The bug was around passing a slice to OpenGL which referenced memory outside of its lifetime. Since the memory location had no owner, vertices would still exist in Dev builds and everything would work fine, but in ReleaseSafe the application would run and just have nothing to render.
Since OpenGL was trying to read the memory, there was no panic from Zig, but it was a cool look into how the different build modes handle memory.
This is the commit where I fixed the issue: https://github.com/quot/donut/commit/8fff107e76278c4bf55007c...
Zig does offer some amount of temporal memory safety.
Link: https://zig.guide/standard-library/allocators/
Text:
> The Zig standard library also has a general-purpose debug allocator. This is a safe allocator that can prevent double-free, use-after-free and can detect leaks.
For more detail, see:
https://github.com/ziglang/zig/issues/3180#issuecomment-5284...
I still don't think that does anything regarding use-after-frees, only double-frees.
Here's the code: https://codeberg.org/ziglang/zig/src/commit/e44e927d33d37c44...
The closest callout in the doc comment is:
>Never reuses memory addresses, making it easier for Zig to detect branch on undefined values in case of dangling pointers. This relies on the backing allocator to also not reuse addresses.
But it's not really clear what this means. "branch on undefined values" would I think indicate that maybe they're doing a fill pattern that the compiler can detect at runtime when dereferenced? But I don't see it in the `free` path. It's not clear if this is deterministic or not either.
I'm not sure what "branch on undefined values" means there, yeah, but never reusing memory addresses is enough to prevent use-after-free.
Or, rather, you can use a value after freeing it, but it will not be exploitable, because it will contain valid data of the right type. This is the same idea as Type-After-Type,
https://dl.acm.org/doi/10.1145/3274694.3274705
(Also similar to when you use indexes to an array in Rust and happen to read from a wrong but in-bounds index.)
Interesting that OCaml was flexible and expressive enough to be used as a prototype testbed but not chosen as the implementation language, especially given the maturity of both. I would be surprised if Zigs incremental builds could be meaningfully faster than dune's.
Cross compilation is great, but not mentioned in the "why Zig" section. Is memory control that crucial for a compiler?
Rust itself was originally written in OCaml, same with WASM. I'm curious about what milestone gets reached where the maintainers collectively decide to transition away.
Rust moved away from OCaml when it decided to be re-written in Rust. The post alludes to this as being a usual time for a wholesale re-write, and I'd agree.
I appreciate the insight, and on closer reading the post clearly states that realistically only Zig and Rust were ever considered anyway.
Since you're here, could you comment on the approach Rust took in their rewrite? Was it more of a straight translation like Go did when they self hosted -- similar to the recent Bun transliteration? Or were there architectural changes made along the way like this article describes with Roc?
The Rust re-write happened before I got involved. If pcwalton is around and sees this comment, maybe he can provide a more first-class account.
> Was it more of a straight translation like Go did when they self hosted -- similar to the recent Bun transliteration? Or were there architectural changes made along the way like this article describes with Roc?
From what I remember, it was a whole-sale re-write from scratch, not a transliteration. While Rust took a lot of inspiration from OCaml, especially in those days, it was different enough that I'm not sure that a more direct transliteration would have been particularly possible, though again, see above, I wasn't there, so I don't know for sure.
One of the primary goals for the Roc project is compiler speed. I presume OCaml is out of the running because it's not a systems language.
OCaml compiler is incredibly fast. I wonder how it'd fare with Jane Street's extensions for the borrow checker etc in OxCaml, if it's good enough for their HFT I'm sure it's good enough for a new language.
I suspect this "not a systems language" alludes only to OCaml's rather steeper learning curve and until-recently difficulty with multiple threads. I am sure it could roll just fine as a single-threaded compiler language written by a small team, which indeed, it was.
I wrote a toy Scheme implementation in OCaml by using the Camplp4 preprocessor. In benchmarks, it was faster than Gambit Scheme, which compiles through C.
Sounds interesting, have you put it online somewhere?
Depends on the beholder.
Unix system programming in OCaml
https://ocaml.github.io/ocamlunix/
https://mirage.io/
OCaml has often historically been considered a language that's been appropriate to write systems tooling like compilers, runtimes, and unikernels in, even though GC'd languages were/are not often considered for such projects.
They are considered in many research labs since Xerox, unfortunately there are still too much anti-GC religion among mainstream devs.
I don’t think there’s too many of us on the ‘GC did nothing wrong’ hill.
Reading the average HN opinion, it seems everybody is writing high-performance latency-sensitive systems that would implode if a response would take 1 ms longer than normal.
Sampling bias. Most of the people responding are probably those with a strong opinion because of what they work on. Everyone else is likely relatively indifferent to it.
It is a misconception that GCs only affect latency-sensitive systems. High-performance throughput-optimized systems are also sensitive at ~1µs granularity for different reasons, so GCs are not used there either.
That a GC is adverse to the performance both latency-oriented and throughput-oriented workloads doesn't leave many use cases in "high-performance" systems. Maybe systems that are severely I/O bound but is barely a thing these days.
> Maybe systems that are severely I/O bound but is barely a thing these days.
Any kind of web service is barely a thing today? Which is what 99% of HN posters are working on, hence my comment.
> High-performance throughput-optimized systems are also sensitive at ~1µs granularity for different reasons, so GCs are not used there either
Games are high-performance throughput-optimized systems that have adopted GC languages for 15+ years now, and again a type of application which is much more latency sensitive than most people deal in their day to day.
Nobody is claiming GC is a panacea, but it’s good enough for a lot more use cases people give it credit for.
If you are severely I/O bound it isn't intrinsic, it means your server is badly under-provisioned in the I/O department. Linux on a modern server can push 200 GB/s of I/O. Even if web services were engineered to a standard that could consume that much I/O, which they are not, you would have to be astonishingly wasteful to burn it all.
It is rare to be severely I/O bound because software engineered for I/O performance tends to run out of memory bandwidth first.
Games are not throughput-optimized systems in any conventional sense. They are a canonical example of latency-optimized systems.
I have nothing against GCs, I use them regularly even in performance-sensitive contexts. But too many people understate the adverse impact of GCs on performance contrary to evidence and theory.
I/O bound means waiting on I/O, which isn’t necessarily because it is slow, but because it is simply waiting on data to arrive, like a web service most of its idle time. If your client is dozens of milliseconds away, a GC pause is pretty much invisible, unless you are trying to squeeze every last request/second from a machine (instead of simply scaling horizontally)
That said, from your profile, you seem to work on a very sensitive niche that might colour your opinion, with good reason. What I am claiming is most of us are not building such strict a system.
Even in my toy hobby of OS development a GC isn’t the end of the world unless your goal is to compete with, say, Linux in a some kind of performance challenge, where in that case memory allocation might be the least of your bottlenecks.
So you also are fully skilled in using value types, stack allocation and GC region free memory in such languages?
It is also a misconception that all GC are born alike, and that don't exit languages with support for value types, stack allocation and GC free memory regions with C like pointer fun if so desired, while being mostly GC enabled.
Could you elaborate on "GCs are not used there [high-performance throughput-optimized systems]"? Are you referring to the cascading effects of tail latency on systems with high fanout?
Sophisticated throughput-optimized systems rely on deep latency-hiding. Schedulers see millions of atomic operations into the future, continuously rewriting the schedule globally to maximize locality and minimize resource contention based on real-time changes to workload, resource availability, and system behaviors.
In short, for each of the millions of in-flight operations (which might only map to a handful of user operations), it is trying to precisely optimize the concurrency, timing, and dependency sequencing such that when operations are executed every resource required is hot, uncontended, and available with high probability. When this works well it dramatically reduces the number of hidden stalls in execution. The schedule is constrained by tail latency requirements; a theoretically throughput-optimal schedule can defer execution indefinitely.
For an analytical database engine, an "atomic operation" is typically a query operation on a database page. A modern server can retire 100M ops/sec. While I am oversimplifying a bit, a 1 millisecond GC pause can blindly wreck the schedule for 100,000 operations in an unpredictable way. In these architectures we try to eliminate all context switches for the same reason which are 100x cheaper.
Practically, 1µs stall is a good heuristic for a noise floor. The schedulers have pretty wide concurrency on big systems, so the implied 100 operations are unlikely to have a dependency. Many stalls that are difficult to precisely control like cache line fills fit in here too.
If there was a GC that had a worst-case stall of 1µs then you could probably use it for these cases. Unfortunately, "low-latency" GCs tend to be more like 1000x that. I don't think there is any way of closing that gap short of putting a GC in hardware.
Real time GC as used by the military in weapons control systems, and on factory automation robots, keeping PTC and Aicas in business, people pay to use them.
TBH, physics limits how latency-sensitive weapons systems need to be and you can largely just disable the GC in these contexts. They use CPUs from the 1990s to do hypersonic terminal guidance. You don’t have to do any performance engineering for many latency-sensitive weapon systems. Could probably write it in Javascript.
For throughput-optimized systems, some of which are real-time, you never see a GC. That loss in performance is simply too large such that the computation becomes intractable. A lot of really poor systems admittedly exist but no one considers them “good”.
Throughput-optimized systems like HFT?
https://www.lmax.com/exchange/technology
No, you could not guide battleship weapons in Javascriptt
https://www.lockheedmartin.com/en-us/products/aegis-combat-s...
https://dl.acm.org/doi/abs/10.1145/2402709.2402699
And no they don't disable the GC, they have real time GC implementations,
https://www.ptc.com/en/products/developer-tools/perc
> Most of the people responding are probably those with a strong opinion because of what they work on
Quite the opposite. People here have strong opinion because they work on web apps and CLI toys.
Not to mention that in general GC simply requires more memory.
> Most of the people responding are probably those with a strong opinion because of what they work on.
I can assure you that's not the case on here. The people working in truly low latency environments are not commenting on GC threads to begin with because it's a non-starter for them. For whatever reason, there is just a chunk of people that eat a lot of FUD around GCs who are working in the exact domains they thrive in.
The Web scale meme exists for a reason, yeah.
definitely not suitable for runtimes. After all, OCaml's own runtime is in C, not OCaml! For compilers I agree it's a fine choice.
Zig's incremental builds are DEFINITELY a killer feature. In the short term, I could see why you'd make a switch to get it. But, in the medium term, can we really not expect to see this in Rust in the somewhat near future?
I want to go fast, but I don't want to go fast just to shoot my foot off.
If only somehow we could get Rust's safety with all of Zig's features and Go's runtime without GC...
That's what I'm working on building [=
This is being worked on: https://rust-lang.github.io/rust-project-goals/2026/roadmap-...
Most of the goals on this page are targeted for this year.
Rust's compile times will get faster long before Zig gets safer.
I'm pretty sure Zig has no plans to ever become safe - by any sane sense of the word - so, yes, I would expect...
zig does have plans to give access to IRs when stable so adding a borrow checker to zig will be even easier than it is now
This is cool and will likely enable some cool tooling.
I don't think a borrow checker is likely to be in that tooling. Borrow checking requires shaping the code, and all the dependencies, into easily analyzable (and at least in rust's version annotated) patterns. You can't borrow check arbitrary code not designed for it without false positives.
You can because all allocations are tracked and explicit
That's not sufficient - consider the following pseudocode
Conditions can be opaque and non-analyzable due to rices theorem - in any turing complete language. This code is correct (or at least not memory unsound) if opaque_cond and other_opaque_cond are never both true. Otherwise it isn't.And functionally compiler analyses of whether conditions hold have to be trivial because using some form of theorem prover to decide of code is correct or not leads to code that is brittle against compiler version changes, and slow compile times. Thus opaque_cond could be as simple as `len == 0` and `other_opaque_cond` could be `len > 0` and it's unlikely you'd want the compiler to realize those are mutually exclusive (at the stage where it accepts programs, obviously during optimization it is very likely to take advantage of this).
Rust solves this by simply rejecting the pattern. Very roughly forcing you to write if opaque_cond() { free(x) } else if other_opaque_cond() { use_x } (or something else where the program structure and not just the logic in the conditions guarantees correctness). Zig simply allows it and leaves it up to the programmer not to make a mistake.
And as onlyrealcuzzo suggests aliases are where this type of analysis (accepting enough programs to be useful but still imposing enough structure you can prove correctness) is really tricky.
so yes it is possible to detect those patterns and ban them as unsafe, and have a"safety checked alternative. clr does this currently:
https://github.com/ityonemo/clr#safety-oriented-architecture
It's possible to detect and ban the version of the problematic pattern that I made as short and simple as possible to illustrate the point, sure.
I'm general though, I don't believe it is practical to do so. Not without every library being designed with the checker in mind and annotated to more precisely describe their APIs. Which is why I'm not surprised to see the limitations.md that seems to exclude all the hard cases (aliasing, pointers used as first class values, cross function analysis): https://github.com/ityonemo/clr/blob/main/LIMITATIONS.md#mem...
Obviously if you rewrite the zig world to obey rust like rules and include rust like annotations you can implement a rust like borrow checker, but I don't think that it would still be meaningfully zig. It might be an interesting language worth exploring.
did you miss this part?
> planned to be addressed
i have in mind a strategy to address all of them. this is a side project, a proof of concept, i have other things going on in my life. i dont chip away at it every week.
you make, without any evidence ("Obviously"), an assertion that "it would look like another language". so far if anything applying zig clr would push a user to write more idiomatically ziggy code, away from idiomatically c-ish code. i dont see why continuing with clr wouldn't go further along that trend. so consider what is "obvious" to you might just be flat out wrong.
> It might be an interesting language worth exploring.
worth how much? youre welcome to sponsor my exploration and put your money where your mouth is:
https://buymeacoffee.com/vidalalabs
I wish them the best of luck, but I don't expect them to succeed, and unspecified plans don't constitute a demonstration of feasibility. Plenty of people have made plans to solve unsolvable problems in the past, me included. I strongly suspect that that is the case here (or that they're willing to iterate away from zig).
The evidence is the amount that rust had to iterate on the underlying language to make the borrow checker work well. Something it had the freedom to do since it was co-designing the language and linter.
Edit: Didn't realize this was your project (responded before you added the donation link) - I would have worded my response slightly differently but my opinion is unchanged. Seriously mean it with the best of luck getting this to work.
i dont understand the downvotes here. the point of any safety checker is to flag and ban potentially unsafe code, and force the author to rewrite with existing language patterns that guarantee the desired safety parameters.
in this case, zig has a first class nullable syntax that the checker can use ti guarantee correctness for, so a checker can deterministically sidestep this turing completeness issue, by squeezing indeterminate code into the knowably safer language idiom.
Allocations are less of a problem than aliases.
Without affine/linear ownership - solving the aliasing problem is the Halting Problem.
Rust didn't invent Affine Ownership just to make Rust hard. It did it because it's one of the only ways to have memory safety without a GC.
1. rust didnt invent affine ownership. 2. It's possible to bolt on to other languages (see ada). zig in particular is easy (disclaimer: i think, i haven't implemented it yet)
Apologies for the noob question, but what is an IR?
intermediate representation. attempting to analyze zig code directly would be too hard (especially with comptime). on the way to the compiler backend, the compiler builds a simplified representation that only has "actually existing functions" and is very straightforward, e.g.
you can see how building a data dependency graph from this would be easy.Are compile times that big of a deal? I haven't used Rust a ton, but the few times I have it seemed like the bulk of the compile time was a one off compiling the crates, and then compiling your own code was super fast.
I feel like I'd massively prefer to end up with a binary free of memory exploits than shaving some time off compile.
Instead of waiting for faster compiler in Rust, how about from the other direction, adding some kind of borrow checker to Zig? That sounds more within reach and practically achievable, possibly even in userland.
That's sort of what I'm doing...
I'm writing a language with Affine Ownership that transpiles to Zig and has a built-in FSM-based Green Fiber runtime.
Affine Ownership gives you memory safety + fearless concurrency + eliminates the need for Go's GC.
It's obviously going to slow down compilation - since you need to do Rust's borrow checking, etc. But I can do this incrementally as well...
Can selectively turn off the borrow check for dev builds?
It's impossible to add a borrow checker to any existing language.
The reason Rust has a working borrow checker is because every part of the language from structs, enum, traits, generics and all the way to the syntax itself has been designed to support lifetimes and borrow checking.
It's is not something you can just tack on to an existing language without fundamentally changing it.
Swift, Linear Haskell, Chapel, Ada/SPARK are all counter examples from such claim.
Also OxCaml, from what I hear.
OCaml already starts form a memory safe base being GCd?
I'm not familiar but I think this paper describes how OxCaml works:
Oxidizing OCaml with Modal Memory Management - https://dl.acm.org/doi/10.1145/3674642
> We focus on three mode axes: affinity, uniqueness and locality. Modes are fully backwards compatible with existing OCaml code and can be completely inferred. Our work makes manual memory management in OCaml safe and convenient and charts a path towards bringing the benefits of Rust to OCaml.
https://github.com/oxcaml/oxcaml
Yes, it is getting there, however I would rather count it when they finally manage to upstream everything to OCaml as per plan.
I wouldn't say it's impossible, rather un-ergonomic. TypeScript can add type information to ordinary JavaScript code via JSDoc comments; the result can both be executed as ordinary JavaScript as-is and type-checked with TypeScript. But it's a huge pain to try to write (and maintain) everything that way, it was supported as a hack to help migrate legacy codebases. You could probably take a similar "the lifetimes are embedded in comments" approach with other languages, and the result would be similarly un-ergonomic.
A better comparison would be Python.
The way Python added types is the most disgusting thing imaginable... but it has type hints now, so I guess that makes some people happy.
That is possible (clang has experimental lifetime annotations support), but that is not enough to guarantee memory safety.
As a simple example, Zig has no private fields. That makes encapsulating any unsafety impossible.
> Zig has no private fields
You may have missed the point here. You could add a comment to the struct field that marks the field as private, and build a TypeScript/JSDoc analogue that analyzes all accesses to the field and fails if it finds accesses from functions that aren't part of the struct that owns the field. You don't even need a comment on the field - you could copy Go's convention, add a comment to the struct definition marking it as "follows Go convention", and then fail any access from outside the struct to a field that starts with a lower-case character.
It doesn't prevent you from ignoring that tool and writing Zig code that imports the struct and accesses the field. It is, of course, not part of the Zig language itself. But if you adopted a tool like that, it would be your responsibility to run it across-the-board and pay attention to the results - same as how it is your responsibility to pay attention to the results if you added those JSDoc comments.
I have picked private fields as an example of feature that is needed because it is very simple. You're right that you can build an analyzer (with additional code annotations) to support that, but it's only one example.
Take another example: unsafe traits. They are fundamental to some safety encapsulations, most famously concurrency (`Send`/`Sync`). Here you cannot just build an analyzer to mark something unsafe, because Zig has no traits, its generics are duck-typed.
You can, of course, add traits. But at this point you're essentially creating your own language that compiles to Zig, with all problems this entails (e.g. bad ecosystem support). It's also hard to claim that Zig can be memory safe then.
> You can, of course, add traits. But at this point you're essentially creating your own language that compiles to Zig
I think herein lies the rub. What's the difference between a static analysis tool and an actual separate language that transpiles to the original? Hypothetically - again, very un-ergonomically - you could add traits to Zig code in comments, or in example-traits.typezig files that would be skipped by the Zig compiler (like how *.d.ts files are skipped). How much of a language is writing code in a particular syntax, versus how much of a language is writing code that will pass a tool "building" it, versus how much of a language is about the final compiled output that you get from the tool? All static analysis tools that support line-level exceptions are, essentially, programmed by comments, with their own language (typically highly simplified compared to a "full" programming language), that affect whether or not the "language" passes or not. What Typescript/JSDoc shows is that, actually, much more complicated tooling can be built with this programming-by-comments model than had been done before (to my knowledge), and thus even more powerful still tooling could be built with that model.
Of course there's a difference between static analysis and a language that transpiles. But perhaps it's more a question of degree than a simple binary classification.
no. You don't need private fields. All you have to do is analyze the code, harness the compiler to generate a time-dependent data dependency graph, and map allocation/frees/uses, if you can 'color' branches where data are shared you can also track and check to see there isn't an aliasing violation too.
it is easy to patch the zig compiler to enable this this (export the code graph; about 50 LOC). The analysis is much much harder to get right.
This analysis is undecidable. There is a reason sound static analyzers (including languages like Rust) require in-code annotations.
it is possible to do in-code annotations in zig, if you're clever. you can get pretty far without them too.
as an example, you can check for double free without ownership tagging, by being agnostic about who should free, and flagging if two nondisjoint code paths attempt to free the same allocation.
It seems like it'd be pretty reasonable to get something akin to polonius. I can write up an engine in zig if it'd help?
start by examining zig-clr
It is only feasible to do this if the whole of the codebase idea designed to allow it, and it's still going to blow up in odd ways of you don't have a way to describe lifetimes in your interfaces. The magic of rust's design is that it turns this memory tracking into a local problem, such that you can design an interface and be sure that every use case is safe and verifiably so.
Exactly.
Every part of the language must support memory safety from first principles.
empirically untrue. several projects exist that bolt on extra safety to unsafe languages or unsafe parts of language. SeL4 for C, MIRI for rust unsafe. i guess ada/spark for ada too, is the OG, spark being added to ada 4 years after its first release
Hardening is definitely possible, we've had sanitizers in C/C++ for a long time. It's not full memory safety though. Miri is the same.
SeL4C is formal verification, and while it can prove memory safety (and much more) it is much more difficult, to the point that you're basically programming in a different language.
Ada/SPARK is your best example, and also the example I know the least of, so I won't comment on.
SPARK omits some features of Ada, so it would only reinforce the sentiment that bolting on verifiability after-the-fact is difficult. Expressivity is generally the antithesis of static analysis, and it's very easy and tempting to make a language that is accidentally too expressive to support a given analysis without being required to make breaking changes to reduce expressivity.
i mean in zig-clr it pushes you towards more expressive patterns, for example, making you label pointers as optional if their status is ambiguous
A language is more expressive when it allows more programs and less expressive when it allows fewer programs. I don't know zig-clr, but if it rejects programs that Zig accepts (for example, by rejecting the aforementioned ambiguous pointers), then it is less expressive, not more (keeping in mind that being less expressive is not a pejorative).
no thats not the definition of more expressive. more expressive means the language can encode more programmer intent without making a dog's breakfast of the code.
Now we're just having a semantic argument over the word "expressiveness", which is not especially interesting; see https://chrispenner.ca/posts/expressiveness-spectrum . My argument above remains true regardless of the terms you choose to use.
C# was already a very mature language when it had referenes and later "ref safety" added to it.
> It's impossible to add a borrow checker to any existing language.
Why do you say that. Have you tried and failed? It seems to be possible to add a borrow checker to zig, just as you can add MIRI to rust to get extra safety in unsafe blocks.
> how about from the other direction, adding some kind of borrow checker to Zig? That sounds more within reach and practically achievable, possibly even in userland.
It's doable, and as static analysis. see sibling comment.
No, it would fundamentally change how Zig works.
no, it would not. If you do not believe me, you should try out the repo.
the architecture doesn't make sense. MIRI doesn't perform static analysis on MIR. It is, as the name says, an interpreter. The borrow checker is entirely different from miri.
Rust's borrow checker requires lifetime annotations. Zig code doesn't contain any such annotations. How does your design handle this?
1. it is possible to do code annotations in zig even though i havent implemented it in this iteration of clr (the first poc demonstrated this). i want to see how far i can get without them.
2. let's take double free (easiest to explain).
you dont have to tag ownership, you can be agnostic about who should free, and merely report if two nondisjoint code paths attempt to free the same memory.
Layperson here: what is special about Go's runtime, aside from the GC?
Chief design goals were radically easy concurrency and speed of compilation.
Speed of compilation feels like a distant second in terms of goals given the weird new generic features they keep adding..
I was fine with basic generics they complicated it quite a bit much for my liking.
What weird new generic features? Generic type aliases? Those aren't very complicated.
Is the Go GC that special? Is it even generational yet?
I'm not sure it would ever make sense to be. That makes the assumption tons of allocations get made that don't live long, which was(maybe is still?) more common in some languages. Go is more aggressive about not heap allocating, and has tools to help you avoid them.
Idk, is it? https://go.dev/blog/greenteagc
> Is it even generational yet?
Is there any reason in particular it should be? Or are you just throwing random buzzwords around?
Anyways, https://github.com/golang/go/discussions/70257#discussioncom...
You seem to be really hostile for no apparent reason. There are plenty of reasons to be generational, there are lots of workloads that the current implementation might fall flat if it wasn't.
Goroutines?
It's literally the most sophisticated scheduling engine in the world.
In practice, Go can typically outperform Rust in throughput (using more memory), despite having a mountain of disadvantages against it in theory.
That's how good the Go scheduler/runtime is.
> n practice, Go can typically outperform Rust in throughput (using more memory), despite having a mountain of disadvantages against it in theory
This is a huge claim that disagrees with both my real-world experience and everything I've seen from artificial comparisons.
Every high performance Go system I've worked on has quickly reached the point where we're optimizing memory management and doing things that would have been explicit in a non-GC language like Rust anyway.
The Go runtime is amazingly optimized, but it comes with overhead over doing the same work directly in a lower level language.
Go has few issues with performance (lack of in-line union types, interface overuse, inefficient idioms reg. collections, some missed optimizations) but its seems plausible for a idiomatic Go program to outperform an idiomatic rust program in some situations.
Example: https://news.ycombinator.com/item?id=22336284
This is the first I've heard anyone claim higher throughput for Go than Rust. Any articles you'd point to to learn more?
I think one of the few performance benefits with a GC is that you can defer allocations. You can do that in Rust too though.
> It's literally the most sophisticated scheduling engine in the world.
That seems unlikely regardless of how good it is. This is a domain where state-of-the-art research is not in the public literature. Scheduling is an AI-complete problem.
I think this is interesting and warrants explanation. There are cases where a GC can be faster (sort of, Arenas get you most of the gains) but "the most sophisticated scheduling engine in the world" should be easy to at least partially support.
What benchmarks are you referring to?
Rust itself doesn't have a scheduler of course, I assume this is comparing against tokio or one of the other async executors?
What a joke, ignoring Erlang, and the custom schedulers from JVM and CLR runtimes.
Erlang's scheduler is not sophisticated, which is what makes it AWESOME.
but yeah. i would be surprised if the JVM's scheduler is not more sophisticated than go's if for no other reason than it has way more knobs you can tune. you know they put that knob in there because someone (probably Google cough cough) asked for it
The missing part is that if what is in box isn't enough, both JVM and CLR allow you to fully customise how the scheduling algorithm works.
> if only somehow we could get Rust's safety with all of Zig's features
i periodically throw my unused codex tokens at this:
https://github.com/ityonemo/clr
Nitpicking ahead:
I am not sure, but there might be a bug in their pattern matching example.
What happens if 'verb' is "GET" and 'path' is "/users/1234/posts/1234/extra_path/and/more/"? Will 'post_id' become "extra_path/and/more/"?
I tried running it in the sandbox, and it does indeed seem to buggily result in:
"Post ID: 1234/extra_path/and/more"
I suspect that the reason it is behaving like it is, is due to how it handles characters in the string literal. The example program exploits that only the slashes present in the string literal pattern are matched, to enable matching on 'page' having slashes. But then in the nested 'match', it forgot to account for any possible extra slashes.
Nitpicking end.
I have not read the whole post yet, but the pattern matching not requiring any allocations, seems very nice. The string literal patterns also seem interesting, though I am not completely sold on them, also as per the above possible bug. It seems really clean in some ways, but the specific semantics, I am not fully sure about. Maybe it is excellent, and is so clean and concise that it is overall less bug-prone than alternatives in other programming languages. I do not know.
Tangentially relates, but if any Roc devs are around I'm curious about the use cases for Roc.
It's supposed to be a scripting language right you embed into your C ABI right?
Do you see it competing with WASM for the plugin use case (i.e. a really large Roc platform)? Why would an app author prefer to expose a Roc layer to their app rather than a WASM layer? With a WASM layer, plugin devs can write in any language.
Another use case I've heard from it is as a more app-level language (i.e. a really small Roc platform). Do you see it competing with Gleam for server side http code? Do you see it competing with Elm for client side code?
Same question. I always like learning about languages I had not heard of, especially functional languages, so I was immediately curious what sorts of applications this might have. But after looking over the roc-lang.org website and the FAQ, I still don't know.
This piece would have been a lot more compelling if they had actually done science on selecting a language for compiler development. From what I can tell, they had an untested hypothesis that a low level systems language is necessary for a high performance compiler https://www.roc-lang.org/faq#self-hosted-compiler and from that concluded that their only choice besides rust was zig.
I know from experience that this initial assumption is wrong. Compiler performance is dominated by algorithms. The fastes managed languages tend to be at worst within a factor of two for wall time on any given algorithm. Algorithmic differences can be unbounded in their performance gaps. Zig itself is a perfect counterexample to the theory that writing a compiler in a low level systems language will lead to a fast compiler. Roc seems to compile at around 15k lines per second. That is not fast. There were evidently compilers written in ml that did 3k likes per second in 1998 https://flint.cs.yale.edu/cs421/case-for-ml.html
The zig rewrite of roc looks like the author's second compiler. Compiler and language design is a skill like any other and from my vantage point, they appear to have overcommitted to an initial design at the expense of developing their higher level design skills. In my opinion, the best thing they could do for the future of roc is stop working on their current compiler and use it to write a self hosting compiler for a much smaller subset of roc. They should be able to do that in less than 10k lines of code. They might even find that their self hosting compiler is faster than their zig based bootstrap compiler for the self hosted subset of roc. If the self hosting compiler is inadequate. Now they at least have identified a smaller useful subset of roc and can experiment with different compiler implementations in 10k likes of code rather than 300k lines of code. Then they could actually test the theory of whether or not a low level language is necessary to meet whatever arbitrary compiler performance goals they have.
By self hosting, they would also discover what roc features actually matter and they would spend much more time actually writing roc code. The features that are needed to write a self hosted compiler are all features that are generally useful. By improving the self hosted compiler, they also improve downstream programs.
Your comment is very assertive, but also doesn't offer much in the way of science.
Being able to compile ML quickly in the 90s tells you little about being able to compile Roc or some other language today because the language design enforces hard constraints on the algorithms necessary to compile it and the hardware today is much more complex. It's not hard to write a fast Pascal compiler that targets a 1980s chip with shallow pipelines. But that's not the problem being solved here.
I don't know much about Roc but it looks like it's got some amount of overloading and the linked article alludes to sophisticated algorithms to avoid heap allocating closures. Those can enforce algorithmic complexity in the compiler that is essential and can't be eliminated.
Once you're at the limits of algorithmic optimization, all that's left is reducing constant factors. I've written code in many languages in different performance regimes over the years and it's certainly the case that higher level languages, especially managed memory ones, put a hard floor in terms of how low you can go when optimizing to improve those constant factors.
I have seen in real-world code where explicit control over memory layout improved performance by more than an order of magnitude. I have friends in the game industry where much of their career is this kind of work. Those people would love to live in the luxurious world you describe where all they need to do is find a sufficiently clever algorithm and all of their performance problems will disappear.
Did you read the article at all?
Zig itself is an incredibly fast compiler. And the language and standard library is designed to support writing that compilers. And yeah, that’s all about algorithm and data structures. A large part of why struct-of-arrays is easy to do in Zig is because Andrew wanted it for making the compiler fast. The article also points out that it’s reusing code from the Zig compiler source as well.
Roc may not be super fast now, but that doesn’t mean they haven’t seen ahead and set themselves up for success.
Yeah, I agree there is value in self-hosting as you say. Zig itself is an example of that. But zig is a systems language. If Roc isn’t aiming for that nice it might not be the best choice for writing a compiler in. As an example in the other extreme end of language flavours: Python is still mainly CPython and efforts to make a python interpreter in some variant of python hasn’t been particularly successful.
> Compiler performance is dominated by algorithms
But you can always use the best algorithm no matter what your implementation language is, so it still makes sense to prefer a language that makes it easy to write fast code.
One thing I wish Rust would improve over time is the builds. Its one of the biggest sources of wasted storage space on all my computers, builds a ton of libraries can take tens of gigs, it adds up very quickly. Not sure what the best solution is, one I found is to set the global build folder so dependencies get reused across projects, but imho it should be an OOTB default behavior whatever the real solution should be.
We are trading away disk space for faster builds. We could make them faster in some cases by using even more...
On the other hand, it would be good to garbage collect those caches. We are wrapping up work on a new layout for intermediate build artifacts that will make it easier to GC them.
Sounds like what I was thinking, that or for third party deps to go into the same temp folder, and anything not accessed in over a week, or so just gets auto wiped by rustc / cargo build process?
Can you talk a little more about that? How would that work, and what would the developer experience be like when using it?
The proposal's at https://hackmd.io/@rust-cargo-team/SJT-p_rL2; progress is tracked at https://github.com/rust-lang/cargo/issues/12633.
I always got a kick out of that, coming from a JavaScript background where people constantly harp on the size of node modules.
My Tauri project, where the backend is much smaller code-wise than the frontend, has 9gb of rust artifacts (node_modules is 550mb for comparison)
Rust isn't great, and it shouldn't be a surprised since it's designed after npm. However one metric where nodes_modules is still worse for me is the sheer number of small files in it.
Having nearly one million files in nodes_modules isn't that unusual. The problem is that on most common file systems the minimum allocation is usually at least 4KB. So even if the actual data is less than 500MB, you end up with 4GB disk space used/wasted.
I wish ext4 had a feature to mark a file as "atomic" where it would allocate all atomic files in a long run, without room for expansion, and I suppose with very inefficient compaction upon deletion, but without any padding bytes.
A file “pointer” for byte exact files, pointer gets ditched for files that get updated or the pointer gets adjusted to another common file.
Use sccache.
Zig is a pre-1.0 language while Rust is post-1.0. This alone is settles which one to pick for may developers. The library support is probably favours Rust too. Rust build times are much slower than Zig, I get that, but I rarely optimize software for build times.
That’s totally reasonable. But while each minor version has breaking change, a minor version release itself is generally fairly stable and useable. There are solid production software which is already using Zig, like TigerBeetle and Ghostty/libghostty.
I am waiting for (near) 1.0 myself before doing anything major with it. But that’s mainly because I want at least the async IO stuff to be settled for my use cases.
Zig is not pre-1.0 because it’s not ready for production (bugs or missing features), it’s pre-1.0 because they want to be able to make breaking language changes.
Nowadays when you can just point an agent at release notes and have it update everything, I actually prefer not having to wait through rare major releases to get new language features.
> Nowadays when you can just point an agent at release notes and have it update everything
Except that means that not only you lose compiler bugfixes, you also pretty much has no access to the ecosystem. For most production codebases, this is a deal breaker.
> they want to be able to make breaking language changes
That sounds like it's not ready for production to me.
I invite you to read the release notes and see for yourself the types of breaking changes we’re talking about.
To me it is not much different from Lua, which despite being on 5.x for decades, makes breaking changes on minor releases (because it predates SemVer).
I also don’t see it being much different from any other language or language runtime that has a major release every year.
It’s fine to update at your own pace.
> I invite you to read the release notes and see for yourself the types of breaking changes we’re talking about.
I did, and I immediately found this in the latest release: https://ziglang.org/download/0.16.0/release-notes.html#IO-as...
That seems like it would require changing a lot of code. Calling it "production ready" is dishonest at best
Not really. It’s find and replace work.
If by production-ready you mean you can forever avoid changing code you wrote 8 months ago, sure, pick something else.
To me production-ready means it can be trusted to power production workloads, has all tooling I need, and has a consistent long-term vision. Zig ticks all the boxes.
Changing core features every few months doesn't seem very consistent...
> Zig is not pre-1.0 because it’s not ready for production (bugs or missing features), it’s pre-1.0 because they want to be able to make breaking language changes.
This is a solved problem in other projects. Either use the version numbers as intended and bump the major version number on breaking changes, or use Rust-style editions to opt in to the newer versions of the changes.
Calling a project production-ready but keeping the version number below 1.0 and saying breaking changes are expected is a tired game. We've seen it backfire across a number of language projects like Elm, where the exact same claim was used to both encourage people to use it and then blame them when it backfired.
If it's production ready, go to 1.0 and then follow semver for breaking changes. I don't care if we get to Zig v73.2.0 as a result. At least we can see from a glance which versions need to be checked for breaking changes.
languages ideally should not have breaking changes ever.
on the other hand, a language with frequent breaking changes should not be considered production ready.
people are of course free to live on the edge, and if someone decided that zig is good enough and they are not bothered by breaking changes then they are free to use it for their production system, but that doesn't mean it's ready for everyone. so i prefer the zig approach.
> languages ideally should not have breaking changes ever.
I disagree MIGHTILY. This is how you wind up with C++ and Java.
Languages need to be able to remove features to stay coherent. Occasionally, you get things wrong, it takes time to figure that out, and that's just the way life is.
different preferences i guess. i much prefer language stability. the idea that i should have to test my code with every python version out there for example is disturbing, but there are tools to do exactly that. they should not be needed.
The solution to this is editions/epochs, like Rust. You can set your project to the 2025 edition and be sure that, even if future versions introduce breaking changes, they will not affect your project; it will continue to compile as before.
pike has that. or had it. for two decades. inline in the code, in each file you could specify the version. they decided to drop it because of the maintenance overhead.
Counter Argument - with Zig it's perfectly viable to make programs using just the standard library. You have json serde, zon (zig object notation) serde, system functions (ie libc/rustix equivalent), random number generators... all without a single dependency.
The 35ms incremental rebuild is the part that sold me. I'd be curious to see the same benchmark on ARM once -fincremental gets there.
Zig team member here---obviously I can't say for sure yet, but I'm pretty confident the number will be basically identical. In the Zig compiler, incremental updates (rebuilds) have a small amount of overhead which is roughly proportional to the total size of the codebase (rather than just the amount of code which was changed). This comes from a) detecting which source files changed, and b) traversing a graph to figure out which declarations are referenced (necessary due to Zig's "lazy analysis" feature). But performance analysis reveals that for small updates, this overhead actually dominates the update time, by a lot. Of the 35ms, I would guess that under 5ms are actually spent rebuilding the function(s) that changed. Of that 5ms, code generation---the only thing which would really be different on AArch64---is an even smaller slice of the pie (it often doesn't even impact the overall time, since it runs in parallel with other parts of the pipeline, and those other parts are usually the bottleneck there). So even if the AArch64 backend was significantly slower (which, right now, is the opposite of what we expect---instruction selection and encoding for x86_64 is unusually complicated!), I wouldn't expect the number to change from 35ms.
Quite interesting the hand waving of security issues with Zig, oh well.
If I want to use allocator debuggers I already have the production ready tools that exist for C and C++ for at least 30 years.
Compilers are not security sensitive, usually. And while UB could theoretically poison the generated code, this isn't a bigger risk than logic bugs.
> Compilers are not security sensitive, usually.
The compiler is one of the most significant trust boundaries we have. Its decisions can intentionally or unintentionally create vulnerabilities in programs compiled by the compiler, which means that if you can compromise a compiler you can compromise everything downstream.
Unsafe memory access in a compiler can be exploited in order to hijack the compiler itself (this is reported regularly in production compilers), allowing the attacker to then insert arbitrary code into compiled binaries. Not everything that a compiler absorbs from its environment is meant to be treated as source to be compiled, and in a memory unsafe compiler any of that input can silently turn into machine code in the compiled binary if an attacker is able to exploit the memory safety bug and hijack the compiler.
Most compilers do not consider themselves a security boundary, and will not try to limit malicious code from hijacking the compiler. E.g. LLVM is explicitly not designed to handled malicious code (https://llvm.org/docs/Security.html#what-is-considered-a-sec...).
For this to be useful you would need to modify the compiler binary to make the exploit persistent. Otherwise why put an exploit for the compiler in the source to so that the compiler can put some malware in a binary, when you can put the malware in the source directly?
Because:
1. There are lots of things that compilers load into their memory that aren't actually source code. A memory exploit turns non-source data into executing-in-the-compiler code.
2. Depending on the language semantics, a memory exploit can allow substantially higher privilege than just being loaded as library code. Latent malicious code that never gets called into never becomes active, but if you can exploit a weakness in the compiler you can make your code execute at any time you'd like instead of relying on the main application calling in to your malicious library.
1. What would this be? 2. Good point, but for most purposes I think exploiting some aspect of the build system or the language that moves the exploit code or causes it to be executed before main would be easier than exploiting a memory issue in the compiler (I agree though in principle).
Of course they are, anything can be a gateway to inject backdoors, if security is not taken into account.
And as mentioned, if what Zig offers is already in Purify, there is hardly any added value over C and C++, without the headaches of a niche language.
Considering that you often run the code after you compile it, it might not matter. Anyway, like it or not, most compilers don't consider themselves security sensitive and will not consider malicious code that is able to hijack the compiler a security vulnerability.
Ken Thompson won the award for nothing, yeah.
Totally unrelated; The trusting trust attack was about downloading a malicious compiler because malicious code was injected into it in some early version.
(And if you ask me, it was indeed overrated, but that's unrelated).
That was one way the attack could be done, there are many other ways if the compiler binary can be open for manipulation.
Didn’t know Roc was still being worked on. I think it’s an interesting concept for a language that I personally haven’t seen elsewhere
What made you think it was no longer being worked on?
What concepts do you find interesting, compared to other FP languages?
Does this mean every time you find yourself using lots of “unsafe” Rust blocks, it’s not the right tool for the job? I suspect it’s not that simple, but what are people’s experience?
It really depends. For example, it might mean that you do not know the way to do the same thing, but in a safe manner. It might mean that you could refactor your code to do things more safely.
Of course, reasonable people may also believe that it is easier to use an unsafe language directly rather than change the ways that you code.
In my experience doing embedded, operating systems work, compiler work, and others, you never need a large amount of unsafe code. 1%/4% is really about it.
Makes sense. Thanks!
Compile times are a really underrated thing. My #1 gripe with C++ is waiting 10 minutes on a build, it absolutely kills flow.
My realization first time with Eclipse/Java.
I currently have a problem with Rust that I use Rust-Analyzer for syntax and autocomplete in VS Code and it has to run the compiler when you save a file to "refresh" things.
As you may imagine, this is insanely slow.
So slow that when I switch from a .rs file to a .ts file I feel like I switched computers.
> As discussed earlier, having full control over allocations and deallocations is what I want in our compiler's implementation. And in tests, I also appreciate the testing allocators detecting leaks—it can even detect leaks in compiled Roc code! Unfortunately, to get that benefit requires a lot of "init this, defer deinit" code in tests that has to be correct or else the test fails on a memory leak. None of that is necessary in Rust. I care more about the compiler's implementation being the way I want it than the tests looking nicer, but in a perfect world I could somehow have both.
haha, just for fun... do you want a C++ in your perfect world :D?
even though I don't use Zig - a few things make me excited.
such as new games that are gonna come - written in Zig - since its more ergonomic than C.
the other side is on the distributed software side of things - we have already seen it with TigerBeetle.
on my own end - probably robotics side.
Saw Zig in the headline and worried they were going the way of Bun. Turned out it was the opposite and made for a good read.
Nice. I like this - adds a different perspective without adding drama
Roc seems interesting. But for some reason I find it very grating to have the type definition on a separate line. Very much prefer F# syntax for that.
is this uno reverse for bun post of zig to rust port ?
> I enjoy Rust, I've taught a course on it, and I happily use it daily for my work at Zed. Despite what Internet comments might have us believe, it's extremely normal for one language to be the best fit for one project, while a different language turns out to be the best fit for a different project. One size does not actually fit all!
Amen. I like both Zig and Rust, and if I praise/criticise one of them, people act like I'm "switching", as if they were two exclusionary religions (though I think some people may indeed view them that way).
The stuff on memory safety written in the article is well worth reading, because in a lot of programmer discourse it's talked of as if it's some binary, that Rust Is Memory Safe, and Zig Is Not Memory safe. That's simply not the case.
Seems like Rust unsafe could be improved without changing the language design.
why not rewrite in ROC?? Would be much more cooler.
I think precious cognitive time should be spent more on the language itself rather than wasting it on rewrites.
The article links to https://www.roc-lang.org/faq#self-hosted-compiler to discuss this.
Why? Zig is clearly not memory safe like Rust.
I think there will be soon a wave of rewriting rust to language X coming up.
I've sort of thought this too. Rust adoption feels a lot like Haskell to me in that it's centered around ideological things that don't really improve the final product and arguably slow development. If your goal is to write a bunch of LLM code there are better languages than rust, and rust development is often regarded as too annoy for anyone not enthusiastic about the ideological improvements.
There are cool ideas in the language but I don't see it enduring over time.
The other way around.
Rust is also one of the best languages to use with AI.
I like Rust and I'm an advocate of Rust, but this really isn't true (at least it hasn't been and I doubt much has significantly changed).
The syntax complexity and the ecosystem haven't been ideal for LLM development. And there have been publications on findings of LLM efficacy with different languages. Rust is most often towards the lower end of efficiency/correctness when benchmarked.
https://arxiv.org/html/2508.09101v1
This paper uses models that are over a year old at this point. Many people didn't believe that LLMs were worth using for programming until 6 months after that, and now again this week we've had another huge leap in abilities.
This is beyond the other issues with the methodology of this study. For example, their Rust code was created by asking Deepseek to port their C++ code, not having it try and write Rust itself.
From what I am seeing in big corp, with low code/no code tooling, coupled with agentic orchestration, for many scenarios the actual programming language will become irrelevant.
Sure the programming platform still needs to be programming in something, but everything else on top will migrate to such tools.
This might not come to all corners of programming, but in the domain of orchestrating SaaS products, with MCP tools replacing classical microservices, it is getting there already in 2026.
While I'm a rust enthusiast, I do agree that certain languages lend themselves well to particular domains. So a rewrite from Rust to something better suited is fine by me. In fact, while I do work on a rust project, I would not have and still would not recommend it as the choice for that particular project.
That being said, I had to do some double takes while reading this.
> https://rtfeldman.com/rust-to-zig#memory-safety-post-rewrite
I feel that it's a bit weird to compare a rather well tested 7 (?) year old rust implementation with a brand new not yet released less than a year old Zig implementation. Without that context, this looks like a bad comparison for rust, when it is in fact the complete opposite.
> https://rtfeldman.com/rust-to-zig#build-times
The swiftness of the Zig compilere here is insane, and would would very much shift my recommendation of Rust if it got to similar speeds.
That being said, I do find it funny that currently, the compilation speed is actually worse on Zig than Rust, despite Zig (anonymous commenters at least tbf) claiming the opposite for years.
How did you eventually discover the 35 ms figure for Roc? Did you have to temporarily update the codebase to 0.17?
> https://rtfeldman.com/rust-to-zig#memory-control-zero-parse-...
Nothing negative here. I did play around with implementing a scripting language in this DOD-ish, index-based paradigm and yeah, it is neat.
I was thinking that it might be possible to do resumable computation across the network like this (in the context of frontend frameworks "resuming" UIs), but ultimately I have no use for this so just the experience itself was enough.
One note here is that it does tend to break completely if non-pointer-free data is introduced. It seems like it's either all or nothing.
> https://rtfeldman.com/rust-to-zig#ecosystem-relevance
This is more of an LLVM thing, which is fair, but I find it funny that "LLVM unstable bad" while "Zig unstable whatever".
Overall though, this was an interesting read. And if the folks contributing to roc like zig then more power to them.
Last thing, the link here is broken (points to a TODO):
> Zig's compiler itself is another
> In fact, while I do work on a rust project, I would not have and still would not recommend it as the choice for that particular project.
wondering what type of project is that? I think besides some very embedded projects with very little memory where you need C/assembly, rust is good enough for all kind of projects..
I work both on a pretty much bog-standard web (GraphQL) backend and the frontend that uses it. We switched over from Apollo on node to async-graphql on Rust.
The runtime performance is much better, but the compiler time performance is terrible. To be fair, this is mostly the fault of async-graphql, but that doesn't really matter all that much. For example, it's not uncommon for a single character SQL query change to trigger over a minute long incremental rebuild.
The rust compiler is just choking on the number of generics and codegenned functions.
I've personally looked at how to improve this, but short of breaking up the type graph using federation, nothing can help. Not even cranelift makes a noticeable dent.
Additionally, the team started off composed by a bunch of TypeScript/React/Node developers, so mistakes were made along the way.
Honestly, I would have recommended to just use C#.
That's not to say that I don't think Rust can work for web development. We have some (GraphQL-less) services where Rust is a great fit. Just maybe shouldn't have been the default. That or give up graphql ...
For whatever it's worth, I share some of your async-graphql woes, though I haven't investigated things deeply enough to have strong opinions about what to do instead.
> though (more on this later) and we ended up with about 1,200 uses of unsafe (out of our 300K lines of Rust code
Vs
>Rust code has a different source of memory-safety gaps: the unsafe sections that nearly every Rust program has somewhere in its dependencies. Unsafe Rust has all the memory unsafety risk of ReleaseFast Zig code, but none of the runtime checks to catch issues during development
Well if ReleaseFast would work for you then just write that in rust. The unsafe keyword can be used to write a shitshow, or it can be used to write small pieces of functionality, such as ArcUnion, that are safe to use. I don’t agree that every application has to have any unsafe code. If you find yourself needing it, then you go build the abstraction you need in another crate, miri the fuck out of that, and then just consume it in your application.
If you’re fine with the shitshow, then use zig, because that’ll improve rusts stats.
If you think you’re in the magical “I know what I’m doing and I need the extra performance” then you probably don’t know what you’re doing. You’re young. Knock yourself out. I’ve written assembly language hackery for video games you wouldn’t believe. Would I use any of that for a language others are going to use? You’re not that smart. And if you are, someone else on your project isn’t. Just a matter of time.
Can anybody explain to me why anthropic bought bun in the first place ?
Nobody can.
"Claude Code uses Bun" is the reason that's given. But even though Anthropic tells us that "coding is solved", instead of rewriting Claude Code in something other than JS, they bought a company that made a JS runtime and then did a mass rewrite of that JS runtime.
Too much money, not enough ideas what to do with it.
Claude Code uses bun.
Yeah I get they use it but I don't understand why you would buy it. it's just the runtime for code that makes up the agent.
Bun was a startup. Startups can go out of business, and then you are now scrambling to move your code to something else. They could also be bought by someone who has different priorities regarding future development than you, and that's also a risk.
The simplest solution to these problems, if you have the capital, is to buy them.
I get that but there is always node.js
They aren't exactly the same thing, moving to node would be possible but bun does more than node, so you need to replace the whole thing.
Oh for sure. I guess its just as much a Acqui-hire as well as for the tech.
Irrespective to the technical merits of both language, moving from a stable language to a pre-1.0 one that just lost his most popular open source project is a wild move.
> that just lost his most popular open source project
As they state in the article, they started the migration a year and a half ago, something that happened a few weeks back would never come into the decision making process.
I don't even know what Zig is but I've seen this topic come up so many times on this site that I'm starting to think the people who are actually doing this are unsure themselves whether it's a good idea or not.
Basically the security model of Modula-2 or Object Pascal, with a curly brackets syntax, and compile time execution.
Some folks embrace it as some kind of novelty.
It does seem like they're trying to convince themselves. If you like Zig, that's a good enough reason to use it. Why waste time on language tribalism?
I have the same issue with "use the right tool" rhetoric. The right tool is the one that does the job and that you know best.