Am reminded reading this of an esteemed and since passed away colleague who had written windows driver code since the dos days and may have had decades of insanely archaic knowledge die with him - when working on a difficult piece of windows driver code years ago, he said to me in a thick eastern europe accent as best i can remember “you make the primary mistake of thinking anything in windows makes sense. once you abandon this bias, you may someday hope to get where i am”
If there's anything I've learned from reading Raymond's blogs, it's that Windows internals actually make a fair bit of sense. Every weird thing I've ever seen him address turns out to be something that has had real thought and engineering put into it, and if the solution is ugly that seems to be more due to external constraints placed upon the engineers than it is the engineers doing weird stuff shooting from the hip.
This seems adjacent to Chesterton's Fence, though maybe not the canonical form of it.
For anyone not familiar with the term, Chesterton's Fence is the idea that you should understand why a rule exists before trying to remove it or work around it: https://fs.blog/chestertons-fence/
Here the issue is not that the rule was removed, but that the code followed the wording while missing the reason the rule existed.
2. Understand why that rule is in place before proceeding
This article deals with the second part, but not the first. So it is only about about half of Chesterton's fence at best.
In these examples, a rule (avoid blocking calls) is in place to guide the programmer to a performant system. Programmers apparently thought that if they found a way to avoid directly blocking calls, but managed to indirectly block, they had still obeyed the rule. And strictly by the most narrow reading of the rule they had obeyed it. But they had defeated the purpose of the rule.
So definitely Chesterton's Fence adjacent, but not Chesterton's Fence itself.
It's not about Chesterton's Fence, though I thought so at first due to the title wording. The developers Raymond is referring to actually weren't trying to circumvent a rule. They followed the rules to the letter, but whoever made the rule didn't adequately document the spirit of the rule. So when the letter of the rule turned out to be insufficient, the devs couldn't really know that their solution violated the intent behind the rule.
I feel like a lot of documentation falls into this kind of style, where the motivation behind design is not communicated, just the design itself. Sometimes it's because the documentation writer doesn't want to leak internal details to the end user (closed source libraries are an especially bad source of this). Sometimes it's that the writer is too close to the project, and is struck by the curse of knowledge (can't properly identify which details are actually self-evident, and which are only obvious because they already know them).
Another example of why technical writing is difficult, I think.
I worked on a team where the responsibility of maintaining the onboarding guide was always passed to whoever joined right after they finished going through it (with the current maintainer being the "onboarding buddy" of the new team member and passing the torch at the end). The rationale was that the longest-tenured members of the team would be the most likely to accidentally rely on implicit knowledge when updating it and forget to put in something that wouldn't be obvious to a newcomer, and the person who had needed to learn the processes from scratch most recently would be the most likely to notice gaps like that (and could reach out to the rest of the team to have the gaps explained to them so they could fill them in).
I always liked that mindset, and it helped teach me the important lesson that sometimes being the person who asks very basic questions because I don't understand things can make me valuable to the teams I work on. There are certainly times when the answer makes me go "oh, duh!", and I feel a little sheepish, but the feeling doesn't last very long, and no one has ever held it against me, whereas the times when it leads somewhere interesting and potentially to better documentation for people coming after me are far more frequent and memorable to everyone involved.
This isn't to say that the burden should fall on the ones who are reading the documentation rather than writing it, but to encourage people who are in the position of being frustrated and confused to take advantage of those moments, because they can make you very valuable to your team in addition to helping you learn. If you're on a team that operates in good faith, the burden for documenting things well can be shared, and in the long run it will matter less who's job it is to keep it updated and more about whether everyone is contributing however they can to maintain the quality (and if you're on a team that operates in bad faith, you have my permission to keep quiet and do whatever you can to get through that experience, not that you need it from me!)
There is this famous experiment with 9 monkeys in a room with a banana attached to the ceiling and a scale.
First day, a monkey climbs the scale, gets the banana and is happy.
Second day, they start spraying whomever gets on the scale. Monkeys hate this. They learn not to climb.
Third day, they take a monkey out and replace with another. The new monkey sees a banana up there and tries climbing the scale. He literally gets beaten out by the others, like "seems like you're new here".
Days 4-12, they've replaced one monkey per day, so that no monkey was here when it was possible to get the banana. None of them have ever been sprayed either. Still, they enforce the rule not to climb up there.
I am putting this example because in our society as well, there are many rules that are enforced without anyone questioning the "why". Yet the "why" is often more important to know than the rule itself.
Designers know this dichotomy between the "why" and the "how". Most people don't.
I have more than once in life realized that one of the two contradictory lessons was correct for this situation, but in a different one the other was correct...
It's funny how this monkey experiment is often trotted out as "people blindly follow the rules without knowing why," when the rule learned by the monkeys is a really good rule (it prevents a malevolent entity beyond your understanding from attacking you!), and the only reason the monkeys don't understand the "why" at the end of the experiment is because they don't have language.
The lessons I'd take away from the experiment would be 1) be sure to tell people why the rules exist, but also 2) follow the rules even if there's no apparent reason for them, otherwise you might get smacked down by some unimaginably powerful entity you're barely even aware of.
I really wish Windows had a way to show what kernel callbacks are registered and execute on each call. Even better would be to have actual timing data on these callback. Using that one could much more easily to debug and potentially uninstall misbehaving software.
You may already know about it, but for those that aren't aware of it, Windows' Event Tracing system can give a pretty deep view into identifying misbehaving software. It probably doesn't cover everything you wish you could track, and not quite with the level of directness you're asking for, but it's helped me track down weird slowdowns and hangs on my computer on a few different occasions.
There's a third party tool for collecting and examining such traces easier called UIforETW
Microsoft is so broken that an employee finds it easier to write a blog post about a documentation improvement than simply making that improvement? Explains a lot. "Conway's Flaw?"
Raymond's important and high up enough that he probably only needs the approval of three AIs and five managers to publish a blog - updating documentation likely needs twice that.
> The documentation should open with something like this:
>> The callback function must perform its work quickly without blocking. If you need to do complex work or synchronize with other threads or processes, do the work asynchronously, such as by using System Worker Threads.
A change was made, but not the change that Raymond thinks would explain why the list is there anyway.
Am reminded reading this of an esteemed and since passed away colleague who had written windows driver code since the dos days and may have had decades of insanely archaic knowledge die with him - when working on a difficult piece of windows driver code years ago, he said to me in a thick eastern europe accent as best i can remember “you make the primary mistake of thinking anything in windows makes sense. once you abandon this bias, you may someday hope to get where i am”
If there's anything I've learned from reading Raymond's blogs, it's that Windows internals actually make a fair bit of sense. Every weird thing I've ever seen him address turns out to be something that has had real thought and engineering put into it, and if the solution is ugly that seems to be more due to external constraints placed upon the engineers than it is the engineers doing weird stuff shooting from the hip.
We all knew this kind of guys. Bless them
This seems adjacent to Chesterton's Fence, though maybe not the canonical form of it.
For anyone not familiar with the term, Chesterton's Fence is the idea that you should understand why a rule exists before trying to remove it or work around it: https://fs.blog/chestertons-fence/
Here the issue is not that the rule was removed, but that the code followed the wording while missing the reason the rule existed.
Agreed that on first blush it looks like Chesterton's Fence but it may be more of a cargo culting situation.
Came here to say this. How can you write an entire article about Chesterton’s Fence without mentioning Chesterton’s Fence?!
Chesterton's fence has two parts:
1. I want to remove a rule
2. Understand why that rule is in place before proceeding
This article deals with the second part, but not the first. So it is only about about half of Chesterton's fence at best.
In these examples, a rule (avoid blocking calls) is in place to guide the programmer to a performant system. Programmers apparently thought that if they found a way to avoid directly blocking calls, but managed to indirectly block, they had still obeyed the rule. And strictly by the most narrow reading of the rule they had obeyed it. But they had defeated the purpose of the rule.
So definitely Chesterton's Fence adjacent, but not Chesterton's Fence itself.
It's not about Chesterton's Fence, though I thought so at first due to the title wording. The developers Raymond is referring to actually weren't trying to circumvent a rule. They followed the rules to the letter, but whoever made the rule didn't adequately document the spirit of the rule. So when the letter of the rule turned out to be insufficient, the devs couldn't really know that their solution violated the intent behind the rule.
So we can call this one Chen's Fence
I feel like a lot of documentation falls into this kind of style, where the motivation behind design is not communicated, just the design itself. Sometimes it's because the documentation writer doesn't want to leak internal details to the end user (closed source libraries are an especially bad source of this). Sometimes it's that the writer is too close to the project, and is struck by the curse of knowledge (can't properly identify which details are actually self-evident, and which are only obvious because they already know them).
Another example of why technical writing is difficult, I think.
I worked on a team where the responsibility of maintaining the onboarding guide was always passed to whoever joined right after they finished going through it (with the current maintainer being the "onboarding buddy" of the new team member and passing the torch at the end). The rationale was that the longest-tenured members of the team would be the most likely to accidentally rely on implicit knowledge when updating it and forget to put in something that wouldn't be obvious to a newcomer, and the person who had needed to learn the processes from scratch most recently would be the most likely to notice gaps like that (and could reach out to the rest of the team to have the gaps explained to them so they could fill them in).
I always liked that mindset, and it helped teach me the important lesson that sometimes being the person who asks very basic questions because I don't understand things can make me valuable to the teams I work on. There are certainly times when the answer makes me go "oh, duh!", and I feel a little sheepish, but the feeling doesn't last very long, and no one has ever held it against me, whereas the times when it leads somewhere interesting and potentially to better documentation for people coming after me are far more frequent and memorable to everyone involved.
This isn't to say that the burden should fall on the ones who are reading the documentation rather than writing it, but to encourage people who are in the position of being frustrated and confused to take advantage of those moments, because they can make you very valuable to your team in addition to helping you learn. If you're on a team that operates in good faith, the burden for documenting things well can be shared, and in the long run it will matter less who's job it is to keep it updated and more about whether everyone is contributing however they can to maintain the quality (and if you're on a team that operates in bad faith, you have my permission to keep quiet and do whatever you can to get through that experience, not that you need it from me!)
There is this famous experiment with 9 monkeys in a room with a banana attached to the ceiling and a scale.
First day, a monkey climbs the scale, gets the banana and is happy.
Second day, they start spraying whomever gets on the scale. Monkeys hate this. They learn not to climb.
Third day, they take a monkey out and replace with another. The new monkey sees a banana up there and tries climbing the scale. He literally gets beaten out by the others, like "seems like you're new here".
Days 4-12, they've replaced one monkey per day, so that no monkey was here when it was possible to get the banana. None of them have ever been sprayed either. Still, they enforce the rule not to climb up there.
I am putting this example because in our society as well, there are many rules that are enforced without anyone questioning the "why". Yet the "why" is often more important to know than the rule itself.
Designers know this dichotomy between the "why" and the "how". Most people don't.
I've heard that story. I've yet to see evidence it actually happened though. I don't the experiment with pass a modern ethics panel either.
I stand corrected: https://blogs.cornell.edu/info2040/2012/11/03/monkeys-and-in...
parables that didn't happen can still be useful
The parable-replication crisis is real though
They can be, but you need to take care as sometimes they are misleading and can even drive wrong decisions.
Aesop wrote a lot of fables that had contradictory lessons.
I have more than once in life realized that one of the two contradictory lessons was correct for this situation, but in a different one the other was correct...
There's a meta-lesson there.
It's funny how this monkey experiment is often trotted out as "people blindly follow the rules without knowing why," when the rule learned by the monkeys is a really good rule (it prevents a malevolent entity beyond your understanding from attacking you!), and the only reason the monkeys don't understand the "why" at the end of the experiment is because they don't have language.
The lessons I'd take away from the experiment would be 1) be sure to tell people why the rules exist, but also 2) follow the rules even if there's no apparent reason for them, otherwise you might get smacked down by some unimaginably powerful entity you're barely even aware of.
I really wish Windows had a way to show what kernel callbacks are registered and execute on each call. Even better would be to have actual timing data on these callback. Using that one could much more easily to debug and potentially uninstall misbehaving software.
You may already know about it, but for those that aren't aware of it, Windows' Event Tracing system can give a pretty deep view into identifying misbehaving software. It probably doesn't cover everything you wish you could track, and not quite with the level of directness you're asking for, but it's helped me track down weird slowdowns and hangs on my computer on a few different occasions.
There's a third party tool for collecting and examining such traces easier called UIforETW
https://randomascii.wordpress.com/2015/09/24/etw-central/
How many of these winapi callbacks should have been just messages?
Microsoft is so broken that an employee finds it easier to write a blog post about a documentation improvement than simply making that improvement? Explains a lot. "Conway's Flaw?"
Raymond's important and high up enough that he probably only needs the approval of three AIs and five managers to publish a blog - updating documentation likely needs twice that.
the blog says the documentation improvement was made in 2020? presumably not by the post author either
From the article:
> The documentation should open with something like this:
>> The callback function must perform its work quickly without blocking. If you need to do complex work or synchronize with other threads or processes, do the work asynchronously, such as by using System Worker Threads.
A change was made, but not the change that Raymond thinks would explain why the list is there anyway.
He used his blog to queue the work to be executed asynchronously by another MS worker that reads it
And, since he isn't waiting on the reply, he is following the rule for the callback.
Always been that way
of course they had to add dont wait for your worker thread in 2020