Overall cyclomatic complexity is a useful metric, but it does have one shortcoming when used with modern languages: it was invented before polymorphism really became a thing.
That means that it really only counts explicit branching. So, for example, in an OO language like C#, calling a virtual method doesn’t increment cyclomatic complexity even though the method invocation could go down many code paths. Potentially thousands if you’re dealing with a common interface like IEnumerable. If you’re working on a library then the number of potential code paths in this kind of situation is unbounded.
As an aside, it’s interesting to think how it might apply to a language like Smalltalk that doesn’t even have if or switch statements.
OO isn’t the only monkey wrench, either. Higher-order functions also introduce forms of branching that cyclomatic complexity doesn’t measure.
Again that doesn’t make it a useless metric. Just don’t think that a cyclomatic complexity limit in your codebase is some sort of maintainability panacea. Some of the least comprehensible functions I’ve deciphered had quite low cyclomatic complexities.
It also depends how leakproof the abstractions are. Goto statement and if-statement might both compile to similar jump instructions in assembly, but it's abstracted away. Similarly dynamic dispatch can abstract the branching so it can be ignored.
In reading the article, it did feel somewhat wrong in a way that I couldn't quite describe. But after reading this comment, maybe it just seems old and maybe obsolete.
Taking the example provided in the article, I don't feel like the new code is meaningfully less complex. In fact, since it added some additional indirection, I could argue it's slightly more complex.
The core code with the nested if statements is something that I would probably refactor in some other way entirely. Maybe by taking advantage of other language features. It is a toy example so it's hard to say but that's part that feels like it needs simplification and untouched in this example.
Agreed. I also don’t really agree that the refactored code is any easier to test. ProcessOrder’s behavior hasn’t changed, only its implementation details, so the minimal test surface is the same for both: just test ProcessOrder.
You could additionally test the three helper functions. But the original tests against ProcessOrder would still be needed for completeness, so they wouldn’t necessarily add much except in an Uncle Bob style, “He who dies with the largest burden of gratuitous micro-tests wins,” sort of way.
Now if I really wanted to make that code easier to test, I’d instead be looking into ways to make the whole thing less stateful. Temporal coupling is much more confusing than if statements.
> Some of the least comprehensible functions I’ve deciphered had quite low cyclomatic complexities.
In agreement with your post, this research that measures cognitive load via EEG and time spent shows that the metrics we use for complexity and readability are only partial matches to what is going on:
https://pmc.ncbi.nlm.nih.gov/articles/PMC9942489/
I don't disagree that functions with many execution paths are harder to read, but I do believe that the solution given here; 'just split it into multiple functions' frequently makes a function even harder to read. It forces you to scroll back and forth between code.
I think generally when you run into something like this, the better way to handle it is to sit back, and reconsider how your overall handling is designed.
Fun story: at my previous aaaawful company CC was discovered as a thing to care about at about the same time as PMs and managers were encouraged to land code changes using the _then_ quite terrible AI tooling (this was a year or two ago). Cue an avalanche of completely unreviewable diffs.
I made CC part of my deterministic quality gate[0] for agents. Not sure how much of it is placebo, but overall I've seen the gate catch a lot of cases where the agent strayed from the constraints of the particular project.
FWIW In my setup I have just the one coding agent, and umpteen others for support and/or testing. I still do a human testing pass on everything though, if only to find errors in the testing process.
From a security perspective cc is highly relevant. I use it to get a solid rating of the security aspects of Python code. I use [1] which is solid and proven.
Worst things happen always when 2 or more systems are combined because each system might be simple on its own, yet a combination is always much more complex.
For something the the prior statement it is never a weird question to ask of there actually evidence of this or just it seems like it should be true so we believe it.
There are tons of things that seem like they would obviously be true, but it turns out they aren't.
System is more complex based on possible states it can have or inputs/outputs.
That is just maths here working. Two systems combined always will have more states and inputs/outputs.
There is nothing to check here as it can be proven purely by maths.
Complex systems having more attack surface are obviously less secure.
They might be less interesting for attackers if they have to scan huge attack surface like IPv6 vs IPv4 but no one is claiming IPv6 network is more secure.
It’s not obvious to me because cyclomatic complexity is not a straightforward proxy for the number of systems that are being combined.
It’s also the case that some of the most common sources of vulnerabilities, such as SQL injection, introduce no additional cyclomatic complexity. Heck, buffer overflows are good for your cyclomatic complexity - those array bounds checks are all extra branches.
Buffer overflow checks are really only going to be a linear growth in CC. It's when things move towards exponential growth or higher that it gets really easy to introduce flaws of many kinds.
Now, it's probably not a direct correlation. I'd think security bugs are more likely from programmers that unintentionally raise CC without really realizing it. Aka, overreaching their own knowledge when simpler structures are avaliable.
Sure. It’s just that there’s also so much research that has found that cyclomatic complexity is theoretically ill-founded, and that it tends to underperform other ways of measuring complexity. Most notably, just counting lines of code. (Not per function, in total.)
I'd argue that assuming something is obvious without any empirical validation is the root of a huge number of misconceptions that humanity has historically had. There's a reason science suddenly started moving a lot faster after we moved past Aristotle and started measuring things in experiments.
Anyone using tools like ndepend or others to help guide agents in refactors?
Personally I have a some tools that build dependency graphs (C# and Python) and store the results in a local database. Agents seem quite good at poking at this and coming up with refactor ideas. Graph analysis tools are useful here, simple application will detect cyclical dependencies, but I encourage the agents to use more complex tools like clustering to poke at the data.
I've been feeding agents dependency graphs plus CC and coverage data from a local store, and it works well for spotting cyclical deps and high-CC hotspots
Overall cyclomatic complexity is a useful metric, but it does have one shortcoming when used with modern languages: it was invented before polymorphism really became a thing.
That means that it really only counts explicit branching. So, for example, in an OO language like C#, calling a virtual method doesn’t increment cyclomatic complexity even though the method invocation could go down many code paths. Potentially thousands if you’re dealing with a common interface like IEnumerable. If you’re working on a library then the number of potential code paths in this kind of situation is unbounded.
As an aside, it’s interesting to think how it might apply to a language like Smalltalk that doesn’t even have if or switch statements.
OO isn’t the only monkey wrench, either. Higher-order functions also introduce forms of branching that cyclomatic complexity doesn’t measure.
Again that doesn’t make it a useless metric. Just don’t think that a cyclomatic complexity limit in your codebase is some sort of maintainability panacea. Some of the least comprehensible functions I’ve deciphered had quite low cyclomatic complexities.
It also depends how leakproof the abstractions are. Goto statement and if-statement might both compile to similar jump instructions in assembly, but it's abstracted away. Similarly dynamic dispatch can abstract the branching so it can be ignored.
In reading the article, it did feel somewhat wrong in a way that I couldn't quite describe. But after reading this comment, maybe it just seems old and maybe obsolete.
Taking the example provided in the article, I don't feel like the new code is meaningfully less complex. In fact, since it added some additional indirection, I could argue it's slightly more complex.
The core code with the nested if statements is something that I would probably refactor in some other way entirely. Maybe by taking advantage of other language features. It is a toy example so it's hard to say but that's part that feels like it needs simplification and untouched in this example.
Agreed. I also don’t really agree that the refactored code is any easier to test. ProcessOrder’s behavior hasn’t changed, only its implementation details, so the minimal test surface is the same for both: just test ProcessOrder.
You could additionally test the three helper functions. But the original tests against ProcessOrder would still be needed for completeness, so they wouldn’t necessarily add much except in an Uncle Bob style, “He who dies with the largest burden of gratuitous micro-tests wins,” sort of way.
Now if I really wanted to make that code easier to test, I’d instead be looking into ways to make the whole thing less stateful. Temporal coupling is much more confusing than if statements.
> Some of the least comprehensible functions I’ve deciphered had quite low cyclomatic complexities.
In agreement with your post, this research that measures cognitive load via EEG and time spent shows that the metrics we use for complexity and readability are only partial matches to what is going on: https://pmc.ncbi.nlm.nih.gov/articles/PMC9942489/
I don't disagree that functions with many execution paths are harder to read, but I do believe that the solution given here; 'just split it into multiple functions' frequently makes a function even harder to read. It forces you to scroll back and forth between code.
I think generally when you run into something like this, the better way to handle it is to sit back, and reconsider how your overall handling is designed.
Fun story: at my previous aaaawful company CC was discovered as a thing to care about at about the same time as PMs and managers were encouraged to land code changes using the _then_ quite terrible AI tooling (this was a year or two ago). Cue an avalanche of completely unreviewable diffs.
I made CC part of my deterministic quality gate[0] for agents. Not sure how much of it is placebo, but overall I've seen the gate catch a lot of cases where the agent strayed from the constraints of the particular project.
[0]: https://github.com/ivmirx/agentic-quality-loop
FWIW In my setup I have just the one coding agent, and umpteen others for support and/or testing. I still do a human testing pass on everything though, if only to find errors in the testing process.
From a security perspective cc is highly relevant. I use it to get a solid rating of the security aspects of Python code. I use [1] which is solid and proven.
[1] https://nocomplexity.com/documents/codeaudit/complexitycheck...
Is there research that show if and how much a low complexity improves security?
Weird question to ask, that is pretty obvious.
Worst things happen always when 2 or more systems are combined because each system might be simple on its own, yet a combination is always much more complex.
> Weird question to ask, that is pretty obvious.
For something the the prior statement it is never a weird question to ask of there actually evidence of this or just it seems like it should be true so we believe it.
There are tons of things that seem like they would obviously be true, but it turns out they aren't.
System is more complex based on possible states it can have or inputs/outputs.
That is just maths here working. Two systems combined always will have more states and inputs/outputs.
There is nothing to check here as it can be proven purely by maths.
Complex systems having more attack surface are obviously less secure.
They might be less interesting for attackers if they have to scan huge attack surface like IPv6 vs IPv4 but no one is claiming IPv6 network is more secure.
No. Watch the simple made easy talk.
Just more I/O isn’t more complicated nor a bigger risk. More entanglement is more complicated.
It’s not obvious to me because cyclomatic complexity is not a straightforward proxy for the number of systems that are being combined.
It’s also the case that some of the most common sources of vulnerabilities, such as SQL injection, introduce no additional cyclomatic complexity. Heck, buffer overflows are good for your cyclomatic complexity - those array bounds checks are all extra branches.
Buffer overflow checks are really only going to be a linear growth in CC. It's when things move towards exponential growth or higher that it gets really easy to introduce flaws of many kinds.
Now, it's probably not a direct correlation. I'd think security bugs are more likely from programmers that unintentionally raise CC without really realizing it. Aka, overreaching their own knowledge when simpler structures are avaliable.
Sure. It’s just that there’s also so much research that has found that cyclomatic complexity is theoretically ill-founded, and that it tends to underperform other ways of measuring complexity. Most notably, just counting lines of code. (Not per function, in total.)
Here’s an oldie but goodie: https://cs.du.edu/~snarayan/sada/teaching/COMP3705/lecture/p...
I’ve personally had better success thinking of it as more of a measure of readability than of quality.
I'd argue that assuming something is obvious without any empirical validation is the root of a huge number of misconceptions that humanity has historically had. There's a reason science suddenly started moving a lot faster after we moved past Aristotle and started measuring things in experiments.
Anyone using tools like ndepend or others to help guide agents in refactors?
Personally I have a some tools that build dependency graphs (C# and Python) and store the results in a local database. Agents seem quite good at poking at this and coming up with refactor ideas. Graph analysis tools are useful here, simple application will detect cyclical dependencies, but I encourage the agents to use more complex tools like clustering to poke at the data.
I've been feeding agents dependency graphs plus CC and coverage data from a local store, and it works well for spotting cyclical deps and high-CC hotspots