> you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
This has never happened and never will. You simply are not omniscient. Even if you're smart enough to figure everything out the requirements will change underneath you.
But I do still think there's a lot of value into coming up with a good plan before jumping in. A lot of software people like to jump in and I see them portray the planning people as trying to figure everything out first. (I wonder if we reinforce the jumping in head first mentality because people figure out you can't plan everything) A good plan helps you prevent changing specs and prepares you for hiccups. It helps by having others but basically all you do is try to think of all the things that could go wrong. Write them down. Triage. If needed, elevate questions to the decision makers. Try a few small scale tests. Then build out. But building out you're always going to find things you didn't see. You can't plan forever because you'll never solve the unknown unknowns until you build, but also good prep makes for smoother processes. It's the reason engineers do the math before they build a bridge. Not because the math is a perfect representation and things won't change (despite common belief, it's not static) but because the plan is cheaper than the build and having a plan allows you to better track changes and helps you determine how off the rails you've gone.
It is also perplexing to me that people think they can just plan everything out and give it to a LLMs. Do you really believe your manager knows everything that needs to be done when they assign jobs to you? Of course not, they couldn't. Half the job is figuring out what the actual requirements are.
>> you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
> This has never happened and never will. You simply are not omniscient. Even if you're smart enough to figure everything out the requirements will change underneath you.
I am one of those "battle-scarred twenty-year+ vets" mentioned in the article, currently working on a large project for a multinational company that requires everything to be specified up-front, planned on JIRA, estimates provided and Gantt charts setup before they even sign the contract for the next milestone.
I've worked on this project for 18 months, and I can count on zero hands the times a milestone hasn't gone off the rails due to unforeseen problems, last-minute changes and incomplete specifications. It has been an growing headache for the engineers that have to deliver within these rigid structures, and it's now got to the point that management itself has noticed and is trying to convince the big bosses we need a more agile and iterative approach.
Anyone who claims upfront specs are the solution to all the complexity of software either has no real world experience, or is so far removed from actual engineering they just don't know what they're talking about.
Working on a project for 18 months doesn't give you enough insight into it to know what is good or now about it. You need several more year before you can usefully figure out what changes will help get you to make milestones. (other than trivially obvious things, which might be the low handing fruit - but sometimes they are the better way to do things but the real problem makes that stand out instead).
Nothing will get you to hit every milestone. However you can make progress if you have years of experience in that project and the company is willing to invest in the needed time to make things better (they rarely are)
> A lot of software people like to jump in and I see them portray the planning people as trying to figure everything out first.
My approach, especially for a project with a lot of unknowns, is usually to jump in right away and try to build a prototype. Then iterate a few times. If it's a small enough thing, a few iterations is enough to have a good result.
If it's something bigger, this is the point where it's worth doing some planning, as many of the problems have already been surfaced, and the problem is much better understood.
Yes it did, however it never works in pratice when it comes to integration testing two years later after the 2000 pages specification document was written, and passed down from the architects to the devs.
2000 page specification documents are rarely useful (if ever?).
You need smaller documents - this is the core technology we are using. This is how one subsystem is designed - often this should be on a whiteboard because once you get into the implementation details you need to change the plan, but the planning was useful. This is how to use core parts of the system so new comers can start working quick.
You need disciple to accept that sometimes libfoo is the best way to solve a problem in isolation, but since libbar is used elsewhere and can solve the problem your local problem will use libbar despite making your local problem uglier. Have a small set of core technologies that everyone knows and uses is sometimes more valuable than using the best tool for the job - but only sometimes.
> There should be some balanced path in the middle somewhere, but I haven’t stumbled across a formal version of it after all these decades.
That's very simple. The balanced path depends directly on how much of the requirements and assumptions are going to change during the life time of the thing you are building.
Engineering is helpful only to the extent you can forsee the future changes. Anything beyond that requires evolution.
You are able to comment on the complexity of that large company only because you are standing in the future into 50 years from when those things started take shape. If you were designing it 50 years back, you would end up with same complexity.
The nature's answer to it is, consolidate and compact. Everything that falls onto earth gets compacted into a solid rock over time, by a huge pressure of weight. All complexity and features are flattened out. Companies undergo similar dynamics driven by pressures over time, not by big-bang engineering design upfront.
“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”
Gall’s Law
I think the important part here is "from scratch". Typically when you're designing a new (second, third, whatever) system to replace the old one you actually take the good and the bad parts of the previous design into account, so it's no longer from scratch. That's what allows it to succeed (at least in my experience it usually did).
These days software has been done a lot. You should be able to find others who have done similar things and learn lessons from them. Considering microservices - there are lots of people who have done them and can tell you what worked well and what didn't. Considering using QT - lots of others have and can give you ideas. Considering writing your own framework - there are lots of others: look at what they do good and bad.
If you are doing a CRUD web app for a local small business - there are thousands of examples. If you are writing control software for a space station - you may not have access to code from NASA/Russia/China but you can at least look at generic software that does the things you need and learn some lessons.
This is often quoted, but I wonder whether it's actually strictly true, at least if you keep to a reasonable definition of "works". It's certainly not true in mechanical engineering.
The definition of a complex system is the qualifier for the quote. Many systems that are designed, implemented and found working are not complex systems. They may be complicated systems. To paraphrase Dr. Richard I. Cook’s ”How Complex Systems Fail” where he claims that complex systems are inherently hazardous, operate near the edge of failure and cannot be understood by analyzing individual components. These systems are not just complicated (like a machine with fixed parts) but dynamic, constantly evolving, and prone to multiple, coincidental failures.
A system of services that interact, where many of them are depending on each other in informal ways may be a complex system. Especially if humans are also involved.
Such a system is not something you design. You just happen to find yourself in it. Like the road to hell, the road to a complex system is paved with good intentions.
Then what precisely is the definition of complex? If "complex" just means "not designed", then the original quote that complex systems can't be designed is true but circular.
If the definition of "complex" is instead something more like "a system of services that interact", "prone to multiple, coincidental failures", then I don't think it's impossible to design them. It's just very hard. Manufacturing lines would be examples, they are certainly designed.
The manufacturing lines would be designed, and they'd be designed in an attempt to affect the "design" of the ultimate resulting supply chain they're a part of. But the relationship between the design of some lines and the behavior of the larger supply chain is non-linear, hard to predict, and ultimately undesigned, and therefore complex.
The design of the manufacturing lines and the resulting supply chain are not independent of each other -- you can trace features from one to the other -- but you cannot take apart the supply chain and analyze the designs of its constituent manufacturing lines and actually predict the behavior of the larger system.
AFAIK there's not a great definition of a complex system, just a set of traits that tend to indicate you're looking at one. Non-linearity, feedbacks, lack of predictability, resistance to analysis (the "you can't take it apart to reason about the whole" characteristic mentioned above"). All of these traits are also kind of the same things... they tend to come bundled with one another.
IMHO, the key is where you add complexity. In software you have different abstraction layers. If you make a layer too fat, it becomes unwieldly. A simple system evolves well if you're adding the complexity in the right layer, avoiding making a layer responsible for task outside its scope. It still "works" if you don't, but it's increasingly difficult to maintain it.
The law is maybe a little too simplistic in its formulation, but it's fundamentally true.
You built this gear using the knowledge from your last gear. You didn't start with no knowledge, read a manual on operating a lathe, grab a hunk of metal and make a perfect gear the first time.
That's what happened though? First humans built sheds, then we built 2-story buildings, then taller and taller, until we built skyscrapers. Obviously it wasn't a single structure, but we did have to evolve our thinking on how to build things, we didn't just start building a skyscraper before we built a shed.
You can't do that. A small bike shed is often just put some concrete blocks on the ground, and then build on top of them with wood. A correct house needs a stronger foundation at higher costs (sheds larger than bike shed are build the same way), but is still made of wood. A skyscraper is built with a very different foundation, and needs a steel frame that would not be affordable in a house. In between the two there are also building made of brick which allows building taller than wood. (and there are lots of other options with different costs - engineered wood is different)
Point is though eventually some system runs out of ability. It works different in programming from physical construction, but the concept is the same, eventually you can't make a bad early design work anymore.
But you didn't upgrade the shed into a skyscraper. The iterative process you describe involves a human respecifying from scratch using the knowledge developed building the previous instance and seeing it's limitations first hand. That part can't be automated, no LLM is going to challenge your design assumptions by itself. Hence people pushing agent-built projects way past what their inherent architecture should support, delivering an unmaintainable code spaghetti.
That's exactly why software is so bad. No one ever knows their shed would ultimately have to become a skyscraper, and management doesn't allocate any budget to lay stronger foundations when expectations change; you make do with what you have.
See also: "there is nothing more permanent than a temporary solution"
It’s actually the opposite - you actually can. The feel I'm getting reading anti-AI sentiment is people are expect one shot results out of limited context.
I'm pretty sure that you can't gradually upgrade a shed into a skyscraper unless you pour a skyscraper-ready foundation before even starting on the shed. But if you're doing that, why start with a shed and not with a skyscraper?
Not sure why you're trying to bring AI development into this.
But this is about the first systems? I tend to tell people, the fourth try usually sticks.
The first is too ambitious and ends in an unmaintainable pile around a good core idea.
The second tries to "get everything right" and suffers second system syndrome.
The third gets it right but now for a bunch of central business needs. You learned after all. It is good exactly because it does not try to get _everything_ right like the second did.
The fourth patches up some more features to scoop up B and C prios and calls it a day.
Sometimes, often in BigCorp:
Creators move on and it will slowly deteriorate from being maintenaned...
I disagree with most of this article, but this part stood out:
> the size of the iterations matters, a whole lot. If they are tiny, it is because you are blindly stumbling forward. If you are not blindly stumbling forward, they should be longer, as it is more effective.
You are not blindly stumbling forward, you're moving from (working software + tiny change) to (working software including change). And repeat. If there's a problem, you learn about it immediately. To me that's the opposite of moving blindly.
> you really should stop and take stock after each iteration.
Who is not taking stock after every iteration? This is one of the fundamental principles of agile/lean/devops/XP/scrum. This one sentence drastically lowers my impression of the author's ability to comment on the subject.
> The faster people code, the more cleanup that is required. The longer you avoid cleaning it up, the worse it gets, on basically an exponential scale.
Unsafe tempo is as likely to happen in big-spec design projects as in small iterations. In fact, working in careful small iterations helps us manage a realistic tempo because we know we can't move faster than we can get things into production and evaluate.
The terrible outcomes listed in the same paragraph are linked to unwise practice and have nothing to do with small iteration size.
exactly - and this is what good agile attempts to address - implement (importantly) user-facing function in exactly the smallest size possible (also important) with those two things you can build, identify issues / questions / problems in the fastest feedback loop as possible
indeed, i would argue 'big iterations' are the ones where all the problems which the author mentions crop up in the first place!
> There are two main schools of thought in software development about how to build really big, complicated stuff.
> The most prevalent one, these days, is that you gradually evolve the complexity over time. You start small and keep adding to it.
> The other school is that you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
I think AI will drive an interesting shift in how people build software. We'll see a move toward creating and iterating on specifications rather than implementations themselves.
In a sense, a specification is the most compact definition of your software possible. The knowledge density per "line" is much higher than in any programming language. This makes specifications easier to read, reason about, and iterate on—whether with AI or with peers.
I can imagine open source projects that will revolve entirely around specifications, not implementations. These specs could be discussed, with people contributing thoughts instead of pull requests. The more articulated the idea, the higher its chance of being "merged" into the working specification. For maintainers, reviewing "idea merge requests" and discussing them with AI assistants before updating the spec would be easier than reviewing code.
Specifications could be versioned just like software implementations, with running versions and stable releases. They could include addendums listing platform-specific caveats or library recommendations. With a good spec, developers could build their own tools in any language. One would be able to get a new version of the spec, diff it with the current one and ask AI to implement the difference or discuss what is needed for you personally and what is not. Similarly, It would be easier to "patch" the specification with your own requirements than to modify ready-made software.
Iceberg is, primarily, a spec [0]. It defines exactly what data is stored and how it is interacted with. The community debates broadly on spec changes first, see a recent one on cross-platform SQL UDFs [1].
We have yet to see a largely llm driven language implementation, but it is surely possible. I imagine it would be easier to tell the llm to instead translate the Java implementation to whatever language you need. A vibe-coded language could do major damage to a companies data.
If I had a spec for something non-trivial, I probably would ask AI to create a test suite first. Or port tests from an existing system since each test is typically orders of magnitude easier to rewrite in any language, and then run AI in a loop until the tests pass.
> I can imagine open source projects that will revolve entirely around specifications
This is a really good observation and I predict you will be correct.
There is a consequence of this for SaaS. You can imagine an example SaaS that one might need to vibecode to save money. The reason its not possible now is not because Claude can't do it, its because getting the right specs (like you suggested) is hard work. A well written spec will not only contain the best practices for that domain of software but also all the legal compliance BS that comes along with it.
With a proper specification that is also modular, I imagine we will be able to see more vibecoded SaaS.
Interested in ideas for this. I've mulled over different compact DSLs for specs, but unstructured (beyond file-specific ownership boundaries) has served me better.
There are parallels of thought here to template and macro libraries.
One issue is that a spec without a working reference implementation is essentially the same as a pull request that's never been successfully compiled. Generalization is good but you can't get away from actually doing the thing at the end of the day.
I've run into this issue with C++ templates before. Throw a type at a template that it hasn't previously been tested with and it can fall apart in new and exciting ways.
> The WHATWG was based on several core principles, (..) and that specifications need to be detailed enough that implementations can achieve complete interoperability without reverse-engineering each other.
But in my experience you need more than a spec, because an implementation is not just something that implements a spec, it is also the result of making many architectural choices in how the spec is implemented.
Also even with detailed specs AI still needs additional guidance. For example couple of weeks ago Cursor unleashed thousands of agents with access to web standards and the shared WPT test suite: the result was total nonsense.
So the future might rather be like a Russian doll of specs: start with a high-level system description, and then support it with finer-grained specs of parts of the system. This could go down all the way to the code itself: existing architectural patterns provide a spec for how to code a feature that is just a variation of such a pattern. Then whenever your system needs to do something new, you have to provide the code patterns for it. The AI is then relegated to its strength: applying existing patterns.
TLA+ has a concept of refinement, which is kind of what I described above as Russian dolls but only applied to TLA+ specs.
Here is a quote that describes the idea:
There is no fundamental distinction between specifications and implementations. We simply have specifications, some of which implement other specifications. A Java program can be viewed as a specification of a JVM (Java Virtual Machine) program, which can be viewed as a specification of an assembly language program, which can be viewed as a specification of an execution of the computer's machine instructions, which can be viewed as a specification of an execution of its register-transfer level design, and so on.
Lots of wisdom in this post about some of the realities of software development.
The core point they're trying to make is that agile (or similar) practices are the incorrect way to approach consolidation of smaller systems into bigger ones when the overall system already works and is very large.
I agree with their assertion that being forced to address difficult problems earlier on in the process results in ultimately better outcomes, but I think it ignores the reality that properly planning a re-write of monumentally sized and already in use system is practically impossible.
It takes a long time (years?) to understand and plan all the essential details, but in the interim the systems you're wanting to rewrite are evolving and some parts of the plan you thought you had completed are no longer correct. In essence, the goal posts keep shifting.
In this light, strangler fig pattern is probably the pragmatic approach for many of these re-writes. It's impossible to understand everything up front, so understand what you reasonably can for now, act on that, deliver something that works and adds value, then rinse and repeat. The problem is that for sufficiently large system, this will take decades and few software architects stick around at a single company long enough to see it through.
A final remark I want to make is that, after only a few years of being a full-time software developer, "writing code" is one of the easiest parts of the job. The hard part is knowing what code needs to be written, this requires skills in effective communication with various people, including other software developers and (probably more importantly) non-technical people who understand how the business processes actually need to work. If you want to be a great software developer, learn how to be good at this.
The paradox in post is resolved by limiting the planning to Russian doll like nested timeframes and scopes, upgrading from endless 2 week sprints and quarterly or annual "planning" to cycles within cycles, scoped to human magnitudes of time, and JIT re-planned at the Nyquist interval of each cycle by those at the corresponding level of the enterprise org chart who must also be domain leads with mastery at that level and who have retained ability to probe two levels below while practiced at bringing along at least one level up.
The 1970 Royce paper was about how waterfall didn't work, and most "Agile" is a subset of DSDM, each flavor missing a necessary thing or two whether working in large systems or growing them greenfield from nothing. But DSDM wasn't "little a" agile (and SAFE just isn't). There is a middle way.
If you like applying this stuff (e.g. you've chatted with Gene Kim, follow Will Larsen, whatever, sure, but you've deliberately iterated your approach based on culture and outcome observability), feel free to drop me a note to user at Google's thing.
100%. I came here to find something new from a field that I don't know but imagine has some good lessons for software. Instead I found someone commenting on small iterations vs big design which is quite ho-hum by comparison.
Something i see pop up in large Orgs and software solutions is as follows.
- you create large number of working small apps .
- you create a spec from these apps .
- create a huge app .
- make a Dsl to make extensible .
- extend the Dsl to fit what you need in the future .
- optimize the Dsl remove obvious N+1 stuff.
The hard part is throwing away the code in each step . Both managment and devs cant stomach the reality that the code is useless at each stage prior to dsl. They cant molt and discard the shell and hence the project dies.
"It’s not that you could cut the combined complexity in half, but more likely that you could bring it down to at least one-tenth of what it is today, if not even better. It would function better, be more reliable, and would be far more resilient to change. It would likely cost far less and require fewer employees as well. All sorts of ugly problems that they have now would just not exist."
Incidentally this highlights a problem when using chatbots to build large software projects that are intended to be used for a long period of time.
The key is not how much code you can add but how little you can get away with.
Chatbots only solution ever is to ADD code. They're not good at NOT writing code or even deleting it because after all the training set for the lines of code that do not exist is an empty set. Therefore it's impossible to train a robot to not write code.
What's better than generating 10kloc really fast? Not having it in the first place.
Grady Booch said that any large system that works is invariably found to have evolved from a smaller system that worked. I've seen this cited as Gall's Law, from John Gall's 2012 book Systemantics, but I read it in a book by Booch back in the late 80's/early 90's. At that time the "waterfall model" was the conventional wisdom: to the extent possible, gather all the requirements, then do all the design, then do all the coding, then do all the testing, doing the minimum of rework at each step.
It didn't work, even for the "large" systems of that time: and Booch had worked on more than a few. The kind of "system" the OP is describing is vastly larger, and vastly more complex. Even if you could successfully apply the waterfall model to a system built over two or three years, you certainly can't for a system of systems built over 50 years: the needs of the enterprise are evolving, the software environment is evolving, the hardware platform is evolving.
What you can do, if you're willing to pay for it, is ruthlessly attack technical debt across your system of systems as a disciplined, on-going activity. Good luck with that.
Big upfront designs are obviously based on big upfront knowledge which nobody has.
When they turn out to be based on false assumptions of simplicity the fallout is that the whole thing can't go forward because of one of the details.
Evolutionary systems at least always work to some degree even if you can look after the fact and decide that there's a lot of redundancy. Ideally you would then refactor the most troublesome pieces.
Big upfront design always tries to design too many things that should be implementation details. Meanwhile the things that are really important are often ignored - because you don't even realize they are important at the time.
Nah I’m good. I’ve watched system architecture framework views be developed. Years of prep and planning. System is released and half the employees that had requirements no longer work there and the business already pivoted to a new industry focus.
There’s a reason we went this way in software development a quarter century ago.
>In a sense, it is the difference between the way an entrepreneur might approach doing a startup versus how we build modern skyscrapers. Evolution versus Engineering.
There are core differences in software engineering that, unlike in construction work:
- making changes is often cheaper
- we might not know beforehand everything that is needed to be built, especially unknown unknowns
I would still agree that the truth is somewhere in between, but I would argue that, for software, it's closer to the evolutionary approach.
Everything that is touching hardware, for example. Bluetooth stack, HDMI, you name it.
Everything W3C does. Go is evolving through specs first. Probably every other programming language these days.
People already do that for humankind-scale projects where there have to be multiple implementations that can talk to each other. Iteration is inevitable for anything that gains traction, but it still can be iteration on specs first rather than on code.
Prototype and the specs go hand in hand. Write a spec - prove you can implement it. Write an implementation - write a spec so we can talk about what is important (vs details of how you implemented it but someone else is allowed to implement differently) Often parts that are "obvious" are not implemented, or only the trivial version is implemented. You need to do both.
> Also, the other side of it is that evolutionary projects are just more fun. I’ve preferred them. You’re not loaded down with all those messy dependencies. Way fewer meetings, so you can just get into the work and see how it goes. Endlessly arguing about fiddly details in a giant spec is draining, made worse if the experience around you is weak.
IMO the problem isn't discussing the spec per se. It's that the spec doesn't talk back the way actual working code does. On a "big upfront design" project, there is a high chance you're spending a lot of time on moot issues and irrelevant features.
Making a good spec is much harder than making working software, because the spec may not be right AND the spec may not describe the right thing.
I suppose it's primarily a matter of experience. And as the article alludes, it's very important to deeply understand the subject matter. I highly value some of my non-programmer colleagues responsible for documentation. But can't put my finger on what exactly they brought to table that made their prose exceptionally good (clear, concise, spot on)...
The Evolution method outlined also seems born from the Continuous Delivery paradigm that was required for subscription business models. I would argue Engineering is the superior approach as the Lean/Agile methods of production were born from physical engineering projects whose end result was complete. Evolution seems to be even more chaotic because an improper paradigm of 'dev ops' was used instead of organically emerged as one would expect with an evolving method.
Ai assistance would seem to favor the engineering approach as the friction of teams and personalities is reduced in favor of quick feasibility testing and complete planning.
I think that a comparison with Engineering is not that helpful for software.
Software has 0 construction cost but that it does have is extremely complicated behavior.
Take a bridge for example: the use case is being able to walk or drive or ride a train across it. It essentially proves a surface to travel on. The complications of providing this depend on the terrain, length etc etc and are not to be dismissed but there's relatively little doubt about what a bridge is expected to do. We don't iterate bridge design because we don't need to know much from the users of the bridge: does it fulfill their needs, is it "easy to use" etc AND because construction of a bridge is extremely expensive so iteration is also incredibly costly. We do, however, not build all bridges the same and people develop styles over time which they repeat for successive bridges and we iterate that way.
In essence, cycling is about discovering more accurately what is wanted because it is so often the case that we don't know precisely at the start. It allows one to be far more efficient because one changes the requirements as one learns.
> There should be some balanced path in the middle somewhere, but I haven’t stumbled across a formal version of it after all these decades.
Well, there isn't a formal version of it, because the answer is not formal, it is cultural.
In enterprise software, you have an inherent tension between good software engineering culture, where you follow the Boy Scouts' principle of leaving a codebase cleaner than you found it, and SOC2 compliance requirements that expect every software change to be tracked and approved.
If every kind of clean up requires a ticket, that has to be exhaustively filled out, then wait for a prioritization meeting, then wait for the managers and the bean counters to hem and haw while they contemplate whether or not it's worth it to spend man-hours on non-functional work, then if you're lucky then decide you can spend some time on it three weeks from now and if you're unlucky they decide nope, you gotta learn to work within an imperfect system. After once or twice or trying to work By The Book, most engineers with an ounce of self-respect will decide "fuck it, clearly The System doesn't care," and those with two ounces of self-respect will look for work elsewhere.
Or, you get together with the members of your team and decide, you know what, the program managers and the bean counters, they're not reading any of the code, not doing any of the reviews, and they have no idea how any of this works anyway. So you collectively decide to treat technical debt as the internal concern that it anyway was in the first place - you take an extra half hour, an extra day, however long it takes to put in the extra cleaning or polish, and just tack it on to an existing ticket. You give a little wink and you get a little nod and you help the gears turn a little more smoothly, which is all the stakeholders actually care about anyway.
You cannot replace culture with process. All attempts to replace culture with process will fail. People are not interchangeable cogs in the machine. If you try to treat them as such, they will gum up and get stuck. Ownership and autonomy are the grease that allows the human flywheel to spin freely. That means allowing people to say, "I'm going to do this because I think that it is Right And Good For My System Which I Own", and allowing them to be responsible for the consequences. To pass SOC2, that means treating people like adults and allowing them to sometimes say, instead of "can I get this reviewed because I legit need another set of eyes to take a serious look?", to say "can I get a quick rubber-stamp on this please?"
> There are two main schools of thought in software development about how to build really big, complicated stuff.
That feels like a straw man to me. This is not a binary question. For each small design decision you have a choice about how much uncertainty you accept.
There are no "two schools". There is at least a spectrum between two extremes and no real project was ever at either of the very ends of it. Actually, I don't think spectrum is a proper word even because this is not just a single dimension. For example, speed and risk often correlate but they are also somewhat independent and sometimes they anti-correlate.
Borrowing from mechanical/electrical etc. Limit the number of things you can build with. An example in the comments here was a gear. You make a new gear based on examples of gears that work. So whats the software equivalent of a gear? an axle, a bearing, etc.. Using OO or some ABI, you specify an object is a gear and behaves like a gear and magically you know how it does or doesnt fit together with other objects. I know this idea has been used before but im wondering if theres a well known software framework or library. We have things like the stl in cpp or built in libraries in python but im thinking of a higher level abstraction.
> if theres a well known software framework or library
Those are called data/structures and design patterns (not only the ones in the GoF book). If you have a good understanding of those and you know your data and the operation you will apply to it, it’s easy to model your data using those structures. Software are state machines specifications. Knowing how to model states and thus figuring the transitions is helpful. And there’s a lot of samples out there.
A major factor supporting evolution over big up-front design is the drift in system requirements over time. Even on large military like projects, apparently there's "discovery"--and the more years that pass, the more requirements change.
This isn't my experience. Requirements tend to settle over time (unless they're stupidly written). Users tend to like things to stay the same, with perhaps some improvement to performance here and there.
But if anything, all development is the search for the search for the requirements. Some just value writing them down.
Software cannot be built like skyscrapers because the sponsors know about the malleability of the medium and treat it like a lump of clay that by adding water can be shaped to something else.
You're mixing up design and manufacturing. A skyscraper is first completely designed (on paper, cad systems, prototypes), before it is manufactured. In software engineering, coding is often more a design phase than a manufacturing phase.
Designers need malleability, that is why they all want digital design systems.
Yep! Manufacturing is the running of the software, either via testing or via deployment. That’s when you’ll find bugs or design defects. Operational errors (misconfigurations, under allocation of resources) are not related to the design of the software itself.
Splitting coding and design is a bad idea. It’s like asking engineers not to draw and measure.
But software is in fact not very malleable at all. It's true the medium supports change, it's just a bunch of bits, but change is actually hard and expensive, perhaps more than other mediums.
How rapidly has business software changed since COVID? Yet how many skyscrapers remain partially unoccupied in big cities like London, because of the recent arrival of widespread hybrid working?
The buildings are structurally unchanged and haven't been demolished to make way for buildings that better support hybrid working. Sure office fit outs are more oriented towards smaller simultaneous attendance with more hot desking. Also a new industry boom around team building socials has arrived. Virtual skeet shooting or golf, for example.
On the whole, engineered cities are unchanged, their ancient and rigid specifications lacking the foresight to include the requirements that accommodate hybrid working. Software meanwhile has adapted and as the OP says, evolved.
This missed the point that they are ignoring evolution is literally the way you build things. There is no other way. You don't know what is actually needed or what might work really. You try things and then compress later. If you can try bigger things, bigger leaps great.
> Since the foundations like tech stacks, frameworks, and libraries are always changing rapidly these days, there are few accepted best practices, and most issues are incorrectly believed to be subjective.
Huh? at least in web, the "big ones" (angular/react/vue/svelte) have been around for YEARS at this point and IMO mostly stablized (though i still don't understand why angular needs to releast a breaking change version every 6-12 months)
the major 'issue' is often near 0 understanding in the fundamentals i'm talking like, super basic, what is an onclick function, how do we get our website to talk to our backend. if you can have clean domain and abstraction cuts, the rest really is a 'technical detail' - i.e. language / framework / technologies truly are all subjective. there are probably combined over 1000 valid tech stacks for a standard "show me table entries in a web dashboard" - not one is 'more correct' or 'more wrong' than the other, but rather in a given _organization_ or with a _given set of devs_ etc. that makes it 'more wrong'. the tech doesn't care, its the team of humans and HOW that team of humans interacts with the stack that is where things can go off the rails
there are so many ways to skin a cat and there ARE tradeoffs (positive AND negative) to each of these tech decisions... i'm not sure what the author is getting at - he seems to hint there are a select few sets of known best practices and tech choices, but fails to list them explicitly... this is dubious at best, and in some way counteracts his claim that it is "incorrect" that tech choices are subjective.
if you want to be non-subjective, be objective. name exactly the tech decisions and best practices you are talking about!
But I do still think there's a lot of value into coming up with a good plan before jumping in. A lot of software people like to jump in and I see them portray the planning people as trying to figure everything out first. (I wonder if we reinforce the jumping in head first mentality because people figure out you can't plan everything) A good plan helps you prevent changing specs and prepares you for hiccups. It helps by having others but basically all you do is try to think of all the things that could go wrong. Write them down. Triage. If needed, elevate questions to the decision makers. Try a few small scale tests. Then build out. But building out you're always going to find things you didn't see. You can't plan forever because you'll never solve the unknown unknowns until you build, but also good prep makes for smoother processes. It's the reason engineers do the math before they build a bridge. Not because the math is a perfect representation and things won't change (despite common belief, it's not static) but because the plan is cheaper than the build and having a plan allows you to better track changes and helps you determine how off the rails you've gone.
It is also perplexing to me that people think they can just plan everything out and give it to a LLMs. Do you really believe your manager knows everything that needs to be done when they assign jobs to you? Of course not, they couldn't. Half the job is figuring out what the actual requirements are.
>> you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
> This has never happened and never will. You simply are not omniscient. Even if you're smart enough to figure everything out the requirements will change underneath you.
I am one of those "battle-scarred twenty-year+ vets" mentioned in the article, currently working on a large project for a multinational company that requires everything to be specified up-front, planned on JIRA, estimates provided and Gantt charts setup before they even sign the contract for the next milestone.
I've worked on this project for 18 months, and I can count on zero hands the times a milestone hasn't gone off the rails due to unforeseen problems, last-minute changes and incomplete specifications. It has been an growing headache for the engineers that have to deliver within these rigid structures, and it's now got to the point that management itself has noticed and is trying to convince the big bosses we need a more agile and iterative approach.
Anyone who claims upfront specs are the solution to all the complexity of software either has no real world experience, or is so far removed from actual engineering they just don't know what they're talking about.
Working on a project for 18 months doesn't give you enough insight into it to know what is good or now about it. You need several more year before you can usefully figure out what changes will help get you to make milestones. (other than trivially obvious things, which might be the low handing fruit - but sometimes they are the better way to do things but the real problem makes that stand out instead).
Nothing will get you to hit every milestone. However you can make progress if you have years of experience in that project and the company is willing to invest in the needed time to make things better (they rarely are)
Agreed. Since this blog has posts since 2007 I can only think that the author is in the "so far removed" group.
> A lot of software people like to jump in and I see them portray the planning people as trying to figure everything out first.
My approach, especially for a project with a lot of unknowns, is usually to jump in right away and try to build a prototype. Then iterate a few times. If it's a small enough thing, a few iterations is enough to have a good result.
If it's something bigger, this is the point where it's worth doing some planning, as many of the problems have already been surfaced, and the problem is much better understood.
Yes it did, however it never works in pratice when it comes to integration testing two years later after the 2000 pages specification document was written, and passed down from the architects to the devs.
2000 page specification documents are rarely useful (if ever?).
You need smaller documents - this is the core technology we are using. This is how one subsystem is designed - often this should be on a whiteboard because once you get into the implementation details you need to change the plan, but the planning was useful. This is how to use core parts of the system so new comers can start working quick.
You need disciple to accept that sometimes libfoo is the best way to solve a problem in isolation, but since libbar is used elsewhere and can solve the problem your local problem will use libbar despite making your local problem uglier. Have a small set of core technologies that everyone knows and uses is sometimes more valuable than using the best tool for the job - but only sometimes.
> There should be some balanced path in the middle somewhere, but I haven’t stumbled across a formal version of it after all these decades.
That's very simple. The balanced path depends directly on how much of the requirements and assumptions are going to change during the life time of the thing you are building.
Engineering is helpful only to the extent you can forsee the future changes. Anything beyond that requires evolution.
You are able to comment on the complexity of that large company only because you are standing in the future into 50 years from when those things started take shape. If you were designing it 50 years back, you would end up with same complexity.
The nature's answer to it is, consolidate and compact. Everything that falls onto earth gets compacted into a solid rock over time, by a huge pressure of weight. All complexity and features are flattened out. Companies undergo similar dynamics driven by pressures over time, not by big-bang engineering design upfront.
“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.” Gall’s Law
I think the important part here is "from scratch". Typically when you're designing a new (second, third, whatever) system to replace the old one you actually take the good and the bad parts of the previous design into account, so it's no longer from scratch. That's what allows it to succeed (at least in my experience it usually did).
These days software has been done a lot. You should be able to find others who have done similar things and learn lessons from them. Considering microservices - there are lots of people who have done them and can tell you what worked well and what didn't. Considering using QT - lots of others have and can give you ideas. Considering writing your own framework - there are lots of others: look at what they do good and bad.
If you are doing a CRUD web app for a local small business - there are thousands of examples. If you are writing control software for a space station - you may not have access to code from NASA/Russia/China but you can at least look at generic software that does the things you need and learn some lessons.
This is often quoted, but I wonder whether it's actually strictly true, at least if you keep to a reasonable definition of "works". It's certainly not true in mechanical engineering.
The definition of a complex system is the qualifier for the quote. Many systems that are designed, implemented and found working are not complex systems. They may be complicated systems. To paraphrase Dr. Richard I. Cook’s ”How Complex Systems Fail” where he claims that complex systems are inherently hazardous, operate near the edge of failure and cannot be understood by analyzing individual components. These systems are not just complicated (like a machine with fixed parts) but dynamic, constantly evolving, and prone to multiple, coincidental failures.
A system of services that interact, where many of them are depending on each other in informal ways may be a complex system. Especially if humans are also involved.
Such a system is not something you design. You just happen to find yourself in it. Like the road to hell, the road to a complex system is paved with good intentions.
Then what precisely is the definition of complex? If "complex" just means "not designed", then the original quote that complex systems can't be designed is true but circular.
If the definition of "complex" is instead something more like "a system of services that interact", "prone to multiple, coincidental failures", then I don't think it's impossible to design them. It's just very hard. Manufacturing lines would be examples, they are certainly designed.
The manufacturing lines would be designed, and they'd be designed in an attempt to affect the "design" of the ultimate resulting supply chain they're a part of. But the relationship between the design of some lines and the behavior of the larger supply chain is non-linear, hard to predict, and ultimately undesigned, and therefore complex.
The design of the manufacturing lines and the resulting supply chain are not independent of each other -- you can trace features from one to the other -- but you cannot take apart the supply chain and analyze the designs of its constituent manufacturing lines and actually predict the behavior of the larger system.
AFAIK there's not a great definition of a complex system, just a set of traits that tend to indicate you're looking at one. Non-linearity, feedbacks, lack of predictability, resistance to analysis (the "you can't take it apart to reason about the whole" characteristic mentioned above"). All of these traits are also kind of the same things... they tend to come bundled with one another.
IMHO, the key is where you add complexity. In software you have different abstraction layers. If you make a layer too fat, it becomes unwieldly. A simple system evolves well if you're adding the complexity in the right layer, avoiding making a layer responsible for task outside its scope. It still "works" if you don't, but it's increasingly difficult to maintain it.
The law is maybe a little too simplistic in its formulation, but it's fundamentally true.
You built this gear using the knowledge from your last gear. You didn't start with no knowledge, read a manual on operating a lathe, grab a hunk of metal and make a perfect gear the first time.
> It's certainly not true in mechanical engineering.
Care to exemplify?
People misinterpret this and think they can incrementally build a skyscraper out of a shed.
That's what happened though? First humans built sheds, then we built 2-story buildings, then taller and taller, until we built skyscrapers. Obviously it wasn't a single structure, but we did have to evolve our thinking on how to build things, we didn't just start building a skyscraper before we built a shed.
You can't do that. A small bike shed is often just put some concrete blocks on the ground, and then build on top of them with wood. A correct house needs a stronger foundation at higher costs (sheds larger than bike shed are build the same way), but is still made of wood. A skyscraper is built with a very different foundation, and needs a steel frame that would not be affordable in a house. In between the two there are also building made of brick which allows building taller than wood. (and there are lots of other options with different costs - engineered wood is different)
Point is though eventually some system runs out of ability. It works different in programming from physical construction, but the concept is the same, eventually you can't make a bad early design work anymore.
But you didn't upgrade the shed into a skyscraper. The iterative process you describe involves a human respecifying from scratch using the knowledge developed building the previous instance and seeing it's limitations first hand. That part can't be automated, no LLM is going to challenge your design assumptions by itself. Hence people pushing agent-built projects way past what their inherent architecture should support, delivering an unmaintainable code spaghetti.
That's exactly why software is so bad. No one ever knows their shed would ultimately have to become a skyscraper, and management doesn't allocate any budget to lay stronger foundations when expectations change; you make do with what you have.
See also: "there is nothing more permanent than a temporary solution"
You can't physically but the logic is the same: you need beams, foundations, walls and roofs, with strenghts adjusted for scale. Software mindset :-)
In this sense, web applications haven't changed so much in the last twenty years: client, server, database...
It’s actually the opposite - you actually can. The feel I'm getting reading anti-AI sentiment is people are expect one shot results out of limited context.
I'm pretty sure that you can't gradually upgrade a shed into a skyscraper unless you pour a skyscraper-ready foundation before even starting on the shed. But if you're doing that, why start with a shed and not with a skyscraper?
Not sure why you're trying to bring AI development into this.
Ah, the Second System Effect, and the lesson learned from it.
But this is about the first systems? I tend to tell people, the fourth try usually sticks.
The first is too ambitious and ends in an unmaintainable pile around a good core idea.
The second tries to "get everything right" and suffers second system syndrome.
The third gets it right but now for a bunch of central business needs. You learned after all. It is good exactly because it does not try to get _everything_ right like the second did.
The fourth patches up some more features to scoop up B and C prios and calls it a day.
Sometimes, often in BigCorp: Creators move on and it will slowly deteriorate from being maintenaned...
So true.
Came here to add this very comment. https://en.wikipedia.org/wiki/John_Gall_(author)#Gall's_law
I disagree with most of this article, but this part stood out:
> the size of the iterations matters, a whole lot. If they are tiny, it is because you are blindly stumbling forward. If you are not blindly stumbling forward, they should be longer, as it is more effective.
You are not blindly stumbling forward, you're moving from (working software + tiny change) to (working software including change). And repeat. If there's a problem, you learn about it immediately. To me that's the opposite of moving blindly.
> you really should stop and take stock after each iteration.
Who is not taking stock after every iteration? This is one of the fundamental principles of agile/lean/devops/XP/scrum. This one sentence drastically lowers my impression of the author's ability to comment on the subject.
> The faster people code, the more cleanup that is required. The longer you avoid cleaning it up, the worse it gets, on basically an exponential scale.
Unsafe tempo is as likely to happen in big-spec design projects as in small iterations. In fact, working in careful small iterations helps us manage a realistic tempo because we know we can't move faster than we can get things into production and evaluate.
The terrible outcomes listed in the same paragraph are linked to unwise practice and have nothing to do with small iteration size.
exactly - and this is what good agile attempts to address - implement (importantly) user-facing function in exactly the smallest size possible (also important) with those two things you can build, identify issues / questions / problems in the fastest feedback loop as possible
indeed, i would argue 'big iterations' are the ones where all the problems which the author mentions crop up in the first place!
> There are two main schools of thought in software development about how to build really big, complicated stuff.
> The most prevalent one, these days, is that you gradually evolve the complexity over time. You start small and keep adding to it.
> The other school is that you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
I think AI will drive an interesting shift in how people build software. We'll see a move toward creating and iterating on specifications rather than implementations themselves.
In a sense, a specification is the most compact definition of your software possible. The knowledge density per "line" is much higher than in any programming language. This makes specifications easier to read, reason about, and iterate on—whether with AI or with peers.
I can imagine open source projects that will revolve entirely around specifications, not implementations. These specs could be discussed, with people contributing thoughts instead of pull requests. The more articulated the idea, the higher its chance of being "merged" into the working specification. For maintainers, reviewing "idea merge requests" and discussing them with AI assistants before updating the spec would be easier than reviewing code.
Specifications could be versioned just like software implementations, with running versions and stable releases. They could include addendums listing platform-specific caveats or library recommendations. With a good spec, developers could build their own tools in any language. One would be able to get a new version of the spec, diff it with the current one and ask AI to implement the difference or discuss what is needed for you personally and what is not. Similarly, It would be easier to "patch" the specification with your own requirements than to modify ready-made software.
Interesting times.
Iceberg is, primarily, a spec [0]. It defines exactly what data is stored and how it is interacted with. The community debates broadly on spec changes first, see a recent one on cross-platform SQL UDFs [1].
We have yet to see a largely llm driven language implementation, but it is surely possible. I imagine it would be easier to tell the llm to instead translate the Java implementation to whatever language you need. A vibe-coded language could do major damage to a companies data.
[0] https://iceberg.apache.org/spec/ [1] https://lists.apache.org/thread/whbgoc325o99vm4b599f0g1owhgw...
If I had a spec for something non-trivial, I probably would ask AI to create a test suite first. Or port tests from an existing system since each test is typically orders of magnitude easier to rewrite in any language, and then run AI in a loop until the tests pass.
> I can imagine open source projects that will revolve entirely around specifications
This is a really good observation and I predict you will be correct.
There is a consequence of this for SaaS. You can imagine an example SaaS that one might need to vibecode to save money. The reason its not possible now is not because Claude can't do it, its because getting the right specs (like you suggested) is hard work. A well written spec will not only contain the best practices for that domain of software but also all the legal compliance BS that comes along with it.
With a proper specification that is also modular, I imagine we will be able to see more vibecoded SaaS.
Overall I think your prediction is really strong.
Interested in ideas for this. I've mulled over different compact DSLs for specs, but unstructured (beyond file-specific ownership boundaries) has served me better.
I think it has to be modular and reusable. Like GDPR compliance spec should be opensourced and reused by all SaaS specs.
There are parallels of thought here to template and macro libraries.
One issue is that a spec without a working reference implementation is essentially the same as a pull request that's never been successfully compiled. Generalization is good but you can't get away from actually doing the thing at the end of the day.
I've run into this issue with C++ templates before. Throw a type at a template that it hasn't previously been tested with and it can fall apart in new and exciting ways.
You can look at the Web as a starter: https://html.spec.whatwg.org/#history-2
> The WHATWG was based on several core principles, (..) and that specifications need to be detailed enough that implementations can achieve complete interoperability without reverse-engineering each other.
But in my experience you need more than a spec, because an implementation is not just something that implements a spec, it is also the result of making many architectural choices in how the spec is implemented.
Also even with detailed specs AI still needs additional guidance. For example couple of weeks ago Cursor unleashed thousands of agents with access to web standards and the shared WPT test suite: the result was total nonsense.
So the future might rather be like a Russian doll of specs: start with a high-level system description, and then support it with finer-grained specs of parts of the system. This could go down all the way to the code itself: existing architectural patterns provide a spec for how to code a feature that is just a variation of such a pattern. Then whenever your system needs to do something new, you have to provide the code patterns for it. The AI is then relegated to its strength: applying existing patterns.
TLA+ has a concept of refinement, which is kind of what I described above as Russian dolls but only applied to TLA+ specs.
Here is a quote that describes the idea:
There is no fundamental distinction between specifications and implementations. We simply have specifications, some of which implement other specifications. A Java program can be viewed as a specification of a JVM (Java Virtual Machine) program, which can be viewed as a specification of an assembly language program, which can be viewed as a specification of an execution of the computer's machine instructions, which can be viewed as a specification of an execution of its register-transfer level design, and so on.
Source: https://cseweb.ucsd.edu/classes/sp05/cse128/ (chapter 1, last page)
Lots of wisdom in this post about some of the realities of software development.
The core point they're trying to make is that agile (or similar) practices are the incorrect way to approach consolidation of smaller systems into bigger ones when the overall system already works and is very large.
I agree with their assertion that being forced to address difficult problems earlier on in the process results in ultimately better outcomes, but I think it ignores the reality that properly planning a re-write of monumentally sized and already in use system is practically impossible.
It takes a long time (years?) to understand and plan all the essential details, but in the interim the systems you're wanting to rewrite are evolving and some parts of the plan you thought you had completed are no longer correct. In essence, the goal posts keep shifting.
In this light, strangler fig pattern is probably the pragmatic approach for many of these re-writes. It's impossible to understand everything up front, so understand what you reasonably can for now, act on that, deliver something that works and adds value, then rinse and repeat. The problem is that for sufficiently large system, this will take decades and few software architects stick around at a single company long enough to see it through.
A final remark I want to make is that, after only a few years of being a full-time software developer, "writing code" is one of the easiest parts of the job. The hard part is knowing what code needs to be written, this requires skills in effective communication with various people, including other software developers and (probably more importantly) non-technical people who understand how the business processes actually need to work. If you want to be a great software developer, learn how to be good at this.
> "writing code" is one of the easiest parts of the job. The hard part is knowing what code needs to be written.
I highly applaud this idea. IMO this is why big upfront design is so risky.
The paradox in post is resolved by limiting the planning to Russian doll like nested timeframes and scopes, upgrading from endless 2 week sprints and quarterly or annual "planning" to cycles within cycles, scoped to human magnitudes of time, and JIT re-planned at the Nyquist interval of each cycle by those at the corresponding level of the enterprise org chart who must also be domain leads with mastery at that level and who have retained ability to probe two levels below while practiced at bringing along at least one level up.
The 1970 Royce paper was about how waterfall didn't work, and most "Agile" is a subset of DSDM, each flavor missing a necessary thing or two whether working in large systems or growing them greenfield from nothing. But DSDM wasn't "little a" agile (and SAFE just isn't). There is a middle way.
If you like applying this stuff (e.g. you've chatted with Gene Kim, follow Will Larsen, whatever, sure, but you've deliberately iterated your approach based on culture and outcome observability), feel free to drop me a note to user at Google's thing.
I find the title misleading. While the author is "thinking about systems", this is not about https://en.wikipedia.org/wiki/Systems_thinking
100%. I came here to find something new from a field that I don't know but imagine has some good lessons for software. Instead I found someone commenting on small iterations vs big design which is quite ho-hum by comparison.
Build what you know you need now. Refactor when things grow. Doesn't have to be rocket science.
Something i see pop up in large Orgs and software solutions is as follows.
- you create large number of working small apps .
- you create a spec from these apps .
- create a huge app .
- make a Dsl to make extensible .
- extend the Dsl to fit what you need in the future .
- optimize the Dsl remove obvious N+1 stuff.
The hard part is throwing away the code in each step . Both managment and devs cant stomach the reality that the code is useless at each stage prior to dsl. They cant molt and discard the shell and hence the project dies.
"It’s not that you could cut the combined complexity in half, but more likely that you could bring it down to at least one-tenth of what it is today, if not even better. It would function better, be more reliable, and would be far more resilient to change. It would likely cost far less and require fewer employees as well. All sorts of ugly problems that they have now would just not exist."
Incidentally this highlights a problem when using chatbots to build large software projects that are intended to be used for a long period of time.
The key is not how much code you can add but how little you can get away with.
Chatbots only solution ever is to ADD code. They're not good at NOT writing code or even deleting it because after all the training set for the lines of code that do not exist is an empty set. Therefore it's impossible to train a robot to not write code.
What's better than generating 10kloc really fast? Not having it in the first place.
Grady Booch said that any large system that works is invariably found to have evolved from a smaller system that worked. I've seen this cited as Gall's Law, from John Gall's 2012 book Systemantics, but I read it in a book by Booch back in the late 80's/early 90's. At that time the "waterfall model" was the conventional wisdom: to the extent possible, gather all the requirements, then do all the design, then do all the coding, then do all the testing, doing the minimum of rework at each step.
It didn't work, even for the "large" systems of that time: and Booch had worked on more than a few. The kind of "system" the OP is describing is vastly larger, and vastly more complex. Even if you could successfully apply the waterfall model to a system built over two or three years, you certainly can't for a system of systems built over 50 years: the needs of the enterprise are evolving, the software environment is evolving, the hardware platform is evolving.
What you can do, if you're willing to pay for it, is ruthlessly attack technical debt across your system of systems as a disciplined, on-going activity. Good luck with that.
Big upfront designs are obviously based on big upfront knowledge which nobody has.
When they turn out to be based on false assumptions of simplicity the fallout is that the whole thing can't go forward because of one of the details.
Evolutionary systems at least always work to some degree even if you can look after the fact and decide that there's a lot of redundancy. Ideally you would then refactor the most troublesome pieces.
Big upfront design always tries to design too many things that should be implementation details. Meanwhile the things that are really important are often ignored - because you don't even realize they are important at the time.
lol this article is like the waterfall manifesto.
Nah I’m good. I’ve watched system architecture framework views be developed. Years of prep and planning. System is released and half the employees that had requirements no longer work there and the business already pivoted to a new industry focus.
There’s a reason we went this way in software development a quarter century ago.
Software is not a skyscraper.
>In a sense, it is the difference between the way an entrepreneur might approach doing a startup versus how we build modern skyscrapers. Evolution versus Engineering.
There are core differences in software engineering that, unlike in construction work:
- making changes is often cheaper
- we might not know beforehand everything that is needed to be built, especially unknown unknowns
I would still agree that the truth is somewhere in between, but I would argue that, for software, it's closer to the evolutionary approach.
It's strange that engineering is considered to be in opposition to evolution when we get concepts like prototypes and working models from engineering.
Show me an example of a large complex software system built from spec rather than evolved.
Everything that is touching hardware, for example. Bluetooth stack, HDMI, you name it.
Everything W3C does. Go is evolving through specs first. Probably every other programming language these days.
People already do that for humankind-scale projects where there have to be multiple implementations that can talk to each other. Iteration is inevitable for anything that gains traction, but it still can be iteration on specs first rather than on code.
I would expect them to build prototypes to iterate over the specs.
Prototype and the specs go hand in hand. Write a spec - prove you can implement it. Write an implementation - write a spec so we can talk about what is important (vs details of how you implemented it but someone else is allowed to implement differently) Often parts that are "obvious" are not implemented, or only the trivial version is implemented. You need to do both.
> Also, the other side of it is that evolutionary projects are just more fun. I’ve preferred them. You’re not loaded down with all those messy dependencies. Way fewer meetings, so you can just get into the work and see how it goes. Endlessly arguing about fiddly details in a giant spec is draining, made worse if the experience around you is weak.
IMO the problem isn't discussing the spec per se. It's that the spec doesn't talk back the way actual working code does. On a "big upfront design" project, there is a high chance you're spending a lot of time on moot issues and irrelevant features.
Making a good spec is much harder than making working software, because the spec may not be right AND the spec may not describe the right thing.
Yeah I’ve noticed the same problem. Do you know any resources for writing good specs?
Nope, sorry. I known I'm not good at it.
I suppose it's primarily a matter of experience. And as the article alludes, it's very important to deeply understand the subject matter. I highly value some of my non-programmer colleagues responsible for documentation. But can't put my finger on what exactly they brought to table that made their prose exceptionally good (clear, concise, spot on)...
Wow that's a good piece. It propelled me into writing this: https://dev.to/itay-maman/the-elephant-in-the-room-systems-t...
In short: the tension described in "systems thinking" is the same one as the one between "spec driven" vs. "iterative prompting"
The Evolution method outlined also seems born from the Continuous Delivery paradigm that was required for subscription business models. I would argue Engineering is the superior approach as the Lean/Agile methods of production were born from physical engineering projects whose end result was complete. Evolution seems to be even more chaotic because an improper paradigm of 'dev ops' was used instead of organically emerged as one would expect with an evolving method.
Ai assistance would seem to favor the engineering approach as the friction of teams and personalities is reduced in favor of quick feasibility testing and complete planning.
I think that a comparison with Engineering is not that helpful for software.
Software has 0 construction cost but that it does have is extremely complicated behavior.
Take a bridge for example: the use case is being able to walk or drive or ride a train across it. It essentially proves a surface to travel on. The complications of providing this depend on the terrain, length etc etc and are not to be dismissed but there's relatively little doubt about what a bridge is expected to do. We don't iterate bridge design because we don't need to know much from the users of the bridge: does it fulfill their needs, is it "easy to use" etc AND because construction of a bridge is extremely expensive so iteration is also incredibly costly. We do, however, not build all bridges the same and people develop styles over time which they repeat for successive bridges and we iterate that way.
In essence, cycling is about discovering more accurately what is wanted because it is so often the case that we don't know precisely at the start. It allows one to be far more efficient because one changes the requirements as one learns.
https://en.wikipedia.org/wiki/Conway%27s_law is very relevant here
HN comments: 53
Comments on actual blog post: 0
Why are people so afraid to leave replies on the author’s OC
> There should be some balanced path in the middle somewhere, but I haven’t stumbled across a formal version of it after all these decades.
Well, there isn't a formal version of it, because the answer is not formal, it is cultural.
In enterprise software, you have an inherent tension between good software engineering culture, where you follow the Boy Scouts' principle of leaving a codebase cleaner than you found it, and SOC2 compliance requirements that expect every software change to be tracked and approved.
If every kind of clean up requires a ticket, that has to be exhaustively filled out, then wait for a prioritization meeting, then wait for the managers and the bean counters to hem and haw while they contemplate whether or not it's worth it to spend man-hours on non-functional work, then if you're lucky then decide you can spend some time on it three weeks from now and if you're unlucky they decide nope, you gotta learn to work within an imperfect system. After once or twice or trying to work By The Book, most engineers with an ounce of self-respect will decide "fuck it, clearly The System doesn't care," and those with two ounces of self-respect will look for work elsewhere.
Or, you get together with the members of your team and decide, you know what, the program managers and the bean counters, they're not reading any of the code, not doing any of the reviews, and they have no idea how any of this works anyway. So you collectively decide to treat technical debt as the internal concern that it anyway was in the first place - you take an extra half hour, an extra day, however long it takes to put in the extra cleaning or polish, and just tack it on to an existing ticket. You give a little wink and you get a little nod and you help the gears turn a little more smoothly, which is all the stakeholders actually care about anyway.
You cannot replace culture with process. All attempts to replace culture with process will fail. People are not interchangeable cogs in the machine. If you try to treat them as such, they will gum up and get stuck. Ownership and autonomy are the grease that allows the human flywheel to spin freely. That means allowing people to say, "I'm going to do this because I think that it is Right And Good For My System Which I Own", and allowing them to be responsible for the consequences. To pass SOC2, that means treating people like adults and allowing them to sometimes say, instead of "can I get this reviewed because I legit need another set of eyes to take a serious look?", to say "can I get a quick rubber-stamp on this please?"
> There are two main schools of thought in software development about how to build really big, complicated stuff.
That feels like a straw man to me. This is not a binary question. For each small design decision you have a choice about how much uncertainty you accept.
There are no "two schools". There is at least a spectrum between two extremes and no real project was ever at either of the very ends of it. Actually, I don't think spectrum is a proper word even because this is not just a single dimension. For example, speed and risk often correlate but they are also somewhat independent and sometimes they anti-correlate.
Is this philosophising about software or does it have something to do with real life?
Borrowing from mechanical/electrical etc. Limit the number of things you can build with. An example in the comments here was a gear. You make a new gear based on examples of gears that work. So whats the software equivalent of a gear? an axle, a bearing, etc.. Using OO or some ABI, you specify an object is a gear and behaves like a gear and magically you know how it does or doesnt fit together with other objects. I know this idea has been used before but im wondering if theres a well known software framework or library. We have things like the stl in cpp or built in libraries in python but im thinking of a higher level abstraction.
> if theres a well known software framework or library
Those are called data/structures and design patterns (not only the ones in the GoF book). If you have a good understanding of those and you know your data and the operation you will apply to it, it’s easy to model your data using those structures. Software are state machines specifications. Knowing how to model states and thus figuring the transitions is helpful. And there’s a lot of samples out there.
A major factor supporting evolution over big up-front design is the drift in system requirements over time. Even on large military like projects, apparently there's "discovery"--and the more years that pass, the more requirements change.
This isn't my experience. Requirements tend to settle over time (unless they're stupidly written). Users tend to like things to stay the same, with perhaps some improvement to performance here and there.
But if anything, all development is the search for the search for the requirements. Some just value writing them down.
Completely irrelevant but this exchange reminded me of two Greek philosophers saying "everything is change" versus "nothing ever changes" LOL
Even if the requirements are indeed fixed your understanding of the problem domain evolves.
Software cannot be built like skyscrapers because the sponsors know about the malleability of the medium and treat it like a lump of clay that by adding water can be shaped to something else.
You're mixing up design and manufacturing. A skyscraper is first completely designed (on paper, cad systems, prototypes), before it is manufactured. In software engineering, coding is often more a design phase than a manufacturing phase.
Designers need malleability, that is why they all want digital design systems.
Yep! Manufacturing is the running of the software, either via testing or via deployment. That’s when you’ll find bugs or design defects. Operational errors (misconfigurations, under allocation of resources) are not related to the design of the software itself.
Splitting coding and design is a bad idea. It’s like asking engineers not to draw and measure.
Funny you bring up the clay analogy.
It was discussed here just 2 days ago intensively.
https://news.ycombinator.com/item?id=46881543
But software is in fact not very malleable at all. It's true the medium supports change, it's just a bunch of bits, but change is actually hard and expensive, perhaps more than other mediums.
I'd argue it's more malleable than a skyscraper.
How rapidly has business software changed since COVID? Yet how many skyscrapers remain partially unoccupied in big cities like London, because of the recent arrival of widespread hybrid working?
The buildings are structurally unchanged and haven't been demolished to make way for buildings that better support hybrid working. Sure office fit outs are more oriented towards smaller simultaneous attendance with more hot desking. Also a new industry boom around team building socials has arrived. Virtual skeet shooting or golf, for example.
On the whole, engineered cities are unchanged, their ancient and rigid specifications lacking the foresight to include the requirements that accommodate hybrid working. Software meanwhile has adapted and as the OP says, evolved.
With LLMs it's becoming very malleable.
This missed the point that they are ignoring evolution is literally the way you build things. There is no other way. You don't know what is actually needed or what might work really. You try things and then compress later. If you can try bigger things, bigger leaps great.
> Since the foundations like tech stacks, frameworks, and libraries are always changing rapidly these days, there are few accepted best practices, and most issues are incorrectly believed to be subjective.
Huh? at least in web, the "big ones" (angular/react/vue/svelte) have been around for YEARS at this point and IMO mostly stablized (though i still don't understand why angular needs to releast a breaking change version every 6-12 months)
the major 'issue' is often near 0 understanding in the fundamentals i'm talking like, super basic, what is an onclick function, how do we get our website to talk to our backend. if you can have clean domain and abstraction cuts, the rest really is a 'technical detail' - i.e. language / framework / technologies truly are all subjective. there are probably combined over 1000 valid tech stacks for a standard "show me table entries in a web dashboard" - not one is 'more correct' or 'more wrong' than the other, but rather in a given _organization_ or with a _given set of devs_ etc. that makes it 'more wrong'. the tech doesn't care, its the team of humans and HOW that team of humans interacts with the stack that is where things can go off the rails
there are so many ways to skin a cat and there ARE tradeoffs (positive AND negative) to each of these tech decisions... i'm not sure what the author is getting at - he seems to hint there are a select few sets of known best practices and tech choices, but fails to list them explicitly... this is dubious at best, and in some way counteracts his claim that it is "incorrect" that tech choices are subjective.
if you want to be non-subjective, be objective. name exactly the tech decisions and best practices you are talking about!