It Can Be Done (2003)

This 100%. The issue is that much of the time, project planning is done terribly. The stakeholders have no idea what they want, and will demand extra features/sweeping changes at seemingly random. Lots of edge cases are needed for things that probably shouldn't be done with this tool, but are being done so anyway. The designers often create designs that are at odds with the features needed, how the existing system works, what said stakeholders want, etc. The various teams developing parts of the system don't communicate well and build things in silos. Poorly thought out 'agile' processes end up trying to cram everything into a certain timeframe based on some questionably decided points score. Etc.

For the most part, it's those types of issues that lead to a system not working as intended, or being buggy as hell. If you can get a good setup going where the requirements are clear and don't change and the people on the project communicate well and have full control over the process then things will work out fine. That's seemingly what happened in the source article.

even if they do it keeps changing because "agile".

If you think "agile" is the reason requirements change...I don't know what to tell you. Requirements will always change, full stop. It's like a force of nature, there is no universe in which everyone just "knows" what to build up front and has a fully spec'd out API that you can go off into a cave and implement. Real life never works that way, and software engineering is not the right profession for you if you need that.

That plus a thousand special cases, that are quite often absolutely meaningless to implement in code, but still have to be done.

What do I mean by that: I work for big corporations mostly and quite often there is the average case that is about 90%-99% of all incoming work.

Then there is a myriad of special cases some of which happen every third year on a blood moon, if an eclipse is happening at the same time and the witches chant in the woods...

Instead of managing these unicorn-cases by hand, they have to be implemented in code and lead to bugs and a ton more code to review and maintain.

Speaking of maintenance... oh, don't get me started on that one, it's a sore spot!

The reality that most people don't want to hear is that he could do this because he himself was the domain expert.

Requirements isn't just making lists and tickets in swimlanes. It's actually learning the domain you are building for. Sure you can still build for something you don't really understand, but it'll be garbage. In fact, writing this garbage is OK if you accept that it's just a drafting process for learning the domain challenges and potential solutions. There are no shortcuts to quality.

My advice for stakeholders is to make sure you have access to the person(s) building your application and see if they understand your domain (you can do this without constantly pestering them!). Beware bullshitters using buzzwords.

My advice for designers / programmers is to make sure your stakeholders & domain experts are keen and engaged, otherwise getting requirements and understanding is like pulling teeth. Sometime a stakeholder doesn't even want to solve the problem. They may have wanted to go in a different direction entirely or maybe there are politics involved that has them miffed. Nothing sinks a project like lazy or uninterested stakeholders.

I'm building plain old boring desktop software. The bugs all (ok not all, but most) come from edge cases with users doing "dumb" (totally unexpected) stuff.

If it weren't for users my software would be perfect ;)

Clear requirements and a singluar task, doesn't sound like he also had to balance a load of other projects, meetings, rituals, etc. He was able to do deep work as the hype books call it.

I'm probably romanticising it, but it was a time where software engineers and their skills, time and attention span were still respected.

> Give a dev a clear API and a well defined set of criteria

If you've got that - why do you need the developer? Genuine question. If the technology requirements are that well defined why on earth do i need to pay engineer level salaries for such basic work? As you said, and i agree:

> Give a dev a clear API and a well defined set of criteria, and most will write code that works very well

At that point of having clear criteria, there are only 2 tasks left:

    1. write some code that satisfies this outcome
    2. write it in a way such that it can be cheaply changed in future

But but but performance! durability! ... <many more of the -ilities of software that an experienced engineer will claim to just take in their stride...>

As a customer of this software developement process, these are just the next cycle of requirements to feed into the machine - hey take this thing you did, make it faster while obeying rule #2 above.

It would not surprise me if ChatGPT were able to cover 80%+ of a problem that well defined.

I totally get the attraction to focus on the fun bit, but it's not how the business world is designed to work in most cases.

For what it's worth i think you nailed the actual value opportunity for a strong dev:

> no one actually knows what they're building

If you can nail that, you're always gonna be valuable. We talk about programming and coding but actually, this is more of the role. The coding part of the puzzle is easy in comparison.

As for:

> and even if they do it keeps changing because "agile"

If your understanding of why requirements are in constant flux amounts to pinning it on "agile", you're not even on the field yet never mind winning the game.

I wish we had give the ability to push back on nonsense like even when making a web page, ok make this whole div clickable but inside this div, put another button that when you click the button something else happens.

Or at least spend the time to tell us why we are doing this nonsense and maybe I can come up with a less asinine solution.

and even if they do it keeps changing because "agile".

Agile is precisely the response to the fact that requirements keep changing, not the cause of it.

Depending on the definition of "dev", they can be the ones who define the requirements and API.

People always seem to be scoffing when they say stuff like this, like it's a bad thing. But I don't get that attitude at all. I don't want to work on things where the whole shape of the project is known up front. To me, that feels like doing art with a stencil. I love the process of seeking the most valuable next step to take while we feel our way through a dark room with an ambiguous destination.

Now, the (common) version of this that I don't like is when there is no way to know whether a sequence of steps ended up in a better place than where it began. So feedback is very important to me, but there are lots of ways to get it; there are quantitative measures of growth in usage or revenue and there are qualitative measures like surveys or for things like internal platforms, seeing the roadmaps of other teams be unblocked or accelerated by your work.

But I find significantly less joy in being told "we need this exact specific thing, please go build it and report back", and I have rarely seen it be the case that what they needed was actually that exact specific thing.

Nope, he had a well documented set of APIs to program against. In one instance he didn't, they had a crash.

He did "invent" internal and external APIs for his service, just like developers do today: that's the part requiring developers to be like "engineers".

I fully agree. Yet I'm in the same boat.

Started as a Windows line of business software developer almost 20 years ago. At first we got clear specifications with UI mockups and a description what each and every button should do. When there were questions, these specs where updated and we implemented and tested and fixed until our boss was happy.

Over the years we got more and more customers yet less and less time for tests and fixes. So we switched to "agile" and dropped the specs, instead wrote quick notes about what must (roughly) be done. At first all were happy. But now we have a huge amount of features that not one dev knows. You have to ask and search around until you find all the lose specs.

Now I'm managing such a team of developers and have the same issue. I don't have the time to write a clean specification, yet alone discuss it with the actual customer, which anyway doesn't really understand all the implications. So they start coding by adding more if and else blocks to an already bloated code base.

Pity the days we would start with a class diagram or just some quick drawing about how the components would interact and be testable.

Give a dev a clear API and a well defined set of criteria, and most will write code that works very well

Someone still needs to do this work, and sometimes that's the developer. Isn't that what he was doing in this anecdote?

Also because many devs do not understand multiple execution layers. From react devs who never heard of javascript native loops to Spring devs who have no idea how sql looks to c devs who have never used a memory heap tool to node devs who never seen io performance impact to ..... You need an all seeing eye to make things performant. Those are scarse. Probably one in the 20 devs max.

If you don’t understand a problem well enough to write it down, you don’t understand it well enough to solve it.

If you don’t understand a solution well enough to write it down, you don’t understand it well enough to implement it. (Much less get someone else to implement it.)

An overwhelming number of engineers I have worked with seem to think any kind of written design document is busy work, and not actually a part of the process of making something worth using. My experience shows it is not that at all: design docs are a tool to externalize your own thinking, and reflect on its quality without the overhead of keeping it in your mind. That’s their first and foremost purpose. To explain your rationale to others is a secondary objective.

I’m not talking about lengthy requirements documents or specifications, either. I mean a two or three page white paper outlining a problem or a proposed solution in plain English. This is something you can bang out in a half hour or hour if you’re diligent.

Many of the folks I work with never even seem to bother.

If you think there were good old days were all software was well specified, you are wearing very rose tinted glasses.

We are inventing the field as we go. Have been for the last decades.

Also it's one of those fields where the expert in the topic must build a system for something he is not an expert with.

Again and again.

And most project are custom.

Agile has nothing to do with it.

It's the nature of IT right now.

Modern devs are frustrated. They don't do anything of importance or significance anymore. They're not the guy designing a virtual memory manager for a novel OS. They're just some cog in a machine converting SQL queries into HTML to show ads to kids and old people.

One can only become cynical and bitter.

But you don't understand, I once had to replace chart.js 3 with chart.js 4 and the documentation didn't cover all of the changes. This man would have collapsed into a heap if he'd had to face the challenges I've faced.

I understand this attitude, but FWIW, when I got here 5 hours after your post, all the top comments were very positive and not picking on it at all. And that's how it usually is when I see one of these comments like yours. It takes a little time for the sieve of user moderation to bubble the better comments up to the top.

Every time someone posts a counter argument, people pick on it. Please stop.

Oh really, don't: that's what the discussion should be about.

You are certainly right to a point, but a better way to redirect positively is to ask: how can we come to a state where similar feats are possible today in a company setting?

Every time comments are posted, a real interactive discussion, other commenters pick on it. Please stop.

Alternatively: only people who were into it enough to handle doing things on paper and with limited access to computers kept working afterwards.

So: was it a good way to teach or a good way to filter out less motivated people?

He must have been one of the privileged few with access to pencil and paper then.

I learned on computers where cards and printouts were the I/O system.

I remember a number of years where the edit->compile->print loop was much better with listings than terminals, simply because you couldn't see/navigate/edit your code as efficiently. It eventually improved with better/visual editors, larger terminals and faster compiling and running.

An analogy might be early firearms, where things like wet gunpowder, unreliable flintlocks and pushing everything down the barrel made for an inefficient transition time from the longbow.

I can attest to this. Once, a college teacher gave us a programming exercise, and then went on to suggest how we might go about solving it.

While he was talking, I started outlining a solution on a piece of paper. At school, I couldn't use a computer during class, so I'd gotten used to programming (and debugging!) on paper.

When I arrived at home, I typed it in and tried it out. Could I possibly reproduce that Multics story I'd read about?

It didn't compile the first time, but after correcting a variable name, it worked perfectly.

When we learned COBOL in high school we did it in pen and paper mostly because the school labs didn't had enough PC's for every student, and we ended up working 3-4 students per computer.

OTOH, there were practically no examples to draw from. There was no virtual memory class in their OS-101 course. These people pioneered OS design.

i was surprised to find that my natural-language notes from my electronics notebook from the last month total a quarter megabyte already. about 37000 words of markdown

a small fraction of that is urls for electronic parts, but nearly all (90%, say) is just english text. another 10% is stuff i quoted from other people (web pages, manufacturer app notes, books, etc.) it is a single file and will probably continue to be a single file forever, at least for the rest of the year. if i continue at the same pace, that will be 12 megabytes. there is nothing impossible about that at all

(if you're interested, git clone http://canonical.org/~kragen/sw/leatherdrink.git. it's currently almost 200 megs because i checked in the avr toolchain. there are also photos and videos and schematics and stuff)

in a programming language the growth would be fewer bytes per month, but perhaps by a factor of five

The software at that time was just much smaller.

Do go check the code: https://multicians.org/vtoc_man.html

This is both bigger and more complex than most stuff people work on today (which is glorified glue code for bullshit CRUD/REST/etc). It might be smaller than the overall LoC of their project, but it's more than the units they deal with (and this is part of an overal OS code anyway).

This is a whole "manager, to manage file description information. It had to transport the file information between disk and memory, manage a shared memory buffer pool, and manage space on disk for the information."

Dare most current programmers to write such a thing, with the requirements and semantics it has on his version, today, in their language of choice.

Most would be lost at even contemplating that. Much less write it in paper and type it in and have it work.

I'm sorry but the line of business software most of us work on is not more impressive than writing a major component of a file system from scratch.

It might also have to do with how programming was taught to a certain generation? At least from my own experience, when I was at university OOP and UML was all the rage, so we had to specify through diagrams in fancy diagram editors, then generate the code from that, and then still re-write everything by hand because it never really worked out quite that well.

I never touched UML again after university ...

My experience is that it's a skill like any other, I was bad at it, I wanted to be better at it so I did it more often and now I'm good at it.

Being able to anticipate and find nice design level general solutions for hairy real-world problems is where systems design becomes most rewarding.

I think the modern-day equivalent would be test-driven development; in both cases (unless I don't get it) is that you make sure you understand the problem and write verifications before you start on a solution.

Of course, this doesn't work for every problem / task.

So the answer to "how did he do it?" is obvious -- it simply wasn't too complex of a task (by modern standards).

Hmm, have never seen much of PL/1.

Control flow aside, the syntax looks surprisingly clean.

pebkac ?

s/HW/HN/?

This comment really deserves more attention.

I don't even smoke, and think that photo is amazing.

Scams and ad-pushing? Persistent Invigilation by ad-serving companies also surprisingly started out in commercial space, not military, from what I know.

Im not sure that I like this approach. Documentation very rarely cover all 100% of nuances. There always will be "test and see" part of learning in my opinion.

For me its like you can use paper for drawing but only 10 minutes a day. Or piano but only every second Monday for few hours.

It works because normally you do most of the thinking in your head, so the few occasions you are doing deep thinking using paper and pen, you have the discipline and care to only write what is important and keep the rest in your head. If you do everything on paper, it will start being not much different than doing everything directly on computer because you will be writing everything and you will lose that abstraction and care.

This is exactly it. People are lamenting the bygone days of having to think a lot about your software before you wrote it, but the truth is that, today, the same programmer would have been able to iterate his way to the same program in perhaps half the time.

I remember an assignment in my university course, where I had to write a (simple, admittedly) OS kernel. I didn't know C very well, and I didn't know what kernels did beyond the theory, but we had to write a few hundred lines of C code to manage tasks. I knew that there would be absolutely no way I could debug this program if it didn't work, as it was all parallel code and a bug would mean mysterious race conditions.

I reasoned about the system as a whole, and I spent a few days writing small, self-contained functions that I thought a lot about. I then compiled and ran it, and, after the obligatory "missing semicolon" compilation errors, it worked first try.