Such a contradiction - world-class research, patents filed, etc. At the same time unable to stay up to date and relevant. After many years in the valley I joined them, thinking I’d learn how they ran truly global projects. Spoiler, it’s just chaos - throw people at it and when that fails throw more people at it. A giant red flag was when they decided they couldn’t compete in the commoditized x86 server market and sold off that division to Lenovo, for who it instantly became their highest margin business. IBM’s cost structure and bloat just meant they couldn’t compete in all but the most profitable product lines. Then came all the financial engineering like recasting their software as Cloud revenue.
What ive learned in my career is that all companies are operating on a knife's edge. I have accepted offers at some very prestigious companies where I assume that "surely this company will be organized and have good systems in place". Then, after I start I realize every company is operating on chaos, barely putting one foot in front of the other.
They put on a good front for the public investor meetings, but behind the scenes I have decided that every company is just running one day at a time.
I’ve come to the same conclusion. But not just about companies, but everything. Talk to someone in the military, or government agencies, or working on large infrastructure projects. It’s the same story everywhere - chaos. And it’s generally not because of malice, being dumb, or lack of trying.
I think as a civilization we’re much more like ants than a troop of smart chimps. In that our civilization is a kind of a proverbial emergent anthill. And most likely the relative individual understanding of our respective anthills doesn’t differ too much between ants and us.
And it’s kind of freeing. You just move your proverbial bit of earth that you’re just compelled to put somewhere else, and don’t stress the rest. Emergence will take care of it.
I think this is the cause of my depression. It's horrendously sad to think about what we could be accomplishing if we were just a little efficient.
Efficiency, trust and cooperation, what an amazing place it could be.
Finding a hobby that provides a little of that helped me. What I also noticed, having spent far too much time in front of a screen and programming, that I began to think of the world as a giant program that is really really buggy. Trying to fix those bugs got me down ... just gotta live with those bugs.
I think this is an excellent way to phrase the experience of "programmer brain" and I will be blatantly ripping it off in the future. It's definitely an occupational hazard, and something we should be wary of, but it's only a small symptom of the larger technologized worldview that permeates Western thought (and via export, a lot of global thought). There are definite upsides to technology and programming, but: "we shape our tools, our tools shape us". I think we technologists think a lot about the former, and rarely about the latter (that's for those squishy humanities types!) -- at our own peril.
Thank you for ripping it off - I call it sharing :)
I always like to quote the frog in water. As the water is heated to boiling point, apparently the frog doesn't spring out. That is, in fact, an urban legend - the frog does spring out. However we are the frogs that don't spring out.
Trust and self sacrifice for the greater good.
It is exceeding difficult to sacrifice yourself for someone you barely know even if you know it is for the betterment of everyone. Ants have 0 problem with this.
You should consider that you might be the one wrong here. What you think is efficient is just an individual perspective.
I’ve come to accept this after years of fighting “the system”.
The system doesn’t care about you or what you want. It is a ruthlessly collective thing and it makes short term mistakes you will pay for but long term builds foundations you benefit from.
Also over and over efficiency in many cases proves to be maladaptive as it kills flexibility.
Well, everything exists along a gradient.
The example often given is Japan vs the West.
Especially people in the West laud how great Japan its collectivist society is, how streets are clean, relatively little gets stolen, personal responsibility is still a thing, etc; But they completely skip over the other side of the coin: collectivist societies crush much of the independence out of a person.
So, in this one sense, you can trade independence for social “efficiency”. I imagine it is much the same for humanity. We could become more harmonized, at the cost of becoming more drone-like.
An interesting book that deals with this exact dilemma (among other things) is “A Deepness In the Sky” by Vernor Vinge. Worth a read!
Sorry to hear this! I hope you can manage to get out of it, depression can be a real bitch! Do seek help, it’s nothing to be ashamed of.
We blame large, intractable systems for our problems when it's too dangerous to look at the small, local causes.
Efficient how? Ants are extremely efficient and successful, second only to humans in biomass. Turns out making good global decisions is hard to impossible, so you see decentralized decision making pop up everywhere.
Perfect isn't efficient. The loss from wastage and chaos is less than the resources it would take to eliminate it.
Emergence is an agent that isn't me. Why am I even conscious if all my agency is subsumed by the egregore. This isn't freeing in the slightest, maybe only for the most conditioned megacity dwellers.
Within a group of n people, everything else being equal, why would you expect to be able to steer its behavior by more than 1/nth?
But that's the problem with emergence, you don't even get the 1/n. You get zero, or very near it.
Well, you had the agency to read HN today, to comment here, and to choose the contents of your comment. So I’m not sure what you’re getting at, beyond the well-trodden free-will debate.
I think you misunderstand what emergence is. It’s really not an argument about if you have agency or not in your individual actions.
This couldn’t be farther from the truth though. As in, I don’t think any place I’ve even been to would count as a megacity. Let alone where I live.
The way I see it is that emergence is not an agent. More like some basic law of nature. Think about planned gardens and suburbs versus natural forests and medieval city centers.
You can mow your lawn every week, expend obscene amounts of water during the summer, pour everything with herbicides in a misguided effort to have the perfect lawn, but the moment you stop doing it, nature takes over and introduces a fractal amount of complexity in just a few seasons.
In my opinion an old forest where nature has been let alone for some time is much more beautiful and interesting than any planned garden, and a medieval city centre much more beautiful and interesting than any planned neighborhood.
And then, emergence explaining our civilization doesn’t necessarily imply that you don’t have agency as an individual. It just says that the total of whatever we collectively do, irrespective of if we’re compelled to do it, or have free will, results in more than just the sum of individual parts.
Tell it to the choanoflagellates.
The emergent agent isn't a harmonious oak old growth. It's a ravenous, homogenizing beast, primally unable to die; accelerating forever in the red queen race; killing everything that isn't itself, fundamentally unaware and uncaring of the individuals it's made of.
But you don’t know that. As in, it’s impossible for you (or me) to know that.
I do care about my cells. Like, not individually, but I do care that they are in as good an environment for them as I can muster. I don’t think they have any concept of me. Or if they do, I don’t think they can reason about how murderous am I.
For me, it's a good argument against conspiracy theories. There's no way there could be secretive organisations running for years and controlling the world without screwing up in stupid and obvious ways.
To the contrary does it not imply any well organized group should have an easier time of accomplishing their aims while the rest are mired in the chaos?
The claim is that there are literally no observable organizations running without chaos, so are we to believe the only one to achieve it is nefarious instead of just regular profiteering?
I like the meta conspiracy theory that the only organizations running without chaos are covert ones though.
It’s more that within that chaos there are short bursts of organization that can pull off narrow conspiracies.
There are also natural long term aligned players who conspire in ways that they can use their energy for their benefit. “Market makers”
There are also random alignments that appear to be conspiracies ex post facto.
This take makes sense to me. It's not the org as a whole that is accomplishing secret aims, but a sub-org within a chaotic org, or spread across multiple chaotic orgs, that is able to do so since no one is paying close attention to anything.
Right, the conspiracy is to destroy things, not build them.
Yeah. I think anyone who has had experience with school/kindergarten parents chats trying to organize _anything_ would find the idea of shadowy organizations running the world laughable.
It’s kind of a side effect of capitalism. You get the culture of doing more with less and drive it until the point you are doing less than adequate with less money than you’d need to do it properly.
The end result is software that puts postal workers in jail and doors that fall off planes.
Like democracy, it sucks, but it's all we have. Depending how you define capitalism (currency, trade, investing in capital?) and software (something executed on a digital or analog computing machine) there was no software 10,000+ years ago before capitalism
In the end it's about process and dumbing it down enough that it can be followed by the cheapest resource available (while also pressing down the cost of such resources through managed poverty). This race to the bottom is never good.
As for democracy, it's great, much better than any alternative. It's inconvenient (to the powers of the day), however, that it tries (at least the functional ones) to prevent the widening of the chasm between the haves and the have nots. One thing any functional democracy must aggressively prevent is the acquisition of power from any means other than popular vote.
It's inconvenient (to the powers of the day), however, that it tries (at least the functional ones) to prevent the widening of the chasm between the haves and the have nots.
If there was a central lesson of the twentieth century, it's that the chasm between the haves and have-nots cannot be closed without doing more damage than simply allowing it to exist. People are not created equal, no matter how tightly you shut your eyes and how loudly you chant.
Meanwhile, the central lesson of the twenty-first century is beginning to come into focus: giving smart people and stupid people the same voting rights is maladaptive to civilized society. You can forget any notion of progress while this state of affairs prevails; just ask your local Trumpers. Does this refute democracy? Very well, then, it refutes democracy.
I'm not super optimistic on either front, I guess. Optimism simply requires too much denial. Preservation of equal opportunity is likely to be the best goal we can aim for.
I don’t think there is evidence supporting this conclusion.
And yet they have equal rights, including the same right to influence political decisions, which is clearly not being respected.
Democracy requires an informed electorate. That’s why public schools exist and why they should be well (and equally well) funded, without some schools being privileged by donations that’ll only exacerbate unequal rights to education.
My conclusion is that this "chaotic scenario" happens when humans work in big groups. It's extremely hard/pretty much impossible to coordinate big group of humans and made them to work efficiently. In fact, this is one of the reasons by which nowadays, I personally prefer to work on a small start up environment instead of a big corporation.
It is possible, they just need to spend more than half of the work day in coordination meetings and doing paperwork.
And not many organization are willing to pay thousands of people to do paperwork to help other departments do their paperwork more effectively and so on.
Or somehow they found a group of extremely trustworthy and reliable employees who will never fudge the truth even at the cost of their jobs/reputation/etc...
Where have you seen this where the group is > 500 people?
If you mean the latter, I haven't.
"emergent anthill"
Wonder if this concept could also be Moloch.
Moloch is sometimes used to represent how humans just consume.
And, what do ants do, expand and consume.
More and More I think the ant-human analogy is best.
We are organizing, we form structures, but it isn't a plan, it's just twitching on our feedbacks. We have some loose internal functions to respond to inputs. That when stacked up by millions form some pattern.
Ant's don't have an 'anthill plan' and humans don't have a 'city/town plan'.
Like ants, we just kind of group together and follow the chemical paths laid down by others (coffee, beer).
Coffee, beer and sex hormones.
Sounds Daoist to me.
I believe this describe most of the industry.
Conferences about high performance teams and practices are wet dreams.
If I'm wrong then I want to meet these teams.
I work at a very high performance company that I co-founded. But we are less than 150 people and ~50% have been with us for 10 to 20 years as loyal employees we have brought through a few firms.
I think small, lucrative, focused companies can enjoy work-life balance and strong employee and customer loyalty. You need a niche that is big enough to support a 90th percentile team, but small and esoteric enough to avoid attention of giant firms. It is a delicate balance. I think the German Mittelstand model is similar, except translate this to software.
I work in a "Mittelstand" company and we are definitely operating mostly on chaos.
Well to be honest most companies of any size are operating on chaos. But the person I was replying to "wanted to meet [the high performing] teams" so I gave an example of mine, where I feel things are fairly well-oiled. But this is a unique situation, that is actually the result of 20 years of building. We even raised VC but they have accepted whatever it is they have invested in, which won't be 1000x ROI.
I thought I was insane reading all of these chaos comments. I’ve worked at several companies that are VERY well organized. I think that above or below a certain scale chaos is the rule, but somewhere around 100 people can be a smooth running machine.
Thanks, there's so many questions coming to mind:
- how did you find your team mates
- how do you plan the work (fully self organized ? xp ? agile ? another method ?)
- how do you resolve human issues (intra or inter teams frictions, loss of motivation)
- do you assess per employee performance or not ? (have you ever run into a situation where someone was faking, or faking too much, but nobody checked ? or is you group tight enough so that any such case will be detected and fixed rapidly)
- do you have allocated time for team performance improvements ? remove friction, adjust processes
- do you allow creative attempts (if someone thinks he could chase a new idea for a day or two)
also, i didn't ask, how do you define high performance ? i have my own definition in a way: ability to understand most parts of your system in a few minutes with high confidence, ability to try new ideas, ability to split work between people for parallelism so that integrating is nearly guaranteed and lastly people who can generate new ideas multiple times per day
It seems that the bigger a company is, the higher percentage of dead wood.
yeah it's something one can read regularly, and especially in military or govt settings where the bureaucracy is so heavy it takes a month to add a file.
" Then, after I start I realize every company is operating on chaos, barely putting one foot in front of the other."
So true. I have been involved in some projects where we had to figure out how certain processes. Every time it turned out that things are run on a mix of SAP, printed Excel sheets and some E-mails. Somehow it all (mostly) works out but nobody really knows how the company works.
And this is why AI will not take our jobs, Why would AI let itself do this work when it's got such a bright future.
I think AI will be way more able to understand convoluted process than humans.
What made IBM worse to me than the rest of the industry was the crazy crunch/burnout culture both in my project and utilization-wise. Also the absolute insanity of putting fresh out of college juniors on major aspects of multi-million euro projects.
They also seem to have zero organizational understanding of modern requirements engineering even though they talk a big buzzword game with "Design Thinking".
Suffice it to say, I didn't stay long.
Kinda the opposite of some of my experiences in big blue - in some of the software sides of things, productivity expectations were so low as to be really kinda funny.
You could (and people did) coast along producing not very much for years at a time. Which is probably why entire business areas just got shitcanned every so often. One office in particular operated more like an old-fashioned university campus. There was a yearly release cadence, with 3-5 months of that dedicated purely to merging together the work that the teams had produced over the past 7-9.
IBM Technology people seemed like that, yeah. I'm talking about IBM Consulting, where most (if not all) hiring seems to be happening recently.
Most companies I’ve worked for in 20 years doing this were a mess in one way or another, but also generally delivered an acceptable if inconsistent outcome.
I’ve had a few brief spells at companies in their “golden days”, where it just sings. Every once in a while there’s plenty of budget because the product market fit is hand-in-glove, and the first people set an aggressive but achievable bar for quality, and a mentoring culture emerged, and leadership still knew their trade.
That’s a quarter of my career at best, but it’s the stuff you remember.
Yeah, I was at a company when we were on top of the world. For a few years we could do no wrong but even when we hit our peak, behind the scenes it was chaos (and we assumed "the other guys" actually knew what they were doing while we were just winging it). How we were able to achieve so much during so much craziness is still amazing to me. It didn't last very long, of course, but at least I got to experience something like that--most people don't.
The only one I've seen otherwise was one of the big banks.
They were pretty well organised. They had relatively low productivity expectations for developers because they had nailed eveything down in terms of network security and legal compliance, but they accounted for that and expected it.
It may come from them having no shortage of funds, I suppose, and as a result no shortage of staff, skills and time to throw at things.
I worked at an oil major which was fairly well organized, but I think much of that was down to the nature of the business which is both highly integrated and highly regulated.
That's kind of re-assuring, it means nobody has some secret huge advantage over their competitors. (Well it's not re-assuring for me riding in a Boeing plane.)
I guess it's some combo of the Efficient Market Hypothesis and "Every complex system is always running in a broken state" (https://how.complexsystems.fail/#5)
Great excerpt
I call this the "shiny window effect".
30 years ago, when I started my first business from a converted garage, I used to think about all of the chaos I had to deal with. I'd look at the big companies with their shiny, mirror glass windows and think "when we're that size, we'll have it all figured out".
Then we started working for these large companies. We were supplying IT solutions to try help them deal with their chaos. I realised then that the only difference between them and us was the shiny windows and the scale of their chaos.
I apply this on a human level too. There's not one of us that has it all figured out, despite outward appearances.
Even in a segment as rigorously regulated as aerospace, Boeing ended up with planes that fly into the ground and lose doors while flying.
The horrors we see in any corporate-grade software-intensive company where development is a cost would drive H. P. Lovecraft to madness.
Surely it’s a matter of degree through? I was at a company with a recently acquired-in product line and the amount of chaos on release day each month was notably higher. I would have described the original product’s development process as awkward but way less bad than the new one.
I second this. When you think about it it’s a fallacy of the ideal to assume some non-trivial assemblage of humans would be able to be in perfect harmony with each other. That fiction of perfect execution of godlike companies is really a youthful impression when all your exposure is slick marketing campaigns, Hollywood movie representations and other artifacts that a company generates that enter the public square. When you see how the sausage is made it’s a shock to all who get to see behind the veneer. Once you get over it you’re a lot more forgiving for anything that doesn’t meet one’s unsubstantiated expectations because as they say life is hard.
I've seen and been in enough situations where a quite moderate changes could had improved QoL of the company people and conseq. the company itself.
How many times I was paid for that? Zero.
Add to that 'yes, we see it would be better for us, but [bullshit reason]'.
I guess you never worked for a Large Drug Company. I have a relative working for one (low-level employee) and pretty much on the way home, you pick up your bag of money bonus leaving every day :)
The benefits are 100x better that anything I have ever had in tech. Plus everyone gets stock bonuses and money bonuses when a new product is successful. And working from home, sure, why not.
And every day, one decision at a time, it is the quality, skills, and integrity of the people that make an organisation's culture, capabililty, and sustainability.
An organisation without a culture that attracts talent and maintains integrity in its decisions is nothing but a licence to burn money squatting in a building.
I know a pretty senior person who went to Apple and they boomeranged after about 6 months. At scale, there's a high level of chaos just about everywhere.
In general I would agree it’s not great to compete in a commodity segment if you can focus on differentiated products. You can make the same amount of cash from high-volume low-margin product or a low-volume high-margin product, with the latter generating sizeable IP you can also generate money from.
The split between HP and HPE is one example: HP gets the high-volume no-added-value segment and HPE tries to rebuild what was systematically killed by its descent into generic x86 hardware. They have very little headroom there, as HP/UX is on life-support and their high end has been stagnant for years.
The user experience for random x86 servers is utter garbage, especially at the low- to mid-tier. Bug-riddled firmware, weird licensing schemes for some features, IPMI is complete crap. Salespeople stuck in the 90s who can't get their head around the fact that I want an HBA for my ZFS, not some convoluted "RAID solution". Updating firmware is an adventure. Middlemen who all seem to think you'll be running Windows, again, stuck in the 90s or 2000s. I'd gladly pay a premium for something better here.
Unfortunately, the people who know better rarely are the same people who signs the checks.
Yeah, agree about the sale to Lenovo. That’s when they became just another “body shop” like Accenture or Deloitte.
This business - which they run under IBM Consulting in EMEA - they also bought off of PwC in the early 2000s.
That's the key to IBM success and longevity. Despite blatant incompetence, they have been pretty good at riding every single high margin fad in the industry.
Just another sad example of the MBA caste ruining an organization built on engineering excellence.