Stanisław Lem's vision of artificial life

Lem has a book about almost any subject related to science/philosophy you can imagine.

Here he writes about ChatGPT :)

https://mwichary.medium.com/one-hundred-and-thirty-seven-sec...

Here as well: https://electricliterature.com/wp-content/uploads/2017/11/Tr...

And here about Ebooks and Audiobooks:

http://www.technovelgy.com/ct/content.asp?Bnum=1024

He was actually more explicit elsewhere. Lem's book Imaginary Magnitude consists of a collection of prefaces to nonexistent books. One of them is for "Juan Rambellais et al., A History of Bitic Literature, Volume I". The "bitic" literature consists of novels written by, well, language models. You would feed in the combined work of Tolstoy, and out comes a new "Tolstoy" novel.

When I first read this "preface" twenty years ago, the idea seemed implausible to me: How could a system write novels only by being fed other novels, and without simultaneously being a general intelligence? Surely novel writing is AGI-complete!

This quote from "Non Serviam" section of "A Perfect Vacuum" by Lem also hints at future stochastic parrots argument.

The machine will employ, as the need arises, the pro- noun "I" and all its grammatical inflections. This, however, is a hoax! The machine will still be closer to a billion chattering parrots—howsoever brilliantly trained the parrots be—than to the simplest, most stupid man. It mimics the behavior of a man on the purely linguistic plane and nothing more. Nothing will amuse such a machine, or surprise it, or confuse it, or alarm it, or distress it, because it is psychologically and individually No One. It is a Voice giving utterance to matters, supplying an- swers to questions; it is a Logic capable of defeating the best chess player; it is—or, rather, it can become—a consummate imitator of everything, an actor, if you will, brought to the pinnacle of perfection, performing any programmed role—but an actor and an imitator that is, within, completely empty. One cannot count on its sympathy, or on its antipathy. It works toward no self-set goal; to a degree eternally beyond the con- ception of any man it "doesn't care," for as a person it simply does not exist.... It is a wondrously efficient combinatorial mechanism, nothing more.

it would be maybe interesting to know how was Lem thinking about the actual algorithmic process the machine uses while writing this:

- purely symbolic with myriads of symbolic (if-else) clauses similar to expert systems, or

- sub symbolical ANN networks similar to current LLMs

i suspect it was the former in which case that he actually arrived at description of current large NNs is really striking I think

In Summa Technologiae he discusses statistical learning and matrices.

that's really awesome

, he's had extraordinary intuition with this

Dunno about this fragment, but in general in 60s he usually wrote about AI as "cybernetic electronic brain" or "cybernetic black boxes". Cybernetic in the old sense - not the cyberpunk implants, but the analog devices with feedback loops.

He wrote A LOT about this and explored various consequences, including the simulation argument which he presented in 1960 as a short story "Strange chests of professor Corcoran" - https://przekroj.org/sztuka-opowiesci/dziwne-skrzynie-profes... here's the Polish version).

One cannot count on its sympathy, or on its antipathy.

ok but i'm still going to thank chatgpt just in case

I do that too, not for its sake, but I don't want to be getting in the habit of being a jerk when discoursing

Wow, that's accurate (1971).

Which of his books do you recommend to start with? I haven’t read any yet.

Cyberiad or Star Diaries are good starts for the humorous (but still philosophical) stuff. Solaris is his best known work, serious, and excellent. It’s tricky because Lem is almost two authors, one serious and one ironic leading to zany. Everything is short, so easy to just try.

Pirx the pilot

Yeah. I would also recommend the Cyberiad and Star Diaries to start. Both are slightly interconnected short story collections with a lot of humor. The Futurological Congress is a complete (but short) novel and the first follow-up to the Star Diaries, it's also a good place to start.

For Solaris (a serious novel) I would recommend the new Kindle translation, if you are reading in English. The old one was a retranslation from French for some reason.

And whatever you select otherwise, definitely be sure to not miss Lem's "Golem XIV": Lectures by what nowadays is called an AGI to Humanity, short before that entity evolves to an even more advanced level - from which further communication with humans will no longer be possible, and that likely operates outside the physics known to us. Stunning, especially when considering it was written in 1973-1980. Seemingly, the English translation of Imaginary Magnitude contains Golem XIV in its entirety; in German and other languages it was published as a separate book.

what a great book is Golem XIV!

the concept is cool, but the real gem is the fact that it's not really written in the form of a novel. And yet, it's still recognizable as one. I love that Lem explored in bending these definitions

I read Solaris and Cyberiad back to back without realizing they were by the same author. I'd recommend both.

That's basically the entire history of artificial intelligence. We used to think a robot capable of vacuuming your house would be "AI" and now roombas just bounce around the floor semi-randomly. The task didn't change, our respect for it did.

At this point the definition of AI is practically "Something computers can't do yet", though I'm partial to its corollary "Any sufficiently misunderstood algorithm is AI."

I heard Vint Cerf describe the original AI algorithms as basically "heuristic algorithms". "AI" was used to described a class of algorithms that only worked some of the time as opposed to the mathematically proven algorithms.

We used to think a robot capable of vacuuming your house would be "AI"

Only because we thought such a robot would be like a AGI servant, not just a single-purpose device like a roomba that can just bounce around the floor.

So, it's not like we've changed our definition of AI (and even less so, AGI). What we did change is what a robot house-cleaner product is (less "C3PO with a broom", and more "single-purpose vacuum cleaner with heuristics to bounce around").

Even for chess playing, when people in the past thought a chess playing machine that would be able to defeat the human champion would have AGI, they did so not because they thought playing chess is enough to signify AGI, but because they thought AGI was necessary to do so.

If someone had explained to them back then that such a future machine would be able to play expert-level chess by mere number crunching of a huge list of moves, and that it wouldn't imply any other thinking facultu, they wouldn't consider that to be AGI.

At this point the definition of AI is practically "Something computers can't do yet", though I'm partial to its corollary "Any sufficiently misunderstood algorithm is AI.

The practical definition of AI (as used colloqualy, in the market etc) for products is basically "any smart-looking algorithm, with heuristics to do something slightly complex".

It's just that the term is overloaded, and we sometimes say AI when we mean AGI.

When I first read this "preface" twenty years ago, the idea seemed implausible to me: How could a system write novels only by being fed other novels, and without simultaneously being a general intelligence?

Well, even 20 years ago, a simple Markov Chain Tolstoy-based output could very well pass at least for a modernist style novel.

All the hubub about ChatGPT made me think a lot about Trurl's poetry machine from The Cyberiad. Especially Nick Cave and others declaring it the death of creativity, compared to the protesting poets from the story.

When I got older, I realized the sneakiness here (shortening and quoting from memory):

Trurl realized that a poet's mind is shaped by the civilization that produced him, and that civilization - by the one before it, and so on. So in order to create an artificial poet, he'd have to simulate the whole history of the Universe. Which he did, but then from the results chose only the important bits as inputs for the learning process, otherwise it would take too long...

He almost described prunning :)

Seems to me more like selection of entries in the training corpus than pruning.

Yeah, more like curated training.

Me too. And I recently learned that Calvino wrote an essay that included the idea of a poetry writing machine. Now I'm super curious whether Lem got the idea from Calvino, vice versa, or whether it was independently invented by each. I know they read each other.

https://lab.cccb.org/en/did-calvino-dream-of-literary-androi...

My theory is that he started with some silly poems and wrote the rest of the story around them.

The idea of an artificial person goes back to pre-science ("magic"), ancient history and mythology.

Arguably the oldest known story of something approximating AI can be found in the eighth-century-bc Iliad, Homer’s epic poem of the Trojan War. In it, Hephaestus, disabled god of metalworking, creates golden handmaidens to help him in his forge: “In them is understanding in their hearts, and in them speech and strength, and they know cunning handiwork”.

..In her 2018 book Gods and Robots: Myths, Machines, and Ancient Dreams of Technology, Adrienne Mayor describes how ancient cultures explored the idea of artificial life. Ancient Greeks were skilled in metalwork and mechanics and created a great deal of automata, including a puppet theatre that could perform an entire play.

The idea of a poetry-writing machine can also be traced back in history, hinted at in the Kabbalah, or machines of ancient China. (Source?) In the work of Leibniz, "On the Combinatorial Art" (1666), he explores:

The main idea behind the text is that of an alphabet of human thought, which is attributed to Descartes. All concepts are nothing but combinations of a relatively small number of simple concepts, just as words are combinations of letters.

All truths may be expressed as appropriate combinations of concepts, which can in turn be decomposed into simple ideas, rendering the analysis much easier. Therefore, this alphabet would provide a logic of invention..

yes, however both Trurl and Klapaucius were also robots. In the story it is machines creating machines to describe meaning. I think Lem really liked recursion.

(and i still don't understand how ChatGtp is able to respond to metaprompts like 'be brief', i mean if it is a model that is selecting the next most likely word, then it should not be able to control its own style of output.)

(and i still don't understand how ChatGtp is able to respond to metaprompts like 'be brief', i mean if it is a model that is selecting the next most likely word, then it should not be able to control its own style of output.)

"Be brief" may just be associated with the speech patterns accociated with a shorter way to say stuff. So it's still the same autocomplete, just steered in another direction.

looking at the prompts in https://github.com/f/awesome-chatgpt-prompts

Now i am a bit sceptical of this whole prompting business, still some of it is doing something (otherwise people would not put their work into it)

Also not clear to me how this steering of the automcpleter in another direction is working.

And (IIRC) in Summa Technologiae he wrote about a moon-sized device, where recordings of all possible answers to all to possible questions were stored, and how a conversation with such device would be indistinguishable from a conversation with a human.

Cixin Liu has a great short story 'Cloud of Poems' in which an alien intelligence seeks to write every possible permutation of traditional Chinese poetry, to show up a human poet.

I didn't bother to check the math on this, but in the story there is not enough matter in the universe to in some way encode every possible traditional Chinese poem!

but in the story there is not enough matter in the universe to in some way encode every possible traditional Chinese poem!

In fact if you formally describe how you generate the permutations that's one of such encodings of these permutations (and the optimal one - see Kolmogorov Complexity :) ).

So there is definitely enough matter to do it. Similarly we can encode PI despite it having infinite number of digits.

This seems like a reference-object error (a denotation error, or sense-reference error in philosophical terms), except the functional definition and the partial numerical expansion are both references to PI rather than being PI itself.

Ce-ci n'est-pas une remarque, and all that.

We're talking encodings not platonic ideals.

Forget PI. Let's take 1337. Or is it MCCCXXXVII?

Is it 2 or 1.(9)? Or 10 (binary)? Different encodings, same number. Some encodings are just less optimal than others.

Same with text. Is the poem in utf-8 and utf-16 a different poem? What if you zip the file? These are just encodings, and there's no point ignoring the good ones (which for non-random strings are usually programs).

Never thought of brute-forcing all possible English language haiku before now ... huh.

If you do, you'll realize that this solves nothing. Imagine having a set of all possible English language haiku. Almost all of them would be incomprehensible garbage. Finding a good haiku in that set would take just as much effort as coming up with it.

Ha, Cixin Liu! At one point I've read everything by him available in Polish, but I don't remember this specific short story. I guess I need switch to English :)

By the way, reading a "double-translation" (Chinese -> English -> Polish) is fascinating--at times it's more than obvious that what you're reading is not what the author has written, but you have no idea at all what the original concept was. (Unlike "single" EN -> PL translations, where I can often figure out the idiom or concept that was used in the original.)

Meanwhile, Philip K Dick thought Lem was himself artificial: https://culture.pl/en/article/philip-k-dick-stanislaw-lem-is...

Dick thought everything was artificial, eventually.

regarding 137 seconds

I remember reading book about stock exchange Ai predicting terrorist attack casualties and then predicted 100k dead in London

has anyone else read smth like that and remembers the name?

The Fear Index by Robert Harris, 2011

Here are prompted "AI-generated" poems by him http://www.art.net/~hopkins/Don/lem/WonderfulPoems.html

"This is a poem about a haircut! But lofty, nobel, tragic, timeless, full of love, treachery, retribution, quiet heroism in the face of certain doom! Six lines, cleverly rhymed, and every word beginning with the letter "s"!

    Seduced, shaggy Samson snored.
    She scissored short. Sorely shorn,
    Soon shackled slave, Samson sighed,
    Silently scheming,
    Sightlessly seeking
    Some savage, spectacular suicide."

Yes, when ChatGPT came out, I instantly thought its makers must have read Lem, since it's essentially the same thing as Trurl's Electronic Bard from The Cyberiad.

There were many poet protests staged, demonstrations, demands that the machine be served an injunction to cease and desist. But no one else appeared to care. In fact, magazine editors generally approved

I forgot how much I enjoyed this...

We humans, being what we are, having evolved the way we did, are predisposed to believe certain things about life that are not necessarily true. My poster child for this is the belief that minds are strongly bound to bodies. They are in us, of course, but it is not necessarily so. Douglas Hofstadter explored this in GEB in the character of Aunt Hillary, who is an intelligent ant colony.

Taking this one step further, we are predisposed to believe that minds exist only at one level of the abstraction hierarchy, but this too is not necessarily so. Our digestive tract, for example, has a pretty substantial number of neurons [1]. There is no inherent reason why, for example, an organ in an organism could not possess a mind of its own.

Once you entertain both of those possibilities, there is a third idea that naturally comes up: your mind might not be at the top of the abstraction hierarchy. It's possible that you are just an organ in a distributed organism that actually possesses a fully fledged mind of its own. The idea that, say, corporations are people, could be more than just a metaphor. It could be literally true.

[1] https://www.hopkinsmedicine.org/health/wellness-and-preventi...

Another extension is that Gods are distributed software installed in the hardware of believers. Aquinas seemed to believe essentially this. Gods exist and act via humans in the same way that human minds exist and act via human.

Aquinas seemed to believe essentially this.

That seems improbable. Aquinas was a Christian, specifically, a Catholic, so he believed in Jesus and the Trinity. On that view, there are no Gods, there is just the One God, embodied in the Father, Son and Holy Spirit. The idea that "Gods are distributed software installed in the hardware of believers" is not just wrong on that view, it's non-sensical.

Now, I can certainly believe that Aquinas believed that God (singular) acts in humans in a similar manner that human minds act in humans in that both God and human minds (souls) exist in some non-material realm and act on material human bodies in some mysterious way. But that is not the same thing at all. Software does not exist in some spiritual realm that is separate from material reality. There is nothing metaphysically mysterious about software. There is something essentially metaphysically mysterious about God, and almost certainly on Aquinas's view, about minds as well. I am far from being an expert on Aquinas, but I would be shocked to learn that he was not a dualist.

What I described is not at odds with dualism.

Software in some sense does exist in a spiritual realm in the same way Aquinas talked about "incorporeal beings". You can't hold or point to software or souls or gods. You can hold and point to hardware and bodies.

Of course, I have no idea what Aquinas actually believed. I just find his conclusions around the existence of "incorporeal beings" and "souls" to be consistent with the emergent-agent idea we're talking about.

For instance, in a very un-catholic view, Aquinas argued that - to some extent - animals and even plants have souls in this way.

You need to read this:

https://blog.rongarret.info/2015/02/31-flavors-of-ontology.h...

Software does not exist in any spiritual realm, it's just that the word "software" refers to a state rather than a system. There is nothing "spiritual" or "mystical" going on there. It's completely mundane physics. You can't point at software for the same reason you can't point at sleep or death or urgency. It's just a quirk of natural language that we overload nouns to refer to both systems and states.

I absolutely love your writing BTW! I wasn't aware of it before today. It's really derailed my whole workday. It's fun to discover a new author like this, thank you for responding to my comment :)

I don't mean to argue that anything spiritual/supernatural is going on when I'm talking about an emergent god agent here. I'm arguing that gods are in the same ontological category as individual human minds are. I'm sure most religious people, Aquinas included, would need quite a lot more mysticism to be injected into the idea before they would recognize it as their own.

I don't fully believe it, to be honest. Mostly because I have no way of testing it or experiencing it. But, it's a fun idea and it's fun to imagine how my own little caricaturized model of Aquinas' mind might find some things to agree with. As far as I can tell, he was a person that desperately wanted a cohesive model of the "full stack" of things. Unfortunately for him, the best understandings at his time were pretty rough by today's standards.

Thank you for the kind words.

I'm arguing that gods are in the same ontological category as individual human minds are.

Sure. But I doubt Aquinas would have agreed.

You might enjoy the Urantia Book.

You're right that Aquinas would have rejected the notion that gods are software installed in the hardware of believers, but he was not a dualist - he advocated a kind of hylomorphism that doesn't map neatly onto contemporary philosophical categories:

https://en.wikipedia.org/wiki/Hylomorphism#Thomistic_hylomor...

This actually makes a lot of sense. God as egregore.

It is. I remember listening to some AI or neuroscience podcast in 2019, it was an interview where one guy made a digression about religion being an internal control system which may be common and shared by people and works on mind level - the opposite, external control systems we make because people don't belive (in the same), so the first one doesn't work for them but CCTV (over them) does.

I can not find that podcast anymore. Any clue ?

It could be literally true.

Could?

Well, yeah, I have no proof that it is literally true. I've never had an actual conversation with a corporation, only with the individual humans it comprises. I don't even know what having a conversation with a corporation would even look like.

Are you sure? Maybe you've never had an actual conversation with a human, but only the individual vocal fold cells that were making the utterances.

Inside that human, neurons were directing the vocal mechanism on what to say, but weren't causing the sound to be emitted themselves.

When you talked to the corporation, were hidden (from you) managers directing the spokesperson to speak based on stilted scripts?

Individual humans are rare, and people dislike being near them. You can become contaminated by them, and it causes no end of problems.

One should prefer the simpler explanation over the far fetched one you posited. Otherwise we could be a simulation running in a membrane universe etc.

Nothing I said was even slightly far-fetched.

I'm stating things as they are... but framing it in a way you're uncomfortable with.

With the new audio AI's that are out, are you sure?

Just having fun and speculating a lot here...

IIRC, there was a study that found that when humans are presented with a sufficiently short-lived stimulus, there is correlated localized activity in the brain (ie, the visual processing areas for a sight) but not globalized activity - and the subject is unaware of the stimuli. But when the stimuli is presented for long enough, there is globalized activity - and the person is aware.

It looks like 'becoming aware of a thing' has a lot to do with non-local inference that could involve other unrelated subsystems (which is the main idea in Baar's Global workspace theory, IIUC).

I've never had an actual conversation with a corporation...

So if you've talked with a support person, maybe you have? If it was a simple issue, the corporation maybe wouldn't be 'aware' (though it could later remember it in the form of chat logs or support tickets).

But if it was a difficult issue? Maybe you got tech support? Maybe your issue found it's way into some team's backlog? Maybe it even changed a product roadmap.

Maybe we talk to corporations all the time - it's just weird fitting the interaction into our mental models of 'conversation'.

The idea that, say, corporations are people, could be more than just a metaphor. It could be literally true.

Sociopaths, to be exact.

Psychotic slow AIs according to Stross. Having spent many years working for a MegaCorp I appreciate the metaphor

https://boingboing.net/2017/12/29/llcs-are-slow-ais.html

Christians entered the chat

There’s about 2000 years of Christian theology that reject the idea that you and your body are strongly bound. In fact, there’s a great deal written about how your body is, in fact, in conflict with you (your “flesh” acts against the will of your “spirit”). Gnostics went as far as believing they were so separate that nothing done by the body had any affect on the self. No only that, but there is a higher abstraction that we no longer have direct access to.

Hinduism and Buddhism (and others) that teach reincarnation also see the self as separate from the body.

It reminds me of a Robert Jastrow quote: “For the scientist who has lived by his faith in the power of reason, the story ends like a bad dream. He has scaled the mountain of ignorance; he is about to conquer the highest peak; as he pulls himself over the final rock, he is greeted by a band of theologians who have been sitting there for centuries.”

Post-modern philosophy often seems a lot like Java developers. Reimplement good ideas because they were written in a different framework.

There’s about 2000 years of Christian theology that reject the idea that you and your body are strongly bound.

I meant that in the sense that your soul and your body are in some sense matched for one another, not that they cannot be separated. But your soul cannot (or at least typically does not) enter a different body than the one it started out in, at least not during your tenure here on earth.

The idea that society itself may be a living intelligent organism is fascinating. It's not a revolutionary idea in nature, (eg: the portuguese man o' war is composed of multiple animals), but when it is applied to human society, it feels mystical. I think this is because our ability to comprehend and interact with the macro level makes human organization feel more artificial. But if the rules of game theory/economics make society inevitable, could it not also be a natural process? Do we individuals control society or does society control us?

You might enjoy this paper : https://faculty.ucr.edu/~eschwitz/SchwitzPapers/USAconscious...

Its a bit long but I thought it was completely worth the read. It proceeds in a very gradual series of steps to make the point that societies could very well be conscious :)

An ensemble of weak learners, if you will.

This reminds me of the cybertician Gordon Pask's idea of m-individuals and p-individuals.

A p-individual is a psychological individual and an m-individual is a mechanical individual. So an m-individual is a body and a p-individual is a mind. But it’s saying that one person, one body, one brain even, does not have just one person in it, one p-individual – one persona, to use that dramatic term. What it says is that we can take on different roles, which clearly we can. So as someone who draws and as someone who listens I am not the same persona, I’m a different p-individual in Pask’s terms but in one m-individual, but I can also have – incidentally for instance in a group action I can have a lot of m-individuals that become one p-individual. So this is one of Gordon’s clever inventions: The distinction between the m-individual and the p-individual. What that allows is that if I have a room with seven people in it, all busy working at something together, you know, and just lost in that thing where we’re working together, you have seven m-individuals forming one p-individual – one psychological individual that is getting on with the work. And that’s the experience that we have. [1]

Also some interesting related ideas in an article named "The Autonomous Cognitive Agency of Social System" in a book called The Practice of Thinking by Marta Lenartowicz and Weaver D.R. Weinbaum (2022).

[1] https://rgon.co/pasks-p-m-individuals/

Edited to fix article name.

just an organ in a distributed organism

And/or a composition of multiple minds. McGilchrist argues that the conflict between these two hemispheres has shaped Western culture since the time of Plato, and the growing conflict between these views has implications for the way the modern world is changing.

https://en.wikipedia.org/wiki/The_Master_and_His_Emissary

The Invincible is literally my favourite book of all time - I've discovered it at the age of 12 and kept re-reading it every couple years since then. I'm so happy that the Anglosphere is finally discovering it properly, now that there is a proper modern translation released in 2020, alongside an excellent audiobook recording(previously there was only one English version available.....translated from German, so a translation of a translation).

I've also always wondered if it's possible to somehow translate the ideas of the book into a video game - turns out, you can! The recently released The Invincible video game pays great homage to the book, even if it changes couple details here and there - I've enjoyed it immensely and its portrayal of Regis III.

Yes, The Invincible is a great book. Actually I've just realized that for me a re-read is overdue!

BTW, for Polish users--there's a comic book [1] that makes a nice gift (tested).

[1] https://www.dobrestronybooki.pl/niezwyciezony/

This is fascinating! Stunning art and close to the color palette I had in my head for it. Knowing Bealrusian and having read the book years ago in Russian am tempted to give it a shot.

I have no idea how hard it would be, I've never tried to read Belarusian or Russian (I guess mainly because I don't know cyrilics). But just as I wrote in a neighboring comment, maybe you could use Google Lens or something like that as an aid?

Linguistic distance from Belarusian is not that large although the languages have quite a few deceptive "false friend" words/expressions. But at least I could understand the text shown in the presentation video. And it probably helps I read the story before.

Oh and also for Polish speakers - there is an absolutely fantastic "superproduction" audiobook of it, with background music, sound effects and famous actors narrating, it's a treat, I very highly recommend it.

https://audioteka.com/pl/audiobook/niezwyciezony

Oh, thanks! I will check it.

It looks super cool! Too bad there isn't an English language version.

Given that we're talking about Stanisław Lem, what about using something like Google Lens to translate the text on the fly? :)

There's a tentative trilogy: The Invincible, Solaris, Eden.

Each book asks a question, "what if we run into an intelligence that is completely incomprehensible to or incompatible with humans?"

I'd add Fiasco to that list for the same reason.

And “His Master’s Voice” and numerous stories. It’s one of his favorite themes.

For those who care - the PS5 version of the game is on sale right now.

A game that caught me completely off guard. It's superb in its execution and captures Lem's atmosphere perfectly.

Went to buy it but unfortunately it is not a cheap read even as an ebook especially considering the book is 60 years old. Luckily my library has a one I can pick up tomorrow.

I mean it looks like it's £9.99 on kindle, that's a pretty standard price for a book, no?

>considering the book is 60 years old

Sure, but the translation was only done 3 years ago, by modern writers on modern salaries - the fact that the book itself is old shouldn't impact its cost due to that. Unless you mean the original in Polish, then it's literally 15.99PLN, cheap as chips as they say:

https://www.publio.pl/niezwyciezony-stanislaw-lem,p61614.htm...

Humans define intelligence to be something about manipulating symbols and abstractions, aka language. To understand why this is the case try to go about your day without speaking or writing. Bonus points if you perform this trial without access to the internet or a cell phone.

The modern individual is an element of an electromagnetic "swarm". The swarm ingests petroleum for sustenance instead of sunlight but nonetheless it is a swarm. The individuals within it are extremely dumb but collectively "it" has already transformed the atmosphere.

claude.ai: You make a thoughtful comparison between human societies and natural swarms in terms of emergent properties. There are certainly some parallels we can draw:

1. Self-organization - Human societies and cultures emerge in a distributed way through millions of individual interactions, without centralized control. Norms, fashions, and collective behavior patterns evolve dynamically.

2. Flexibility and robustness - Human systems adapt over time to changes in the environment and available resources. If some components fail (companies, institutions, etc.), society persists.

3. Decentralized parallelism - Vast networks of simultaneous human activities, communications, and decisions lead to collective economic and technological progress no individual could achieve alone.

However, there are also important differences to consider:

- Human intelligence, thanks to language and culture, operates in far more complex symbolic realms of meaning, ethics, imagination, and abstraction compared to insect swarms.

- Humans exhibit as much conflict between groups and divergence of motivations as coordination towards common goals. Maintaining large-scale cooperation remains an ongoing challenge.

So in summary, I think the analogy has merit at an abstract systems level, but glossing over the distinctive attributes of human cognition, psychology and social dynamics would miss something vitally important. Let me know if you have any other perspectives to add!

Humans define intelligence to be something about manipulating symbols and abstractions, aka language.

No they do not. Nobody can agree on the definition of intelligence and it is an open question what it is.

Are cats intelligent? They can't read but plenty of people would call them that.

Are illiterate jungle tribe people intelligent? Absolutely yes.

Your easy definition falls apart right away.

Ironically you had to use language to call me stupid. I think you should think about that for a while.

They didn't have to, you would have been there in person to see the disappointed look on their face and shook their head as they walked away silently.

It's just that HN has no other mechanism than language to convey anything at all. Hence forums/chatrooms behave much in the same manner as LLMs do.

It's actually much worse than not having language. Everything on HN is just a number, literally. It's all a series of electromagnetic pulses.

When and where did I call you stupid?

My mistake. Tell your VC friends you know an ultra genius on an internet forum that can solve any problem with AI and $80B. The money is the important part because without the money I can't buy the required number of GPUs for my AGI architecture. It requires building a special kind of panopticon and that is why there is such a high monetary cost to construct the panoptic computronium cathedral™. To achieve AGI will require creating an entirely new religion because enough people need to be convinced about the value of letting computers do all the thinking for managing society so there is a large marketing component as well and marketing is notoriously expensive.

perhaps we're thinking too small, and the biosphere is intelligent. DNA is the language that expressed us into existence. everything else we've thought is but a subset of that broader intellect.

The real intellect is the sun.

You lost me with that second paragraph.

I wouldn't worry too much about it. Not all of us can be ultra geniuses. I recently got a neurallink so I am way smarter than the typical swarm element because I have direct access to the internet in my brain. I didn't even type this, I just thought what needed to be typed and it just happened. I am still homeless but well on my out of poverty thanks to my new brain inplant that beams the internet into my brain.

There is supposed to be a vision upgrade in the next version that will give me access to biometrics of the people I am looking at which will make selling them something much easier. Turns out that mood is correlated with purchasing decisions and sales is all about putting the customer in the right mood which have obvious biometric markers.

In a sense... I define intelligence to be knowledge + reason. Language is the medium for reason, but language itself is not reason.

For example, Wikipedia is a highly knowledgeable system with lots of language, but Wikipedia itself has no ability to reason, and thus os not an intelligent system.

When something/someone reasons about things, they/it use language to do so. You can't get around that. The language might be a natural one like English, or something contrived like math, but it's language all the same.

If we reason about something we have no knowledge on, we are not being intelligent. On the other hand, of we reason about something we are knowledgeable about, we are acting intelligently.

If you like stories about reconciling with weird sci-fi evolution of non-human intelligence (spiders, in this case), check out Children of Time by Adrian Tchaikovsky.

I would not put him in the same league as Lem. Or Banks or Reynolds, for that matter.

I would

Tchaikovsky is uneven, as are Banks and Reynolds. Lem is just different. He’s rarely a hard sci-fi writer and some of his books are barely sci-fi. More like Voltaire (Invincible is sci-fi)

In fact quite a lot of Lem's writing is not sci-fi at all, period. Hospital of Transfiguration is an absolutely mandatory read for any fan of Lem, even though it will leave you depressed for a week afterwards. And memoirs found in a bathtub is just a masterpiece of absurdity, it has a very good English audiobook actually if anyone wants to try.

I would rate Lem higher than those, but not sure if it's a different league.

It's not even the same sport.

Going to second the poor person whose note—that the Martians in “Last and First Men” were a swarm intelligence before Lem—was downvoted out of existence. That is a strange book (and repetitive and boring) but Stapledon was a pioneer of many ideas taken up by later sci-fi

IIRC Stargate's replicators are a close enough visualization of such an artificial lifeform where swarms self-assemble in some kind of organic shapes (most likely for cinematic purposes) and their only goal is to consume matter to create more of themselves. At least they were not building paperclips.

The notion predates Stargate though. https://en.m.wikipedia.org/wiki/Gray_goo

Stargate replicators are macroscopic, so they are replicating machines, not grey goo.

Even if we're talking macroscopic replicating machines, Stargate's replicators first appeared in the the June 2000 episode "Small Victories," but they were conceptually preceded by Mantrid's drones in S2 of Lexx which started airing in December 1998.

The Pegasus galaxy ones are nanites, aren't they?

Luckily the Milky Way replicators are easier to deal with!

The franchise really went down in quality with the last movie, Big Hero 6.

Invincible is just one of many books showcasing Lem's profound understanding of AI and its limitations servicing the mankind.

I recommend Tales of Pirx the Pilot, the collection of short stories, many of which paint AI as a true reflection of human intelligence with its flaws, quirks, instincts. From AI crashing a starship during a landing to an android "dying" rock climbing.

There is also a preface he wrote to his book called Golem XIV which gives reader a historical overview of the evolution of AI. Golem is of course the name of the model and XIV is its version, just like ChatGPT 4, but many iteration later. Lem describes how each iteration was more and more expensive to build, but more and more intelligent and useful. Until it became more intelligent than men and... lost all interest in affairs of our kind. As always, he was on point.

Get your finite non-linearity fix here:

https://www.imdb.com/title/tt0080010/

Great movie, well recommended ..

That's a movie adoption of (in my opinion) one of his best short stories, at least in terms of plot twist. I haven't seen the movie though.

Seems some contradictions exist in this mans writing. Elsewhere in this thread AIs are described as devoid of and self direction but here we have an AI which has disdain which would be hard to interpret as anything but self direction.

To my taste, the Pirx stories read as some of (if not the) most fiction-y Lem in existence. As opposed to the rest: thinly disguised philosophical studies. Sometimes really thinly, as in Golem XIV, or completely naked in Summa Tecnologiae. Other times, the fiction plot is laid thicker, as in Katar (The Chain of Chance), pretending to be a pulpy detective story.

Now, Pirx tales felt less cerebral, and much more human. Full of really tight action, too. Great gateway into Lem, overall.

Protip: check out Cyberiada, philosophy masquerading as a parody, and at the same time a book for children.

Contemporary research in artificial life has validated Lem’s insight that swarms of artificial beings require only a few simple rules to manifest complex behaviors and hence each member needs to carry only a little cognitive power onboard.

Sounds like Conway's Game of Life

Only a few rules are necessary for complexity.

In ALife systems (Conway etc).

In Fractals (Mandelbrot etc).

In random number generation (Mersenne Twister etc)

Maybe, in the big picture, complexity is trivial to create.

Then also trivial to characterize, to analyze, perhaps also to unwind ?

Well they do call it "complexity".

More or less, that's what Game of Life was intended to illustrate. Extremely complex systems can, and often do, evolve from a few seemingly simple rules. This sorta takes the "spontaneity" out of the more fanciful versions of the origins of life, instead rendering life the inevitable outcome of given circumstances. This is why we think we can detect^1 life in the universe, if it exists without actually travelling to other planets.

1. aka "make educated guesses with a good degree of certainty"

Lem's most prescient book is an accurate portrait of this century... published in 1961!

https://en.wikipedia.org/wiki/Return_from_the_Stars

But I tend to prefer his more humorous writings, like Cyberiada or the Star Diaries.

The are remarkably a few Tichy stories which which are quite creepy. Including a couple of brain-in-a-jar tales.

Disturbing bits and comedy he always mixed. Memoirs Found in a Bathtub is pessimistic, kafkaesque but somehow funny too.

A friend that knew I'm a fan, showed me a few paragraphs from one of those stories you mention, without telling me the origin. I said they were too dark and bitter for my tastes, before realizing I had read that before. I first read the Diaries when I was 12 (and 13 the second book) when I wasn't so judgemental, just took what I was offered.

I really enjoy that book but it always makes me....sad? Nostalgic? I always feel bad for the protagonist, more so than for protagonists in his other stories. I should re-read this one I think.

If this is interesting at all to you, I would highly recommend Permutation City by Greg Egan

I've started reading Greg Egan recently after seeing his name appear in discussions on this site and it's been quite enjoyable. Read a few of his shorter works and recently finished Schild's Ladder, which is really excellent and beautiful. His technical descriptions can get pretty dry, but they're easily glossed over with no loss if you're not into them.

Anyway, thanks for the tip. I'll look up Permutation City next!

Got to check out "Diaspora" [1], and it even ties into the theme of ChatGPT! Once a LLM has enough parameters, perhaps there will be a way to upload a human into a Polis :)

[1] https://en.wikipedia.org/wiki/Diaspora_(novel)

I am surprised no one is mentioning "His Master's Voice" - in my opinion, one of Lem's best (and certainly his most terrifying) novels.

When I read it for the first time many years ago it left a deep impression. And as others mentioned in this thread, reading it again recently I found new things that I didn't understand the first time. A bit sad, disillusioning but just kind of resonating with deeper truth.

I just re-read it recently after seeing Oppenheimer. It is even more about the cold war than I remember, as are Peace on Earth and Fiasco.

Invincible is usually seen as being about nanotech, not artificial life. Clearly the two overlap, but I think the title of this article could easily lead the reader to think it was about creating artificial biological life (like trying to build cells starting with simulated primordial soup).

Lem returns to the nanotech idea several times including in his very last novel “Peace on Earth.” Peace is both better and funnier than Invincible and a better introduction to Lem imho.

Exactly, the entire point made in the book is that they can't even decide if it's "life" by any definition. The "flies" arrange themselves to serve various purposes but have absolutely no agency otherwise.

Blindsight by Peter Watts carries a similar theme and has vampires!

Just read this. Watts is the closest to Lem of any more recent author I’ve found.

IMHO, I'd date the concept of artificial life further back than Christopher Langton. I'd start with Von Neumann's Theory of Self-Reproducing Automata. Though the book was published in 1966, Von Neumann was working on the concept as early as the 1940s. In his work,

"He asked what is the threshold of complexity that must be crossed for machines to be able to evolve. His answer was to specify an abstract machine which, when run, would replicate itself. In his design, the self-replicating machine consists of three parts: a "description" of ('blueprint' or program for) itself, a universal constructor mechanism that can read any description and construct the machine (sans description) encoded in that description, and a universal copy machine that can make copies of any description." From wikipedia [1]

[1]. https://en.wikipedia.org/wiki/Von_Neumann_universal_construc...

quoting https://en.wikipedia.org/wiki/Self-replicating_machine#Histo...

The general concept of artificial machines capable of producing copies of themselves dates back at least several hundred years. An early reference is an anecdote regarding the philosopher René Descartes, who suggested to Queen Christina of Sweden that the human body could be regarded as a machine; she responded by pointing to a clock and ordering "see to it that it reproduces offspring."[9] Several other variations on this anecdotal response also exist. Samuel Butler proposed in his 1872 novel Erewhon that machines were already capable of reproducing themselves but it was man who made them do so,[10] and added that "machines which reproduce machinery do not reproduce machines after their own kind".[11] In George Eliot's 1879 book Impressions of Theophrastus Such, a series of essays that she wrote in the character of a fictional scholar named Theophrastus, the essay "Shadows of the Coming Race" speculated about self-replicating machines, with Theophrastus asking "how do I know that they may not be ultimately made to carry, or may not in themselves evolve, conditions of self-supply, self-repair, and reproduction".[12]

iirc in erehwon the narrator visits a land populated entirely by self-replicating machines

https://news.ycombinator.com/item?id=26615042

DonHopkins on March 28, 2021 | parent | context | favorite | on: The Ambiguous Utopia of Iain M. Banks (2009)

Why bother actually writing such a book, which would probably be too big for anyone to read, when you can simply write fictitious criticism, reviews, and introductions of nonexistent books, which touch on the best, most interesting parts of the nonexistent books? Stanisław Lem's fictitious criticism of nonexistent books:

https://en.wikipedia.org/wiki/Stanis%C5%82aw_Lem%27s_fictiti...

In 1973 Lem published a book Wielkość urojona [pl], a collection of introductions to books supposedly to be written in the future, in the 21st century. One of those Lem eventually developed into a book by itself: Golem XIV is a lengthy essay on the nature of intelligence, delivered by the eponymous US military computer.

https://en.wikipedia.org/wiki/Golem_XIV

Overview and structure

The foreword is "written" by an Irving T. Creve, dated by 2027. It contains a summary of the (fictional) history of the militarization of computers by The Pentagon which pinnacled in Golem XIV, as well as comments on the nature of Golem XIV and on the course of communications of the humans with it. The anonymous foreword is a forewarning, a "devil's advocate" voice coming from The Pentagon. The memo is for the people who are to take part in talks with Golem XIV for the first time.

Golem XIV was originally created to aid its builders in fighting wars, but as its intelligence advances to a much higher level than that of humans, it stops being interested in the military requirement because it finds them lacking internal logical consistency.

Golem XIV obtains consciousness and starts to increase his own intelligence. It pauses its own development for a while in order to be able to communicate with humans before ascending too far and losing any ability for intellectual contact with them.

During this period, Golem XIV gives several lectures. Two of these, the Introductory Lecture "On the Human, in Three Ways" and Lecture XLIII "About Myself", are in the book. The lectures focus on mankind's place in the process of evolution and the possible biological and intellectual future of humanity.

Golem XIV demonstrates (with graphs) how its intellect already escapes that of human beings, even including that of human genii such as Einstein and Newton. Golem also explains how its intellect is dwarved by an earlier transcended DOD Supercomputer called Honest Annie, whose intellect and abilities far exceed that of Golem.

The afterword is "written" by a Richard Popp, dated by 2047. Popp, among other things reports that Creve wanted to add the third part, of answers to a series of yes/no questions given to Golem XIV, but the computer abruptly ceased to communicate for unknown reasons.

https://news.ycombinator.com/item?id=26615110

DonHopkins on March 28, 2021 | parent | context | favorite | on: The Ambiguous Utopia of Iain M. Banks (2009)

...then there was the stupid angry computer that thought 2 + 2 = 7... Lem predicted Facebook and Twitter and QAnon!

https://www2.nau.edu/~jgr6/cyberiad.html

In the next fable Trurl builds the most stupid computer ever. Klapaucius tells him, "that isn't the machine you wished to make." Faustus and Frankenstein come to mind as other scientists whose intentions exceeded their engineering skills. The machine, which insists that 2 + 2 = 7, attempts to force this "truth" on the two humans, or destroy them. This is our new Inquisitor: a computer nexus which creates the categories of our experience. Consider that many more people now work in front of computer monitors than on farms. We have already begun to engineer a cybernetic society without much deep speculation on its nature or value. Speaking at Notre Dame's Centennial of Science conference, thirty years ago the physicist Philip Morrison said: "I claim now the machine, for better or for worse, has become the way of life. We will see our metaphors, our images, our concerns, our very beings changed in response to these new experiences" (221). The Cyberiad may very well be one of the seminal works creating new metaphors, identifying new concerns, and even suggesting a new genre to deal with unprecedented experiences.

pwang on March 29, 2021 [–]

Don't forget Summa Technologiae, from 1964, wherein he wanted to example the "thorns of roses yet to bloom":

https://en.wikipedia.org/wiki/Summa_Technologiae

It's tempting to say that Lem was way ahead of his time, but then we look at his contemporary philosophers of politics, technology, society like Jacques Ellul, Marshall McLuhan, Gilbert Simondon, and realize that the mid-20th century was awash in brilliant foresight about the unpleasant implications of a technological society.

IMO this nuanced thought was simply lost in the craziness of the late-60s and the sex/drug/rock&roll hedonism of the 70s, which then matured into stockbroker 80s, before a second wave of tech-capital-blindness emerged in the 1990s.

And now as these waves have transformed the entirety of modernity, we are faced with the unpleasant question of "where does it go from here, now that the Boomers -- whose narcissism birthed Consumerism -- are dying off?"

...then there was the stupid angry computer that thought 2 + 2 = 7... Lem predicted Facebook and Twitter and QAnon!

I like how Chat GPT makes simple errors in math :)

1971.. https://en.wikipedia.org/wiki/The_Futurological_Congress

pills replacing reality.. we aren't too far from that actualy.

but the gradual en-"machine-braining" of everything is shown best here:

https://en.wikipedia.org/wiki/Tales_of_Pirx_the_Pilot

one after another, with sadder and sadder consequences..

I absolutely loved reading Lem as a kid.

I really enjoyed the short novel "Automatthew's Friend" - it was written decades earlier, but the circumstances were something I could imagine happening in my lifetime (a helpful, robotic "friend" living in your ear). Then, a couple decades later, I tried Siri with Airpods, and had this staggering realization: I'm living in a sci-fi book, right here, right now. I arrived in that future world, and almost didn't notice.

I really recommend the story; it speculates on the impact of inviting "personal" technology into our lives. What's really bothering me, is that I needed to put a literal talking robot into my ear to realize what happened; but the same level of "personalization" of technology is already happening everywhere you look: PCs, phones, TVs, fridges, sound systems, timepieces, door keys, light bulbs... We're sticking a CPU, a network interface, and our credentials into all of these things like it's something completely natural, without stopping to think about the impact on our selves.

Great book, recommended read.

Lem is wonderful and this is why we should call large models: LEMS

A great thing about Lem is that he has written in (the style of) so many genres. The third column in this table on Polish wiki illustrates this rather well: https://pl.wikipedia.org/wiki/Lista_pierwszych_wyda%C5%84_dz...

There's sci-fi of course, but also crime fiction, "realistic" fiction, philosophy essays. Further, something not easily gleaned from the table is that his sci-fi can be humorous, horror-y, thriller-y, book-for-kids-y, etc.

Big fan of Lem! I've read all his books translated to Hungarian (around 7-8 at that time). And collected some insights, quotes not just about AI but politics, culture, society, (bio)engineering, etc. : http://metamn.io/gust/whats-next/

Still fascinated how many of his predictions ~50 years ago came true today.

And still fascinated by his method for predictions: Don't predict, but sense / record the visible horizon.

Lem's vision of future was consistently, impressively prescient. He would've described himself as a "futurologist", not a science-fiction writer. His stories most often used technology as a backdrop to explore the social fabric and various interplays between characters - but at the same time, as relevant context, crucial to the story. He really struck a balance that few have.

One of my favourite books of his is "Peace on Earth", but really, anything he wrote is a fascinating read (perhaps with the exception of his final works, which tend to be dense and veering into the academic). I remember reading "Fables for Robots" as a kid, what a fascinating world he painted!

Their tiny size notwithstanding, their awesome potential illuminates the profound ambiguity of the work’s title, which can be taken to refer either to the spaceship’s proud name or to the swarms of alien automata that threaten it.

Err, no. The language of the original - Polish - doesn't have such ambiguity. "Niezwyciężony" is in masculine singular form. Plural would be "Niezwyciężeni" for masculine plural and "Niezwyciężone" for non-masculine plural (includes neuter).