The Best Essay

Much of the restricted depth of this nature is a consequence of deliberate obfuscation or neglect by the people who are involved in the doing, such as trade secrets, or simply choosing not to write about or share the actual things that impact their craft.

You can easily accumulate depth of knowledge from reading in areas that are well documented.

I misread that for a second as "while breath comes from reading and talking and seeing, death comes from doing" which I thought would be a great line from a Zorro film, even if not that sensible.

Excellent point on "Death of a pig" by E.B White. It's the perfect example of a timeless essay, without a "big scientific idea".

For others unaware of it, that essay was written in 1948[1], go read it in full. It starts like this:

"I spent several days and nights in mid-September with an ailing pig and I feel driven to account for this stretch of time, more particularly since the pig died at last, and I lived, and things might easily have gone the other way round and none left to do the accounting."

[1] https://web.archive.org/web/20240227003736/https://www.theat...

When he mentioned lipstick and evolution right after I figured he was going to remark on the obvious connection later in the essay, but no.

I'd add to this category Haldane's "On being the right size". Great essay then, still great now.

Did you see this part later on?

"Almost any question can get you a good essay. Indeed, it took some effort to think of a sufficiently unpromising topic in the third paragraph, because any essayist's first impulse on hearing that the best essay couldn't be about x would be to try to write it. But if most questions yield good essays, only some yield great ones."

Sounds like you have a challenge in front of you.

This wouldn't be a classic Paul Graham essay without his two great hallmarks -- many passages of provocative valuable insights -- paired with periodic, bewildering attempts to sabotage his own argument.

I'll add some fan mail later if necessary (because I do get value from reading him), but for the moment, here's where I believe he went off course.

1. When he says the best essays are "ineffective," he's chosen the wrong word. They are "premature." They arrive before the world is fully ready to acknowledge their power. But they catch at least a modest following right away. And then their work grows and grows.

2. Essays about new technology can be quite powerful, and that's Graham's wheelhouse, so it's fine for him to talk up this cohort. But any serious survey of legendary essays needs to look wider. The most powerful essays redefine our social, moral, political and religious norms. Here are a few favorites that didn't just win their year; they stood out for centuries.

70 AD: The Gospel of Mark. Chronologically the first book of the New Testament, and look what that unleashed

1778: Common Sense by Thomas Paine. The boldest, fiercest justification for the American Revolution, and one that's still a touchstone today for anyone with a deep interest in the theory or practice of democracy.

1963: Letter From Birmingham Jail by Martin Luther King. Worth being on the list simply for its effect on the U.S. civil rights movement; even more significant as the unbreakable tuning fork for any civil or human rights movement anywhere.

We'll keep inventing new technologies, because that's what humans are good at, and I'm sure many strong essays will ensue. But it's the redefining of our social institutions that's likely to make the future so incredibly different from today. Anyone who can write a prescient reflection about society's new rules will get my vote for "Great Essays."

is French for "to try."

Which looks to be where the English word assay comes from too: https://www.etymonline.com/word/assay

I think the concept of a best essay is somewhat illogical, but exploring the idea like this makes an interesting post.

pg does specifically address this objection btw:

I love questions that seem naughty in some way — for example, by seeming counterintuitive or overambitious or heterodox. Ideally all three. This essay is an example. Writing about the best essay implies there is such a thing, which pseudo-intellectuals will dismiss as reductive, though it follows necessarily from the possibility of one essay being better than another.

Good comment. It surprised me that a definition of essay wasn't sought before asking questions about the supposed best. But that tentative approach actually struck the essence in an interestingly performative manner.

So you're saying that Paul Graham is trying? :)

From that same Wiki definition I notice that "In English, essay first meant a trial or an attempt.

That implies an iteration on some seed, or maybe just the intent to grow and iterate on the initial essay.

Personally I find that true. Most of my "essays" I keep for myself, or sometimes share just with close friends who I trust to bounce ideas off as they mature.

Later they become collections of essays in a folder, and I might join the dots between them to perhaps see the potential for something of a book, or a new essay that synthesises them.

So I think calling something an "essay" grants the author one special privilege; the right to revise and "re-attempt" the same piece without moral judgement.

That's problematic in the Internet age, where the provenance of a document is judged by dates, by diffs to archived versions etc. It wouldn't seem right to actually change the contents at the same URL.

Nor would it seem right to insist on tracking edits, because a writer always has the right to forget and disown old and weak ideas they've moved beyond.

So I think essayists have our permission to keep approaching the same idea over and over. The way composers would sometimes write ten different versions of the same piece.

What makes a "best" essay then is its improvement with respect to the author's previous attempts.

All we can ask is, "Is this Mr Graham's best essay so far?"

The best possible essay at any given time would usually be one describing the most important scientific or technological discovery

That sounds like a trekie concluding that the best literature is "Star Trek" licensed novels...

Sorry, but PG didn't invent or define the essay.

It is a form that is centuries old (and in prior incarnations, millenia old) and have its own long cultural history, classics, and canon, few of the latter being about "important scientific or technological discoveries".

Maybe if instead of "best" he changed it to "most impactful scientifically" he'd have a point.

Relationships are key to our survival, so I suggest personal relations could be just as good an essay as anything scientific

Some of my favorite books are old literature that describe relationship dynamics I still see playing out between people today. Swann’s Way, The Brothers Karamazov, even Canterbury Tales…

I think it was Vonnegut that wrote that the best books are the ones that tell you what you already know [but maybe you didn’t know you knew till you read it]. That’s something like surprise, and yet something like the opposite.

I'm suggesting that social science isn't really a science.

In reality and practice, no effort at science is 100% "really a science".

But for "social science":

(1) It can be a goal to have some of social science be real science.

(2) My wife's Ph.D. was in social science, and her dissertation was real science. The work started with a real question, in social science, gathered some real data, analyzed the data statistically, and came to some real conclusions. The conclusions were "new, correct, and significant". Some science? Yup.

Anyone can nitpick anyone by listing whatabouts they didn't intend. Whatabout this OTHER sense of the word "best". It's in the dictionary. You didn't address it, therfore you're a lazy man of straw. The only defense to that type of attack is to write everything in a way that never chooses any one path, spends all it's time mounting a defense to every whatabout, vaguely floating over eggshells, immune to attack. Who wants to read something like that? Or, you can trust the reader to try to understand the context. The context is a man who calls himself an essayist, has about a hundred essays spanning a decade, all in a very distinct style, where most folks reading them are fans of the prior ones. The context is the Paul Graham Essay sense of the word essay. It's not discussing emotional, spoken, money grossing essays. And to help the reader not get hung up on those whatabouts, he even explicitly spelled what sense of best he is talking about.

Hemingway once said just write the truest sentence you can think of and I think he was right. Good writing and good essays are truthful in a way that wouldn't be expected from commercial and marketing copy. But to write truthfully one must know what is true and this is surprisingly hard.

Well, yes exactly.

On the most important topic you could tell people something surprising about. And all those things you said. And … and (yes, that “and”) … so much more!

We are talking about the join of all essay joins. The essay supremum. By definition

I think we can agree there is a limit to the length of this essay. Proof by absurdum: if the essay is too long to read, that would imply unread word choices make a difference. So there is finite length N.

Now we just search for it. Unleash the monkeys! Or today, unleash the competitive adversarial AI cohort!

We can find it, just not copyright it. Which I think, is the best outcome for humanity. Assuming this essay is aligned with our interests, but we have set that unboxing in motion now so it’s not worth worrying about

I think it’s best to maintain some humor when talking about mythical bests, one way or another

Not surprised that pg proves once again that he is little more than a narrow-minded capitalist, who doesn't actually understand anything about culture, the humanities, literature, or what it is people actually value in those spaces.

His writing is consistently flat, uninspired, and egotistical. He's ok at writing clear prose and this plus his success in relation to YC has made him popular among technical people that also embrace capitalism. Anyone outside of this little cult realizes that pg says very little of value, particularly when he decides to comment on subjects outside of his capitalistic and technical wheelhouses. I have not read a single pg essay that touches on culture, ethics, or general epistemology that has given me anything other than a complete groan of a reaction and astonishment that this is the sort of thinking that passes the "high-quality" bar for the average technologist.

pg can try as hard as he can to write the "best essay". He will never convince this reader since he occupies a completely different world and basically isn't even capable of speaking to this audience in the first place.

...just to lengthen this comment more than is due, the quote is astonishing. I mean, how arrogant do you have to be to claim that the past hundreds of years of literary development, in which the notion of "best essay" is considered at best completely relative and at worst nonsense, must be wrong and that anyone who thinks otherwise must just be a "pseudo intellectual" (read: someone who disagrees with paul graham). He's really high on his own supply.

Not to mention that "best" is subjective, so there is no single ordering, but "How to write Paul Graham's favourite essay" wouldn't get as many clicks on HN (ironic for a site with anti-clickbait rules).

In earlier drafts I wrote "the existence of some set of best essays follows necessarily" but it sounded so pedantic that it didn't seem worth keeping just to protect myself from being accused of not understanding the distinction between a partial and total order. I assumed anyone reading it in good faith would understand what I was saying.

Paul has been putting me off lately. Everyone that disagrees with him is a "pseudo-intellectual" or some other hinted-at bad or incomplete person. The guy loves the smell of his own farts and isn't the towering intellectual or writer than he congratulates himself as.

Seems to me something like songwriters in popular music. Proven hitmakers and their expert associates still can't say for sure which of their offerings will resonate like nothing before.

You always have to try or you wouldn't have a chance. But you can't just sit down and determinedly write your best material ever, with any kind of certainty, right when you decide to.

Good fortune is going to need to smile.

Even worse, it mistakes the possibility of partial ordering to imply an actual total ordering.

I found your comment by searching for "it follows necessarily from the possibility of one essay being better than another" because it stood out as a fallacy.

you can't argue that a Ford F150 is objectively better

Perhaps if you limit yourself to a hidebound traditionalist view of the form but this topic has been explored by the Moving Picture Essay https://www.youtube.com/watch?v=oMDcIApBtL8

Or formalized, a partial ordering over some set doesn't imply a total ordering or a maximum value.

The pareto optimal essay?

This struck me too. I think we can do better than appeal to insult. Especially given the potential technical mistake. It’s quite disappointing, in a way.

me: write a story that is so good you get a 1 billion dollar book deal

Claude 3 Opus:

Here is a story that I hope captures your interest and imagination:

The Memory Thief

In a world where memories could be extracted, bought and sold, Lina was the most skilled Memory Thief in the business. The wealthy elite paid handsomely for the chance to acquire the experiences and skills of others. And Lina had no qualms about stealing memories from the unwitting. It was just business.

Her latest mark was Jonathan Thorn, a reclusive tech billionaire. The bounty on his memories was the biggest Lina had ever seen. Enough to set her up for life.

Infiltrating Thorn's mind was surprisingly easy. But as she slipped into his memories, Lina found herself drawn in by the raw beauty and staggering brilliance she discovered there. She witnessed Thorn's greatest triumphs and his deepest heartbreaks. And she began to question everything about her life as a thief.

Then, in the innermost recesses of Thorn's mind, Lina uncovered a memory unlike any she had ever encountered - a memory of the future. In it, she saw a world in flames, its population enslaved by a ruthless dictator. And she saw herself and Thorn, leading the last desperate resistance against the tyrant's armies.

Shaken to her core, Lina fled Thorn's mind without the bounty memories. But she took something else with her - a newfound resolve and a dangerous knowledge.

Now, Lina must track down Thorn in the real world and convince him that his prophetic memory is true. Together, they must figure out how to prevent the apocalyptic future only they know is coming.

Even if it means Lina must do the one thing a Memory Thief is never supposed to do - reveal who she really is.

What follows is a race against time and a battle against fate as an unlikely pair - the world's most notorious thief and the man whose mind she invaded - become the only hope for salvation. All the while, a forbidden love blossoms between them, forged by the intimate knowledge of each other's deepest secrets and the heavy burden they now share.

In the end, they will have to sacrifice everything to rewrite the future. But in the process, they might just find that the most valuable memories are the ones you make with someone else.

This response represents some confusion on your part and not a paradox or counterargument to rationality.

very very roughly the ideas you need are:

1. survivorship bias

2. self-interested optimisation is not guaranteed to lead to a global maxima, e.g. self-interested agents may be systematically less risk-taking than would be societally optimal.

3. it's fine for rational agents to account for private information, e.g. "I did really well at Stanford probably I have a better shot at a company than a random person." is a fine thing to include in reasoning.

I feel like the missing ingredients are risk, variance, cost, time horizon etc.

"Moonshot" type endeavours are not paradoxes. They do involve considerable uncertainty and usually some kind of large investment (time, money or something else). Most can't afford them and don't have any particular reason to believe they will be more successful than everyone else who has tried. Relative to most people's resources, time preference and risk tolerance they are bad bets.

If you are starving then your strategy for getting food should usually be the one with the highest overall chance of success, you care much less about factors like effort/reward ratio or getting the highest possible payoff.

The most common strategy for people who want a tiny chance at an enormous payoff is to buy a lottery ticket. Millions of people do that every week.

Such bets absolutely do feel paradoxical at times because multiplying a very small chance of success times a huge (usually easier to reason about) payoff seems to break people's brains a bit. This is often described as a "bug" in human cognition, though I have always felt it's an instinct for "exploration" in the sense of occasionally making small bets with huge potential payoff "just in case" your world model is wrong. Also that the way humans process probabilities and payoffs seems more logarithmic than linear (making small probabilities "feel" bigger than they really are).

I had a tangential insight recently. Your "best" isn't actually all you can do, because your idea of what your best is, is limited by your self-concept. Similarly, it is possible to do some things that are widely considered impossible.

From this it seems that the best strategy for doing impossible things is to assume that impossible things are actually possible. It's only by doing that that you'll find out.

It's a risky strategy though, since many things are actually impossible, and even for possible things, they may be difficult, or the easy way may be hard to discover.

Maybe there's some hueristic for sorting the actually possible, and from there, for sorting the reasonably doable.

But I'd be wary of hueristics because (barring unusual personal experience) they're likely to suffer from the same bias that led people to dismiss things as impossible in the first place.

...I don't think I get it. Like, if the angel is going to be a jerk, he could be a jerk with any question.

The philosophers could have asked "What is the meaning of life?" and the angel would answer "Of course, it is the meaning of life," without any of this ordered pair nonsense.

This is like trying to access information buried inside a data structure. You need the right incantations to get the right information content and not just the headers.

You can click on [4] which brings you to the footnote and when done you can just use the back button which brings you back to [4] in the text. Works for me.

I couldn't help but think of Nicholson Baker's first novel [0].

The Mezzanine tells this story through the extensive use of footnotes—some of them comprising the bulk of the page—as the narrator travels through his own mind and past. The footnotes are quite detailed and sometimes diverge into multiple levels of abstraction. Near the end of the book, there is a multi-page footnote on the subject of footnotes themselves.

[0] https://en.wikipedia.org/wiki/The_Mezzanine

This is the use case for Tufte-format layout. Notes, footnotes, and other stuff go into a comfortably-wide right-hand margin.

With books, I tend to not bother with the notes if they are bundled up at the end of the book or chapter. I prefer to have them on the bottom of the page. A web page doesn't even have the benefit of easy page turning to a bookmark, so I think some medium adapted solution like mouse hover would be appropriate.

I agree, having them separated is quite bad. They could have just put them into brackets ( ) and include it in the normal body of the text, that way if you're not interested you can just skip it, but if you are interested then you don't have to list through/switch tabs/search at the bottom of the page/...

I believe that the reading flow is quite important, probably near the top of importance, because if the reader gets into the reading flow they're much more likely to read it all and enjoy it!

That quote is also just factually wrong because we can have partial orders, or even just have a lot of totally different essays that happen to be equally good.

Sam went through the garage startup stage in 2005-2007, and again in 2015-2017 with OpenAI. He was good at it -- I saw both firsthand. At OpenAI he was scrounging GPUs when that's what we needed most, and wangled a prototype DGX-1 long before you could buy one. More importantly, he convinced people to leave multi-million dollar salaries at Google & DeepMind for a tiny startup no one had heard of. Now OpenAI needs to build a trillion dollars of infrastructure and lobby governments for favorable regulation, so he's doing whatever that involves. It wouldn't make sense for him to be running Ethernet cables today.

YC's public content focuses on early-stage startups. Should it have more on late-stage startups? Maybe, but it doesn't seem as impactful. The reason to produce public content is that it can scale to reach more people than the partners can talk to one-on-one. There are fewer late-stage startups, so the scaling problem hasn't hit yet. YC follows its own advice of doing things in ways that don't scale until scaling becomes necessary.

PG would probably never admit it, but any reference to a person such as @sama is not only talking about him as a person but also to him.

Of course one can only guess what the message for him might be.

He asked us, "Be you angels?" And we said nay. We are but men, write!

Tenacious reference.

Unashamed to admit my first action on the comment page was to Ctrl-F for 'tribute'

Disney can predict with a high degree of certainty that its superhero movies will do well, as there is a large, built-in market for those movies, and they tend to be conceived on the same creative blueprint or foundation. Even its 'duds' are still profitable.

Google something like "Marvel superhero fatigue". It seems that either the movie quality is decreasing or the audience is becoming more and more bored by these movies.

Is it important to predict how it will be received? If you are trying to tailor your work for your audience, then you are on your way to being yet another content farm and when’s the last time any of those published something great?

I know Paul does have some trusted readers that he shares early drafts with. Maybe he uses those readers as his focus group?

This was brilliant

What a weird comment that could extend to literally any post here.

Maybe we should just delete HN and replace it with a link to a climate change non profit, since apparently we're better off platforming that than discussing anything else.

Most people in society throughout the ages have not done the majority of their thinking. It has been handed to them from a pulpit or something similar. Even without an explicit religion, people still want someone to follow, so now successful people are followed. Successful people don't have to be intelligent or even moral.

As a commoner, you can accept that pollution is terrible, but then immediately think that it can't be a big deal because the people on high are flying around in private jets and living in mansions. Do you think we can convince people of the importance of having a sustainable environment, while the people on high pay lip service to the cause and then blatantly contradict themselves by showing us how they live? How do you get common people to care about their environmental footprint when they're constantly told not to be content with what they've got?

How does humanity rein in its covetousness? Do you think humans can live sustainably without solving that problem?

how?

I don't know if it looks different, or what it looks different from. To me, it looks like it was designed in 1999 (which it may have been).

There's value in people trying to derive things from first principles sometimes. It can lead to something unique, like outsider art.

For example, Darwin first described the idea of natural selection in an essay written in 1844. [1] Talk about an important topic you could tell people something surprising about. If that's the test of a great essay, this was surely the best one written in 1844.

I think he's speaking broadly about what counts as an essay, and does have at least one example in there to illustrate his point. He's also an esteemed essay writer, and I think it's fair for him to talk about what he values in an essay without needing to justify any of it by saying "and also, this person that some people respect did it like that"

Yeah. Imagine writing Darwin's essay on evolution in the 15th century. People wouldn't have been merely outraged, they would have killed him right away. And also today some possible essays may contain true and important insights, while being too taboo to not be immediately dismissed by society.

How do you organize those notes?

An essay is a search for an answer. The more easily the answer can be found, the shorter the essay will be.

I wondered that too. A subtle endorsement of Sama perhaps. A way of saying there's no bad blood there despite what the press was saying about his 'firing' from YC. Sama did a great job running YC, and certainly made PG a boatload of money. Also seeing the success at OpenAi, hard to argue Sama didn't make the right call to leave and focus on it, could hardly have done that plus keep running YC. (my guess at what happened is Sama started openai on the side and gradually it became more and more of his focus until there was a conversation about whether he should focus exclusively on it, which they mutually decided should happen).

What is the correct cadence at which Paul Graham should thank Sam Altman?

Maybe that's the process PG describes themselves, but I think that the amount of different "processes" or "human equivalent to LLM decoding/sampling techniques" used by humans is likely very large - far larger than the number of techniques available to LLMs today.

Today within LLMs, there typicality sampling, contrastive search, top-p/top-k (nucleus sampling), and a whole massive amount of other more obscure techniques which didn't make it into huggingfaces model.generate() function.

I believe that the ontological configuration of humans has wide variety. As such, I believe that humans go through a diverse and vibrant amount of different writing processes - and often radically different processes can still result in excellent works!

Specifically, beam search is really only good at dealing with sequence-sequence tasks, i.e. summarization or translation. It's also horribly, horribly inefficient in the worst case scenarios. That said, it's also the only good way we know of for enforcing sequence level constraints (i.e. like this https://huggingface.co/blog/constrained-beam-search)

He talks plenty about taboos elsewhere (e.g. "What You Can't Say").

Thanks for leaving the link to the PDF and web site. Already had some fun playing with the site and think I will give the book a shot.

Somehow a 233 page manuscript is not what I think of when I think of "essay".

Speaking of tributes

https://www.youtube.com/watch?v=_lK4cX5xGiQ#t=217

https://www.youtube.com/watch?v=PoAGasPLh30#t=213

Related: https://paulgraham.com/getideas.html

“Write the best essay in the world, or I'll eat your soul.”

— some shiny demon in the middle of some long and lonely road

All it takes is motivation. A lesson there for all of us, I think