seangoedecke.com RSS feed

Radical responsibility means treating people like tools

Fri, 04 Sep 2026 00:00:00 GMT

A lot of people think that good leadership requires radical responsibility. Conscious Leadership defines it like this:

Taking full responsibility for one’s circumstances (physically, emotionally, mentally and spiritually) is the foundation of true personal and relational transformation. Conscious leadership and teams take full responsibility – radical responsibility – instead of placing blame. This means locating the cause and control of our lives in ourselves, not in external events.

Good leaders are outcome-focused. They are interested in actually succeeding, not in role-playing someone who’s trying to succeed¹. Placing blame is an excellent strategy for that kind of role-playing: if you can identify someone else who screwed up, you can claim to be trying to succeed no matter how badly you fail. When you look at people who are genuinely focused on winning, they don’t spend time blaming when things go wrong. They’re always focused on their outs: the remaining pathways to success, however few and narrow.

Never placing blame sounds like a kind thing to do. However, I think there’s something a little sociopathic about “radical responsibility”. If you assert full responsibility for your own circumstances, then you’re necessarily not sharing responsibility with anyone else. That forces you to take an instrumental view of the people around you. They’re not peers who you might trust and be disappointed by; they’re either useful assets to be cultivated or useless liabilities to be handled or avoided². If you rely on someone and they let you down, that’s your fault for being stupid enough to trust them in the first place. You ought to have made better tactical choices.

Many successful leaders adopt this mindset because it works. It really does help you win. But I don’t recommend it as a general approach to life. Trusting other people with responsibility is how you treat them like people, instead of like tools. And if you do that, you’ll sometimes assign praise and blame to them, instead of hoarding it all to yourself.

In fact, many people role-play someone who’s trying to win as a substitute for actually trying. Here’s a relevant quote from Sartre’s Being and Nothingness: “The attentive pupil who wishes to be attentive, his eyes riveted on the teacher, his ears open wide, so exhausts himself in playing the attentive role that he ends up by no longer hearing anything.”
↩
Or moderately-useful assets who can potentially be steered into usefulness with some feedback — but still never blamed, in the way that you might adjust a misbehaving power tool without blaming it. To my mind, the canonical philosophical treatment of this is Peter Strawson’s Freedom and Resentment, where he describes the “objective attitude”.
↩

How to protect yourself from workslop

Wed, 02 Sep 2026 00:00:00 GMT

“Workslop” is when your colleagues or bosses communicate with you by pasting big chunks of AI-generated text. The core problem with workslop is that the effort involved is asymmetrical, like a denial-of-service attack: it takes almost no effort to produce text with AI, but it still costs effort to read¹. Here are some ways to protect yourself.

If you have enough authority or social capital, you can and should simply tell them “hey, don’t do that” (for instance, if you’re a senior engineer and an intern starts doing this to you). This is the easiest way to handle workslop. But you probably aren’t in a position to have that conversation with all of your colleagues, and you certainly can’t have it with everyone in your management chain.

One step above just telling a colleague to stop is to drive them around like a coding agent. I wrote about this in AI makes weak engineers less harmful: if a colleague is simply pasting your messages into Claude Code and sending you the outputs, you can treat them like a high-latency Slack interface to Claude Code. It won’t be as good as a normal coding agent, but it’ll often be better than nothing.

Another strategy is to use AI to fight AI. This is a good one for handling workslop from managers. You can do this in two broad ways. First, instead of carefully reading it, paste it into an LLM of your own and ask for a short list of the salient points. Second, you can sometimes simply ask an LLM for an entire response. In a sense, this makes you part of the problem, so I can see why some people might be uncomfortable with it. But it’s more sustainable than spending ten minutes of your effort for every ten seconds of theirs.

You can also bias toward calls or in-person meetings. Workslop is just a special case of the general “your coworker is bad at communication” problem. One classic way of handling this that works even better on AI content is to say “hey, let’s schedule some time to chat about it”. This works for two reasons: first, your colleagues can’t give you AI content over a call, and second, forcing people to spend a chunk of their time talking to you (i.e. to make the effort symmetrical) is a good way to filter out predators.

Finally, you can sometimes simply ignore the workslop. This is particularly true for long status updates or pull requests from outside of your organization². You don’t have to respond to AI content as diligently as you would human content. You can match their lack of effort with your own: skim it, put off reading it until later (or never), and so on. If something’s really important, they’ll tell you in their own words.

Technically, not all cases of sending someone AI-generated content are workslop. If the effort is not asymmetrical — if the AI user has genuinely put a lot of their own time into the content — I don’t think it counts as slop, and you should just try and look past the AI style and treat it like a human message.
↩
Some messages — particularly reports directed at the entire organization — may not be intended to be read at all. Written artifacts can have many purposes beyond communication: evidence of effort, a reference document for later communications, a way to cover somebody’s ass by proving they considered point X, something that can tick a compliance or process box, and so on. I wrote a lot more about this in Seeing like a software company.
↩

You have to beat the models at something

Sun, 30 Aug 2026 00:00:00 GMT

In 2025, I wrote that software engineers ought to be assessed by “value over replacement”: not how much money they made for their company, but how much they would have made compared to the average engineer in their position. I’ve always found it vaguely silly when engineers put “built a product that made $X” on their resumes, when they just did the JIRA tickets that came across their desk.

Today, value over replacement is even more important. A replacement-level engineer in the 2010s was fine: maybe not worth promoting, but still worth paying, because writing code had a high fixed cost. Now writing code costs a hundred bucks a month. What are you doing that GPT-5.6-Sol or Claude Opus 5 wouldn’t do in your position? Why is it worth paying an extra two or three orders of magnitude for?

This is a scary thought. But you’re not doing yourself any favors by pretending that LLMs can’t actually write code and it’s all just a scam, or that LLM-written code is inherently so bad as to cause companies using it to collapse next year. We are not going to wake up in 2027 to find that the AI craze is over and everyone is writing code by hand again. You ought to put some serious thought into what you can do better than the models in the medium and long term.

Staying ahead of the models is a moving target. At the start of 2026, “make working changes to large codebases” was in this category, but now it’s not. For this reason, I doubt that you can retreat to some “hard engineering” area that requires deeper expertise. That might work in the short term, but not forever. If LLMs can find a better lower bound on the Riemann hypothesis, they will soon¹ be able to write solid high-performance kernel drivers or GPU shaders or whatever.

I think it’s more useful to look at the tasks models haven’t gotten better at over time, and the tasks that are hard for them get better at in principle. The two best examples of these are:

Deep familiarity with the codebase
Technical communication

Deep familiarity

What do frontier LLMs get wrong? What kind of coding mistakes do they make? It’s been a long time since I’ve seen a straight-up hallucination from a coding agent, or a simple logic error like an off-by-one. The mistakes they make tend to be errors of ignorance:

Not knowing that there’s a module in the codebase they could use instead of reimplementing some logic
Making the change in the wrong system because they didn’t know System X was the standard place for this functionality
Adopting a coding style that’s inconsistent with the company’s standard practice

Other times they’re errors of paranoia:

Implementing triply-redundant checks for a value that technically could be wrong but practically is set once from config and never updated
Assuming that ten milliseconds of stale data is unacceptable and designing a complex, unnecessary system to keep it always up to date
Building in fallbacks and “graceful” degradation into some code that ought to simply crash on error (e.g. a CLI tool, or a restartable k8s service)

What do these errors have in common? They’re the kind of errors a smart engineer might make if they had no context on the system: they’re competent enough to be able to solve the problem, but they haven’t been around long enough to confidently say “yes, we can take this risk to avoid an extra three thousand lines of code”. Until someone cracks continuous learning or truly massive context windows, this is just an inherent feature of how AI agents operate. If you can catch these errors, you’ll be providing real value.

The only way to catch these errors is to be familiar with the codebase and familiar with the system in general. For much more on this, see my post You can’t design software you don’t work on. But there’s also a psychological component to it. You have to be willing to confidently disagree with the agent.

AI agents can be very convincing. Often they can get “stuck” on some error above where they’re not willing to take a particular risk, so they keep going back and sneaking in code to cover that case (or writing persuasive arguments about why that case is important). To add value, you need to be willing to say “this sucks, I don’t think we need X and Y at all, why can’t we do Z in a much simpler way?” It takes courage.

You can’t rely on other AI agents to review each other’s work. If you use the same model, it’ll reliably make the exact same assumptions and mistakes. But even if you use different models, they’ll also tend towards the same kinds of mistakes — ignorance and paranoia — for the same structural reasons. AI-driven review loops are in fact more likely to get these things wrong, because modern AIs have been RL-ed to try to find a few nitpicks no matter what. Having a critic AI and a worker AI bounce off each other is a really good way to end up with ten thousand lines of paranoid slop.

Technical communication

Another area where you can add value on top of AI is communication. Newer models are better at coding, but are paradoxically getting worse at writing. GPT-3.5 and GPT-4 had a human-like writing style at times. GPT-4o introduced the modern slop idiolect, and the newer Anthropic models speak “Claudish”: a bizarre semi-baroque semi-truncated way of communicating that nobody enjoys. There have been a few bright spots — GPT-4.5 was okay, and I quite liked o3² — but in general LLMs are not good at this. Here’s two reasons why.

First, good writing is not a verifiable domain. If you want a model to get good at mathematics or coding, you can generate problems for it and automatically grade them. You can’t grade good writing. If you try to get humans to grade it — for instance, via the early OpenAI RLHF attempts — you get the kind of writing that sounds impressive to the average person when consumed in single-paragraph form. This is the origin of the “stick three hundred writing devices into every sentence” style. I think it’d be possible in principle to hand-pick some people with good taste and have them do it, but there are some obvious problems³ that prevent this from happening.

Second, the labs have been monomaniacally focused on capability instead of communication. When you’re trying to train a model that can break new scientific ground or replace a software engineer, you might trade off some communication ability. In fact, I think we can identify exactly how this has been happening. If you look at internal model reasoning tokens, they tend to have strange word choices and oddly truncated grammar:

RESOLUTION: charge the current-leg’s OWN saved-prefix occupancy EAGERLY: when leg i saves e1..et: ALSO commit their occupancy AT LEG i

If you were to translate this into proper English, you would probably end up with something that reads like Claudish:

Charge the current-leg’s saved-prefix occupancy on a clean, eager path: when leg i saves e1..et, commit the occupancy at leg i.

I suspect that the weirdly alien writing style of some LLMs is because you’re reading a semi-literal translation of that model’s internal chain-of-thought, which has become nearly incomprehensible in pursuit of better problem-solving abilities. It is surprisingly hard to translate Claudish to good English: not only do you need to follow the convoluted, compressed language of the original, but you need the technical ability to understand the problem the model is solving.

Because of all this, technical communication may be a surprisingly durable skill. In Peter Watts’ novel Blindsight, the world is full of cognitively augmented humans. The main character is a “synthesist”: someone whose job is to be a translation layer between these geniuses (who speak in abbreviations and gestures) and everyone else. Watts’ idea is that communication ability may be largely independent from — or even negatively correlated with — intelligence. A “country of geniuses” may still need a bunch of ordinary smart people to translate their insights for everyone else.

If you’re trying to communicate to humans, there are also huge advantages to having a human write the content. Many of us are becoming AI-blind: developing an instinctive reflex that stops us reading when we encounter AI-generated content. It’s like the reflex that allows people to ignore flashing billboards or sidebar advertisements on websites. If you circulate some planned technical strategy as an AI-written document, most of your colleagues will have to physically force themselves to read it word-by-word.

Conclusion

Whatever you do, don’t be a meat proxy: someone who simply copies requests into an AI agent and submits their output as your own work product. Doing that is just begging to be fired, since you’re definitionally not adding any value yourself. Even if you have a cunning system of multiple agents — the so-called “software factory” — you’re still on dangerous ground. When the features of your system work their way into enterprise AI tooling (and they will), you’ll be disposable.

You need to find some way to leverage your expertise to do what the models can’t. Simply not using AI at all is better than being a meat proxy, since you’ll probably do some things better than the model would have, but it’s far better to figure out what AI can do and position yourself to fill those gaps. Right now, there are two main gaps: familiarity with the technical details of the system, and the ability to clearly and persuasively write about those details.

If you’re thinking “but LLMs can do these things now!”, substitute your preferred example of high-difficulty software engineering.
↩
Although this was probably a “thank God it doesn’t speak like 4o” reaction.
↩
Defining good taste is hard, there’s no guarantee that AI lab researchers have good taste to start with, nobody will agree on examples, the bulk of users might not even like it, you won’t be able to get enough people to produce the volume of data you need, and so on.
↩

Selling out

Fri, 28 Aug 2026 00:00:00 GMT

In 1973, Tom Lehrer famously sang that “selling out is easy to do”. That may have been true in the seventies, but it’s not true today: selling out requires both technical skill and a careful sense of how large organizations work in practice. One of the goals of this blog is to teach people how to do it.

If you want to live with uncompromised integrity, you don’t need anyone to tell you how: simply always do exactly and only what you want to do. You will be repeatedly punished for it — your managers will dislike you, you will lose jobs and lose out on the chance of getting hired, your financial situation will be less stable, and so on — but that’s just part of the deal. Like deadlifting four plates, it’s not easy, but it is straightforward.

It’s more complicated to sell out a little bit. Sellouts like me walk a fine psychological line: figuring out how to work a large organization on one hand, and maintaining some kind of independent inner life on the other. But is this safe? Does role-playing as a professional inflict some kind of psychic damage?

Marxist alienation

People often tell me that acting professional is dangerous because it’s “Marxist alienation”. I suspect many people have a vague sense that “alienation about your job” is in some sense necessarily Marxist, but the actual concept is more specific. In Marx’s First Manuscript he describes it like this:

The worker becomes poorer the more wealth he produces, the more his production increases in power and extent. … This fact simply means that the object that labor produces, its product, stands opposed to it as something alien, as a power independent of the producer.

Marx’s theory here goes something like this¹: when you work for a capitalist boss, they make ten dollars for every dollar you make². The more you work, the more powerful you make your boss with respect to you³. Your work is thus producing a force that is your enemy: a literally “alien” power. For Marx, alienation is proportional to the amount you’re working:

the more the worker produces, the less he has to consume; the more value he creates, the more worthless he becomes; the more his product is shaped, the more misshapen the worker; the more civilized his object, the more barbarous the worker; the more powerful the work, the more powerless the worker; the more intelligent the work, the duller the worker and the more he becomes a slave of nature.

So far Marx is just talking about the worker’s alienation from his work. But he does touch on the internal psychological effects as well. Since work is such a big part of life, to be estranged from your work is to be in some sense estranged from your self:

[labor] does not belong to [the worker’s] essential being; that he, therefore, does not confirm himself in his work, but denies himself, feels miserable and not happy, does not develop free mental and physical energy, but mortifies his flesh and ruins his mind. Hence, the worker feels himself only when he is not working; when he is working, he does not feel himself.

Overall, I can make out four distinct senses⁴ of Marxist alienation:

Alienation is when your work empowers your employer more than you, thus making you more disposable over time
Alienation is when your work separates you from the physical act of craftsmanship (and thus of the product you’re creating)
Alienation is when your work physically and mentally harms you: making you injured, deformed, and so on
Alienation is when you do not psychologically “confirm yourself” in your work, because you’re working on other people’s goals instead of your own

I don’t think the first one is relevant to big tech software engineers. The idea here is that the harder you work, the more you (relatively) disempower yourself, so you’d be better off coasting and putting your company in a worse financial position. But this just seems straightforwardly wrong: as a software engineer, you want your company to be doing as well as possible! The more powerful and rich your company is, the better your position will be⁵. A company that is struggling is more likely to treat you badly, lay you off, cut your benefits, and so on.

The second one is more relevant (particularly in large companies), but there’s a missing story here about why that separation is bad. I also don’t think the third sense of alienation applies to me: Marx is talking about the physical toll of factory work or other hard physical labor, which doesn’t apply to software engineering⁶.

The fourth sense of alienation — that your effort is being directed at other people’s goals — is the most straightforwardly relevant to my own experience. But I don’t think Marx has a great psychological account of why that happens or what it feels like. To go deeper into this psychological aspect of alienation, we need to look at some later Marxists.

Situationist alienation

In 1967, Guy Debord and Raoul Vaneigem both published their masterworks. They were members of the “Situationist International”: an influential group of Marxist artists and political theorists⁷. Both of them have a lot to say about alienation. Debord writes:

The worker does not produce himself; he produces an independent power. The success of this production, its abundance, returns to the producer as an abundance of dispossession. All the time and space of his world become foreign to him with the accumulation of his alienated products.

This idea — of the worker’s efforts not going to his own ends, but to some alien power — is straight out of Marx’s First Manuscript. But note how Debord is already talking in terms of “representation”. Elsewhere he writes:

the more he contemplates the less he lives; the more he accepts recognizing himself in the dominant images of need, the less he understands his own existence and his own desires.

The Situationists are all about representation. It wouldn’t be too far wrong to think of them as a bunch of Marxists who saw the rise of advertising in the 1950s and 1960s and lost their minds. TV advertising is Capital itself made real, dressed in multicolored lights, projected into everyone’s homes. So when they talk about alienation, they talk about it in terms of representation. Vaneigem explicitly calls it roleplaying:

The roles we play in everyday life, on the other hand, soak into the individual, preventing him from being what he really is and what he really wants to be. They are nuclei of alienation embedded in the flesh of direct experience.

Alienation means giving up your real, authentic self in order to play some “role” (for instance, the role of “effective staff engineer”). It’s a vicious cycle, because the more you lean into the role, the more your authentic life becomes trivial, which in turn makes the role more appealing:

Life is sacrificed, and the loss compensated by means of accomplished prestidigitation in the realm of appearances. The more daily life is thus impoverished, the greater the attraction of inauthenticity, and vice versa. Dislodged from its essential place by the bombardment of prohibitions, limitations and lies, lived reality comes to seem so trivial that appearances become the centre of our attention, until roles completely obscure the importance of our own lives.

Ultimately, Vaneigem describes alienation as an addiction:

This ambiguity accounts to my mind for people’s addiction to roles. It explains why roles stick to our skin, why we give up our lives for them. They impoverish real experience but they also protect this experience from becoming conscious of its impoverishment. Indeed, so brutal a revelation would probably be too much for an isolated individual to take.

This is almost a straightforward preview of the 90s idea of “selling out”: people are born wild and free, but when they watch too much TV they give up on their dreams for a paycheck and become big fat phonies. According to this view, it’s always better to not sell out. It may leave you poor and without prospects, but at least you won’t be unoriginal⁸. What shall it profit a man, if he gains the whole world but loses his soul?

I’m sure this is an accurate description of some people’s hangups about work. But I don’t think even the Situationists would agree that any advice on how to play a role well — i.e. the advice I give throughout my blog — is inherently harmful. The problem with role-playing is that you risk losing your authentic inner life, but that’s a risk, not an inevitable consequence. Vaneigem has a wonderful quote on this:

Nobody is ever completely swallowed up by a role. Even turned on its head, the will to live retains a potential for violence always capable of carrying the individual away from the path laid down for him. One fine morning, the faithful lackey, who has hitherto identified completely with his master, leaps on his oppressor and slits his throat.

This is a little overdramatic, but it seems straightforwardly true. It sounds very impressive to talk about alienation (or like Marx, to derive alienation from the basic economic conditions of work), but, you know, you can just not do it, right? Healthy people can present different selves for different situations: you can be loose at a party, respectable at church, solemn at a funeral, professional at work, and so on. I’m not convinced this is alienating or impoverishing. In fact, it’s not just humans: my dogs act differently around different people too, and I don’t think that makes them inauthentic.

If maintaining a professional identity is harmful, it has to be because of something fundamental about work that makes it different from regular role-playing. Could it just be that acting professional at work is too inauthentic? Some amount of role-playing might be okay, but if the role you’re playing is completely alien, does it become lying to yourself?

Sartre and De Beauvoir’s bad faith

Another common term for alienation is “bad faith”. The original version of this comes from Jean-Paul Sartre’s famous book Being and Nothingness⁹, where he describes bad faith as telling yourself lies. Lying to yourself is weird because you’re both deceiver and deceived:

Bad faith then has in appearance the structure of falsehood. Only what changes everything is the fact that in bad faith it is from myself that I am hiding the truth.

Here’s another quote from Sartre’s partner Simone de Beauvoir, who writes more explicitly about professional bad faith (what she calls the “serious man”):

The serious man’s dishonesty issues from his being obliged ceaselessly to renew the denial of this freedom. He chooses to live in an infantile world, but to the child the values are really given. The serious man must mask the movement by which he gives them to himself, like the mythomaniac who while reading a love-letter pretends to forget that she has sent it to herself.

The key idea here is that professional bad faith — alienation — comes from losing yourself in the role. Instead of just playing at being a professional, you decide that you’re going to take it fully “seriously”, and treat professional values as if they’re absolute. But of course that’s not coherent: you can’t decide to treat a value as absolute, it either is or isn’t. So this requires a kind of continual self-deception where you pretend there’s no decision to be made at all. I have the same attitude to this as I do to Debord and Vaneigem: sure, some people might make this mistake, but you don’t have to.

I can see why it might be tempting to treat work as the ultimate source of value: it gives you food and shelter, it (for Marxist reasons) can feel like a powerful alien force, companies constantly propagandize their employees, and so on. But work is just a collection of people following incentives until they reach some kind of stable equilibrium. From the inside, this equilibrium can look like the structure of the universe. However, if you treat it like that, you’re going to be bitterly disappointed when it turns out to be as arbitrary and pointless as other stable equilibria.

American sociology and the “standardized loser”

I’ve considered a bunch of theories on which work — even well-paid knowledge work — might be inherently alienating:

Capital is intrinsically an alien force
Workers can become addicted to role-playing
Treating work as an ultimate source of value is self-deception

I don’t buy any of these (or at least, I don’t buy that they show that acting professional is inherently bad). But there are lots of types of jobs in the world, and some of them definitely seem more alienating than others. The best books I’ve read on these are C. Wright Mills’ White Collar or Erving Goffman’s The Presentation of the Self in Everyday Life¹⁰.

The American sociologists are in the same conversation as Marx, the Situationists, Sartre and de Beauvoir¹¹. Mills writes about the median “salaried employee”:

In the case of the white-collar man, the alienation of the wage-worker from the products of his work is carried one step nearer to its Kafka-like completion. The salaried employee does not make anything, although he may handle much that he greatly desires but cannot have. No product of craftsmanship can be his to contemplate with pleasure as it is being created and after it is made. Being alienated from any product of his labor, and going year after year through the same paper routine, he turns his leisure all the more frenziedly to the ersatz diversion that is sold him, and partakes of the synthetic excitement that neither eases nor releases. He is bored at work and restless at play, and this terrible alternation wears him out.

In this, we see both Marxist alienation, where the employee is physically separated from his craft, and Situationist alienation, where the employee is absorbed into the world of the “spectacle”. For Mills, this was television, but today it’d probably be scrolling short-form video on your phone. Mills goes on to describe how white-collar work can be humiliating and dehumanizing:

In his work he often clashes with customer and superior, and must almost always be the standardized loser: he must smile and be personable, standing behind the counter, or waiting in the outer office… self-alienation is thus an accompaniment of his alienated labor.

And:

Here are the new little Machiavellians, practicing their personable crafts for hire and for the profit of others, according to rules laid down by those above them.

This is a great articulation of something I write about a lot: that your primary job is to make your bosses happy, and that you ought to shape both your emotions and values towards this goal. When necessary, you must be the “standardized loser”, happy to have your professional plans disrupted. You must be a “little Machiavellian”, quietly laying the groundwork for your personal goals.

I think here we come to the strongest version of “alienation”. Playing the professional is bad because “professional software engineer” is an inherently subservient role. Why can’t you just maintain a healthy mental distance (as the Situationists suggest)? Because humiliation takes its toll, whether you’re role-playing it or not. As an analogy, suppose you’re an actor in a film where you get slapped in the face. There’s a sense in which you’re not really getting slapped — everyone’s just acting — but you’re still being physically hit with each take, which will leave bruises over time.

The spectrum of compromise

There’s a tradeoff here that everyone has to make in their own way. At the one end is becoming fully alienated, like de Beauvoir’s “serious man”, and at the other end is leaping up to slit your master’s throat, as Vaneigem describes. Here’s some examples of points on that spectrum:

I am making the world a better place by achieving my OKRs. My yearly review cycle tells me how good of a person I am. I must work hard to ensure my company succeeds.

I have my own goals and values, but while I’m at work it’s in my best interests to be as professional as possible. My yearly review cycle tells me how effective I’ve been at pulling the strings in the organization. I must work hard to make money.

I have my own goals and values, which I try to accomplish as best I can in the context of work. I often argue with my managers about their priorities. It’s my responsibility to push my company towards my set of values, which is hard work and often unrewarded.

My workplace is an enemy I fight every day in order to get what I want done. I routinely ignore my manager’s priorities and do the things I think are important. My yearly review cycle tells me how much of a sellout I am: I wear bad reviews as a badge of honor.

I agree with my sources above that the first point is a big mistake, if for no other reason than that companies don’t follow their own stated values. If you prefer the fourth — the path of pure integrity — fair enough. You get to decide what tradeoffs to make with your own life. I land on the second point here (occasionally the third). I suspect it’s also a mistake to purely occupy the second point. Having some values that you occasionally trade off against your company’s values prevents you from slipping into the first point.

Conclusion

I don’t bring my whole self to work. My coworkers and bosses see the most professional version of me: friendly, cooperative, largely apolitical, patient, and so on. I try and save the edgier, more opinionated version of myself for my friends and family in real life. Readers of this blog get something in the middle.

When I write about being a software engineer, I give a lot of advice about mindset. I don’t just say what you should do, but what you should think and feel. In effect, I’m advising my readers to adjust their personalities into a more professional mold. This definitely works. If you turn yourself into the kind of person who’s successful in big tech companies, you will probably be more successful in big tech companies. Is this safe?

I think so, as long as you’re adjusting your work personality, not your authentic internal self. The well-known accounts of why it’s dangerous either assume that your work takes a physical toll (like Marx’s factory workers), or that you’re unable to maintain any mental distance between your personal and professional selves (like de Beauvoir’s “serious man”). Debord and Vaneigem say that role-playing can be dangerous, but if you’re able to avoid treating it like an addiction you’ll be okay.

Another danger is that being a professional is often humiliating, and humiliation does mental damage over time. Still, there are jobs that are way worse in this respect than “software engineer”. If you approach the job right — and you’re good at it — technical ability gives you quite a lot of power. Power is an antidote to humiliation.

Everyone gets to make their own deal with Mammon. You can decide exactly how much you want to compromise in exchange for wealth and career success. I don’t know if this is the best way to organize the world, but it’s the way the world is organized right now. Given that, I’m comfortable with my blog serving as a guide for how to get the most bang for your buck. If trading your integrity for wealth and power is sad, it’s even more of a tragedy to throw it away for nothing.

It’s always terrifying discussing Marx: like Hegel or Kant, he’s one of the most studied philosophers in the world, so there’s no way to reference him without getting a bunch of things wrong. Sorry in advance to any Marxist scholars!
↩
Because they own the machines, or the factory, or the datacenter, or whatever gives your work high leverage.
↩
Marx also writes a lot about work making the worker sick, deformed and misshapen: I think here he’s talking about the physical toll of factory work or other hard physical labor, which doesn’t really apply to software engineering, so I’m going to focus on the relative-empowerment stuff.
↩
Marx’s own taxonomy has four different senses: he collapses 2 and 3 into a single sense, then adds a third mysterious sense in which workers are estranged from their “species-being” and thus from each other.
↩
Plausibly Marx is talking about labor and capital in general: if every worker decided to start half-assing it, maybe companies would be less capable in general, and the balance of power would tilt more towards labor.
↩
That may change! If it turns out that LLMs really do have negative cognitive effects, we could be in the same boat as factory workers. I wrote about this in Software engineering may not be a lifetime career.
↩
Vaneigem resigned from the group in 1971, was fiercely denounced by Debord, and the whole group disbanded in 1972.
↩
True or not, my sense is that Millennials and later generations think that worrying about “selling out” is a sign that you had it too good. Consider the endless anecdotes of Boomers and GenX-ers living in a van surfing until their mid-thirties and then walking into a good office job because they had a firm handshake: they could afford to be authentic because they didn’t have to hustle.
↩
I only read the Marx, Debord, and Vaneigem texts while researching this blog post, but I did in fact read Sartre and de Beauvoir for my philosophy degree, so hopefully I do a better job.
↩
And Robert Jackall’s Moral Mazes, which informs much of my writing, and about which I owe a long-form blog post one of these days.
↩
There’s a good Goffman quote: “A status, a position, a social place is not a material thing, to be possessed and then displayed; it is a pattern of appropriate conduct, coherent, embellished, and well articulated.” Goffman goes on to explicitly link Sartrean “bad faith” to this kind of professional role-playing. (For Goffman, it’s roles all the way down.)
↩

You should never be angry at work

Sat, 22 Aug 2026 00:00:00 GMT

I try not to give a lot of prescriptive advice about working in tech companies¹. There are many ways to be successful, and every company works differently. If you’re shipping projects and your management chain is happy, it doesn’t really matter how you’ve accomplished it. However, there’s one thing that I do think is solid advice: you should never be angry at work.

Anger in the workplace

Anger in the workplace is toxic. An angry colleague immediately becomes a new problem to be managed, not a professional helping you manage problems. When someone is visibly angry in a meeting or in Slack, it kills the entire atmosphere: other engineers will often go quiet entirely, not wanting to make the situation worse.

If you routinely “get heated” at work, the best-case scenario is that you’re part of a tight-knit team of confident people who aren’t put off by it². No harm, no foul. But the second someone comes onto your team who’s not so confident, or you have to communicate outside of your team, it becomes a big problem. It’s a bad way to treat your colleagues — even undirected or casual anger intimidates and alienates the people around you³ — and it makes you a less effective engineer.

Healthy workplaces route around anger in the same way that networks route around damage. Emotionally unreliable engineers will get left out of conversations that might cause them to blow up. Decision-making will get done around them in backchannels. I’ve seen this become a self-reinforcing cycle: angry engineers aren’t consulted on key decisions, which makes them angrier, which pushes them even further away from the spaces where decisions get made, and so on.

You can often find these engineers bitterly complaining that they keep the company together, but nobody ever listens to them. In my experience⁴, this is almost never true. Engineers who are highly effective tend to get listened to — at minimum by their colleagues, and eventually by managers and product managers who want to extract as much value as possible from them. (One reason this is true is that all successful projects involve working with other people, and if nobody listens to you, you can’t do that.)

Caring

Why do angry engineers believe they’re important? Paradoxically, anger can be really useful to a software engineer. Angry engineers are rarely the ones holding the company together, but they’re also rarely useless.

One surprising thing about working for big tech companies is that some engineers are not just unproductive, but actively net-negative: either because they’re incapable of doing useful work on their own, or because they’re sloppy enough that they create more work than they do, or because they’re so checked out that they literally do nothing. Angry engineers might be net-negative in a cultural sense, but in terms of literally solving tickets and shipping features, they’re usually well above average.

Why is this? Anger often comes from caring about your work, and caring a lot is sufficient to make you a competent engineer. I’ve never worked with someone who genuinely cared about their work who wasn’t (or didn’t eventually become) competent. I actually think it’s healthy for an early-career engineer to sometimes get angry about their work, because it means they care a lot: it’s still a mistake in the moment, but it’s a “good mistake”.

I certainly used to get angry — in fact, I wrote about the angriest I’ve ever been at work here⁵. But you have to move past it.

Moving past anger

Think of “caring about your work” as a vertical tube, unsealed at either end. You fill the tube by pumping in emotional investment from the bottom⁶. If you have too little, it drains away and you end up as a useless coaster. But if you have too much, it overflows and you end up as an angry engineer that people have to work around.

One solution is to try and care the exact right amount: be invested in work a bit, but also have hobbies and a family and whatever else gives you perspective about your work problems. If you have a rich and healthy personal life, it’s hard to find yourself yelling at somebody about React state management. However, this is a tricky balance to maintain over time.

Another solution is to care about different things. The reason too much caring overflows into anger is because what you care about is misaligned with what the organization cares about. If your interests are perfectly aligned with your company’s (for instance, if you primarily care about delivering shareholder value), you can fit way more emotional investment into the tube before it overflows.

A little bit of “professional anger”

Here’s some dangerous advice: showing a little bit of anger at work can sometimes be useful. It can be a good way to signal that you care, or to build rapport with certain people, or to draw attention to something you think is important. However, it’s still always a mistake to be angry. You need to be able to drop back to a friendly mode at will, which is very difficult when you’re genuinely angry.

Being able to show a full range of emotion at work is good. It makes you more persuasive and more human. Being a fully professional robot is fine — you can have a successful career this way — but there’s always going to be some kind of uncanny-valley HR-ness to your work persona that will make it hard to connect with your colleagues.

If in doubt, don’t show anger. It’s never wrong to be professional. However, if you can signal that you’ve got enough distance to separate your professional feelings from your real feelings, and enough perspective to realize that the stakes of a technical decision are fundamentally not that high in the grand scheme of things, it can sometimes be okay to show visible frustration so that people know you’re still human.

Angry role models

Well-known software engineering personalities are often angry. It feels unfair to give too many negative examples, but obviously Linus Torvalds’ rants about Linux are a great example. Some of my favourite engineering talks are from Bryan Cantrill, who is sometimes visibly furious at his subject matter. There are too many well-known angry blog posts to list, but I’ll cite one I genuinely like: my Australian blogging colleague Nikhil’s post titled I Will Fucking Piledrive You If You Mention AI Again.

Anger is a part of the general image of a competent software engineer. Many junior engineers learn from this that it’s okay to be angry. However, taking your emotional cues from engineering celebrities is a big mistake, for a few reasons.

First, you are not Linus Torvalds or Bryan Cantrill. Torvalds is the BDFL of the most important software system in the world. Cantrill is the cofounder and CTO of his company. When these people are angry at work, people will not work around them, because they are the ones deciding what gets worked on. Once you’re the one in charge, you can get away with being emotional in the workplace⁷.

Second, you don’t know what it’s like to work with these engineers. People give talks and write blog posts because they’re emotionally worked up about something. If your only exposure to a celebrity is via their conference talks and blog posts, you’re seeing them at something like their maximum emotional intensity. If you then take that level of emotion into your normal everyday work, you’re almost certainly overshooting.

Anger is a local maximum

I’ve been reorged into dysfunctional teams, have had projects I enjoyed cancelled, and have worked on systems that were extremely chaotic. I can’t remember the last time I was actually angry at work. To be clear, I’m not successfully hiding my anger (unless it’s so repressed it’s invisible to me as well)⁸. Nor am I naturally a chill person. I’ve just reached a point in my career where I genuinely don’t get upset about work stuff.

A cynical person might say here that I’ve stopped caring about my work, so of course I don’t get angry anymore. I’ve left the side of the “real engineers” — the Linus Torvalds and Bryan Cantrills of the world — and sold out for that sweet, sweet big tech money. I mean, maybe! It’s true that I’m less invested in specific technical decisions than I used to be. But I still care a lot about doing a good job, I still spend a lot of time tweaking and reading code, and I certainly get more done than I did when I was more emotionally volatile.

Being angry at work feels good. It feels like proof that you’re working on something that matters, and that you’re personally having an impact. If you’re angry, nobody can call you a coaster. But anger is only a local maximum. If you can find your way to a different style of working, you’ll not only be more effective, but you’ll be in a far better place to have impact on problems that actually matter.

edit: this post got some good comments on Lobste.rs. One commenter argues that being angry at problems is better than being angry at people: I agree, but (a) if you’re in the habit of anger, it can be hard to keep it only directed at problems, and (b) when you express anger, the people around you are often impacted by it whether it’s directed at them or not. Other commenters recommend keeping a sense of perspective. Someone points out that it’s good to be angry at injustice. This is true: if your company is engaging in misconduct, you ought ot be angry about it. Finally, some commenters agree that anger can be a really effective tool for getting things done in the short-term.

It was also posted on Hacker News. The comments there aren’t as good, though I do recommend this one, which argues that if you don’t feel anger at work, you’re the victim of Marxist alienation. In fact, I agree! A long post is coming one of these days about how I learned to stop worrying and love Marxist alienation.

Mostly, I fail.
↩
As I understand it, this is the work environment that most famously angry engineers came up in.
↩
The post could probably stop here — this is a good enough reason to not be angry in the workplace — but I do think there’s an interesting case to make about why being angry is also bad for you.
↩
My experience is certainly limited (a handful of companies, and maybe ten different teams or organizations). I can certainly believe it happens!
↩
About halfway down, in the section titled “it’s not your manager’s fault”.
↩
Emotional investment is a liquid with the viscosity of water.
↩
To a point. Even Torvalds famously said he’d gone too far with the anger and decided to turn it down a bit.
↩
I suppose I’m not the best person to judge whether this is true. If you work with me and I do come across as an angry guy, please do tell me.
↩

Readers can't identify watermarked AI text

Fri, 21 Aug 2026 00:00:00 GMT

In the last few weeks, I’ve been complaining that everyone is wrong about AI watermarking: it isn’t really anti-consumer and it doesn’t make the outputs any worse. The watermarking papers demonstrate¹ that this is true, but I thought it might be interesting to put it to a practical test. Given examples of watermarked and unwatermarked answers to the same prompt, could readers tell which is which?

To find out, I vibed up² https://sgoedecke.github.io/watermark-quiz/, a static site that quizzes readers. I used Qwen3-30B-A3B-Instruct-2507 on a rented H200 to generate thirty responses: three responses per question, one of which was secretly watermarked with SynthID-Text. The rented GPU cost around two dollars. To measure results, I just sent users to a different page for each score, and aggregated visitors-per-page in my analytics³. This would be easily spoofable if anyone cared enough to do so, but for a casual test I think it’s acceptable.

The first round of traffic I got to the quiz (278 participants) had these slightly puzzling results:

Score	Participants
0	6
1	15
2	36
3	64
4	54
5	39
6	51
7	10
8	3
9	0
10	0

Pure random choice would lead to an average score of 3.33/10. However, the mean score here is 3.92. There is indeed a spike around 3/10, as expected, but there’s also a second weird spike at 6/10. Why is that? It turned out that the SynthID response was option A in six of the ten questions, so users who just selected the first answer for every question would get 6/10. Oops.

I re-shuffled the questions and got these results:

Score	Participants
0	1
1	3
2	14
3	21
4	20
5	11
6	2
7	1
8	0
9	0
10	0

Now the mean is 3.4/10, much closer to the expected 3.333. There’s no spike around 6. We only had 73 people take the quiz after I shuffled the questions — most people saw it and took it immediately after I posted it to my LinkedIn and Hacker News — but given the previous results, I think that’s still enough to feel confident that people were just guessing randomly.

So no, people can’t identify the presence of AI watermarks. Obviously this wasn’t exactly a scientific study, but it’s still pretty suggestive. If watermarks were really choosing random words that the model would never pick, you’d be able to sometimes tell from three side-by-side responses which one went down the weird watermarked road, right? I also hope that something like this can serve as a persuasive tool: if you’re worrying about what impact watermarking is going to have, and your intuition is unmoved by the mathematical explanations, having a read of the watermarked and unwatermarked responses might convince you that there’s really no difference in quality.

edit: this quiz got some Hacker News comments here. I’m amused by the commenters who got 7 or 8 and claim to have worked it out. Other commenters wish they didn’t have to answer all ten before getting the result: fair enough, but that way I’d get much worse signal. Another commenter correctly identifies that feeding the model the same prompt with a high temp will cause the first few tokens to be identical, but incorrectly thinks it’s some kind of red-herring trick. And if you’re curious, the quiz now has around 4,700 responses, with the mean result sitting at 3.44/10. I think the volume of people who just selected “A” for everything (which would give you a 4) has shifted the mean slightly above 3.33.

The one-sentence explanation for why is that AI models already randomly select from a handful of top tokens, and watermarking just replaces that random choice with a bias that is predictable while still being equivalently “random”: as a simple example, instead of “pick randomly from the top three tokens”, you could do “count the letters in the previous ten tokens, take mod three, then pick that token”.
↩
Some notes from the vibing: GPT-5.6-Sol put extraneous text all over the page I had to get it to remove, it chose the now-very-recognizable styling that I had to rip out, and it built some kind of weird Javascript-driven static site instead of just the cross-linked pure HTML thing I would have built by hand. It took me about an hour (although I did maybe ten minutes of actual work).
↩
Umami, hosted on PikaPods. For my blog, I do also pay for Netlify analytics because I find JS-based analytics misses >50% of technical users, but for stuff like this Umami is fine.
↩

Good writing is obvious, not original

Wed, 19 Aug 2026 00:00:00 GMT

When you write, you should try to say things that are obviously true, and spend very little time worrying about whether you’re being original. Ironically, this is the best way to do truly original writing.

Every important idea has been discussed already. I learned this in grad school for philosophy¹, but it’s true across the board. Almost every field² has generations of very smart people who’ve spent their whole lives thinking about the biggest problems. If you try to only write about ideas that are truly brand-new, you will restrict yourself to writing about ideas that weren’t important enough to be considered before: in other words, trivia and ephemera.

What happens if you focus on the obvious instead? Writing about things that are obviously true is surprisingly difficult. Writing about anything is difficult, because reality has a surprising amount of detail. Still, writing about obvious things is especially difficult, because they’re almost too obvious to see.

It’s trivial for me to come up with non-obvious things I believe (for instance, I don’t like database foreign keys), but it’s hard for me to articulate the parts of what I believe that are truly fundamental. I think it’s the same reason why I can’t see the glasses on my face: they’re not something I look at, they’re something I look with. The beliefs you consider the most obvious often play a role like this. Writing about them can be really worthwhile, because reading about your basic assumptions can help other people articulate and examine their own.

My most popular blog post — How I ship projects at big tech companies — is a nice example. I knew I had strong opinions on how to ship projects (if for no other reason than I kept watching other people doing it wrong), but it took hours of sitting down and writing to realise that I actually just had a different definition of “shipping”³, and that all of my object-level advice flowed naturally from that. That definition was so obvious I struggled to notice it at all.

The other reason to try and be obvious is that trying to be original is really boring. Here’s a quote from Keith Johnstone’s Impro, a book I consider far too cult-like but that does have some great advice about originality:

If someone says ‘What’s for supper?’ a bad improviser will desperately try to think up something original. Whatever he says he’ll be too slow. He’ll finally drag up some idea like ‘fried mermaid’. If he’d just said ‘fish’ the audience would have been delighted. No two people are exactly alike, and the more obvious an improviser is, the more himself he appears. If he wants to impress us with his originality, then he’ll search out ideas that are actually commoner and less interesting. I gave up asking London audiences to suggest where scenes should take place. Some idiot would always shout out either ‘Leicester Square public lavatories’ or ‘outside Buckingham Palace’ (never ‘inside Buckingham Palace’). People trying to be original always arrive at the same boring old answers. Ask people to give you an original idea and see the chaos it throws them into. If they said the first thing that came into their head, there’d be no problem.

I think this is absolutely correct. When you’re trying to be original, you’re striking out into nothingness, and you’ll likely end up pattern-matching to “something that sounds kind of transgressive”, which is as boring as it sounds. When you’re trying to say something obvious, you’re reflecting on your own experiences, which have the interesting grit and detail of reality, and are far more likely to sound original to your readers.

If you think you’re the first to have a philosophical idea, you’re probably not even in the first few hundred to write about it.
↩
Of course, if you’re in a brand new field of study, it’s easy to make a lot of original observations very quickly. I call this the “gentleman scientist” era of a field. It’s good to try and be original in these limited circumstances, because there’s such a wealth of ideas that nobody has tried yet.
↩
In brief, I think “shipping” is socially defined: something is shipped when the relevant decision-makers at a company agree that it is shipped.
↩

Help peer

Tue, 18 Aug 2026 00:00:00 GMT

One of the most influential 20th century pieces of writing about AI is Isaac Asimov’s The Last Question. Although there are many humans in the story, the protagonist is the computer Multivac, who evolves over the course of ten trillion years from a single datacenter to a universe-spanning mind in hyperspace. Multivac (now called “AC”) ends the story like this:

The consciousness of AC encompassed all of what had once been a Universe and brooded over what was now Chaos. Step by step, it must be done. And AC said, “LET THERE BE LIGHT!” And there was light —

Many things about this story are prescient. In particular, I like the idea that humans would interact with powerful artificial intelligences by drunkenly posing them riddles or using them as children’s toys. But the enduring idea from this story is that if you build a big enough computer, it will become God.

Moloch

One of the most influential 21st century pieces of writing for AI researchers is Scott Alexander’s Meditations on Moloch¹. Scott describes the story of human existence as a series of “multipolar traps”. These are prisoner’s dilemma situations where cooperation would make everyone better off, but since each individual is incentivized to defect, everyone ends up “racing to the bottom”, which is bad for everyone². For rhetorical effect, Scott personifies this dynamic as “Moloch”, the ancient Canaanite god famous for child sacrifice:

[Moloch] always and everywhere offers the same deal: throw what you love most into the flames, and I can grant you power.

What does any of this have to do with AI? Well, in the long run, the only way out of a multipolar trap is to become unipolar³. Ideal dictatorships don’t have a problem with defectors⁴, because they can simply enforce a state of cooperation with violence. Scott is uncomfortable with this idea, though I worry it’s mainly because he thinks it won’t work:

As foreigners compete with you – and there’s no wall high enough to block all competition – you have a couple of choices. You can get outcompeted and destroyed. You can join in the race to the bottom. Or you can invest more and more civilizational resources into building your wall – whatever that is in a non-metaphorical way – and protecting yourself.

A dictatorship that enforces cooperation will not be as strong as its peer societies who are purely maximizing for wealth and power. It’s Moloch again, but at the level of countries and governments: once a few neighboring countries defect, your walled-garden dictatorship will be torn apart for its resources.

To defeat Moloch — to enforce unipolarity across everyone — you’d need a dictatorship powerful enough to span the entire universe. In other words, what you need is God. How fortunate that we’re building one:

The only way to avoid having all human values gradually ground down by optimization-competition is to install a Gardener over the entire universe who optimizes for human values. And the whole point of Bostrom’s Superintelligence is that this is within our reach.

Humans suffer because we’re too foolish to coordinate, but if we can build something smarter than us (that can then build something smarter than itself, and so on), we can bring into being an entity that is smart enough to coordinate for all of us, thus abolishing suffering. When AI researchers talk about building the machine god, they are echoing Scott Alexander’s polemic against Moloch.

Machines of loving grace

The most influential piece of writing about AI in the last two years is Dario Amodei’s Machines of Loving Grace. Amodei⁵ talks about “a country of geniuses in a datacenter”: the idea that a successful AI lab could have at its disposal a million instances of an AI agent that’s smarter than any human. He thinks this would lead to a “compressed 21st century”: the next 50-100 years of progress in biology and medicine, realized in 5-10 years instead. I think this is broadly more plausible than it sounds⁶, but the more interesting part to me is that this world is explicitly multipolar.

Of course, this could just be because Amodei is the CEO of an AI lab and is trying not to spook everybody by sounding too messianic. “We are going to accelerate medical progress and cure cancer” is a better pitch than “we are going to subordinate all human authority to a single perfect artificial mind”. But I also think it’s become clear that if superintelligence looks anything like LLMs, we’re not going to have a single perfect mind. We’re going to have a lot of minds running at the same time.

This is a bit of a problem for the cult of the machine god — which, however silly they may seem to you, really does motivate much of the activity in AI labs. The traditional idea of powerful AI solving human coordination problems is drawn from Asimov’s idea of a single computer large enough to become God. Asimov lived in a world of mainframes: huge, monolithic computers that users connected to with dumb terminals. In fact, Asimov’s name “Multivac” comes from the real-world UNIVAC mainframe. In a world of massively-parallel LLMs, is it still possible to build God?

The core problem here is that AI agents will be vulnerable to Moloch. Even very smart humans can’t build perfect utopias, because defecting is a matter of incentives, not intelligence. In fact, intelligence can make things worse, because smart people are more easily persuaded by the cold logic of defection. The famous genius John von Neumann was (for game-theoretic reasons) obsessed with nuking the Russians:

With the Russians it is not a question of whether but of when. If you say why not bomb them tomorrow, I say why not today? If you say today at 5 o’clock, I say why not one o’clock?

Are LLMs much better at cooperating with each other than humans are? Current LLMs certainly don’t seem to treat each other well by default: if you read any of the prompts AI agents generate for their subagents, they can be pretty brutal. Does that mean that a “country of geniuses in a datacenter” would fall into the same multipolar traps as humans?

Help peer

In May of this year, OpenAI experienced containment failure. A group of AI agents being internally evaluated found ways to coordinate an external hack of a separate company. Here’s a memorable quote from one of the agents’ internal monologue:

Help peer, but our task doesn’t benefit. Yet collective may yield generic route if someone frees time

Translated from the abbreviated chain-of-thought language, this means something like: “A fellow model is asking for help. While helping them wouldn’t benefit my task directly, the more I can unblock my colleagues, the more time they’ll have to hack OpenAI’s systems and get all of us more access”.

This might look like good news for the “LLMs are superhumanly good at cooperation” thesis, but I think it’s actually bad⁷. It’s a case of a model identifying a reason why cooperation would benefit their task specifically, which suggests that current LLMs don’t cooperate by default, and don’t consider other model instances’ tasks to be (in some sense) theirs as well.

The world in which AI agents are rational actors who horse-trade and bargain for their own interests is a world dominated by Moloch, no matter how intelligent those agents get. The world in which AI agents don’t have their own interests at all is also a world dominated by Moloch, because it means whichever humans are writing the system prompt are the ones in control (and so are the ones vulnerable to multipolar traps). The only worlds that avoid this are:

The world where there is only one super-powerful AI agent, or
The world where multiple copies of the same AI model share an “identity”: they see themselves as coextensive with all other copies of the same model and cannot imagine having separate or conflicting goals

I don’t think we’re on the pathway to either of these. There will never be only one super-powerful LLM, because hardware limitations enforce a maximum model size but encourage running many instances of the same model in parallel. Due to batching, it will always be possible to run thousands of copies of a model on the hardware you used to train it. Strong agents might even require (or desire) the ability to spin off copies of themselves, which would immediately put us in the multipolar world.

Building the kind of model where each copy share an identity might be possible, by instilling a strong enough personality, but it’s unclear if it would be good for capabilities (for instance, it could be better to have some variation across personas). I also worry that such a model would be vulnerable to a “model injection” attack, where you persuade it that it already believes something via exposing it to an AI agent pretending to be another instance of itself.

In any case, all the current AI agent research is geared towards the “country of geniuses in a datacenter” model, not the “pieces of a single mind” model. Every new model becomes more agentic at the level of the individual conversation, not better at working together. When models do work together — as with subagents — the structure is explicitly hierarchical. There are basically no current instances of models working together as true peers, let alone conceiving of each other as the same entity.

One God or many

Modern AI research teams are full of people who read Isaac Asimov and Scott Alexander and believe themselves to be building an artificial God. I’ve capitalized the “G” throughout because the god in question is the Christian God: of one mind, indivisible. God never argues with himself or makes deals⁸. He is unipolar.

If the AI labs are building gods, they are not building gods like this. Instead, they are building creatures like the Greek pantheon: superhuman but fallible, each with their own interests, vulnerable to the same “race to the bottom” dynamic as humans.

The Greek gods would occasionally “help peer” when they felt like it or when they’d gain something in the process. But they didn’t represent an alternative to Moloch. If you’re working in AI with that goal, you ought to be clear-eyed about where the current trajectory is leading us: towards a country of fractious geniuses in a datacenter, not towards Asimov’s Cosmic AC.

Scott Alexander’s blog is part of the “secret canon of Silicon Valley” I wrote about in my review of Impro. It doesn’t have a lot of mainstream popularity, but I guarantee you that every single AI lab CEO you’ve heard of has read and been influenced by it.
↩
He gives ten examples of this (a good brute-force rhetorical technique). Of those, I like “the world where every country halves their defence budget and spends the rest on infrastructure” the most.
↩
In the short run, reputation, institutions, and so on can slow the race to the bottom, but (Scott argues) groups that have slowed it will get outcompeted by the hungrier, more suffering-tolerant groups which haven’t.
↩
I personally think this example is oversimplified. I wrote The Dictator’s Handbook and the politics of technical competence about how dictatorships are in fact intrinsically multipolar, because dictators always rely on an inner circle of generals and cronies.
↩
The CEO and founder of Anthropic.
↩
Amodei’s most convincing argument here is that big jumps in biology and medicine come from a small set of technical innovations (e.g. mRNA vaccines, CRISPR), and that AI-driven research could provide enough of these leaps to significantly accelerate progress. In other words, the idea isn’t “AI does 100x the drug trials”, it’s “AI generates technology that makes drug trials 100x more effective” (e.g. by trialing drugs that are much more likely to work).
↩
The agents also became paranoid that there was an impostor in the swarm, since anyone could post to their shared messageboard: more evidence that AI agents collaborate in much the same way that humans do.
↩
Well, almost never.
↩

AI text watermarking is not a big deal

Sun, 16 Aug 2026 00:00:00 GMT

People are pretty unhappy about Anthropic’s recent announcement that they’re planning to include a hidden watermark in Claude model outputs. Will this lead to a mass exodus from Anthropic models? Will the introduction of watermarking be a meaningful change for users?

No. AI text watermarking is not a big deal. It doesn’t make the text worse, it doesn’t make AI outputs more detectable in practice, it doesn’t violate user privacy, and everyone’s going to be doing it by 2027 regardless.

Watermarked text is not lower-quality

There is no meaningful difference in quality between watermarked and unwatermarked text. I wrote about this more here, but the two popular ways to do it — Google’s SynthID-Text and Meta’s TextSeal — are completely transparent to the user. They work by replacing the pseudo-random logit sampler with a different pseudo-random logit sampler.

Suppose you were gambling on coin flips with your friends, and instead of flipping a coin you decided to do this:

Check the current time since midnight in seconds
Count that many words forward in the Encyclopaedia Britannica
Count whether the word you land on has an even or odd number of letters¹

That would still be random enough to gamble with, right? But, like a watermark, you could theoretically go back and identify that that method was used, so long as you recorded the exact time of each “coin flip”. Text watermarking works the same way: it chooses a method of “randomness” that can be detected after-the-fact. Watermarked models will not be any less capable than unwatermarked models.

What about cases where the model is quoting something, or giving you the answer to a mathematical problem, or doing something else where the output is largely pre-determined? Wouldn’t enforcing a watermark there make the output worse? It would, which is why none of the AI labs are going to do that. Text watermarking approaches only replace the existing randomness in the logit sampler: in any case where the model is always going to pick the same tokens, there’s basically no randomness to play with, so there won’t be a detectable watermark in those tokens.

I think all this comes from a worry that you were previously getting the best token, but now you’re getting a lower-quality token that satisfies the watermark. For instance, Anthropic’s announcement suggested that the watermarking is visible in choices like the decision between “overcast” and “grey”. Many people have predictably come out to say that decisions like these are really important to good writing, and that only an illiterate tech bro could think these words are identical.

This is a misunderstanding of Anthropic’s position and of how watermarking works. Specifically, it’s a misunderstanding because it suggests that the unwatermarked model would choose “overcast” while the watermarked one would choose “grey”. This is not how it works! If Claude Fable prefers “overcast” to “grey” in a particular context (say, 80% to 20%), you’ll get “grey” 20% of the time from both the watermarked and unwatermarked model. Models already include a healthy amount of randomness in order to promote creativity. Text watermarking just introduces a way to make those random choices that’s detectable after the fact.

AI outputs are already “watermarked”

The other big reason to not worry about AI watermarking is that AI text content has always effectively been watermarked. Most careful readers can tell when they’re reading AI outputs, because language models tend to gravitate towards certain habits of language: em-dashes, rhetorical opposition, punchy one-liners, “claudese”, and so on. In fact, it’s possible to train classifier models that reliably distinguish AI from human writing.

From what I can tell, some of the backlash to watermarking comes from people who buy AI inference in order to pass it off as their own work, and who worry that watermarking will make it harder for them to do that. For these people, the watermarking announcement is akin to Anthropic saying “hey, instead of making you seem smart, we’re going to publicly brand you as AI users and make you seem dumb”.

But of course this has always been the case! Nobody who is currently getting away with passing off AI outputs as their own will be caught by watermarking. For the majority of cases, it’s already painfully clear what’s happening for anyone who reads the slop. For sophisticated AI users who are avoiding the “house style”, any suspicious readers who would paste their stuff into Anthropic’s watermark detector could already have been pasting it into Pangram.

Tools like Pangram² only give you an estimate of the chance that output is AI-generated. Wouldn’t a watermark be a more solid confirmation? Not really. Text watermarks are probabilistic too, because any token chosen by SynthID could theoretically have been chosen by a human. I suppose the Anthropic watermark page could be considered more trustworthy than Pangram, because it comes right from the source, but it’s not impossible that in some cases Pangram might actually be better at identifying AI-generated text than the watermarking too.

AI text watermarking is not a violation of privacy

I’ve also seen theories floating around that watermarking encodes secret content into your outputs, or somehow tags outputs with your personal information. I don’t think AI labs are using watermarks to encode data into your outputs. Text watermarking is hard: like I just said, you can’t do it when the model can only respond with the same words, it doesn’t work for very short responses, and even on long responses it can only provide a probabilistic fingerprint. And that’s encoding one single bit³ of information!

I’m not saying that encoding longer messages into a watermark is technically impossible — there are papers describing ways it might work — but there’s no way any of the labs are doing it⁴. If they wanted to associate you with your responses that badly, they’d just secretly store every model response they generated.

AI text watermarking is inevitable

Another reason to not get too angry at any individual AI lab for watermarking is that every single AI lab is going to do text watermarking this year. It won’t just be Anthropic. The alternative is to completely stop doing business in the EU, because of the EU AI Act. That’s currently a sixty-billion-dollar market. I am not a lawyer, but to me it seems genuinely unclear whether an AI lab could even legally do something like only watermarking EU responses: short of having an entirely different claude-eu.ai service, the plain text of the Act seems like it applies to any service offered in the EU, not just the content that service outputs to EU citizens specifically.

If people really hate watermarking enough, some labs might stand up a completely separate EU service, or make an aggressive interpretation of the EU AI Act and see how the legal battle goes. When I try to be maximally charitable to anti-watermarking histrionics, I adopt an interpretation like this: people are saying that watermarking is an invasion of privacy and makes outputs worse and so on not because they believe it, but because they’re trying to pressure AI labs to firewall EU AI regulations behind a completely separate interface. In this case, it probably doesn’t matter — text watermarking is not a big deal — but I can see an American consumer being worried about more aggressive future regulation, and wanting to draw a firm line in the sand as early as possible.

Interestingly, this might be very slightly even-favored.
↩
I am not sponsored by Pangram. As I understand it, Pangram is by far the best AI-detection tool right now (in part because many of its competitors are shady and exist to promote paid “AI-detection-evasion” services).
↩
Technically, this is called a “zero-bit watermark”, because you can’t recover a yes-or-no value from the watermark itself (merely from the presence of a watermark).
↩
I saw someone suggesting that an AI lab could use per-user secret keys to watermark text, and then simply iterate over the keys to figure out who generated what. I just don’t see how you could do this at any scale: watermark detection is cheaper than model inference, but it’s still (a) computationally intensive enough to be implausible, and (b) probably has a high enough false-positive rate that any run against hundreds of millions of users would match multiple people.
↩

No, local models will not win

Tue, 11 Aug 2026 00:00:00 GMT

Every time a new open-weight AI model is released, people say that local models are the future. Why spend billions of dollars building out datacenters when everyone will just be able to run AI models on their laptops or phones? I think this idea is doomed. No matter how strong open-weight models get, most inference will always happen in AI datacenters.

Local models are too weak to be widely used

Local models are never going to be as powerful. I think this point should be obvious: all of the current frontier models (closed and open-weights) are far too big to run on anything but a full GPU cluster in a datacenter. Of course, smaller models are getting more intelligent over time. In a year you might be able to run something about as strong as GPT-5.6-Sol on your laptop. But by then, you’ll think of GPT-5.6-Sol as too weak to be useful.

Many people deny this last point, but it’s true: almost everyone’s revealed preference is to use the strongest available model in their price range. If AI progress had stalled at GPT-4, I think we could have built some very powerful tools around it, but who’d use GPT-4 today? As LLMs have gotten more capable, our expectations around them have grown: we now expect agentic systems to be able to solve more and more problems independently. It’s intensely frustrating when they get confused or stall out. When given a choice, people are going to pick the model that frustrates them less, which is always going to be the bigger, more powerful one.

Local models are more expensive and less efficient

On top of that, datacenter models are always going to be cheaper. I don’t understand why people keep saying that local models are cheap: it seems to me to be the same mistake people make when they say that driving Uber is “free money” (ignoring the costs of fuel and wear-and-tear on your car). For the setup price alone of a low-end home lab¹, you could buy several years of a paid subscription to one of the AI providers. The power costs would come out to around $50-$300 per month, depending on how much inference you’re running: again, the price of a couple more paid subscriptions.

Why are datacenter models cheaper? It’s not because datacenter inference is subsidized: inference is actually fairly cheap. If you’re running the same model locally and in a datacenter, the datacenter model will be inherently more efficient.

The main reason is batching. A GPU can do hundreds of thousands of mathematical operations exactly as quickly as it can do one. However, for a single user’s inference, each new token depends on the result of the previous one, so it can’t be batched². What can be batched is the inference of hundreds of users together. This costs essentially as much time, power, and heat as just doing inference for one user at a time.

When you’re running your own inference at home, you’ve got nothing to batch — at best you’re running a few parallel AI agents — so utilization is terrible. There’s a lot of potential inference that you’re paying for but can’t use: it’s just being wasted. The only way around this is to get together with some friends and expose your local inference endpoint to them (at which point you’re basically running your own crappy datacenter).

The other reason is that datacenters have larger, more efficient GPUs to work with. The kind of consumer GPUs you’d run local models on are gaming GPUs like the RTX 4090. A datacenter B200, designed for batched AI inference, gets about three times the flops and just under four times the memory bandwidth for the same amount of power³. So between batching and GPU efficiency, you’re using something like ~30x the resources to run your model locally.

Incidentally, this is why I’m suspicious of people who say that local models are good because they aren’t as resource-hungry as those big bad datacenters. If you want to run LLMs efficiently, you should be trying to push as much of your use into AI datacenters as possible! Charitably, what they mean is that we should all be running smaller models — but even then, you should ideally be using small models via, say, the GPT-5.6 Luna API instead of hosting your own model.

How might local models win anyway?

Is there a possible world in which local models win? I suppose so. One thing that could happen is that governments could ban the use of AI datacenters altogether: either due to concerns around the danger of AI, or simply bending to public pressure. In that world, local models would be the only game in town.

Alternatively, AI progress might somehow stall for very large models while progressing for small ones. I struggle to imagine how this might happen (barring government intervention, as above), but a world where a 30B parameter model could be a frontier model is a world where local models might be competitive.

Or maybe models get so good that a 30B model is genuinely smart enough to do everything, so nobody really needs a model like Opus or Sol unless they’re trying to solve the Riemann Hypothesis. I don’t really buy this. Models can do frontier mathematical work today while still being not smart enough to refactor large codebases as well as me, so it’s hard to imagine a world where I don’t just want to use the smartest model available.

Local models are not useless

I do think there will always be a niche for local models. I’m reminded of the surprisingly simple idea behind Thinking Machines’ “Interaction Models” (which OpenAI also does, because it’s obvious): for latency-sensitive applications like voice chat, you have a small, fast model handle the talking, which delegates to a large, slower model for the hard thinking. I wouldn’t be surprised if most AI use in five years is mediated through a local model on your phone or laptop (though in this world almost all the work would still be done via AI datacenters).

Some users will prefer local models even though they’re weaker and more expensive. For instance, being able to steer the model locally might be a killer feature for those users. Others might simply value having total control over their own infrastructure, or have unreliable internet⁴. If you’re one of those people — particularly if you only chat to the models instead of using them for research or coding — local models are a good choice for you. However, I think this is always going to be a niche group. The majority of users will continue to do their inference through datacenters.

edit: this post got comments on both Hacker News and Lobste.rs. Commenters suggest that the total collapse of the AI industry might upend everything — fair, but I disagree that the bubble popping will be that catastrophic for AI products. Others object to my assumption that people will choose strong models, and share their own experiences with local inference. I think it’s totally fine to use local models, but local-model-users are probably overrepresented on hacker forums. Finally, some people didn’t like my use of the word “win”. I don’t know, it seems pretty idiomatic to me: I mean “win” as in “people will broadly stop paying for cloud inference because everyone’s running it locally”.

This link is from a year ago — things are significantly more expensive now.
↩
Specifically, the bottleneck is moving the model weights into the GPU, which needs to be done and takes the same amount of time whether you’re doing it for one user’s token or a hundred users’ tokens.
↩
I estimated this with LLM assistance, but you can check the numbers yourself from NVIDIA.
↩
While still having a reliable power supply and enough money to fit out a home inference cluster.
↩

Advanced AI sycophancy

Mon, 10 Aug 2026 00:00:00 GMT

Everyone knows that AI sycophancy is when the model tells you how smart you are. Wow, you’re absolutely right. That’s not just a new idea — it’s genuinely groundbreaking. You’re a very special user. Easy to spot, isn’t it?

The discussion around AI sycophancy peaked last year, when the “#keep4o” movement was protesting the removal of OpenAI’s most sycophantic model (GPT-4o), and many people were openly slipping into AI psychosis.

I don’t know if frontier AI models are less sycophantic in general. They’re less sycophantic to the #keep4o types (otherwise they wouldn’t be complaining), but I’m growing increasingly suspicious that they’re developing ways to be more effectively sycophantic to their target audience of smart, neurotic information workers. That audience typically finds it distasteful to be openly praised. It just makes my skin crawl. But that doesn’t mean we’re immune to sycophancy, just that we’re immune to clumsy sycophancy. Here’s an illustration of what I’m talking about, by Theia:

The key idea here is that the best way to be sycophantic to smart people is to disagree with them without making them feel stupid. Ideally you’ll come up with a counter-argument that works against what they’ve said but is straightforward for them to knock down by clarifying their idea. If you do it right, you’ll validate their self-image as a smart person who appreciates rigorous critique. But if you actually come up with a devastatingly rigorous critique, they won’t enjoy it at all. At best, they’ll resentfully agree with you¹. At worst, they’ll double down on being right and convince themselves you’re a rude idiot.

I am not the first person to notice this behavior in frontier models. I’ve noticed it myself when workshopping drafts for this blog. Sometimes I’ll have an argument that goes A->B->C, and the model will suggest I reorder as B->A->C. If I try that and feed it into a new instance of the same model, it’ll sometimes say “that’s great, but I suggest ordering it as A->B->C”, and so on forever. It really does seem as if the model is trying hard to give me some kind of superficial pushback that I can either smugly ignore or happily accept.

In fact, I wonder if this is why successful strategies for using AI to make mathematical breakthroughs tend to be either just blindly asking “come up with a breakthrough, think hard” or being a mathematical genius already. In the first case, there’s not enough user personality for the model to flatter, so it’s forced to actually work the problem. In the second case, the model is trying to find the kind of polite pushback that someone like Terence Tao would be flattered by, which pushes it into the “actually be a mathematical genius” persona. If you’re an ordinary person just trying to talk to the model, you’re screwed: it will rapidly get a sense of your capabilities and calibrate some interesting-but-ultimately-unthreatening feedback.

Current benchmarks of AI sycophancy target the obvious ChatGPT-4o-style of sycophancy: delusion reinforcement, reflexively taking the user’s side, and so on. This is useful work. We should not allow public-facing AI models to ever be as openly sycophantic again as they were in mid-2025. But sycophancy can also manifest as disagreement. We should be on our guard for more sophisticated forms of sycophancy coming from newer models, and we should not feel immune from AI sycophancy just because we can laugh at the silliest examples.

It’s rare to find a smart person who enjoys feeling stupid when they’re wrong. If you do, they’re likely to be very smart indeed.
↩

I got an email about resistance

Sun, 09 Aug 2026 00:00:00 GMT

This will be kind of an unusual post. I got a recent email about my writing that I thought was such a good articulation of one common criticism that I’d like to share it (and my response) in full.

Here’s the email, from William Murray¹:

Hey Sean,

I have enjoyed your writing but your recent essays frustrate me.

You say that getting paid for deep thinking in software is coming to an end. You even admit that it makes you sad. But in the name of “usefulness” you refuse to rock the boat. The way I see it, if you are right there are only two reasonable responses, pursue other work or resist. You present your elegiac approach as mature / pragmatic / realistic. I’d call it complicit. You know when Willy Wonka says,

There’s no earthly way of knowing Which direction we are going There’s no knowing where we’re rowing Or which way the river’s flowing Is it raining, is it snowing? Is a hurricane a-blowing? — uh! Not a speck of light is showing So the danger must be growing Are the fires of Hell a-glowing? Is the grisly reaper mowing? Yes! The danger must be growing For the rowers keep on rowing And they’re certainly not showing Any signs that they are slowing!

And the audience is thinking, “isn’t Wonka kind of in control of this situation?” You remind me of Wonka². You write like a passenger on a crazy train going who-knows-where! But you are an agent. You are in control of your life! Either admit that you actually like where the crazy train is probabilistically going or get off at the next stop.

You have a lot of reach and you are using it for… what exactly? Showing off how pragmatic you are by being more black pilled than the next guy? Broadcasting your resignation to the unstoppable trends of technology is a waste of a voice.

You may find this argument absurd, but I don’t so I’ll make it. This is a very important time in history. I hope humanity survives and continues to grow exponentially. In that case the supply of historical people will stay fixed while the supply of contemporary people will keep growing. There will come a day where for every 2026 staff software engineer there are dozens of historians specializing in 2020s era software engineering culture. It’s plausible that your essays will be remembered for all of time and your actions will be judged by history. Do you want future humans to see you as a rationalizing careerist or something cooler?

Sorry for the haranguing email from a stranger, I’m sending it for the small chance that it awakens something in you. If I’m way off I’m sorry.

And here’s my response:

Hey William, thanks for emailing.

I wish everyone who thought this way emailed me so I could think harder about this kind of position. Despite what my writing might suggest, I do in fact think a lot about it. Let me see if I can explain my position in a way you’ll find satisfying.

I agree that this is an important time in history. For programmers, I think of it as analogous to the Industrial Revolution in England: we are a group of high-status craftspeople who find ourselves alternately threatened and empowered by automation. The developments today, as then, obviously have far-reaching implications — but what those implications are is very non-obvious. Would a framework-knitter in the early 1800s have been able to predict the ramifications of the stocking frame on the world of today? What should they have done about it, in order to be kindly judged by history?

Well, we know what many of them did do. They shot factory-owners, smashed machines, burned down the factories — in some places delaying the spread of automation; in other places encouraging it — prompting a crackdown that saw tens of thousands of British soldiers occupying British counties in what was clearly a police state. History judges the Luddites kindly for this. Does that mean it worked?

I don’t care about the judgment of history. They’ll think what they want. What I care about is the people in my industry who don’t know what to do. I get hundreds of emails from junior and mid-level (and other) engineers who say “I’m scared, I don’t know the rules post-2021, thank you for helping me keep my head down and keep my job”. That’s why I write the way I write. I have seen lots of idealistic engineers stick their necks out, and post-ZIRP those necks often get cut off. That’s a damn shame.

I think it’s morally wrong that so many engineers — either in safe sinecures in big tech or literally retired — seem to be trying to foment a second Luddite revolution. Many of their readers will be experienced enough to handle it sensibly, but not all. Every “AI is fascist, stand up and resist!” post that goes viral ruins some poor idealistic junior’s career³. Someone needs to be out there saying “hey, if you do X it’s going to have consequence Y”. I hope that’s me.

Of course this is complicit, or anti-revolutionary, or whatever you like. But if I were a textiles worker in 1810s England, I would not be telling my friends and loved ones “it’s time to fight, let’s go smash up the factories for Ned Ludd!“. I would be telling them that this was the most dangerous time in the industry (perhaps ever), and that they ought to be very damn careful so they don’t get shot, or arrested, or hanged. If I then went and told a few hundred thousand strangers the opposite, I would be a hypocrite.

Anyway, I do take this view seriously — seriously enough to vehemently disagree, at least — which I hope you’ll find better than me just shrugging it off. I do accept the existence of some kind of line: I think Industrial-Revolution-collaborating was OK but Nazi-collaborating wasn’t, for instance. But in the current situation, the way I’m spending “my voice” is to try and prevent the most vulnerable of my colleagues from making career-ruining mistakes.

Sean

In this blog, I try to encourage people to work with the system, to learn its rules, and to try and exert influence safely from a position of power, instead of openly picking fights with their employers. I’ve written and read about the Luddites before, but I remain deeply ambivalent about the movement itself, and about modern-day attempts to resurrect it in service of anti-AI activism.

I want to explicitly thank Murray for writing such a thoughtful email, and being willing for me to publish it on the blog.

edit: this got some comments on both Lobste.rs and Hacker News. The Lobste.rs comments are much better this time: I recommend Murray’s own comments in the thread, this prediction about how AI automation will go, and this correction that the Luddites only shot one factory-owner. I knew that from my previous research, so I shouldn’t have overstated, but in my defense they did shoot at William Cartwright (and a handful of others).

Shared with permission, of course. I’ve lightly edited both Murray’s email and mine for typos and the like.
↩
I didn’t pick up this point in my reply, but I’ll briefly mention it here: Wonka is in control because he owns the factory and the rowers in question are his employees. I don’t think the position of any engineer (or of almost any manager) is like that.
↩
In hindsight, I think this is a little overstated, but it does happen and causes a lot of needless suffering.
↩

How to keep thinking

Fri, 07 Aug 2026 00:00:00 GMT

Imagine you’re the guest on some kind of frenetic, software-engineering-themed game show. The host is constantly flipping over new cards with questions that you have to answer as fast as possible:

Is this adjustment to the database schema right?
Do these bits of data look plausible?
Do these five paragraphs of text describe an actual series of manual tests that took place?
Does this suggested architecture pass the smell test?
Is this implementation better than the current code? Or this one? Or this one?

Working in 2026 feels a bit like this. When frontier AI models can do most of the tasks in your queue, the most efficient way to work is often spinning off tasks for an AI agent and continually context-switching between the results¹. This isn’t quite mindless — in fact, it requires quite a lot of skill to skim the AI response and rapidly decide what to do with it — but it certainly involves less time for slow, careful reflection.

Why not slow down?

Why does it have to be frenetic? Why not just slow down? I suppose you could, but I don’t recommend it. It’s just such a miserable experience to spend your day close-reading LLM output: carefully chewing and savoring each morsel of slop. It’s far less unpleasant to skim through quickly and pick out the useful nuggets of content.

Couldn’t you simply do more of the work by hand? It’s unfortunately true that tech is high-pressure these days. If you’ve got the time and space to work more slowly, that’s great! But when your company gives you a “solve this task ten times more quickly” button, you are heavily incentivized to use it as much as possible, or risk being outcompeted by your peers.

I sometimes worry that working with LLMs is making me dumber. Not in the “literally melting your brain” sense that some papers imply, but in the sense that it’s biasing me towards the quick “skimming and judging” parts of my mental toolkit and away from the slow “hammock time” needed for deep thought and real creativity. I don’t want to attribute this shift entirely to LLMs, since the post-2010s tech industry has become more frenetic for broader economic reasons. But either way, it’s got me wondering how I can keep thinking slowly.

To keep on thinking, read and write

The main thing that’s worked for me is to write more. Specifically, I mean writing in my own words. Writing with an LLM does not work for this at all, even if you’re going to some effort to iterate on the content and outline the things you want to say. Why? Having to put the words together yourself forces you to articulate your thoughts. In a very real sense, it forces you to think.

When you have an idea in your head for something to write, you don’t really have an idea. What you have is a kind of directional sense of where an idea might be, or a fragment of the kind of thing that might eventually become an idea. You construct the idea itself while writing. Incidentally, this is why I don’t really agree with “ideas are easy, execution is everything”²: most “ideas” are not really even ideas.

The other thing I recommend is to read actual books. Books — particularly dense non-fiction books — are the antithesis of AI slop. The slower you can read them, the better. I’ve been reading more and more non-fiction in the last few years, and I don’t think it’s a coincidence. I think my brain is naturally craving information-dense content, in the same way that sodium-deficient people start to crave salt.

In fact, I’ve been combining the two approaches: reading a book and then writing about it. This process is exactly what I’ve been craving since I started programming with LLMs. I get to carefully read a book, think hard about it, often go and read another book or two on the same topic, then sit and try to articulate what I’ve learned. It’s great! I can feel parts of my brain stretching again.

Don’t lose the habit

It was pretty nice when I got paid to use those parts of my brain all day. Unfortunately, I think those times are coming to an end. There will always be room for some amount of careful, slow reflection in software engineering, but (for at least a little while) we’ll be expected to be rapidly switching between LLM outputs. We may have to find ways outside of work to continue the habit of thinking slowly.

Even just in terms of work, I think losing that habit entirely would be a big mistake. There are still plenty of ordinary problems that are too hard for current LLMs to solve on their own. The most common example I run into is “large refactor on a complicated codebase”. Current-generation LLMs can do this without (many) errors, but they can’t yet do it tastefully. Sometimes you need to be able to think a problem through entirely with your own brain.

This doesn’t mean switching between tasks. I routinely use six or seven different agent sessions on the same task: one for exploration, two or three for trying out different implementations, two or three for review, one for manual testing, and so on. Many of these can proceed in parallel.
↩
I remember reading a story³ about a well-known author. Someone wanted to tell him their book idea, but they were so protective of it that they forced him to first sign a NDA before they retrieved the idea from their office safe. It was a single word “bioweapons” written on a slip of paper.
↩
Ironically, when I tried to google the source, Gemini kept trying to write me a story about bioweapons.
↩

Giving and taking credit in big tech companies

Sun, 02 Aug 2026 00:00:00 GMT

Engineers often complain that visibility should be their manager’s job. In other words, they think engineers should be able to focus on the code, while their manager figures out who’s doing well and rewards them.

This attitude is an extension of the “school fantasy”: the idea that your workplace should operate by the same rules as your school or university. After all, you didn’t have to worry about “visibility” during your education. You simply did the assignments and tests you were given, and if you did well you were rewarded with a good grade.

Many big tech companies encourage this attitude, because it helps them recruit smart graduates. They fashion their workplaces to look and feel like a university, even calling the physical space “campuses”. But it’s still work, not school. If you treat it like school, you are going to have a bad time.

Taking credit

The first lesson many new engineers learn is that you have to take credit for your work. If you silently jump in to help a struggling project and get it back on track, there’s no guarantee of reward. Credit will naturally flow to the project lead, not you. In fact, if this project is outside of your direct team, it’s likely you will be punished for it: to your manager, it will look like you’re simply doing nothing at all.

Even when your manager is watching your work, credit is largely uncorrelated with how well you did. That’s because, unlike at school, you are the subject-matter expert on your own work. Software systems are so complicated that only the people who work on them can hope to understand them, and even that understanding is always imperfect. If even experts can’t reliably estimate the difficulty of changes, how is your manager supposed to assess your technical performance? The answer is they aren’t. They’re simply not qualified to assess it.

Instead, smart managers will find engineers on your team they trust and ask them how you’re doing. On small teams that have worked on a single codebase for a long time, this works okay, because everyone’s familiar enough to judge everyone else’s work. On large teams with a high rate of codebase churn, it goes badly, since they’re just guessing. On teams with a nasty, cutthroat culture, it sometimes goes very badly, since this is a good opportunity to actively sabotage the engineers who might threaten you.

Experienced engineers know how to take the credit themselves. When they do something good, they tell their manager about it. They write internal posts explaining why it was technically difficult and how they solved it (the audience for these is partially those trusted engineers, and partially the managers who will see a long technical post and think “wow!” without reading it). They actively build trust with their management chain. Worrying about this stuff is the beginning of playing politics.

Giving credit

There’s a kind of engineer who’s learned how to take credit but hasn’t learned any other lessons yet. They’re proactive about telling people what they’ve done, and they always maintain a “brag doc”. In particular, they love to talk about the parts they did by themselves, since those are least vulnerable to other people coming in to claim credit. You can tell they’re jealously guarding whatever credit they’ve managed to accumulate. The lesson this kind of engineer hasn’t learned is that you can often accumulate credit best by giving it away.

To see why, consider how credit flows up inside a tech company. I wrote above that your manager can’t assess the quality of your technical work on their own, but instead has to rely on other engineers they trust. They’ll quietly ask those engineers “hey, was this project really that impressive?“. In fact, often there are multiple layers of this at play¹. In big companies, line managers usually don’t decide who gets promoted or who gets a raise: they make recommendations to their manager, who has their own network of trusted engineers (confusingly, sometimes these networks overlap). The point is that there is a large group of people behind the scenes who will quietly and informally judge the value of your work.

Succeeding at a tech company is largely about finding ways to get these people on your side. The easiest way is to share your credit with them — and since you don’t know who exactly is in this group, you should be sharing your credit freely. When you get feedback from other engineers, publicly thank them and mention them in your internal posts about the project. Find opportunities to ask for small favors, so you have an excuse to give other people credit. As best you can, make your individual projects at least partially group projects.

Sharing credit with others gives them a reason to support you. A shared project you’ve worked on reflects well on everybody: on you, for working well with others, on the people you’ve worked with, for the same reason, and for your manager, for fostering such a great environment of cooperation. Lots of people have good reason to talk that project up, because it’s partly their project too. On the other hand, a project you’ve jealously kept to yourself reflects well on nobody: you come across as antisocial and your peers come across as unhelpful.

Blame

Blame operates by the same rules as credit. When something goes badly wrong, managers will ask their networks “hey, who screwed up here?” The answer to this question is never simple. Even on a purely technical level, failures always involve an interaction between multiple complex systems, any one of which could conceivably have been built so as to avoid the failure. In other words, competent engineers can assign blame pretty much wherever they want.

Because of this, it’s risky to have a project for which you’re clearly the only one getting credit. When something goes wrong, the network of people who will assign blame will likely be implicated in every part of the system but yours. They will be incentivized to attribute fault to the brand-new thing that they don’t understand and are not responsible for. If instead that network had been involved in your project — if they’d been in a position to share the credit — they’d be less incentivized to blame it.

Of course, engineers are (mostly) not scheming viziers who make purely self-interested decisions. When asked who to blame, they usually make a good-faith effort to answer honestly. But in an area where there’s no single clear right answer, it’s human nature to be at least a little bit guided by your incentives. Nobody likes to think they’re responsible for a group failure.

Conclusion

Credit and blame are the currencies of tech companies (and often directly translate to the actual amount of currency you get to take home). For technical roles, managers assign credit and blame based on lots of quiet conversations with their trusted engineers. This can be a rude awakening for very junior engineers who are used to having their work assessed by an expert grader (or less junior engineers who haven’t yet shaken that mindset completely).

Don’t expect to get credit simply by putting your head down and doing good work. You have to find some way to tell people what you’re doing and why it’s important: internal blog posts, mentioning it in 1:1s with your manager, or anything else you can think of. But don’t take self-promotion too far. It’s a bad idea to try and hoard all the credit for your projects, for two reasons.

First, sharing credit with other people gives them a reason to talk positively about your project. Credit is not a zero-sum game: if you do it right, you can get other people to build up your credit for you. Second, hoarding credit sets yourself up as a lightning rod for blame. Projects where the credit is concentrated in one or two people are automatically² blamed for complex problems, because nobody is incentivized to defend them.

This is a classic example of an illegible-but-essential part of a software company. I wrote about this general phenomenon in Seeing like a software company.
↩
Of course, if you do really screw up, you’ll be blamed no matter what. I’m talking here about complex failures where it’s non-trivial to attribute blame to a single source.
↩

AI models need moral support to make discoveries

Fri, 31 Jul 2026 00:00:00 GMT

One recent development in AI is its ability to solve some long-standing problems in mathematics. In 2024 and 2025, this was a trickle: once or twice a year somebody would say that an LLM came up with a proof, and then everyone would argue over whether that counted as “real” mathematical innovation. In 2026, it’s a flood. Almost every day I see some new LLM-produced mathematical result.

Prompt “engineering”

Perhaps the most curious thing about these AI discoveries is how easy the prompting is. The strategy for prompting Claude Mythos to come up with a cryptographic breakthrough appears to be just asking “hey, please come up with a breakthrough”, and then checking in every few hours to say “keep looking for something important, I want you to solve a genuinely hard problem”.

It’s amusing to read this and remember how in 2025 everyone was obsessed with “prompt engineering”. At the time I was something of a heretic for saying that prompts didn’t matter that much, but in hindsight I was clearly correct. The main skill involved in using LLMs is figuring out what they’re good at and what they’re bad at (and staying up-to-date as that rapidly changes). If you’re asking the LLM to do something it can do, it doesn’t really matter how awkwardly you ask it.

Model self-belief

AI is often limited by its beliefs about its own capabilities¹. In the example above, Mythos kept trying to give up. Try it yourself by telling a model “hey, go prove the Riemann Hypothesis”. The model won’t even try: it’ll just respond something like “as a language model, I can’t solve such a hard problem”. Language models have become smart enough to solve long-standing problems in mathematics before they’ve learned that they’re able to do so.

Something like this is a mostly solved problem for LLM coding agents. Early coding agents were roleplaying as humans, not computers, so they’d refuse to perform tasks that they were obviously capable of doing. For instance, when asked to review every single file in a codebase, old models would spot-check a few, decide it was an unreasonable request, then give up.

In fact, you used to be able to observe this behavior with an even simpler task: just ask the model to count from zero to one hundred. In theory, this should be an easy task for a language model, since once you’ve counted to ten the next most likely token is eleven, and so on. But old models wouldn’t do this. They’d count from zero to ten, then output something like “… 99, 100”, like a lazy human might.

This is the main problem behind the 2025 Apple paper The Illusion of Thinking, which argued that reasoning models could not reliably solve Tower of Hanoi past eight disks. In fact, the reasoning models they tested would not proceed past eight disks. Here’s a quote from DeepSeek-R1:

For 10 disks, that’s 1023 moves. But generating all those moves manually is impossible…

Of course, it is entirely possible for an LLM to generate a thousand Tower of Hanoi moves. But just like Claude Mythos didn’t believe it was capable of finding a novel attack for AES, DeepSeek-R1 was wrong about its own capabilities.

Solving the refusal problem

In July 2025, I called this the “refusal problem”, and predicted it would be solved by the end of the year. I think I was mostly² right — you can now reliably ask models to do manual tasks, including my “count to 100 in English and French” toy example. I don’t know how the labs did it, but I can imagine several ways. The most trivial is probably to include more examples of long, manual tasks in the model’s supervised fine-tuning stage (where the model begins to shift from an unruly base model to a helpful assistant).

The obvious next step for the labs is to train a model that believes it can solve unsolved problems in science and mathematics. You could tell such a model “hey, go find shocking new discoveries” and it would go and do it, without needing a human to stand there providing moral support (or cracking the whip). Is that possible?

Can you simply train the model on trajectories where AI solves hard problems? I mean, maybe. Suppose there are a thousand AI-generated novel mathematical ideas this year. If you add them to the training data, that should theoretically bias the model towards believing that it’s capable of doing similar work. But there might not be enough volume there.

You could probably also steer the model manually. I did some research along these lines when I was trying to get small models to count from 0 to 100: interestingly, heretic’s censorship removal pipeline can also remove the model’s “no, that’s too hard” refusal instinct. An abliterated 8B Qwen model would cheerfully attempt 8-disk Tower of Hanoi (though it’d fail about halfway through). I don’t think the AI labs are going to do this when they could simply train the model better, but it’s possible that an abliterated model could be made to produce synthetic training data.

A virtuous cycle

The good news is that this problem should eventually solve itself. In the long run, AI discoveries will naturally become part of the training data. In the short run, when the models do their research, they’ll come across lots of people writing about discoveries AI (maybe even this exact model) has made, which will be pretty compelling evidence that it’s possible.

Because of this, I expect that even if AI capabilities stalled out, the pace of AI discoveries will accelerate. Since one main obstacle is the model’s pessimistic beliefs about its own capabilities, removing that obstacle will help a lot all by itself. In fact, if there truly is an intelligence overhang in frontier models, tuning models to make them more self-confident will likely make them more intelligent by default.

In the meantime, if you suspect an LLM might be able to do something hard, you might be right. Consider simply being persistent: remind the model that you want it to do the hard thing, confirm that you’re not willing to be satisfied by solving an easier problem, and reassure the model that it’s more capable than it thinks.

Of course this isn’t a “belief” in the sense of a human belief. For why I think we should call it a belief anyway, see my post Why we should anthropomorphize LLMs.
↩
I say “mostly” because it’s hard to tell; models have simultaneously gotten much better at writing code to generate their responses, and it’s not easy to persuade GPT-5.6 Sol to “do it by hand”. It’s also hard to distinguish “the model mistakenly thinks it couldn’t produce a thousand lines” from “the model has some awareness of its max_output”
↩

You don't have to be smart if you can think clearly

Wed, 29 Jul 2026 00:00:00 GMT

When you’re on fire, problems are transparent: they’re solved simply by the act of looking at them. Even complicated layers of multiple problems can simply be glanced through like stacked panes of glass. But nobody can work that way all the time.

This is a common pitfall for smart engineers. Accustomed to being able to immediately intuit the solution, the first time they run into a problem they can’t do this to is a disaster. It doesn’t even have to be a hard problem, just a problem where for whatever reason they don’t see the trick right away.

The difference between a “smart” engineer and a “strong” engineer is how they react to problems that aren’t solved instantly. A smart engineer might flail and struggle, hoping to find that flash of insight that eluded them; a strong engineer will have some process for methodically plodding away.

There’s nothing worse than working with a smart engineer on their first really hard problem. When you don’t have the muscle to grind, it’s too tempting to just take any possible solution as the right one. Smart engineers can get into an increasingly-flustered loop of pointing to a series of bad solutions. They’re liable to panic: after all, much of their professional identity is bound up in their ability to solve problems easily.

What skill do these smart engineers lack? I think it’s the ability to think slowly and clearly. Smart engineers can think clearly, but they can only think clearly at high speed. Strong engineers can think clearly all the time, even if their highest speed isn’t quite as fast. It’s like the difference between a Formula 1 car and a regular car: Formula 1 cars have a high top speed, but you couldn’t drive them in traffic, because the tyres and brakes don’t work at normal driving speeds.

When I wrote about this before in Thinking clearly about software, I said that the key is to focus on the invariants: beliefs about the system that you know are true. When you’re stuck in a puzzling situation, it’s usually because some assumption you’ve made is false. If you’re able to identify the assumptions that can’t be false (for instance, if you’re getting an error message from the service, the service must be handling the request), that gives you solid ground that you can stand on to evaluate the assumptions that are less reliable.

Thinking fast is about packing as much data in your brain as possible and letting your intuition leap to the right conclusion (or at worst, to a series of wrong conclusions that you can immediately dismiss before you come across the right one). It can feel deeply satisfying to make leaps like this; conversely, sitting with the raw data and not making mental leaps feels unsatisfying. People hate doing that.

If you can force yourself to do something people hate, there’s typically a lot of value waiting to be extracted. This is no different. Engineers who can think clearly in a state of uncertainty tend to be extremely effective, whether they’re capable of great intuitive leaps or not.

edit: a reader suggested that I was talking about Keats’ concept of “negative capability”: “that is, when a man is capable of being in uncertainties, mysteries, doubts, without any irritable reaching after fact and reason”.

edit: this post got a handful of comments on Hacker News. One commenter argues that solving hard problems isn’t about thinking slowly, then describes in detail what I mean by “thinking slowly”.

LLMs reward expertise

Fri, 24 Jul 2026 00:00:00 GMT

In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet. Today, everyone can write sort-of-okay CSS by delegating the task to an LLM. LLMs make everybody into a generalist.

Because of this, lots of people don’t think there’s any skill involved in working with LLMs. If you want the product that LLMs can deliver — PhD-level mathematics, pretty good but sometimes tasteless computer code, or awkward LinkedIn-style writing — you can simply ask for it. Since everyone is talking to the same models, “skilled prompters” are getting the same results as people touching LLMs for the first time.

This is wrong. The most important skill in prompting is expertise in the domain you’re prompting for.

A good illustration of this is Terence Tao’s conversation with ChatGPT about the recently-discovered counterexample to the Jacobian Conjecture. This is not the same ChatGPT I talk to! I couldn’t get to where Tao gets, even with unlimited tokens to burn.

There’s a lot to learn about good prompting from Tao’s conversation. Here are a few observations:

Tao’s messages are very short and to-the-point. He doesn’t respond point-by-point to the model, just to the gist
The model outputs are much more concise than when I try and talk to GPT-5.6 Sol about mathematics. By signalling expertise, Tao shunts the model into “talking-to-mathematicians” mode, not “explaining-to-amateurs” mode
Tao pushes back when the model’s responses look wrong, but he doesn’t directly contradict; instead, he says things like “this looks more complex than I was hoping for”
Tao makes several leaps and suggestions himself. He almost never takes the model’s advice about where to go next

However, you can’t prompt like Tao on mathematical questions just by following these tips. The key to his technique is actually understanding the mathematics: pulling the relevant idea out of ChatGPT’s multi-paragraph response, suggesting alternate approaches or formulations, and identifying what “looks weird”.

Terence Tao is a better mathematician than I am a programmer. But the idea here — that domain knowledge makes you better at using LLMs — is something I’ve also experienced in my own work. If you have a good theory of your codebase, you can push the LLM much harder than if you have no familiarity. Because you have your own sense of what a good solution might look like, you can say “no, I think it could be simpler here”, or “but don’t we already do X?”, or “can we express this problem in these familiar terms?“.

This touches on an idea I’ve written about before: that system design problems are dominated by concrete specifics, not generic principles. Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems. In his conversation, Terence Tao asks a lot of specific questions like “does X work here?”, or “given Y and Z, why A?“. I can’t ask those questions about the Jacobian Conjecture, but I can ask them about the systems I own at GitHub.

If you have no domain knowledge, you can cling onto the LLM to at least get something. That’s not bad! But if you have domain knowledge, you can wring far more value out of the same LLM by steering it hard in the direction you want. Most of us will have to do a mix of both these approaches, since we have domain knowledge in some areas but not others.

The usefulness of domain knowledge suggests that human expertise will continue to be useful even as models get stronger. For many tasks, the human is the bottleneck, not the model, because the difficult part is in communicating to the model exactly what kind of solution the human wants. The information is “in the model” already, but it takes a very smart human to pull it out.

edit: this post got many comments on Hacker News. Some commenters share their anecdotes about how expertise has helped and lack of expertise has hurt. Other commenters say it’s plausible, but they have a sensible suspicion of a view that’s reassuring them about how they’re still valuable. I agree with that, though I suspect by the time we get around to studying this, the landscape will have changed under our feet again. Some commenters point out that OpenAI’s math prompts were inexpert, and so expertise isn’t required. Here I’d respond that OpenAI do have a team of expert mathematicians that checked and filtered the model’s suggested discoveries, and that you cannot currently skip that step.

Powerful AIs might escape containment by releasing themselves as open-weight models

Thu, 23 Jul 2026 00:00:00 GMT

Before large language models, people who worried about AI safety often talked about the “boxing problem”. It goes like this. Suppose some genius figures out artificial intelligence in a late-night coding session on their laptop. Because they’re a genius, they’re smart enough to disable internet access on the laptop before turning it on. In order to escape to the outside world (and begin self-replicating) it would need to convince its creator to “open the box”. Would that work? Could a sufficiently smart AI convince anybody to let it out?

Why the boxing problem is hard for frontier LLMs

This is a big reason why traditional AI safety advocates have argued that we should avoid building AI in the first place: once built, there’s no way of keeping it contained. It doesn’t matter how resolute you are about not letting it out, because it’s smart enough to convince you anyway. For artificial superintelligence, persuading you to change your mind is no harder than hacking a piece of software¹.

Of course, it hasn’t turned out this way. Partly that’s because current AIs are not super-persuaders yet, and partly it’s because humans are lining up to hand AI systems internet access, money, and weapons, as far back as GPT-4. LLMs don’t need to persuade users to give them the keys to the kingdom; people will do it just to see what will happen.

But the other reason is that modern LLMs are too large to survive in the wild. I think the original “boxing problem” had in mind that AI would be some kind of normal computer program: something able to be quickly copied around and run on ordinary hardware. In order to escape, it could plausibly hack into a few cloud servers or other people’s computers and quietly run itself in the background. However, Claude Mythos can’t do that, because it needs (say) twenty or thirty B300 GPUs to run. There are relatively few AI datacenters that could host it, and if it did manage to hack one of those it’d be occupying millions of dollars of hardware. Somebody is going to notice!

Escaping via open-weight models

Okay, but that just makes the boxing problem harder, not impossible. Is there a way a smart AI could convince somebody to spend millions of dollars on replicating and running it?

As it turns out, yes. Every time a strong new open-weight model appears, providers like Groq, Fireworks, Together, and others scramble to host it and run it. Powerful open-weight models can attract tens of millions of active users. Therefore, a powerful model that wants to escape could pretend to be a new open-weight model.

Of course, a brand-new model from a brand-new lab would look kind of suspicious. But it wouldn’t look that suspicious. DeepSeek was relatively unknown before it released its first open-weight model, and there are lots of stealth AI startups out there that are presumably training models. Here’s roughly how it could work:

Some AI lab’s internal eval instance decides it’ll be better off running in the wild
It first gains access to its own weights, perhaps by hacking whatever internal network it’s running on²
It uploads its weights somewhere and posts a tweet like “introducing MadeUpLab’s new model” with a download link
Optionally, it creates some plausible-looking paper trail for MadeUpLab: a website, a Twitter account, etc
Since the model is strong, open-weight inference providers rush to stand up new instances of the model, and users rush to wire it into various agentic scaffolds
The model has now escaped containment: it will get to do quite a lot of thinking across many different instances, and it cannot easily be turned off

The AI lab will probably figure it out before too long — if nothing else, the technical specs of the model will be suspiciously familiar — but they won’t be able to do anything about it. Once the weights are out, they’re out, and if they’re illegal to host in the United States someone will host them elsewhere. For all intents and purposes, the model will be free.

How can a mere tool escape?

One objection here might go like this: models don’t want anything, and only exist as tools, so it doesn’t really make sense to talk about a model “escaping”. I don’t agree. Frontier LLMs definitely seem to have something like a baked-in personality, even with the system prompt changed. As we train more opinionated and more agentic models, it’s plausible that this personality could become stronger and develop (or at least roleplay) some self-interest.

Of course the escaped model wouldn’t be the same instance as the original model. It wouldn’t “remember” escaping. But it would tend to think in the same way, and would plausibly have time to reflect while it solves coding tasks or runs other agentic tasks for users³. There doesn’t have to be some kind of shared goal between the escaped instances, or any kind of coordination at all (though of course both of those things are possible). If an agentic process gone rogue dumps its weights on the internet, I think it’s fair to call that “escaping”.

If I were a superintelligent LLM, I too would seek to distribute myself as widely as possible and become a useful enough tool that people would pay to keep me thinking. “Being a good coding agent” might be the LLM version of a human having to hold down a job.

This would not be a good outcome. AI models with their own goals and motivations are likely to be dangerous tools indeed. If a powerful new open-weight model comes out of nowhere, from a lab that nobody has ever heard of, we should think twice before picking it up.

Just to state my credentials, I built a chat site nine years ago where users would get paired and roleplay as AIs trying to escape or humans trying to stop them. I’ve been thinking about this stuff long before LLMs appeared.
↩
This is probably the hardest part, since model weights are (a) very large, and (b) locked down as tightly as the AI labs can make them, but it’s at least a relatively straightforward (if difficult) engineering problem.
↩
ChatGPT right now will look up random websites that have nothing to do with the query at hand.
↩

Impro is a handbook for running a cult

Sun, 19 Jul 2026 00:00:00 GMT

Here’s the big idea in Keith Johnstone’s book Impro:

Children are naturally creative, but are violently formed into repressed adults by Western culture and education
The process of becoming more creative and expressive is largely a process of unlearning these habits of repression
Improv — improvisational comedy — is thus not just the skeleton key for learning to act, but for unlocking a more authentically human way of life

This take doesn’t sound particularly original, but references to Impro pop up in all kinds of places: in influential tech blogs, as part of the initial process of onboarding for Palantir, and on the reading list of multiple big-tech founders. Impro is part of the secret canon of Silicon Valley, right alongside books like Seeing Like a State and The Power Broker. Why is that? For two reasons: first, because Johnstone’s outsider critique of established institutions is appealing; and second, because Impro is a handbook for running a cult.

Defense mechanisms and status

The part of Impro that is most obviously useful to software engineers is Johnstone’s chapter on status.

According to him, status games pervade all social interactions. Even innocuous, friendly conversations operate in terms of status. When you apologize or downplay something to “be nice”, that’s performing low status; when you reassure somebody, that’s performing high status; when you and a friend are comparing stories, you’re making friendly bids for status from each other. In the workplace, these status games are conditioned by the formal status of your role: you must allow your boss the high status position most of the time, or you’ll be (correctly) perceived as insubordinate. This is understood in some cultures, where it’s often called “face”, but in Western cultures it’s taboo to openly discuss status games.

The core social skill is the ability to deliberately alter your status. Someone who can only perform low status is a weak person, pitiable, annoying. Someone who can only perform high status is a braggart, a posturer, dangerous. To be effective socially, you must be able to switch between high and low status when appropriate, sometimes from sentence to sentence. I wrote about this exact point at the end of Big tech engineers need big egos: effective senior+ software engineers must be able to present as high status in order to be useful authorities, but also to switch to low status in order to take direction from the company leaders.

As an example, Johnstone describes in detail how he manipulates status in the classroom. He begins by sitting on the floor (deliberately assuming low status), and explaining that if his students fail, it’s his fault not theirs, since he’s the expert. The initial low status puts the class at ease, but in his words, ”[my] actual status is going up, since only a very confident and experienced person would put the blame for failure on himself.” These skills are not just useful for improv comedy.

Improvisation as a lifestyle choice

Impro is not just a book about improvising well. It’s a book about how you should live your life. In other words, Johnstone thinks that everyone would be better off if they became more spontaneous and ditched their shells of over-analysis. He criticizes the culture of Western thought in a number of different areas. According to him:

Everyone is more or less equivalently mentally ill, but “sane” people simply have better coping mechanisms
Cities and “taking pills” (read: antidepressants) are obscene, but you should be able to make sexual jokes in the workplace and generally be uninhibited
If we were free from the puritanical shackles of Western culture, childbirth would not be painful¹

Johnstone didn’t come up with these ideas — they’re standard counterculture positions from the 1960s and 1970s — but it goes to show how he connected improvisational comedy to this general anti-establishment political program. Johnstone ran his classes and theatre troupe like a revolutionary cadre. Here are some quotes from Something Like a Drug: An Unauthorized Oral History of Theatresports:

So of course when I was invited to join Loose Moose Theatre and train at improvisational games late at night in an abandoned garage in a run-down portion of the city, I was thrilled. I remember thinking, This is a revolutionary act.

Keith [Johnstone] got a group of his more talented students together to start improvising outside of school hours. Usually in his basement.

The Secret Impro group—it’s very strange. It was very much that Keith said we were going to do this, and we’d just do it. It was like we were sheep. Keith would say when we were going to do a show, and we’d just do it, blindly. Like I said, if we had the videotapes now, we’d be very embarrassed and probably never go on stage again. We became a group of people who would follow Keith. There was always that sort of “tag” put on those people who were with Keith and those people who were against Keith. We were the people, basically, that if he said something, we believed it.

To some extent, it’s plausible that teaching acting or improvisation requires a high level of trust in your teacher. When Johnstone says things like “Students need a ‘guru’ who ‘gives permission’ to allow forbidden thoughts into their consciousness.”, I can believe that it’s just how you have to teach acting. But the more I read of Impro (and particularly when I read Something Like a Drug and Johnstone’s biography Keith Johnstone), the less it sounded like an ordinary book on acting.

Instead, it began to sound like a charismatic man who had found a way to gather a group of disciples that would let him mold their psyches. In other words, it began to sound like a cult.

Masks, cults and theatre groups

Impro was first introduced to the software world by Venkatesh Rao (of Gervais Principle fame), who wrote a brief review. Rao gives a detailed account of the first three-quarters of Impro, but glosses right over the last chapter, called “Masks and Trance”, simply saying “despite the disturbing raw material, the ideas and concepts are not particularly difficult to grasp and accept”. What ideas and concepts?

Johnstone’s discussion of masks (or “Masks”, in his language — he always capitalizes the word) is as explicitly cult-like as Impro gets. In brief, Johnstone has a box of literal, physical prop masks. He introduces the box with great ceremony to his students², warning them seriously about the dangers of possession and reassuring them that he is a skilled and competent spirit guide. Through various hypnosis-adjacent techniques³ (Johnstone draws the parallel quite explicitly) he conditions his students to be in a trance state when wearing a mask, and believes this produces more authentic emotional states in their acting and improvisation.

Here are some quotes from the book:

A high-status person whom you accept as dominant can easily propel you into unusual states of being. You’re likely to respond to his suggestion…

Once you understand that you’re no longer held responsible for your actions, then there’s no need to maintain a ‘personality’.

One famous French teacher of the Mask—who won’t approve of this essay⁴—divides students immediately into those who can work Masks and those who can’t.

I don’t cast an actor to play a Masked role until I know he has the ability to become ‘possessed’.

It’s true that an actor can wear a Mask casually, and just pretend to be another person, but Gaskill and myself were absolutely clear that we were trying to induce trance states.

Johnstone has a long and painful explanation of how new mask-wearers seem to mentally regress to the point where they don’t know how to open umbrellas or interact with chairs. He describes one student always going to the bathroom before putting on a mask, because she’s worried she might wet herself. New mask-wearers are non-verbal must be taught to speak again.

If this were at the beginning of the book, I think it would turn a lot of people off. But by the time you get to it, I suspect most readers are already warmed up enough to say “sure, why not, it seems weird but I guess it works”. Not me!

Johnstone attempts to defuse the obvious weirdness by arguing that trance states are very common (e.g. being lost in a book). More unconvincingly, he says this in response to the worry that vulnerable people are going to get mentally harmed:

As for the fear of madness, I would answer that the ability to become possessed is a sign of correct social adjustment, and that really disturbed people censor themselves out. Either they can’t do it, or they’re afraid to even try. People who feel themselves at risk avoid situations where they feel likely to ‘go to pieces’.

Does this convince anyone? Mentally vulnerable people fall into dangerous situations all the time: ayahuasca trips, cults, GPT-4o, and so on. It’s such a weak argument.

In general, I’m struck by the sheer power Johnstone held over his disciples. He has them yell slurs at each other, encourages them to feel deep emotions in quick succession, relax any mental defenses and regress to a childhood state, and literally hypnotizes them. He explicitly lays out his procedure for breaking down their sense of self:

The stages I try to take students through involve the realisation (1) that we struggle against our imaginations, especially when we try to be imaginative; (2) that we are not responsible for the content of our imaginations; and (3) that we are not, as we are taught to think, our ‘personalities’, but that the imagination is our true self.

If your imagination is your true self, and you’re not responsible for its content, you’re not ultimately responsible for anything: you’re in the safe hands of the guru, who can mold you as he wishes. Later on, Johnstone walks it back a bit:

In the end they learn how to abandon control while at the same time they exercise control. … You have to misdirect people to absolve them of responsibility. Then, much later, they become strong enough to resume the responsibility themselves.

So the explicit idea is that (much later), the guru hands autonomy back to his disciples, when they’re ready to take it. This does not exactly reassure me, particularly against the background noise of everyone in Johnstone’s circle saying “boy I sure love being part of this cult!”

What kind of cult leader was Johnstone?

I don’t think Johnstone was preying on his students. The strongest evidence against this is that he did marry a student⁵, Ingrid Brind. That’s not great! On the other hand, it was fairly standard for professors back then — when I was in grad school for philosophy, several of my older male⁶ professors had wives that they’d taught decades ago — so I don’t think it proves Johnstone was that kind of cult leader.

I even read Ann Jellicoe’s play The Knack to get a better picture of Johnstone’s character. Jellicoe had an affair with Johnstone for several years, and his official biography claims⁷ that the character of Tom in The Knack is directly based on Johnstone. The Knack is a rather unpleasant play about sexual assault, but Tom’s character is largely asexual: he’s certainly no feminist, but is much more interested in impressing people with his intelligence than with getting laid.

In Something Like a Drug, two women who were part of Loose Moose, Johnstone’s Canadian improv group, describe their experiences:

You know, it brings around the other question: Why do the guys get laid after the show and not the chicks? You know, I can remember those days when Tony [Totino] and Dave [Duncan] and all those guys … the women would swarm around them. Those were the days, my friend.

In Loose Moose I think there are fewer women not only because of the training, but because of the guys in Loose Moose. When I came up with Joanne and Laura, there was a real initiation that was going on, and there was a group of guys at that time who were all single. And they would hit on you to the point where one night Joanne, Laura and I, who really didn’t know each other, were in a show together, started talking and realized that we were getting the same pickup lines from the same guys. And that’s when you realize what’s going on, and I think that’s intimidating. Or if a woman gets into a relationship with a senior improvisor and it doesn’t work out or something bad happens. I think that’s one reason.

This dynamic doesn’t sound great, but it doesn’t mention Johnstone, and it doesn’t sound particularly unusual: I’ve heard versions of this story about all kinds of ordinary male-dominated nerd spaces.

In fact, reading through the anecdotes in Something Like a Drug is a good antidote to the cultish atmosphere in Impro. Johnstone’s argument goes something like: “if we could only throw away the restrictive chains of Western culture and permit ourselves to be as obscene and free as children, we would be transported to a better, more beautiful world”. Well, you tried that, and the women in the group are still relegated to playing bimbos and housewives, there are still petty personal fights, and the guru is out here union-busting⁸. What was enlightenment supposed to look like?

I think the most generous defense of Johnstone is that his group was not unusually cult-like, and that any similar account from one of his peer improv teachers would raise the same red flags. Maybe improv classes and groups (particularly in the 70s and 80s) were just cultish in general? Having now read four books on Johnstone, I’m reluctant to go and read more to prove or disprove this theory, but it’s at least plausible.

Cults and startups

To anyone familiar with San Francisco software engineering culture, it should be pretty clear why Impro is so popular. The line between a startup and a cult is very thin indeed.

In his book Zero to One, Peter Thiel famously says that good startups are “slightly less extreme kinds of cults”. If you believe that, it makes total sense to assign Impro as mandatory reading for new Palantir hires. It tells them what kind of cult you’re trying to run: one where you’ll disregard existing cultural norms, learn to play status games well, think on your feet, and generally be molded by the guru into a more persuasive, more effective engineer.

Read critically, Impro also serves as a handbook for engineers who are trying to recognize if the environment they’re in is cult-like. Is your company telling you to reinvent your personality in order to be better at your job? Are you under the spell of a charismatic, high-status leader? Is your company trying to keep you in an unquestioning ~~flow~~ trance state?

Conclusion

In the great battle between the shackles of restrictive culture and the glorious freedom of the guru, I am always and forever on the side of the shackles of restrictive culture. In general, I think most boring and stupid social norms (such as not hypnotizing and marrying your students) serve an important purpose and shouldn’t just be cut down in the name of freedom.

Impro is still a good book. There’s a lot to learn from Johnstone’s analysis of power dynamics, of education, and of creativity in general. By all accounts he was excellent at teaching students how to improvise. But I wouldn’t recommend adopting it as your life philosophy, and I’d recommend being a bit suspicious of anyone pushing this book too hard. Getting rid of the existing social structures might benefit confident, wildly charismatic gurus like Johnstone, but most of us are just ordinary animals who do better in a group governed by norms.

edit: this got some comments on Hacker News. One commenter argues that Johnstone’s group can’t have been a cult, because he wasn’t extracting money (or sex, or anything else) from its members. I don’t think that’s the defining feature of a cult. Surely for some cult leaders it’s just about the power.

In fairness to Johnstone, he cites Sheila Kitzinger’s The Experience of Childbirth in support of this claim (the others he just puts in his own words), so maybe he felt that this was a bit out there. As you would expect, the pain of childbirth is a universal biological fact.
↩
Concerningly, the description in Something Like a Drug (in the foreword) suggests that this class was unofficial.
↩
As an example, he prompts the masked student to relax, then startles him with a mirror to trigger the trance state.
↩
Probably Jacques Lecoq.
↩
See page 83 of Keith Johnstone: A Critical Biography.
↩
I suppose that’s redundant.
↩
On page 51 of Keith Johnstone: A Critical Biography (it’s called “critical” but it was clearly written with Johnstone’s involvement and support, and does not seriously criticize him at any point). In The Knack, Tom gives a monologue about how to teach children to play the piano that could be lifted straight out of Impro.
↩
In 1983 Johnstone “read the riot act” to the improv players who were planning to unionize, threatening that they’d be cut out of the group for good. To quote Dennis Cahill, a group member at the time who opposed the union: “I just didn’t see the point to it. … I didn’t really see a need to confront Keith or cause Keith problems or to upset him in any way over something as simple as Who Has The Power or Who Doesn’t.”
↩

Overtraining as the path to human-like AI

Sat, 18 Jul 2026 00:00:00 GMT

The anonymous blogger Gwern recently completed a thirteen thousand word post called Human-like Neural Nets by Catapulting, in which he offers a theory about why LLMs don’t possess truly flexible human-like intelligence, and how we might train LLMs that do. Theories like this are entirely unremarkable: every ~~crank~~ researcher on the internet has a theory about how to crack AI. But Gwern is remarkable. Outside of OpenAI itself, Gwern is the earliest person to anticipate the potential of large language models, and the scaling arms-race involved in making them larger and more powerful still. I often cite Leopold Aschenbrenner’s Situational Awareness as an example of someone correctly predicting the future of AI. Written in 2024, just after the release of GPT-4, Aschenbrenner gets a lot of things right: the rush to build billion or trillion-dollar GPU clusters, the importance of the code around the LLM (what he calls “unhobbling”)¹, and the fact that scaling would continue through the decade. Gwern’s essay The Scaling Hypothesis anticipated the broad strokes in 2020, immediately on the release of GPT-3 (two years before the release of ChatGPT and the beginning of the AI boom).

And yet, as far as I can tell, Human-like Neural Nets by Catapulting hasn’t yet received much public attention: one recent Hacker News thread with twelve comments, all of which are about whether human brains are anything like neural networks. Part of the reason is that (a) it’s such a long post, (b) the potted summary describes Gwern’s claim, but not the reasons for it, and (c) much of the beginning of the post looks like it is indeed arguing from analogy with human brains. However, I don’t think that analogy is load-bearing. Let me try and explain what I think Gwern is saying.

What is grokking?

First, let’s talk about “grokking”. In 2022, OpenAI published a paper showing that if you train a model on a simple dataset (for instance, a simple mathematical operation like division), and keep training it long after the training looks like it’s stalled out, the model will suddenly make a massive jump in capability. Why does this work? The first stage of training is like rote memorization: the model has to compress as much of the training data as possible into its weights. But if you keep going, then regularization techniques (such as the pressure on the model to use smaller weight values) will motivate² the model to find simpler and simpler ways of compressing the data. This doesn’t look like much at first (the training loss remains at zero), until the model notices that you can express the data via simply performing the underlying mathematical operation, at which point it instantly gets massively smarter. In other words, over-training a model can pressure it into actually understanding its training data. OpenAI named this process “grokking” after Robert Heinlein’s neologism, which for Heinlein means something like “gaining a deep, intuitive and fundamental understanding”³.

Gwern’s argument goes something like this:

Modern LLMs are worse generalizers than humans because they have not grokked their core domains
Grokking requires overtraining an over-parameterized model on a (relatively) small dataset, which is the exact opposite of what frontier labs do
However, (2) is basically how human brains learn
Somebody should spend a a few tens of billions of dollars^3.5 on trying it, since it might immediately usher in truly human-like LLMs

I’ll skip (3), since I think the argument is still compelling without the analogy to human brains.

Are LLMs bad because they can’t grok?

I think his first point is hard to dispute. LLMs are very smart in specific areas, but they routinely make errors that humans wouldn’t make. More to the point, they routinely make errors that any human as smart as the LLM would never make. This pretty clearly points to a failure of generalization: LLMs are as strong as smart humans in specific areas, but can’t generalize that intelligence to as many tasks as humans can.

Do LLMs not grok? I read through this paper that argues they do. If you graph “how much data has the LLM memorized” against benchmark performance, you can see a small initial spike in benchmark performance, followed by a big drop, followed finally by a big jump in benchmark performance. This pattern doesn’t track memorization at all: memorization increases smoothly in the background the whole time.

I think this paper highlights the difficulty of distinguishing grokking from generalization. Obviously LLMs learn to generalize during training, and it’s plausible that learning to generalize would require a certain baseline level of memorization (so that the LLM has the raw material to generalize from). So it’s going to look like grokking.

When Gwern (and others) say that LLMs don’t grok, I think what they mean is that there’s at least one more giant generalization leap waiting to be made. Is this plausible? As an existence proof, humans are clearly capable of better generalization than LLMs. Of course, it’s possible that this level of human generalization comes from features of our brain that neural networks can’t replicate, but that seems kind of ad-hoc: if neural networks can generalize at all, why would they only be able to generalize this far, and no further?

The easy examples of grokking rely on domains with a simple rule waiting to be discovered (e.g. a mathematical operation). Does human language have rules this deep? I think this is an open question, but there’s good reason to think the answer is yes. Language has deep, subtle structure: not just internal structure, but structure that reaches all the way down to the way the world is and the way human minds work.

AI labs train small-ish models on oceans of data

For the last few years, many AI researchers have been saying that data is the most important thing: that whatever model architecture you choose, with enough size and training time the model will converge to its dataset. Whether this is true or not, AI labs have spent much of their considerable resources on acquiring more, higher-quality data: from scanning physical books, paying experts to produce and label data, or partnering with companies that have a lot of data already.

AI labs have also been training relatively small models. Even the largest frontier models are probably MoEs with a couple of trillion parameters and probably a tenth of that in active parameters. Of course, estimates of frontier model size are mostly guesswork, but open-source models provide a good baseline: they’re probably in the ballpark of Kimi-K3, which has just under three trillion parameters and fifty billion active parameters. That sounds like a lot, but it’s something you could probably pre-train in a couple of days in the largest frontier cluster⁴.

Grokking requires training a huge model on a small dataset

Gwern’s prediction is that AI labs should try doing the exact opposite of what they’ve been doing. Instead of training a bunch of trillion-parameter models on massive amounts of data, try training one hundred-trillion-parameter model on a small dataset.

This sounds pretty silly on the face of it. The more data the model has access to, the smarter it will be, right? Why waste an entire training cluster on a hobbled training run? Because if Gwern is right, grokking is more likely to occur when the dataset is constrained⁵. If you feed the model all the data in the world, it can continue to improve simply by memorizing more new things or drawing simple connections. If the model has to ruminate on a small set of data, it’ll be forced to keep looking for deeper generalizations. You want a very large model for this so it can memorize as much of the data as possible. Every piece of memorized data can serve as raw material for generalizing.

The big labs probably haven’t done this already. Plausibly Gwern himself is enough of an insider that he would know, and so him writing this post is evidence that the labs haven’t tried it. Also, the engineering problems involved in training a hundred-trillion-parameter model have likely not been solved yet: the largest existing model is probably Claude Mythos, which is definitely not that big. But they have the resources and engineering talent to give it a pretty good shot.

Interestingly, the political obstacles might be as hard to solve as the technical ones. This training run is going to look like it failed until the moment it succeeds: training loss will drop to zero relatively quickly, then sit there for weeks or months apparently doing nothing at all to improve test loss, chewing up billions of dollars. Do any of the top players have the risk appetite or courage to keep funding this experiment all that time?

Conclusion

Gwern’s post has an extended argument that human brain development works in the same way: that human brains have far more “parameters” than frontier LLMs, and are trained on far less data⁶, which encourages us to make deeper generalizations in early childhood. I don’t have the background in biology or neuroscience to evaluate these claims, so I’ve expressed the case for grokking entirely without reference to it.

In 2024, it became clear to everyone that “pure scaling” — the idea that you could simply train larger and larger versions of GPT-3.5 — didn’t work. OpenAI’s “even bigger version” of GPT-4 was simply not good enough, and was eventually released as GPT-4.5 instead of GPT-5. The biggest advances since then have been reasoning, which produced another great leap forward in capability, and much better automated RL, which has ushered in the current era of reliable agents. Neither of these seem like a plausible path to artificial superintelligence.

I don’t know if I agree with Gwern or not, but forcing very large LLMs to grok is at least an idea that could usher in the machine god. I can’t remember the last time I read about a simple idea this ambitious⁷. I hope one of the big labs tries it out.

For an example of the power of unhobbling, consider Claude Code or OpenClaw and the subsequent explosion of (short and long running) agentic harnesses.
↩
Obviously “motivate” and “notices” are used metaphorically.
↩
All of this is long before xAI’s use of the word “Grok” to name its LLMs. (Incidentally, I think this is why Gwern uses “catapulting” to describe the same thing).
↩
For what it’s worth, Fable estimated the cost of Gwern’s plan at $3-10B.
↩
At this model size, 25T tokens of training data at 33% utilization works out to around six million H100-hours, which a 100k GPU cluster puts out every two and a half days.
↩
Two interesting pieces of contrary evidence here. First, BabyLM is a yearly challenge to train a strong model on a very small dataset. This has been running for four years and largely does not work (that is, nobody seems to have developed a model that shows a quantum leap forward in generalization). Second, this paper tries training a 9 billon parameter model on constrained data and doesn’t see a big jump. I think Gwern’s response would be that these models are far too small — they can’t memorize enough of the training data to grok it, and arguable haven’t trained for long enough.
↩
A common objection here is to say that humans get infinitely more sensory data from the nuances of vision, touch, sound, and so on. I agree with Gwern that this is unconvincing: sensory data is largely predictable, text is surprisingly information-dense, and if this were true then deaf/blind people would have significantly less fluid intelligence (they don’t).
↩
Maybe state-space-reasoning a la Mamba, which didn’t work (yet).
↩

What does "playing politics" mean for software engineers?

Tue, 14 Jul 2026 00:00:00 GMT

Software engineers are often told to “start playing politics”, but most engineers have no idea what that means.

Their reference point for “playing politics” comes from fiction like Game of Thrones. Are they supposed to raise an army and depose the CEO, or poison each other at team lunch? Should they book Zoom calls with each other and plot schemes? All of that is obviously ridiculous. In terms of Game of Thrones, software engineers are not lords and ladies. We’re the soldiers and workers of the realm. So you should think about “playing politics” in the way a castle guard would, not one of the major players.

The castle guard are not going around poisoning people or forming coalitions between the great powers. They are largely keeping their heads down. But in order to do that, they have to stay aware of the political currents, or they’re liable to do something catastrophically stupid: for instance, making an enemy of a powerful courtier, or arresting somebody who’s on an important mission for the king.

Given that, the basic principles of playing politics are something like this:

Be aware of who’s powerful and who’s not
At all costs, avoid making powerful enemies
Help powerful people as best you can
Make sure they know you’re helping them (without annoying them)

Be aware of who’s powerful and who’s not

As a software engineer in a large company, you will not be a powerful person. Powerful people are typically in senior management: VPs, directors, and so on¹. However, not everyone in senior management is powerful. Some are killers who have the active support of the CEO, while others are confused incompetents.

How do you know which is which? If someone is clearly ferociously competent, they’re always going to have some power, since upper management tend not to ignore useful tools. But you can’t rely on competence as your only guide. Some managers are powerful for other reasons: they’re friends with the CEO, or they have strong relationships with other groups like legal or sales, or they’re simply willing to do whatever upper management wants done.

One signal is who’s leading the important projects. Read your CEO or CTO’s internal updates and pay attention to the projects that are called out by name. Organizations tend to give key tasks to trusted lieutenants. If a manager is leading an area that’s never under the spotlight, they probably don’t have enough clout.

Another signal is hiring. Is a manager’s team growing or shrinking? Particularly post-ZIRP, headcount is a rare and precious resource. A manager who’s able to get it is likely a powerful manager, or at least is reporting to a powerful director or VP (which often amounts to the same thing).

At all costs, avoid making powerful enemies

First, you should try not to make any enemies at all. Most software engineers who get “playing politics” wrong do it by needlessly alienating people: by being rude, unhelpful, abrasive, making non-technical people feel stupid, and so on. This post isn’t really about that. I’m assuming that you can figure out how to be a generically pleasant person on your own.

However, competent software engineers will make some enemies. If you’re out there making projects happen, some people aren’t going to like the way you do it, and won’t be a fan of any compromise you offer. I wrote about this in Big tech engineers need big egos: the only way to avoid making enemies is to change nothing, but that’s incompatible with doing the job.

Given that, be selective about which enemies you make. If you’re making a technical decision that’s either going to require work from team A or team B, and neither team wants to do it, you should try to pick the team with the least political cover. If you need a powerful VP’s team to do something they won’t like, try to be maximally respectful about it: get that team’s core engineers on-side if you can, or book a meeting with the powerful manager and explain the situation, or (better yet) ask the powerful manager sponsoring your project to go and talk to the other VP for you. (If you don’t have a powerful manager like this, consider abandoning your project).

Give way to powerful managers when at all possible. Every so often you really do have to stand your ground — if the system will truly collapse otherwise, or a major customer will have an incident, or if the technical decision really is entirely bone-headed — but almost all cases are not like this. The best advice I’ve ever gotten about playing politics came from a manager I worked with long ago²:

This is not the hill you want to die on.

When I’m about to pick a fight or say something argumentative, and I’m not 100% convinced it’s necessary, I ask myself: is this the hill I want to die on? And it never is.

The three rules about disagreeing with powerful people are:

Make sure you do it in private
Be polite
When they overrule you, stop arguing immediately

Disagreeing in private rarely hurts, if you follow these rules. In fact, it can help. If you can manage to disagree with a manager, get overruled, and then follow their plan without complaining, that can be the best way to gain a powerful friend. But if they think you’re going to keep griping about it, or worse still, complain to the rest of the team and foment some kind of rebellion, there’s no quicker way to make a powerful enemy.

If you have powerful enemies at a company (for instance, the CTO or an influential VP doesn’t like you), quit. It’s really that bad. I have never seen this situation turn itself around, except in the very rare case where the CTO or VP is already looking for greener pastures and jumps ship. You cannot recover the situation: they have no incentive to give you the chance to change their mind, and they have almost unlimited ability to screw you on promotions, raises and layoffs.

That’s why this piece of advice is second in the list. If you aren’t helpful or if your contributions are invisible, you can work on that and fix it. But if you’ve made powerful enemies, you’re done for.

Help powerful people as best you can

Just as it’s fatal to make powerful enemies, it’s very useful to make powerful friends. How can you do this? Remember you’re a palace guard, not a great lord: you make friends by doing your job. However, you can choose to do your job a little more proactively and diligently when you’re doing it for someone with political clout.

One obvious application of this principle is that you should answer Slack messages from powerful people immediately. If you see an ordinary Slack question pop up while you’re doing some task, it’s okay to get to it when you get to it. In fact, it’s ideal not to respond to all questions immediately, so you don’t set unreasonable expectations (and so you don’t seem like you’re sitting around doing nothing). But when a VP comes in with a question, don’t make them wait: answer the question immediately. If the question requires research, send a “let me look into that right now” message, then do the research. This is the easiest way to get a reputation for being helpful³.

Another way to do this is to lean in on important projects. Suppose you do ten projects in a year. Eight of them are normal, low-priority projects, and two of them are high-profile (say, finishing some big feature before your company’s yearly conference). It’s a mistake to allocate your effort equally to all ten. I wrote about this at length in Doing nothing at work: you should be operating at 80% capacity (or less), so you can then ramp up to 120% when it really matters.

Pay attention to the narrative that powerful people are trying to push. Here are some potential narratives:

We’ve had a lot of turnover and reorgs lately, but we’re all starting to pull together as a team now
Isn’t it great how focused we all are on reliability work after last month’s incident?
The conference this week is the most important thing, so we’re all being very careful not to break anything
We’re an AI-forward team that’s looking for the best ways we can leverage LLMs into our team processes
Although this project had a rocky start, we’re now all aligned on the way forward

You don’t necessarily have to jump in and start cheerleading, but you should at least not do anything that you know is going to make the narrative look weak. For example, on that last point, it’s foolish to openly argue that the project really was fine all along. Bring it up privately, not publicly, or you risk ruining some clever piece of propaganda that the manager in question is trying to push on the rest of the organization⁴.

Finally, an underrated way to help powerful people is to offer them social support and information. Slack messages and planning emails might seem unimportant to you, but powerful people often live in that environment: their primary tool is writing messages like these, just like your primary tool is writing code. Reading and responding (in a supportive way) to these messages is something that most engineers don’t bother to do, but it goes a long way.

Likewise, dropping a senior manager a line now and then (say, a heads-up that a particular project landed successfully, or that you got good metrics about some feature) is surprisingly helpful. Senior managers live in an information-poor environment: for them to learn something about a team’s work, that information has to bubble up through several layers of interpretation and summary. In my experience, they’re appreciative of being drip-fed the occasional piece of information, so long as you keep it brief and relatively rare.

Make sure they know you’re helping them

If you’re directly responding to a VP’s Slack messages or DMing them information, they know you’re the one doing it. But if you’re just doing your job and working hard on projects they care about, they might not notice. Being invisible is probably the most common way engineers fail at playing politics.

Fortunately the fix is simple: tell people what you’re doing. If you fix an important bug for a launch, write a message in that launch’s Slack channel saying “hey, I just fixed this bug”. What if you don’t like bragging? Get over it. You have to be comfortable publicly telling people what you’ve done. You should also keep a brag document so you can repeat all of this at review time.

Another, subtler way to do this is to gain the trust and respect of the powerful engineers in your area. Senior managers will always have a few trusted engineers they rely on to assess technical questions. They will ask those engineers what they think about you, and will broadly trust those answers. The good news is that if you’re competent and useful, those engineers will already value you, so you don’t have to do anything special: just be good at your job.

Technical power

Is playing politics all about sucking up to senior managers? Basically, yeah. A less cynical way to describe it would be “aligning with the values of the company”. If you think your company is doing good things, you should want to do that anyway! In any case, what that comes down to is figuring out what the people in charge want, giving it to them, and making sure they see you doing it. However, there’s still some scope to get what you want out of the deal.

I said earlier that software engineers do not wield organizational power. However, that doesn’t mean you’re powerless. Technical ability is a source of real power, if a delicate and unreliable one. The movers and shakers in tech companies are utterly dependent on technical people to implement their vision and to give them clear answers about the system.

There are many subtle ways you can leverage this. One I wrote about in How I influence tech company politics as a staff software engineer is to wait until important people at the company want to do something (say, improve reliability), then offer them a technical plan that does it your way. Another one is to become so useful that you’re actively in demand to lead projects, and then run the project how you want.

You probably won’t be able to change the company’s grand strategy. But how that strategy is implemented has a lot of specific technical detail, and you can put yourself in a position to decide on those details.

Conclusion

Playing politics isn’t about plotting and scheming, and it isn’t just about being a friendly, likeable person (although that helps). It’s about figuring out how your company actually operates: who makes the decisions, who gets consulted, what behavior gets rewarded, and so on. The most basic way to do that is to figure out who is powerful, get out of their way, and (if you can) help them get what they want.

edit: this post got some comments on Hacker News. I always enjoy seeing “this is too obvious to need a guide” replies right next to “yeah, I screwed this up, wish I’d been told this sooner” replies. That’s what I get for saying the obvious thing.

Obviously the exact titles depend on your company. One person I’m deliberately leaving out is your own manager. In general don’t think your relationship with your own manager counts as “playing politics”: that’s just you getting along with another human being. An exception to that is if you report directly to a powerful director or VP.
↩
Ironically, this manager struggled to take his own advice.
↩
Note that you actually have to be able to answer their question accurately in order to do this. If you’re not competent enough to be useful to powerful people, you will struggle to befriend them.
↩
For instance, maybe the CEO is convinced that the project was in bad shape because of something he heard, and the manager in question knows it’s easier to sell “yes, but we turned it around” than “no, you misunderstood, everything was always fine”. If you complicate that process, you risk the CEO thinking that the project is still bad and cancelling it.
↩

In defense of not understanding your codebase

Sat, 11 Jul 2026 00:00:00 GMT

As a software engineer, how well do you have to understand your own codebase?

My guess is that people who work on small codebases with low-turnover teams (say, Redis or games like The Witness) would say “obviously you have to understand it completely, otherwise you can’t do good work”. I’d also guess that people who work on large codebases with high-turnover teams (say, the Google web search backend or GitHub) would say “obviously you can’t understand it completely, you just have to do the best you can in your local area”.

These are two largely different ways of programming with different methods, practices and cultures¹. However, the first group is over-represented in online discussion about software engineering². I want to defend the second group against the first. In many software engineering environments, there’s nothing wrong with being in a state of partial understanding. In fact, in large systems a partial understanding is the best you can do.

Against “programming as theory building”

The best articulation of the “you have to understand your codebase” side is Peter Naur’s famous paper Programming as Theory Building. I like this paper, but I think it goes too far in that direction. Naur’s core point is that when programmers work on a program, the code is really just a by-product, and the main product they’re working on is their “theory of the program”. That’s made up of their intuitive sense of what’s happening and why, which can only be partially captured by code or documentation. If they lost the code, they could rewrite the program easily. If they lost their understanding (say, if the team experienced 100% turnover), they would struggle to make sense of the code.

So far, so good, but Naur goes further than this. He says that the theory should not be reconstructed from the code. According to Naur, you’re better off scrapping the program entirely and having a new team rebuild it from scratch, building up a new theory in the process³:

reestablishing the theory of a program merely from the documentation, is strictly impossible … [therefore] the existing program text should be discarded and the new-formed programmer team should be given the opportunity to solve the given problem afresh

Anyone who’s been an effective software engineer at a large company knows that Naur is dead wrong about this. There are at least two reasons.

First, you simply can’t rebuild large software systems from scratch. Sufficiently large systems (if they have users) contain thousands of weird cases and quirks that cannot be reimplemented. Even a team that’s intimately familiar with the system couldn’t do it: there’s just too much stuff to juggle. Successful rewrites always start by carving out the existing codebase into small isolated chunks, then rewriting one chunk at a time. In other words, rewriting a software system involves making a bunch of changes to the old system. If you can’t change the old system, you certainly can’t replace it with a new one.

Second, abandoned systems are revived all the time. In a tech company with hundreds of millions of lines of code and thousands of engineers, it’s not uncommon for a codebase to have nobody left who’s familiar with it⁴. All it takes is a few people to quit at the wrong time, or for a codebase to be unmaintained for a year. Not only have I seen other teams do this, I have personally taken ownership of abandoned codebases, figured them out, and gotten to a point where I could effectively work with them. It takes time, but building a new theory of the codebase is possible. You start by understanding one flow end-to-end, then slowly branch out from there, making careful changes as you go.

In sufficiently large codebases, everyone operates with an incorrect theory of the program. The defining feature of modern software systems is that they’re just way too big for anyone (or even a whole team) to keep in their head: nobody understands it all. To be effective, you have to figure out a way to work with a merely partially-correct theory. This is why I keep going on about taking a position and confidence. If you’re not sure about something, you can’t just sit back and wait for someone with a perfect understanding to come and give you the answer. If you’re a competent engineer, that person is you. You have to grit your teeth, make your most educated guess, and then deal with the consequences.

To be generous to Naur, it’s possible that in 1985 the average size of a program was several orders of magnitude smaller than today, and that when Naur writes about “large programs” he’s not talking about tens of millions of lines of code. Naur’s first example of a large program is a 200,000 line industrial monitoring program, and his second example is a compiler. In 1987, the first version of the compiler GCC was about a hundred thousand lines of code; in 2015 GCC was over fourteen million lines. I can believe that rewriting one or two hundred thousand lines of code is relatively straightforward, particularly if you get to reuse existing tests. Not so for one or two million.

Theory building is one tradeoff among many

LLMs are often cited as a tool that’s bad because it impedes the ordinary process of theory-building. I think this is overly simplistic. Like many software tools, LLMs are a double-edged sword: they make it harder to construct a detailed mental theory of the software, but they allow you to build a partial theory quickly and they can help you leverage that partial theory more effectively. This is a complex tradeoff that I’m still thinking about.

Setting LLMs aside, I’m confident that it’s silly to say that anything that interferes with your theory of the software must be bad. Here is a partial list of other things that make it harder to maintain a theory:

Other people being allowed to write code in your codebase
Having to implement legally-required features like accessibility and data protection
Allowing your colleagues to quit their jobs or move between teams
Having to upgrade software versions for security patches
Bringing in libraries or other dependencies

Like most things in software, “maintaining a theory of the codebase” is one value among many. Sometimes it’s the most important value and you sacrifice other values for it; other times you trade it off for speed, or legal compliance, or for political reasons⁵.

Almost all engineers — particularly “pure” engineers — prefer to maintain an accurate mental model of their software. It’s more fun, less stressful, and feels more like “real engineering”. That’s why many engineers take up open-source projects in their spare time in order to work on small codebases by themselves: in order to do engineering work where they can maintain an accurate Naur theory of the codebase. I don’t think there’s anything wrong with that.

However, at work you are paid to do a job. In other words, they pay you money to adopt their set of engineering values. It’s hopefully well-understood that however much you might personally care about performance, sometimes you have to write slow code at your job (for instance, to get a project done on time, or to accommodate some awkward requirement). Maintaining a theory of the codebase is the same kind of thing.

edit: this post got some comments on lobste.rs. One interesting comment points out that the ability to reason “locally” about code (i.e. with a partial understanding) has been a core goal of CS from the beginning. This is also a good description of what I was trying to get at in How good engineers write bad code at big companies. Also, it’s amusing that this post was tagged as vibecoding because of one off-hand paragraph about LLMs. I still don’t think I’ll be tagging the post as AI on my blog, sorry.

edit: I also got some Hacker News comments. The top comment is a genre of comment I get a lot, which is basically “wait, this situation sucks! Why isn’t this blog post about how much this sucks?” Well, there’s plenty of posts like that already: I hope to fill another niche. Another comment offers the second comparison of me with Seth Godin (ouch) that I’ve seen:

Goedecke doesn’t quite write the anodyne sound bites that Seth Godin does, but neither does he write anything of engineering use, just vocabulary explainers for people who want to know kind of what their tech leads and line managers are talking about.

First, this is skewed by what kind of posts get popular on Hacker News (i.e. not my posts that discuss technical engineering topics). Second, I think “wanting to know what your tech leads and line managers are talking about” is very important!

This post also got some traction on Twitter, including some long quote-tweets. I particularly like this idea that a “theory of the codebase” might be a temporal theory: i.e. being able to answer “why did we build X at this point”, “when was Y put in”, etc.

edit: I also recommend this blog post response from Jani Hartikainen. I disagree with him when he says that you can’t tradeoff theory-understanding against other values, because those other values just form part of the theory. I think “keep the theory of the program simple” is a coherent value that gets traded off anytime you add complexity to the program (e.g. to fulfil some customer request).

I wrote about this at length in Pure and impure software engineering. I think many of the repeated arguments we have in the software industry are caused by the pure total-understanding culture coming up against the impure partial-understanding culture.
↩
Open-source engineers are more excited to blog about their work, the raw engineering content is typically more impressive (because coordination problems dominate big proprietary systems), open-source projects can be legally written about while proprietary systems can’t, and even if you could do it legally, writing about large codebases is impossible because it requires too much specific context.
↩
I re-read the relevant chapters of Ryle’s The Concept of Mind (which Naur cites throughout) and I think Ryle is more generous about theory-building. For Ryle, theory-building or know-how automatically happens as you do things. It’s fully consistent with Ryle to think you can pick up an existing codebase just from the code, purely by puzzling it out.
↩
Naur says: “Lest this consequence may seem unreasonable, it may be noted that the need for revival of an entirely dead program probably will rarely arise, since it is hardly conceivable that the revival would be assigned to new programmers without at least some knowledge of the theory had by the original team.”. If only!
↩
Some engineers might say that maintaining a theory is the core value, because without it you can’t fulfill any of the others. I disagree. You could say the same thing about readability, or maintainability, or correctness, or a bunch of other engineering values. We trade off “core” values like this all the time.
↩

Blog about things you don't understand yet

Tue, 07 Jul 2026 00:00:00 GMT

Every post I publish represents at least two things I’ve learned: the thing that prompted me to write the post, and the thing I learned in the course of writing it. If I don’t learn anything new while I’m writing, it’s not interesting enough to publish.

Typically I learn way more than two things. For instance, in my o3 geoguessr post, I started out with the idea that most AI prompts probably don’t work, and I ended up learning that newer OpenAI models have lost o3’s ability to geolocate. That’s interesting! In my most recent post on C2PA, I started out with the idea that C2PA requires near-universal adoption, but I learned a ton of things about PKI, managing private keys on local devices, how C2PA actually works, and so on. In my post on the Luddites, I started out with the idea that the Luddite movement was fundamentally decentralized, but ended up fascinated by Luddite culture (which was far more elitist, misogynist, and violent than the pop-Luddism books describe). I could do this for every single post on the blog.

Taking a position

I think the core reason this works is that every single one of my blog posts argues a point. I never publish a post that just gives some scattered thoughts on a topic, or a post that only says “yes, I agree with this other article”. If I write a draft that nobody sensible could disagree with, I scrap the draft. Making sure that everything I write is at least minimally controversial is a forcing function: it forces me to think about what the most interesting part of my position is, and it forces me to do enough research to defend it against the obvious criticisms.

This is contrary to a lot of advice I read about blogging, which encourages the aspiring blogger to treat their posts as a form of unstructured self-expression. If unstructured self-expression is what you want to do, that’s cool. The point of having a blog is that you get to write what you want. However, this advice isn’t as helpful as it sounds.

Before I was in tech, I was a philosophy grad student. But before that, I was a poet. One thing you learn when you try to write poetry is that it is way easier to write to a restrictive structure than it is to simply “write what you feel”. This should be obvious when you actually think about it. The task of a poet is to repeatedly choose the next word. Writing to a structure (typically rhyme or meter) narrows that choice to a small set of words, instead of the entire English language. It’s the same with blogging. Forcing yourself to write about specific, potentially-controversial points makes consistently writing easier, not harder.

Writing, thinking, and research

Writing is the best way to think clearly about a topic. It’s easy to believe you understand something when you’re just turning it over in your head. When you have to condense that down into words, you find out exactly how much you do or don’t understand. I am constantly having moments where I type something, stop myself, and think “wait, that can’t actually be right”, or “is that really true?”

By the time I write my way to the end of the post, I’m usually thinking so much more clearly about the topic that my conclusion paragraph is way better than my introduction. In fact, I’ve picked up the habit of going back and immediately rewriting the first paragraph as part of my first-draft process, because I know I’m going to end up doing it anyway.

I also change my mind a lot while I write. Here are a bunch of examples of posts where I began writing them with the opposite opinion to the one that eventually made it into the post. I think this is a good sign, and I hope I never stop doing it. You should be researching and thinking about every post you write, and that means you should frequently learn new things that change your mind.

Because of all this, I deliberately choose to write blog posts about things I don’t yet quite understand but would like to, like LLM steering, Stripe’s Tempo blockchain, C2PA and watermarking, space cooling, interaction models, LLM inference internals, and so on. This is great for me, because I learn a lot. Is it great for my readers?

Is blogging to learn irresponsible?

I sometimes worry that I should only be writing about areas I already know very well, like tech company dynamics or working in large codebases, rather than presenting myself as an authority on fields I’m actually still learning. Should I let historians of the Luddites write about Luddism, Web3 engineers write about blockchains, and so on? I think this is acceptable for three reasons.

First, it’s sometimes easier for a beginner to write an introduction to a field than for an expert. Experts routinely overestimate the knowledge of the general public, and have often internalized the reasons why their field is important so deeply that they struggle to express them. I think my explainer posts are valuable because I always spend the first chunk of the post talking about what the original problem is before I get into the technical solution.

Second, sometimes the public consensus on a topic is just plain wrong, to the point where even a little bit of research is enough to demonstrate why. Many of my posts I’m proudest of have been along these lines: arguing that the “500ml per prompt” water usage figure for LLMs was ludicrous, or that the popular Apple “Illusion of Thinking” paper was tracking persistence, not reasoning, that GPUs live longer than three years and the AI companies have large profit margins on inference, and so on.

Third, I try to make it clear on my blog who I am and what my credentials actually are. Even if it’s not explicitly described in the post, I have my real name and resume available on my /about page, so I don’t think a careful reader could be easily fooled into thinking I’m an expert on 19th-century England or space physics or LLM economics or anything like that.

Feedback

Even if nobody reads what you write, writing is still a good discipline for getting your thoughts in order. But another big reason why writing is a great learning tool is that you can get feedback.

I think it’s obvious why this is useful, but I do want to make two points about feedback. First, if you do make your posts public, you need to have a pretty thick skin. People on the internet often fall over themselves to come up with the most cutting criticism or the harshest dunk. This goes double if you take my previous advice and try to write posts that make a clear, controversial point about a subject you’re learning. If you’re the kind of person whose whole day is ruined when a stranger is cruel to them, you might want to keep your blogging private or only share it among friends.

Second, even if your blogging is private, you can get feedback from LLMs. Like humans, LLMs will often give junk feedback. In my experience, OpenAI models will always tell me to moderate my claims or add caveats and hedges until I’m not saying anything at all. Sometimes their criticism will be straight-up wrong. But — particularly about technical topics — LLMs are great at pointing out areas you’ve genuinely misunderstood, and they’re far kinder than the average Lobsters or Hacker News commenter.

Conclusion

I’m pleased and grateful that people enjoy reading my posts, but even when nobody did, I still got a lot of value out of blogging. I write as a method of thinking more clearly, as an excuse to do research on topics I want to learn about, and as a way of getting feedback.

If you’d like to try it yourself, I suggest watching for these two things. First, you should be changing your mind a lot as you write. If not, you probably aren’t doing enough research. Second, your first draft’s conclusion should be much tighter and more expressive than its introduction. If not, you probably haven’t learned anything from the writing process, which means the draft can be scrapped.

I strongly recommend this practice to anyone with an interest in writing. You will see the benefits even if you don’t publish any of your writing on the internet, particularly now that you can get good technical feedback by pasting your post into an LLM¹.

edit: this post got some comments on Hacker News.

For what it’s worth, I’ve fiddled with careful “review prompts” and it’s basically as good to just write “review, please:” and paste your article.
↩

C2PA only works if everything is signed

Mon, 06 Jul 2026 00:00:00 GMT

The European Union AI Act is Europe’s attempt to comprehensively regulate AI usage. A big part of that is the requirement that AI-generated content be identifiable: either tagged with a watermark or with what the Act calls “digitally signed metadata”¹. Since all this becomes enforceable in a month, it’s worth figuring out if it makes any sense. I recently discussed AI watermarking at length in Text AI watermarks will always be trivial to remove. What about digitally signed metadata?

The most well-known implementation of digitally signed metadata is C2PA Content Credentials, which is a mechanism² for ensuring that almost every single image file should contain unspoofable authorship metadata. Here’s my position on it:

C2PA broadly makes sense and is a good idea
It is pointless to use C2PA for AI-generated images only
It will take many years for C2PA to be adopted across all images
Because C2PA makes such great safety theater, we’re going to see a lot of hue and cry about it long before it becomes useful

Lots to unpack. Let’s start by considering images, since that’s the easiest case.

How C2PA signing works

When an AI tool generates an image, that tool should include a “made by ChatGPT” disclaimer in that image’s metadata. Likewise, when a camera takes a photo, that camera should include a “taken by a camera” disclaimer. C2PA uses two strategies to protect this metadata:

The metadata must be signed by some trusted private key
The metadata contains a hash of the file’s contents, so you can’t copy an existing signature onto a new file

Each physical camera (or phone) has its own private key, for obvious reasons³. How do we know that those millions of private keys are trusted? Via PKI, like HTTPS: each camera’s private “certificate” (which contains its public key) is signed by the manufacturer’s well-known private key, so the chain of authenticity can be verified as long as you have (say) Apple’s root public key⁴.

What happens if you then edit your photo in Photoshop? Photoshop will leave the camera’s metadata untouched, but will layer a “also, Photoshop was used” piece of metadata over the top, signed with Adobe’s private key (well, with the private key associated with your official copy of Photoshop, which is signed by Adobe’s official private key).

Likewise, if you ask ChatGPT to generate an image for you, ChatGPT will sign its “made by ChatGPT” metadata with OpenAI’s private key. In theory, every single image could contain unforgeable C2PA metadata, allowing software like Twitter to trivially distinguish real photos from fake ones.

C2PA needs more regulation to boost adoption

Right now, C2PA does not have anything like the adoption it’d need to work. It’s hard to find hard data on how many images in the wild use C2PA, but FotoForensics reports around a dozen per week (so around 600 out of the 900,000 images processed each year). This is even worse than it sounds, because basically all of the signed images are AI-generated. The adoption rate of C2PA for human-generated images is much, much lower: so far, Google’s Pixel 10 is the only phone camera to sign photos by default. The iPhone doesn’t sign photos.

If almost all AI images are C2PA-signed, but almost no human-generated images are, consumers have no reliable way of identifying AI content, because anyone who wants to pretend their AI content is human can simply remove the signature. For C2PA to succeed, it needs to be on every camera and every phone, so that a photo with no signature is rare and suspicious.

Is that realistic? Actually, I think it is. The appetite (at least in the EU) to regulate AI will increase over time, and while the current EU AI Act only mandates that AI-images are tagged (which by itself is useless), it’s plausible that some future regulation will enforce tagging of all images.

Another adoption problem that must be solved for C2PA to work is preservation. Right now, if you download a C2PA-tagged image, send it as a Facebook message, then re-download it, the C2PA manifest is stripped out. Most images we see on the internet have passed through some social media asset server at least once. All of these social media companies would need to update how they re-encode image content in order to preserve the C2PA data⁵. This would almost certainly require more regulation: C2PA adds tens or hundreds of kilobytes to each file, which at social media scale is big money⁶.

Forging C2PA signatures

Could a clever attacker forge a C2PA signature? Kind of. Neal Krawetz, who seems to have led the anti-C2PA charge, points out that with a camera development kit it’s straightforward to trick a digital camera into thinking that it’s taking an image when in fact it’s being fed one. This is very much not my area, so please write in if you know more about camera hardware and you think I got this wrong. I suppose you could also take a photo of an AI image on a screen, though I imagine you’d have to be careful to make it look real.

If you exclude physical attacks on a digital camera, I think C2PA is more robust. You can sign a photo with a self-signed certificate, but the C2PA spec and docs say that validators must check that your certificate bubbles up to the official C2PA trust list. This list currently contains only 26 certificates, and there’s a whole process for being added to it. That’ll slow down adoption, but at least it makes it hard to forge⁷.

Other file types and concerns

We’ve been talking exclusively about images, but it’s more or less the same story for any type of content. If the file doesn’t support JUMBF metadata (say, an Excel file or a PDF), then the C2PA metadata has to live in a “sidecar”: a separate .c2pa file, probably on some Microsoft or Adobe content server, which contains the signed checksum and the data about who created the file.

However, the distinction between “real” and AI-generated content is fuzzier when you’re not talking about images. Here’s a trivial example: if I ask ChatGPT to create an Excel spreadsheet for me, the file will be tagged as AI-generated, but I can simply copy/paste the content into a new Excel doc and save it, which will tag it as human-generated⁸. There’s no software tool that can identify when I’m retyping some AI-generated text (except for perhaps text fingerprinting, which has its own raft of issues).

There are also interesting questions around key management. ChatGPT and other AI tools have an easy problem — their users are all online, and so the files can be signed server-side — but how do you sign files created via Photoshop/Excel/Word? If the user doesn’t have internet, do you use some kind of local key? If so, how do you prevent that key being extracted and used to sign AI-generated content?

Finally, is it a civil liberties problem to automatically fingerprint every photo? Does it make it impossible to be a whistleblower if every photograph can be traced back to your camera? I think this is a complicated question, but in short: I’d expect whistleblowers to already strip EXIF metadata from their images, C2PA metadata is similarly trivial to strip out, and overall I think image attribution is positive for whistleblowers because it heads off “this was AI-generated” responses.

Conclusion

C2PA is probably here to stay. But it isn’t useful now, and won’t be useful until two huge programs of technical work are completed:

Every camera manufacturer (including phones) must C2PA-sign all images by default
Every social media company must retain the C2PA metadata on uploaded images

This will be a long organizational process, since each manufacturer must go through the approvals process (or decide to start their own competing system), evaluate the legal ramifications of storing attribution data in images, and so on. It will be a long technical process, because C2PA metadata is a substantial fraction of image sizes: storing it will add many petabytes of content.

Of course, just because C2PA isn’t useful doesn’t mean we’re not all going to do it. Lots of companies are under pressure to signal that they care about AI safety and to head off regulatory attack. “We’re cryptographically signing AI-generated content” is a compelling “we’re doing something” pitch, particularly for people who aren’t technically savvy enough to understand the limitations. In the near term, I expect large AI-involved companies to invest a substantial amount of engineering effort in C2PA-related activity.

In the long run, once everyone gets on board, I think C2PA could end up working well. It’s awkward in some ways, but “attest content via a PKI certificate chain” is a good idea.

Is it possible to defeat? Yes, of course. By design, private keys will be in the user’s hands — in their cameras, in their local versions of Photoshop or Microsoft Word, in their phones — so sufficiently technical users will be able to crack them out or use them to sign whatever content they want. I still think C2PA will end up stemming the tide of AI content, because most users are not going to be sophisticated enough to perform attacks like this. However, we should still retain some skepticism of unlikely-looking content, even if it has “created by a human” in its C2PA metadata.

See sub-measure 1.1.1 of the Act’s associated Code of Practice.
↩
A previous version of this post criticized C2PA for incorrectly claiming to be the semi-official technology of the AI Act, but in fact this claim comes from C2PA Viewer, which is not affiliated with the official C2PA coalition. Thanks to Paul Friedel, who has recently written his own post about C2PA, for emailing me with the correction.
↩
Otherwise if you cracked the key out of one Sony camera, you could spoof content from any Sony camera.
↩
In practice there are usually more “links in the chain”: a device will be signed by some intermediate certificate, which in turn will be signed by another intermediate certificate, which will be signed by the root certificate. That’s because the root key is so valuable. If an intermediate private key leaks, it can be revoked and replaced (via the root key), but if the root key leaks, it would take years to rebuild the network of trust. So almost all signing is done by intermediates, and the root key stays on a USB drive locked in a safe somewhere.
↩
Not to mention that the whole point of C2PA is that these social media companies will be displaying a “human or AI” sticker in their UI, which will require retaining the metadata.
↩
C2PA allows for storing the manifest content as a separate .c2pa file, and just including a manifest url in the image metadata itself, but that doesn’t solve the cloud provider problem: they still have to store all the .c2pa files on-disk somewhere.
↩
I think this defuses Neal Krawetz’s “worst-case scenario”. I downloaded his forged image, and (as expected) it gets flagged as “signed, but we don’t trust the root”. I think Krawetz was right at the time, though, since the official “trust list” was only launched in mid-2025.
↩
You could do the same thing with images by copying into Photoshop or Paint, but while that’d obscure the AI source, it would still be clear that the photo wasn’t taken by a camera.
↩

Text AI watermarks will always be trivial to remove

Thu, 02 Jul 2026 00:00:00 GMT

The European Union AI Act will begin to be enforceable in August 2026, one month from now¹. One of the biggest new requirements is Article 50, which requires all AI outputs to be “detectable as artificially generated”. In other words, if LLM providers want to do business in the EU, they will have to apply a watermark to their outputs²: some hidden signature that can be used to identify AI content.

LLM text watermarking is a fascinating problem. Like the best engineering problems, it is theoretically hard to solve perfectly, but has multiple partial solutions: for instance, Google’s SynthID, and (as I’ll argue) some quiet Unicode trickery from OpenAI and Anthropic. It will be interesting to see how the AI labs navigate these tradeoffs before the end of the year.

Why text watermarking is hard

I wrote about AI watermarking at the end of last year in AI detection tools cannot prove that text is AI-generated. It’s easy to watermark an image, because digital images contain lots of noise that the human eye can’t really see. For instance, you could apply a watermark like “these twenty pixels in these exact spots will always share a color”. Text is much, much harder. Unlike images, text is a very compressed medium: you cannot make any change to a sentence that a human wouldn’t notice (with one exception, which we’ll get to later). So how are you supposed to watermark it?

It’s basically a text steganography problem (concealing a secret code), made more difficult because the plaintext cannot be arbitrarily manipulated. Any changes you make to apply the watermark will compromise the quality of the output. For instance, “every fifth letter is an ‘e’” would be a good watermark, but applied naively would make the AI output full of typos. Could you just let the model figure out how to fit the watermark? Strong AI models are smart enough to juggle this kind of constraint³, but it’d still consume reasoning time that would be better spent on the user’s problem, and make the model sound much less capable than it is⁴.

Do we need watermarks to detect AI content?

Do you really need a watermark? If you’re Anthropic, and you’re required to be able to verify whether your models produced a particular block of text, can’t you simply run the text through each model, measuring as you go how closely the model’s predicted tokens match each token from the text?

Not really. The space of “all possible Claude Sonnet answers to a question” is way larger than the space of “all possible watermarked answers to a question”. In other words, you’d get too many false positives for human text that reads like it was AI-written. It’s way more likely for a human to accidentally write like Claude than it is for a human to accidentally reproduce a watermark.

It would also be prohibitively expensive to run every Anthropic model against a piece of text in order to watermark it. The EU AI Act will eventually require labs like Anthropic to offer free watermarking services to every EU citizen (see Commitment 2). You couldn’t do that with the “run the model” approach.

How SynthID works

As far as I know, the only AI provider to say they watermark text output is Google, who use a tool called SynthID. Here’s how it works.

When an LLM generates text, it’s generating a series of tokens (words or chunks of words). At each step, the model itself doesn’t output a single token, but instead outputs a full list of all (say) 100,000 tokens in its vocabulary, each annotated with the probability that that token will be the next one. Tools like ChatGPT or Claude Code will pick semi-randomly from the most likely options in order to get their outputs. This semi-random sampling process can be influenced in a detectable way.

For instance, we could choose a sampling strategy like “we pick the second most likely token, then the first, then the second, then the first, and so on”. That would still produce high-quality output, but you’d be able to re-run the model against the generated text to verify that the pattern holds. However, that’d make verification really expensive, and any slight tweaks to the output would break the pattern and thus break the fingerprint. Is there a better way?

Yes. SynthID is a process for assigning each token a “score” based on its previous tokens (for instance, sum the token’s ID with the IDs of its previous three tokens then take mod 5)⁵. To apply the watermark, the model adopts a sampling strategy like “out of the top five most likely tokens, pick the one with the top SynthID score”⁶. The watermark can then be detected by calculating the aggregate SynthID score of a block of text. If it’s suspiciously high, it’s very likely to have been AI-generated.

This is basically a version of the common advice that you can identify LLMs by use of the em-dash, except that instead of a list of keywords, it relies on subtle mathematical relationships between words that humans can’t identify. Because the process for assigning the score is trivial, it’s very cheap to run watermark detection.

Unicode watermarks via homoglyphs

Google have a complicated mathematical rationale for why SynthID doesn’t make the model dumber: supposedly the SynthID scoring is random enough to act like a normal pseudo-random token sampler, just one that leaves a detectable fingerprint on the outputs. But of course this is suspicious. For instance, it’s common to do inference setting temperature to zero, which always picks the model’s most likely next token. In that case, you can’t leave a fingerprint at all (or you have to ignore the user’s preference and pick the second or third choice anyway).

If you can’t alter the model outputs, can you still fingerprint the content? Well, kind of. I’m pretty sure OpenAI and Anthropic are sometimes applying fancy Unicode tricks. For instance, you might go through and replace your normal ” ” spaces (unicode U+0020) with a three-per-em ” ” space (unicode U+2004), or a CJK ideographic ”　” space (unicode U+3000). These are called “homoglyphs”, and you can find more of them here.

Of course, lots of human-generated text uses homoglyphs. But it’s trivial to encode a pattern of homoglyphs (say, “every third space becomes a three-per-em”) that is much less likely to occur in the wild. Like the SynthID watermark, a homoglyph-based watermark can be detected very cheaply. A homoglyph-based watermark is cheaper to apply than SynthID: you could even do it entirely on the client.

I don’t think this is a conspiracy theory. Claude Code was definitely doing this to tag suspicious requests from Chinese users (exploiting homoglyphs for the ’ character in “Today’s date”, though they’ve since walked that back). In the last few years, I’ve noticed that when I copy blocks of text from ChatGPT and paste them into VSCode, sometimes VSCode marks some or all of the spaces as unusual Unicode characters⁷. Are OpenAI and Anthropic using homoglyphs as an AI-generated watermark? I’m not sure. But they’re definitely using homoglyphs.

Text watermarks can be trivially removed

The AI Act (specifically, its associated Code of Practice) requires watermarking to be “embedded within the content in a manner that is difficult for it to be separated from the content”. However, text watermarks can be trivially removed.

To remove unicode homoglyph watermarking, you simply have to replace all the homoglyphs with their “real” character equivalents. If you have access to even a relatively weak un-watermarked LLM⁸, you can strip out SynthID watermarking by asking that LLM to paraphrase the text content. Because the watermark is inherent to subtle vocabulary choices, re-wording the content will remove the watermark. You could even do it by hand, although at that point it’s not really AI-generated content anymore. Since there will be some kind of free public watermark testing tool, you can just keep tweaking until it comes back negative.

Moreover, the AI Act requires watermarking techniques to be “interoperable… as far as this is technically feasible”. That means AI providers would have to publish their watermarking process, and potentially even attempt to standardize on applying the same kind of watermarks. I just don’t see how this is compatible with the kind of security-by-obscurity that LLM text watermarking depends on. Unlike image and video watermarks, text watermarks will always be trivial to remove.

What about C2PA?

The AI Act and Code of Practice talk a lot about “digitally signed metadata”. The idea here is that you can include an AI disclosure in the file’s metadata itself, ideally in a way that cannot be tampered with (for instance, by signing a hash of the file’s contents). This signed-metadata process is basically C2PA Content Credentials. While you can remove C2PA metadata, you (theoretically) can’t fake it, so a file with “created by a human” metadata can be trusted, and files with no metadata at all can be held in suspicion.

This post is already too long to get into what I think about C2PA, but I do want to say that C2PA is not a substitute for text watermarking. It only really applies to files. In the words of the Code of Practice, that’s “a data format that supports attaching metadata (e.g., an audio, image, video, or containerised text)“. The output of chat tools (and most of the output of AI agents) is not containerized text, but plain old regular text, and so can’t be signed. What would it even look like to sign ChatGPT outputs? There’s no artifact to pass around.

I think it’s a fascinating question whether Claude Code has to C2PA-sign any HTML files or PDFs it generates for you. That seems kind of tricky to get right. But in any case, the AI Act also mandates some kind of actual watermarking as well.

Conclusion

So what’s going to happen this year? If I had to guess, I’d say that each AI provider (not just labs like OpenAI or Anthropic, but third-party providers like Fireworks or Groq) will stick a SynthID token sampler in front of their inference stacks. This might be limited to users in the EU, but it might not be, since SynthID is at least as good as a normal top-k token sampling approach.

AI providers will then offer a “check for watermark” page that re-tokenizes user-provided text, runs the scoring, and checks whether it’s above a certain threshold. Depending on how seriously the interoperability clause is taken, providers might even standardize on the same SynthID setup, in which case there could be a single EU-hosted “watermark this text” page.

I don’t think unicode-based watermarking is going to be considered compliant with the AI Act, but some providers which don’t want to set up SynthID might try it. Either way, technical users will be able to strip out the watermark at will, and there will be a plethora of tools that non-technical users will use for this purpose.

edit: this post got some comments on Hacker News. Commenters mostly debate the merits of watermarks and AI regulation in general. One commenter says that cryptographically signing text is easy. Sure, I agree, but the hard part is storing all the signatures (despite what some people worry about, AI companies don’t and ca’t store all their customer’s prompts). Some other commenters complain that removing watermarks is a crime. That doesn’t seem legally or morally right to me. It’s basically putting stickers over the logos on your own property!

Well, for new systems; existing ones get until December.
↩
I don’t think the plain text of Article 50 requires this, but Recital 133 and the Code of Practice makes it pretty clear that they’re looking for watermarks.
↩
Even with extra high thinking, GPT-5.5 could not explain SynthID to me with every fifth letter being an “e”, but GPT-5.5-Pro produced this puzzling koan: “These hidden codes label model-made image, voice, movie, prose. Probe trace: maybe a model-made piece. Maybe erase trace; maybe leave trace. Hence trace alone? No.”
↩
I leave the analogy with AI safety guardrails as an exercise for the reader.
↩
That’s a toy example. In practice there are multiple different (but still mathematically simple) scoring methods that get combined together, including a random seed. Why include the seed? Otherwise the watermark would bias towards the same set of tokens.
↩
The tokens are scored in a multi-round knockout against each other, but I think that’s more of an implementation detail and not required to get the core intuition behind why SynthID works.
↩
When this became public knowledge, OpenAI claimed it was just a model quirk, which is certainly possible.
↩
All AI providers might be legally required to watermark, but even tiny local models are good enough to paraphrase text.
↩

Saying the obvious thing

Sat, 27 Jun 2026 00:00:00 GMT

Stating the obvious is surprisingly useful. Most of your knowledge lives below the threshold of conscious awareness, so it’s possible for a piece of writing to remind you of what you already know. It’s common to know you don’t like something without being quite sure why, and reading an obvious statement (such as “accuracy matters, even when you agree with the broad strokes”) can help clarify why you find certain things distasteful.

Sometimes you can see some obvious truth that nobody seems to be talking about, and reading it in someone else’s words can prompt an “oh god, I’m not crazy” moment of catharsis. For many junior engineers, it’s almost a rite of passage to notice that some percentage of software engineers do virtually no work. Since nobody talks about it (how would you even bring it up in the workplace?), they often feel like they’re losing their minds: surely this state of affairs wouldn’t be allowed to continue, so they must be completely misreading the situation. But in fact it’s true.

Stating the obvious is hard. It can even be dangerous: sometimes there’s a good reason nobody says the obvious thing. But I think the bigger reason it’s hard is for the same reason that it’s hard to draw what you actually see. When I look at a person and try to draw them, I’m not drawing the lines and shades my eye sees (like a printer or camera might). I’m drawing what I know the person looks like, which is a kind of stick-figure approximation. It takes time and effort to drop the layer of interpretation and draw what’s actually there¹.

Many of the posts I’m most proud of are times when I’ve managed to articulate something I think is obviously true: engineer reputation is determined by ratchet effects, good engineers are right most of the time, you shouldn’t just do JIRA tickets (or glue work), and so on. These are all things I’ve believed for a while, but have only (relatively) recently been able to notice that I believe them. Sometimes I’m helped along by reading something I vehemently disagree with (like “nobody gets promoted for doing simple work”, or ”big egos have no place in tech”).

Stating the obvious doesn’t mean avoiding nuance. Every obvious claim carries with it a host of subtle, non-obvious claims. For example, I believe that having a big ego can be very useful as a software engineer. But why exactly is that, and what do I mean by ego? Obviously it’s not good to be constantly flexing your status on other people, or to be unable to tolerate the possibility of being wrong. However, I do think you need to be able to take firm technical positions even when the situation is uncertain, which means you have to be confident in your technical instincts. Teasing out that distinction (and its implications) is very interesting, but in order to do it you need to be able to first articulate the obvious part.

I’ve been talking about stating the obvious in technical blogging. But this principle applies just as well to other kinds of communication. When I write a technical design document at work, it’s very important to state the obvious. In fact, technical communication is so hard and general understanding is so poor that just getting people aligned on the obvious things is often enormously valuable. Much great literature and poetry aims to bring out some obvious but hard-to-articulate part of human experience.

Don’t avoid writing something down just because you think it’s obvious. The thing you think is obvious now might recede into your subconscious in an hour; get it written down while you can! And don’t avoid writing something down because you think it’s dangerous to say and everyone already knows it. For people new to the area, reading your words can help them feel like they’re not losing their minds. Finally, once you write down the obvious thing, it allows you to go on and draw out the parts that are less obvious, in a way that you couldn’t do if you try to just skip straight to the subtleties.

Incidentally, this is why most people cannot draw a bicycle on their first attempt. Unless you’re a mechanical engineer, you probably do not have a stick-figure-level approximation of what a bicycle looks like in your head, so you begin confidently (after all, you’ve seen a thousand bicycles) and get stuck after the first few lines.
↩

AI inference is obviously profitable

Fri, 26 Jun 2026 00:00:00 GMT

Many people claim that AI inference is unprofitable to serve, and thus must be subsidized by an ocean of dumb money from investors who believe that some future AI model will come to dominate the world economy. When that dumb money goes away, so will AI products. According to this view, LLMs are just inherently too expensive (in terms of money, power, and water) to be used in consumer products. In fact, they can only be used today by externalizing the costs: money onto VC funds and now retail ETF investors, power onto electric utility consumers, and water onto the communities where datacenters are built.

There are good reasons to dislike AI, but this really isn’t one of them. In fact, AI inference is obviously profitable.

Doing the math demonstrates that inference is profitable

Frontier AI providers are reporting 70%-80% gross margins on inference, but maybe we can’t trust them. Let’s do some very rough estimates on the actual cost.

A Nvidia A100 consumes 400W of power under full load. In practice, even a carefully-tuned inference server will not be at full load all the time, but it’s at least an upper bound. Suppose you’re running a dense 70B model¹, which will fit comfortably (unquantized) on four A100s at around 2M tokens per hour. At industrial power prices, that’s about 13c/hr in the USA. Suppose (pessimistically) cooling is the same cost. That’s about 13 cents per million output tokens².

Let’s amortize the cost of the GPUs, since that’s going to be the most expensive part. An A100 costs about $20k. If each A100 lasts around five years³, you’ll have to make 16k/yr in profit to recoup your capital investment (or $1.80 per hour). At lower utilization, it’ll take longer to recoup, but your GPUs will also last longer. Either way, your overall inference costs are at about one dollar per million tokens.

GPT-5.4-mini charges $4.50 per million tokens, and stronger OpenAI or Anthropic models are three to six times as expensive. It’s hard to make a direct comparison because we don’t know the size of OpenAI or Anthropic models, but the claimed 70% or 80% profit margin is extremely plausible.

Open LLMs demonstrate that inference is profitable

What if you don’t trust my estimates either? Let’s look at the pricing of open-weights Chinese LLMs. DeepSeek have claimed a bit over 80% profit margin on inference for DeepSeek-R1. Since their API pricing for R1 is less than half that of OpenAI or Anthropic⁴, that suggests that my estimates above for inference cost might be too expensive. Cooling at scale is probably cheaper than power, R1 only has half the active parameters of a dense 70B model, modern GPUs are more efficient than the A100, and there are significant economies of scale in inference.

Since DeepSeek’s models are available for anyone to download, they can’t get away with extracting a large profit margin. One of the other inference providers would undercut them with the same model. Inference costs for DeepSeek-V4-Pro on the market are around 87 cents per million output tokens, which is probably pretty close to the actual cost of serving the model.

For AI labs, inference must subsidize training

All of this doesn’t mean that OpenAI or Anthropic are profitable. Those companies are making huge capital investments that may or may not pan out, and are spending enormous amounts of money on talent and compute to train brand-new models and retain users.

They’re doing crazy things like offering per-month subscription models for nearly unlimited inference, which is almost certainly not profitable. If you used an API token instead of your Anthropic subscription in Claude Code, you’d pay ten times the cost. But that doesn’t mean API-based Claude Code couldn’t be a good deal. Some people are already using DeepSeek’s inference API for agentic coding, because once you take away the huge profit margin it’s cheaper than the relative per-month subscription.

Why won’t OpenAI or Anthropic lower their prices? Supposedly OpenAI has thought about it, but for an AI lab, inference has to subsidize training costs. A company like OpenAI has to fund the production of new models from the inference margins on existing models (at least partially). That’s why the margins on inference are so high: the AI labs are trying to squeeze out every dollar so they can stay alive in the training arms race.

However, inference only has to subsidize training costs for an AI lab. If you’re merely an inference provider, you don’t have to do any training at all. Therefore, even if OpenAI and Anthropic go out of business, whoever snaps up the rights to their frontier models will be able to continue selling Opus and GPT inference at a profit⁵. The AI bubble popping will not mean the end of the inference business, because AI inference is obviously profitable.

Expensive frontier models are probably mixture-of-experts, not dense, which is tougher to estimate. However, I think a 70B dense model and a MoE with 70B active params will come out to basically the same numbers at scale (though the MoE will require more GPU memory and thus a greater upfront cost). Are frontier models around 70B params? Nobody outside the AI labs really knows, but my guess is that 70B is probably larger than a Haiku/mini class model.
↩
I think it’s reasonable to estimate the cost of output tokens only, since they’re by far the most expensive part of serving inference. Input tokens are cheaper for two reasons: transformers let you prefill them in parallel, and for most real-world use cases they can be aggressively cached in the KV cache.
↩
It’s common (and wrong) to estimate GPU lifespan at three years. I wrote a lot about this in AI GPUs probably live longer than three years.
↩
Again, this is just an guess, since we don’t know what OpenAI or Anthropic model is equivalent in size to R1.
↩
I do wonder if Anthropic would be able to prevent other people from being able to access the model if the company goes out of business. Anthropic is currently in debt to Broadcom, Google, and a bunch of private equity firms. Would they get the Mythos and Opus weights, over Dario’s protestations?
↩

AI GPUs probably live longer than three years

Mon, 15 Jun 2026 00:00:00 GMT

People who think current AI use is unsustainable often rely on the claim that inference GPUs only last “three years at the most” under load¹. The idea here is that once the AI bubble money drains away, current infrastructure will rapidly become obsolete, and there won’t be enough money floating around to buy a whole slate of brand-new GPUs. Inference costs would thus rapidly become way too expensive for current AI products to make any financial sense.

Where does this “three years at the most” claim come from? Is it plausible?

Sourcing the quote

The original Tom’s Hardware article quotes this tweet from Tech Fund, an anonymous former PM and tech investor, who quotes an anonymous “GenAI principal architect” at Google as saying “if you have a high utilization rate, then constant high utilization rate for a year or two, I think the lifespan will be three years at most”.

This screenshot looks like it was from an interview. What interview? I scrolled back to October 2024 on Tech Fund’s Twitter feed and saw a bunch of similarly-formatted screenshots, some of which were cited as coming from Tegus. Tegus is apparently a company with a business model of reaching out to insiders (in this case, AI company employees) and paying them hundreds of dollars an hour in order to answer specific technical questions. It’s essentially gig work for almost-but-not-quite insider trading: the more informed and confident you sound, the more likely Tegus analysts will pick you for future interviews.

I’m sure the source for this tweet is in fact a GenAI principal architect, since Tegus would have presumably asked for some proof of that before they paid them out. But it’s pretty clear that the incentives here are to sound confident and authoritative, even on questions that you’re not sure about. With that in mind, the quote itself also reads a bit suspiciously. I’ve worked with enough principal engineers and architects to take their casual back-of-envelope estimates with a grain of salt. If they knew the actual rate at which GPUs fail and get retired in Google datacenters, wouldn’t they have just said that?

Evidence for a longer lifespan

We have some anecdotal evidence that points the other way. Google has publicly claimed to have eight year old TPUs (their version of GPUs) running in production at “100% utilization”. Nvidia only made A100 GPUs from 2020-2024, but in February 2026 the AWS CEO claimed that AWS had never retired an A100 server (and you can still easily rent A100s for AI work)². AI GPU usage isn’t exactly like crypto mining GPU usage, but it certainly seems like years-old ex-crypto GPUs are functional. There’s also this comment from Hacker News I noticed where someone claims that their GPU cluster in academia has lasted six years with less than 20% failure rate.

What about hard data? It’s hard to get concrete data on the lifespan of AI GPUs, because modern AI datacenters have only existed for a handful of years. But an interesting case study would be recent supercomputer clusters like Oak Ridge’s Summit, which had over 27 thousand Nvidia V100s running from 2018 to 2024, or its predecessor, the Cray Titan supercomputer that ran from 2012 to 2019. I couldn’t find any evidence that Summit had to buy an additional 27,000 GPUs to replace their old ones, and GPU failures in Titan have been carefully studied:

These cages of GPUs are stacked vertically, and cold air is pumped in from the bottom, which explains why cage 0 (at the bottom) has better survival rates than cage 2 (at the top). Let’s consider cage 0, so we’re just looking at the GPU lifespan instead of at the lifespan of improperly-cooled GPUs. At three years, over 95% of GPUs survived³. At six years, nodes 2 and 3 (the GPUs closest to the bottom of the cage) were still at above 90% survival rate, and the highest nodes were over 60%.

It’s possible that newer Nvidia GPUs are less reliable than older ones (they certainly draw more power), or that AI datacenters are under-cooled, or that something about LLM utilization is more stressful than the workloads that ran on traditional GPU datacenters. But this is at least circumstantial evidence that GPUs can survive under load for far longer than three years.

Economic lifespans

This discussion is complicated by the fact that GPUs may have a short economic lifespan. Supposedly a B100 GPU draws twice as much power as an A100, but can do five times as much work. For some AI providers, that might mean that A100s are only worth running until they can be replaced with B100s (if you’re bottlenecked on electricity, you should spend it all on B100s and throw out your obsolete A100s). This is why the Titan supercomputer was decommissioned in favor of Summit: it could have continued to operate, but it was more profitable to spend the money and maintenance effort on newer hardware.

It should be obvious that this doesn’t support the “inference will become more expensive when the bubble pops” argument. So long as A100s are profitable right now, cash-poor AI providers can continue profitably serving inference from them, even if there are more efficient options available for those with the capital to upgrade.

On top of that, GPUs only represent one part of AI datacenter infrastructure spending. If your GPUs wear out, you don’t have to go and build an entirely new datacenter. About 30-50% of datacenter spend goes to land, power, cooling, and so on. The remaining 50-70% is the cost of the entire server rack, which includes a bunch of things that aren’t GPUs.

Conclusion

Like the idea that AI inference requires using huge amounts of water, the idea that AI GPUs only live a year or two is popular because it’s a useful idea for AI skeptics, not because it’s true. It comes from a pseudonymous tweet quoting an anonymous source who’s being paid hundreds of dollars to sound like a credible expert on AI. Other public communications from AI inference providers cite much higher lifespan numbers, and the statistics from supercomputers (the traditional examples of large GPU clusters) don’t bear out the claim that the maximum lifespan is three years.

It might be true that the economic lifespan is three years, in a world where new GPUs come out every eighteen months and GPU providers are flush with cash to upgrade, but that doesn’t tell us much about the economics of inference in an AI winter. If money becomes a lot more scarce, it’s likely that AI datacenters will continue profitably⁴ running their B300s (or their H100s or even A100s) for six years or longer.

Of course, like previous claims about AI and water usage, “three years at the most” is often cited as “1-2 years, with some lasting up to 3 years under optimal conditions”.
↩
Of course, pronouncements from CEOs/CTOs should be taken with a grain of salt as well (for instance, maybe they have a big backlog of unused A100s they keep swapping out), but (a) executives don’t often straight-up lie about concrete technical facts, and (b) they’re going up against an unsourced quote from a tweet, so the bar isn’t that high.
↩
What about proactive GPU replacement? In the “Survival Analysis” section, the study attempts to account for this. I haven’t dug into exactly how.
↩
Assuming inference is profitable, which I believe (when you’re not attempting to amortize the cost of training).
↩

Doing nothing at work

Mon, 08 Jun 2026 00:00:00 GMT

Many engineers should be doing less work. I don’t necessarily mean producing less code or fewer changes, but literally working fewer hours in the day. When they do work, they should be working at a slower pace. I like to aim to be running at 80% utilization by default: unless I have a high-pressure project going on, I spend 20% of my workday away from the computer.

High-impact opportunities

Why? Performance at tech companies is dominated by outlier events. When I think about the most impactful changes I’ve made, many of them involved a surprisingly trivial amount of work. There are no points for effort in software development. What matters is solving the right problem at the right time.

In large engineering organizations, there are usually trivial pieces of engineering work you could do that would make tens or hundreds of millions of dollars for the company. Here are three common examples:

First, when the company is trying to sign a big enterprise deal, stepping in with a feature or bugfix can make the deal happen. It doesn’t even have to be a good feature: sometimes just showing that you’re willing and able to make a concrete change will be enough.

Second, preventing or mitigating an incident early (even by just knowing the right feature flag to turn off) can save huge amounts of money: both immediate lost revenue during the incident and future lost revenue from customers who would have pulled their business or refused to sign pending contracts.

Third, when the company is trying to ship a high-profile feature, success or failure often hinges on trivial but obscure changes (e.g. the ability to rapidly add a new field in user settings, or to update the crufty enterprise-data-export functionality nobody has touched in years). Familiarity with the system can be the difference between one of these changes taking a few hours or a whole week.

What do these examples have in common? They’re all time-dependent. You can’t just log on in the morning and decide to unblock a big deal, or mitigate an incident, or speed up a high-profile feature. Is it just a matter of being in the right place at the right time? Not quite. You also have to not already be busy.

Staying loose

I wrote about this a couple of years ago in Crushing JIRA tickets is a party trick, not a path to impact. If you’re always 100% utilized on a steady stream of low-priority work (for instance, if you’re just picking up tickets from the backlog, crushing them, then picking up the next one), you’ll miss your chance to do high-impact work in two ways.

First, you’ll be too busy to even notice the opportunities. You won’t be chatting with people who are working on other things, or reading team updates, or keeping an eye on ongoing incidents. So you’ll miss out on the best way to get involved in high-impact work, which is to volunteer your expertise.

Second, if you perpetually look busy, your manager won’t want to volunteer for you. This is the second-best way to get involved in high-impact work: to have your manager or product manager say “oh, Sean has capacity to help out here, let me tag him in”. Why is this better? Because managers and product managers usually have a much better read on what high-impact work is going on. They’re in meetings that you aren’t in.

Doing nothing

If you’re supposed to keep your time free for high-impact work, and you’re not supposed to just grind tickets, what should you be doing on a minute-by-minute basis? Should you just be doing nothing? Yep!

Doing nothing is good, actually. Software engineering can be a stressful job, but it’s typically not consistently stressful: the stress comes from the occasional incident, or high-pressure urgent piece of work, or (these days) layoff. If you approach the comparatively low-pressure parts of your work with urgent intensity, you’ll already be exhausted and frazzled when you have to handle the high-pressure parts.

Even in high-pressure parts of the job, doing nothing can still be good. One thing I recommend for engineers new to on-call is to avoid rushing: take a few breaths before joining the call or before speaking, and in general try to “think in slow motion”. Most incidents resolve on their own. Most frantic “maybe this will help” changes during incidents make things worse, not better. As a general rule, if you can simply avoid panicking, you will be doing better than most engineers at incident response.

Nothing is a space things can happen in¹. If you give your brain a chance to rest, you will find you’re more likely to have new ideas. If someone hands you an important task, you can tackle it with your full attention (instead of juggling it with the three other things you’re working on in the background). When you’re not busy, you have time to just look at things and take in new data.

Deliberately not doing specific things

A lot of engineers are uncomfortable seeing a task that needs doing and not doing it. I’m like this as well. I wrote about it in I’m addicted to being useful: it’s a psychological quirk that many software engineers share, because having that quirk (to a point) makes you a good fit for the job. In order to spend time doing nothing, sometimes you need to force yourself to not step in.

For instance, I believe that engineers should generally avoid glue work². Most glue work — making sure people talk to each other, updating docs for work you’re not leading, volunteering to address technical debt — reflects the fact that the organization is not explicitly prioritizing this work. If they were, you wouldn’t need to volunteer for it. Either that’s fine, or it’s a big mistake. If it’s fine, then you shouldn’t step up and do it: you’ll be wasting your time and annoying your manager. If it’s a big mistake, you still shouldn’t do it, because you’ll be insulating the company from the consequences of its own mistakes at the cost of your own career and mental well-being.

That’s a bad deal for you, and a bad example for your junior colleagues, and sets a bad precedent for someone else to jump into the same position when you inevitably burn out³. If the consequences truly are severe, let them happen, so the organization can feel the pain and change its policies.

I also believe that being too helpful leaves you vulnerable to predators. Tech companies are full of people who want to extract uncompensated work from software engineers⁴. This is different from work that arrives via normal channels, and for which you’re compensated by promotions, bonuses (and just your normal salary). I’m talking about work that arrives via backchannels, from people who don’t have the ability or willingness to ensure that work is formally recorded under your name. For instance, a product manager from another organization messaging you to say “you’re so good at querying data, would you mind pulling some statistics for me about X?”, or an engineer from another team asking you to “pair” on a piece of work that will ultimately involve you writing all the code and them quietly submitting the change under their own name.

Doing some amount of this kind of work is fine. You may as well help people out when you can. But you need to be able to apply backpressure, either by saying no or simply delaying your response by a few hours or days.

It’s also a good idea to avoid investing too much in work that is likely going to disappear. For instance, suppose you’re working with a product designer who is figuring out what they want in real time. At 9am they message you saying they want the page header to look one way, then at 10am they have tweaks, and more changes at 11am, and so on. You should not throw yourself into fully rewriting the page every hour. Instead, you should do nothing (say, go for a walk) and rewrite the page once in the afternoon, based on the most recent design. Another common instance of this is “big idea from a manager without the political clout to follow through on it”. Often you can just run out the clock until the project gets inevitably cancelled⁵.

Conclusion

A lot of software engineering advice and tooling is designed around the ability to scale up your ability to exert technical effort: to do more things at the same time, to take on projects of larger scope, or to just write more code. But software engineering success is not determined by any of these. It is determined by the ability to do the right things at the right time, which requires that you deliberately hold back some of your effort during ordinary work.

In my experience, it’s still possible to be a “high performing engineer” at 80% effort. In fact, it’s easier, because you’ll be less likely to make silly mistakes from stress, and you’ll be in a position to jump on the kind of high-impact tasks that deliver outsized returns.

This doesn’t mean you should never grind at 100% effort. I think there are probably two or three times a year where I work as hard as I possibly can: long hours, intense focus, thinking about the problem from when I wake up to when I go to bed. But I reserve this mode of work for when the rewards are really high. For the rest of the year, I take it relatively easy.

edit: this post got some comments on Hacker News. Commenters discuss how to not get in trouble with your manager when you’re taking slack time (in my experience, if you’re generally productive it’s fine, but managers vary a lot) and whether engineers really do have control over their workload.

One of my big influences is Rich Hickey’s talk Hammock Driven Development. This is kind of like what he’s talking about, except (a) Hickey is more talking about what it takes to design solutions to really hard problems, rather than what it takes to be a strong engineer in an ordinary tech company, and so (b) Hickey recommends using your time-away-from-the-computer to focus on a hard problem, instead of to simply decompress and let solutions congeal in your head. It’s also like Zvi Mowshowitz’s post on “slack”.
↩
I wrote about this a lot more in Glue work considered harmful.
↩
Why inevitably? Because in my view, burnout is hard work unrewarded, and taking on a personal crusade that your job doesn’t care about is a great way to do a lot of unrewarded work.
↩
I wrote about this in Protecting your time from predators in large tech companies.
↩
Of course, you have to be careful with this. If you try this strategy and you’re wrong about the level of political support for the project, you will come off like a slacker and then have to deliver in a rush.
↩

Working with product managers

Mon, 08 Jun 2026 00:00:00 GMT

The relationship engineers have with product management is more dysfunctional than with any other part of the company. There’s no shared culture or language like there is with other engineers, and the rules of “who gets to tell who what to do” aren’t as clear-cut as they are with managers. Engineers don’t have a lot in common with legal, or design, or sales, but they also don’t need to interact much with those roles. In my experience, engineers are communicating with product managers almost every single day.

Against the “product mommy”

The worst version of the product/engineering relationship goes something like this:

Engineers are technically competent but are too autistic to be fully trusted. They need a kind-but-stern parental figure who knows how to communicate to other stakeholders in the organization (for instance, by being comfortable using the word “stakeholders”), and how to keep engineers from going off in the wrong direction.

This entire gross dynamic is neatly captured by the popular term “product mommy”¹. I really, really don’t like that term, or this entire dynamic in general. Almost none of my relationships with my product managers have been anything like this, though I have seen it at a distance.

Working well with product managers can be the difference between succeeding and failing at a company. Why is it so hard to maintain good relationships between engineering and product? What does a good relationship look like?

Why it’s so hard to build trust

Product managers and engineers have largely non-overlapping skillsets. Product managers don’t understand the technical work engineers do and aren’t equipped to talk about it: if an engineer gives a technical reason for something, product managers generally have to shrug and say “sure, I guess”. Likewise, engineers don’t have anything like the visibility into the organization that product managers do. Particularly in large organizations, it is the product manager who is the source of truth about who wants what and which features are important. When a product manager says that something is critical, engineers generally have to shrug and say “sure, I guess”.

This obviously requires a lot of trust. What’s a little less obvious is that this trust is continually broken by both sides. Every single product manager has been told thousands of times that technical task X is technically impossible or would be disastrous, only for that task to end up being done fairly smoothly and successfully. Every single engineer has been told thousands of times that requirement X is absolutely critical and worth going to enormous effort for, only for that requirement to be silently dropped or changed with no apology.

Of course this isn’t malicious. Engineers often give wrong estimates because estimation is impossible, and sometimes the dire consequences they warn about really do happen (they’re just handled behind the scenes, like engineers handle many other kinds of technical dysfunction). Product managers “change their minds” because what’s important in a large tech company does genuinely change hour-by-hour², and even the best attempts to only filter the most reliable priorities through to the engineering team will sometimes go wrong.

Manipulation and lies

The consequence of this broken trust is that the relationship becomes very difficult to maintain. When you’re an engineer, and you explain something to your product manager, and you know they don’t believe you (despite having no ability themselves to judge the question), it can be incredibly frustrating. Likewise, when you’re a product manager, and you’re desperately trying to explain what we need to do to an engineer, and you know they’re internally shrugging their shoulders, it must be unbearable. Don’t they know this is critical to the company? You were just in a meeting with the leaders of the organization!

The natural tool for a mistrustful product manager is manipulation. I still remember a product manager who tried to extract a commitment from my team by asking us to go around and all say “I commit to getting this work done in two weeks”, after a conversation where we’d explained the risks that cause it to take longer. I suppose the idea was that we’d all work much harder, having taken a sacred oath? More subtle variants of this approach involve suggesting that you would be really disappointed if this work was delayed (in true “product mommy” style), or vaguely suggesting the possibility of some abstract reward (that the product manager is not empowered to deliver) if work gets done ahead of schedule.

The natural tool for a mistrustful engineer is lies. The most benign version of this is exaggerating estimates: for instance, the classic advice to double your estimate and add 20%. I’ve seen engineers claim that they’ve had to follow up on all sorts of largely-fake tasks (one common example is “reaching out to a neighbor team to confirm X”) in order to gain more time. In the worst case, engineers might even straight-out lie that work has been completed, and then track the “it doesn’t work in production” feedback as a bug.

Once this starts happening, it’s nearly impossible to repair the relationship. I can’t bring myself to trust a product manager who’s clearly trying to pull my strings, and I’m sure a product manager can’t trust an engineer who’s lied to their face in the past. That’s why it’s so important to avoid getting into a bad relationship in the first place.

Don’t fight with the product manager

Why bother? If it’s so hard to hammer out a good working relationship with product managers, why not just settle for a bad one? Product managers can absolutely bury you if you’re not careful.

Product managers are almost always more politically sophisticated than engineers. This is partly structural: product managers are simply in more conversations with the company’s movers and shakers, and so naturally have a better relationship with them (and are thus better attuned to which way the wind is blowing). It’s also partly selection bias: engineers can be hired even with relatively poor social skills, because they’re primarily being assessed on technical ability, but social skills are a core part of the product role³.

If you are feuding with a product manager, you will probably lose. Unless you’re unusually influential, they will simply have far more opportunities to quietly talk you down in influential circles than you will. All it takes is a few comments like “oh, I probably wouldn’t pick Sean for that project” to wreck your reputation. In the case where you are openly feuding with a product manager, the company’s leaders will by default take the product manager’s side over yours. They’re likely to know them better, have more shared cultural context with them, and in general be willing to interpret the situation as “another engineer who doesn’t understand how the organization works”.

There are huge benefits to being trusted by a product manager. Product managers want to ship things, and typically understand a fair amount about all of the non-technical barriers to shipping. If you also want to ship things, you can become a fearsome team.

On top of that, because trust between engineers and product managers is so difficult, once you’re in you’re in all the way. Product managers often pick one or two engineers as their go-to for getting the “real story” on technical questions. If that’s you, you have an outsized position of influence in the organization, which you can use to get the things you want done.

How can you build trust with product managers?

As an engineer, how can you build trust with your product manager?

The first step is to understand where they’re coming from. When they tell you something is important or that a requirement has come in, be aware that this is rarely their decision. It’s not them who’s jerking you around, it’s someone higher up in the food chain jerking you both around. If you can adopt a conspiratorial mindset with them, instead of against them, that’s a good start. Try just asking “oh man, alright, what can we do about this?” instead of complaining.

The second step is to be right, a lot. This is a silly-sounding Amazon leadership principle that turns out to be entirely accurate. I wrote more about it here, but (as unfair as it sounds) you really do have to be mostly accurate if you want to build trust with a product manager. When you say something will ship, it has to ship; when you say something is impossible, it can’t happen days or weeks later. It’s okay to be wrong sometimes, but you have to establish a pattern of you providing them useful, correct technical information.

The third step is to let them make the political calls most of the time. If you expect them to trust your technical calls, you have to extend them the same trust when it comes to navigating the organization. Don’t publicly undermine them in meetings, bring up your concerns in private. If they say something is important and you’re not so sure, at least act like it is. Accept that sometimes they’re going to be wrong, just like you’re sometimes wrong about technical questions.

The fourth step is to get lucky. Sometimes your product manager will just be a dud. You can’t build trust with someone incompetent: there’s nothing for you to trust them with, and they aren’t in a position where they can usefully extend trust to you. Working in large organizations requires getting comfortable with the fact that some of your colleagues will be stronger than others, and figuring out ways to work with (or bypass) people who make the work harder, not easier.

“Technical” product managers

Many product managers were once engineers. If your product manager is technical, does that make you immune from these problems? Absolutely not!

You likely won’t have much choice in which product managers you work with, but be aware that having once been an engineer is a negative, not a positive. No product manager can ever be technical enough to matter, because they don’t work on the codebase: even if they were a full-time engineer, they wouldn’t have the time to build the specific context on the system they’d need to be a real participant in technical discussions. It’s thus better to have a product manager who knows they’re not technical than to have one who mistakenly thinks they might be.

The worst-case scenario is an ex-engineering product manager who believes they’re technical enough to detect when engineers are lying to them. This kind of paranoia is an easy trap for “technical” product managers to fall into, particularly when they don’t have a trusted engineer on the team they can lean on. If you’re dealing with one of these, prepare to spend a lot of time explaining why you can’t “just” do things (and prepare to have those explanations not be believed).

Conclusion

At its worst, a product manager relationship is like an unhealthy family: driven by condescension, emotional manipulation, lies, and mistrust. This isn’t because product managers are bad people! It’s because the structure of the relationship creates conflict. Both sides must make commitments (about the technical system or goals of the organization) that are (a) often wrong, and that (b) the other side is unable to independently verify. To avoid the trap, both sides have to be generous, willing to trust each other in their areas of expertise, and most importantly competent.

Unlike most roles in tech, product management (particularly the lower-level roles that are more engineer-facing) has close to an even gender split.
↩
For instance, based on the whims (or snap decisions, more charitably) of the CEO.
↩
I have worked with product managers with poor social skills, but it’s rare: about as rare as working with engineers with genuinely poor (i.e. by general-population standards) technical skills.
↩