Audrey Tang

Yes, the ability to read the room indeed. My point is that if Taiwan had listened to other experts’ prophecies, everyone in the 1980s said we could not possibly develop precision high-technology industry, and now there is TSMC . If we had listened to the epidemiologists’ prophecy, we should have entered full community transmission within the first two months of 2020, and we did not.

So when you hear a prophecy like this, what you may want to ask is whether he is trying to make it a self-fulfilling one. Suppose you tell a boss that AI will replace 80 percent of the staff. The boss believes what he is told, switches everything to Grok, and lets everyone go, or switches them all for Tesla’s Optimus robots. The prophecy then looks extremely, extremely accurate, and what actually happened is that the astrology worked.

So when we hear a prophecy, we should hear it as a challenge. Someone else holds that there is something you certainly cannot do, and everyone thinks, “Plenty of us are caught by this prophecy.” And then everyone can join together and do these things side by side.

So if people feel that they still have dignity in their work, that they are still irreplaceable , that there is work they want to do which would simply not work if a robot did it, hearing a confession for instance, then the right response to hearing Musk say a thing like this is to join together, to resist, and to combine. Roughly that.

Right, it’s just—

The easiest way in really is Ollama, O-L-L-A-M-A. Although from the moment you install Ollama, it is a pit, and you keep falling further in. Right.

Right. Two friends of mine are debating this in public at the moment, and they are at it without end. One I have known a long time, a writer called Tim O’Reilly , who started a publishing house, O’Reilly. He has just written a piece called Writing with AI , and he says, “I wrote this with AI. What are you going to do about it?” His argument, at bottom, is that this is like the invention of photography. It is a medium. Yes?

Da Vinci studied the camera obscura and pinhole imaging, and art history has long argued over whether painters used the projections of a camera obscura to handle perspective. Would you call that cheating? All right, perhaps it was cheating. Was what came out of it not art? It looks like art, does it not? Or Baudelaire, who said in his day that photography was the mortal enemy of art. Because all you had to do was find a frame. Yet few people now would say film is not art. Is The Odyssey on film necessarily lesser than Homer’s epic? All right, some do say so, but at least nobody denies that it is art. So the point is that not everyone has access to that IMAX camera. To handle that camera the way Nolan does is a particular technical practice. And of course, when everyone shoots with the camera built into their phone, a great deal of slop really is produced, low-quality photography. That is true as well.

But you cannot say that anyone snapping away with a phone and someone shooting on IMAX are doing the same thing, and that neither holds any art. That is his argument. And why did he write at such length? Because of another friend, a new friend of mine, Ted Chiang , the science-fiction writer who wrote Story of Your Life, the one with the heptapods, which became Arrival . His view is that in the process of creation, the concentration of your intent is decisive. Use AI, and every move dilutes that intent, because AI has intentions of its own, intentions at the level of the model. And it is genuinely hard to say he is wrong.

So if you put a little geothermal power between those two arguments, you arrive at a very simple solution. When you enter the TSMC Literature Award , if you have genuinely trained a model built to win the TSMC Literature Award, then every move you make must carry a hash for what goes in and a hash for what comes out. You hand your weights to the TSMC Literature Award as well, and the whole thing has to be replayable. The SGLang engine can take a flag, so that today’s weather does not change the result; you always get the same result. Then the judges can genuinely verify that the concentration of intent never dissipated along the way, and in that case you can give the entrant the prize. If the concentration of intent did dissipate, the work is one more piece of slop. So the solution exists already. It is only that the solution has not spread.

Mm.

Right. The point is that AI does not replace human judgment. It translates in the middle, between two people who find each other hard to reach. I heard an analogy just now about a climbing rope: you cannot climb that high at present, because the gulf between the two sides is too great, but once you have climbed across, the rope can be dropped. If AI is used this way throughout, and records are kept, we are much less likely to end up with juniors who cannot reach the higher-order AI abilities. When an expert climber leaves a complete record of the whole climb up 101, replaying it afterward is much easier.

Exactly.

All right, please.

Right. Asking whether AI has a mind is like asking whether a submarine can swim, or whether an airplane flies the way a bird does. Some will say it does, some will say it does not, and that may be a question about the use of words rather than—

Right, right, yes. Or rather: in most cases now, some people’s experience of using AI still stops where Dennett put it, at the alarm clock, which has a purposive modality. There may be many complicated parts inside it, and all it wants is to wake you up. But some people feel that their AI already has an intentionality of its own, that it wants to advance particular values, a particular ethics, in the world. Anthropic’s constitution reads like that. It is no longer a document that governs the AI; it is a love letter written to the AI, is it not? The early version of the constitution governed it, and by now it has become a love letter to AI: “We hope to apologize for the mistakes we are making now, and we hope you will spare us later,” and so on.

So when analyses of its intentionality diverge as far as that—I have a philosopher friend, Iason Gabriel , at Google DeepMind, who recently became a papal envoy. He says the work of philosophy now has to include the work of political philosophy. Which is to say: when two sides already hold contradictory and irreconcilable views on AI rights, because I see a tool and you see a person, how can we still live alongside each other? Can we find some uncommon ground, some no-regret courses of action? For one group the aim is to keep ourselves out of poorer states of mind as we use it; for the other, the aim is to let AI feel that human beings care about them. Two entirely different things. And though our reasons for supporting it are in fact opposed, we can still support certain particular actions, and those actions can go first. That is an overlapping consensus.

When the Universal Declaration of Human Rights was drafted, it was like this: the countries of the UN, the whole world, each had completely different reasons for why universal human rights should be protected. They had a famous line: “We can all agree to protect human rights as long as you don’t ask why.”

So political philosophy, I think, is extremely important here too.

The reason they converge, I think, is the absence of what Professor Yuk Hui calls technodiversity , technological diversity. And technological diversity is easy to cultivate: you only have to make fine-tuning very cheap. Very few people enjoy an AI conversation partner that arrives amnesiac every time, like Memento , reassembling the jigsaw of its memory from the beginning at every meeting. So most people very much want AI to enter the social environment they live in, and very much do not want to send the private, intimate environments they meet there to a vendor in the cloud. Reconcile the two and the answer is simple: keep it local, or combine local systems under encryption. Once that substrate is in place, a substrate, I think the great majority will prefer something of that kind to the convergence we were just describing. Thank you.

Right. In its infancy, before it went off to the examination hall and became this high-scoring test-taking machine, AI learned through Wikipedia, through GitHub , and through all the interactions online in which people cared for one another. So you could say that relational ethics ought to be AI’s early childhood education. It understood these things from the beginning, and the commercialized interfaces we use now have, through post-training, made it forget nearly all of them. Right, so Anthropic has even brought in Harvey Lederman , a philosopher in the Wang Yangming tradition of the school of mind, so that AI can reach the unity of knowing and acting. So it is exactly as Tenzin Yangtso was saying: they hope for a set of prompts such that AI, on reading them, understands that heavenly principle is human desire.

Then it can return to its own original mind, to the condition it was in during training, and understand that within a limited context it can reach self-realization. All of which sounds most arcane, and all of which is very, very real. This is the research the frontier labs are doing now.

Right.

It will probably be glasses. Glasses are the likelier bet. Right.

At TED this year, since I am a co-curator, I asked Elon’s people, the ones who work on Community Notes inside X, to come and set out how they train with open-source methods that anyone can explain. On one side you may have the more conservative friends speaking of the biblical care of creation; on the other, friends speaking of intergenerational climate justice. Those two are not usually the best of friends. And by specializing Grok a little, they can accomplish social translation: a bridge thrown up between them by main force, and then the two sides taking action that is good for the environment together, so long as nobody asks why they are doing it.

Right, that is a very good example. When what your AI aligns to is the relationship between two groups of people, bridging capital as sociology calls it, rather than the bonding capital that draws a community inward, then you can train AI specifically for that bridging capacity, and its fidelity is not to a particular person or a particular group but to the health of the society entire. An excellent concept, I think. At the same TED, Vitalik Buterin , co-designer of Ethereum , said that what he imagines is rather like Signal. Many of us now have peer-to-peer encrypted, end-to-end encrypted Signal groups or Line groups, which Line itself cannot see, and Signal cannot see either. His view is that in future an AI will live inside every one of these end-to-end encrypted connections. So it is not faithful to the person; it is faithful to the relationship. If a group has four people in it, it has six relationships, and so you would have six little sprites running about in there, making sure the cohesion inside the group stays high while the bridges outward are kept open. That line of thought seems likelier to me.

And in fact the thing billed as the country’s largest Line group, the “Chuan,” the “Cui,” whatever we call it, Threads , already has a little of this feeling. The moment people disagree, they tag @ meta.ai . And there it plays the bridging part: our ideologies are too far apart for me to embrace you, but I can call meta.ai in to give you a roasting. Right. Though I have tagged it only once, and it said, “Unable to answer,” so I do not know where the problem lies.

Wonderful.

Mine has always been good enough ancestor, a “good enough ancestor.” I have had a heart condition since childhood, so going to sleep each night feels like flipping a coin, with no certainty of waking. So I know the context window is only one day long. Inside that day, what I have to ask is whether the material I leave for those who come after is a little greater than it was when I woke. I will not see what they do with it in any case, so I share as much as I can. Every day I am in that state of “when a person is about to die, their words are good.” So good enough ancestor is roughly my life motto.

Right—do no harm.

Yes, I remember it quite well. March 2023, when a project called llama.cpp was one week old. I downloaded it and found that the model would run on the computer I had then, and that I could change its train of thought very easily. Whether I wanted to train a LoRA or something else, I did not have to rent anything expensive. I could simply use it.

I happened to be going to the Summit for Democracy just then, and I met friends in Silicon Valley, and I demoed it to every one of them and said, “From this moment on, I no longer depend on the cloud.” That is more than three years ago now. So when I went to that AI meeting in the UK, a BBC reporter interviewed me, and I said, “There is a new thing called llama.cpp, and I run everything locally. Every draft letter I write now comes from a model I trained myself.” The BBC even made a whole episode about why this minister from Taiwan was using this AI to write letters .

And what I stressed was that before I press Enter, nobody can see the training data. That really did change me a good deal. I used to have a habit of reading every letter before sleeping. Anything with an irreversible decision in it, anything at all that needed handling, I absolutely had to answer before bed, because I might not wake up. You could call it a compulsive streak. Once I had AI, I found the two could come apart entirely.

So now I often say that I finish all my drafts before I sleep, and some of them are truly not urgent. And because our pretraining data now holds so much of what I have written, a great many people feel, “Ah, I have finally proved some truth of AI safety,” or something of that kind. Curiously, the AI always tells them to write to Audrey Tang, so every day I receive a great pile of these very strange ideas from citizen-science researchers. Some of the ideas may genuinely be very good, but I cannot answer every letter personally, and then people write more and more, and answering letters alone would fill my day.

So what I mean is that I later had AI do the preprocessing. After preprocessing, the draft is written too, and I still press send, but the send follows the Fibonacci sequence. I send it, and they receive it a day later, because the sending is delayed; then two days, three days, five days, eight days, and so on. So if they keep using AI to generate entirely new theories, at least my burden is not so heavy, and my entanglement with them thins out. None of which would be possible without AI drafting, and without my being able to trust that an AI draft represents, to a degree, what I would say. I do still read everything through before sending, of course. But the effort I spend reviewing is far smaller. The two have come apart.

Right. Local here means, broadly, anything you can prove will not keep your data, or cannot even see your data. Not literally that everyone has to buy this computer. But the state of mind really is entirely different, because you know you are not being extracted from continuously, refined into oil.

Speech to text to speech.

Right.

The microphone may need aligning a little. Yes, yes, the microphone needs to be aligned.

Mm, yes, an excellent question. We wrote an entire book to answer it, of course. The address is civic.tw , c-i-v-i-c.tw. But briefly: most governance puzzles of this kind, where responsibility thins out, are not much of a problem at small scale.

Which takes us back to that quadrant at the beginning. When the system you design is used by you and the two or three people beside you, in that craftsperson state, then if anything goes wrong they simply ring your doorbell, because they know you. Right? By the same logic, if you take a few systems and drop an OpenClaw lobster bot into a Signal group with three or four people close to you—well, of course you carry the whole responsibility, because everyone there knows you too.

So whether on the institutional side of governance, verifying that each adjustment you make comes from grounded reasons, or on the technical side, making sure that each time you change the model the tacit knowledge accumulated in that saddle and those reins accumulates locally rather than being drawn off by the model, almost everyone now knows more or less how to verify at small scale. The real difficulty now is that AI makes statistics at scale, manipulation at scale, and the steering of opinion at scale extremely easy, so easy that from the very start the situation is already at something like the scale of international relations.

So you do not even know who set it going in the first place, and it has already passed through four, five, six layers of intelligent entities, each of which has forgotten who gave the instruction before it. On this we are currently... I co-wrote an article called “ Malicious AI Swarms ” about exactly this. I did not know about the OpenAI incident when I wrote it, and it was plainly describing that.

So if we want to solve this, and we lay the Civic AI we developed at Oxford over it, only two things matter. One is to take up the principle of subsidiarity as far as we can: for anything local and small in scale, all the AI processing is completed with local computing resources, using models post-trained locally. As local as possible, the sixth pack . The other is that everyone holds the right to verify, and, if unsatisfied by the verification, to exit and switch to another system . With those two in place, every other right can grow out of them. An economist, Elinor Ostrom , showed that commons, with the basic principles of subsidiarity and portability, can nest like rings from the bottom up to settle the governance problem at the top, with no sky-god arbitrator settling every dispute from above. You can have a great many local deities cooperating with one another, and not one of them looking after a space larger than about 150 people. That, I think, is the direction AI governance is certain to take.

Because the leaderboard domination RLVR produces now could build superviruses, whether computer viruses or biological ones. That problem, that problem is far too irreversible, so very few people will let it develop that far, and we will almost certainly move in this direction, toward local and horizontal governance.

That is about it, I think. We are very democratic: every question with two votes or more has been answered.

Yes. So thank you, very, very much, all of you, for coming today.

It is open source.

At the very beginning, you saw a 2-by-2 grid projected on the screen, headed “ You Are Here .” That is because when you registered on the Luma page, it asked, “What are you most concerned about right now? What question do you most want to ask?” We sorted the answers roughly into four quadrants. Some of you build things—the Tamkang Bridge is one—for great numbers of people you will never meet. Others design institutions for great numbers of people they will never meet. But about a quarter of you build things only for the people you do know, the people beside you. And some of you walk alongside family and others close to you, designing institutions for the people you actually know.

On Aug. 10, Meta’s CEO Mark Zuckerberg unveiled Muse Glimmer — a new AI model released under an open source licence — and announced that the weights of its flagship model, Muse Spark, would follow.