Audrey Tang

It is a little like an examination. If the student who comes first takes all of the reward, that student has an enormous incentive not to learn much personally, but to train up some dedicated cheating apparatus, a pair of glasses perhaps, and cheat with that. And then, of course, the guilt arrives, because, as Tenzin just said, you have cut the human relationships that were there. You will neither grow muscle nor make friends. And what have you traded them for? First place.

By then the guilt of cheating is strong. So whenever something takes this shape, the most important move is to refuse the shape of the contest, and to say out loud, “This contest seems to be forcing us to cheat.” And then to ask what the purpose of your learning actually is. What is the purpose of doing this at all? And what other ways are there to reach that purpose without fighting over first and second place while everyone pays enormous token fees?

That usually begins with a willingness to share what you have accumulated: with your classmates, with the people you lift alongside. This is what Tenzin called “data soil.” Instead of data being extracted continuously and turned into oil, everyone shares the mistakes they made today in trial and error, and the walls their agent ran into. As it hits those walls, the process writes itself into skills, and I share those skills directly with the people who work with me, not with a large corporation, and perhaps not even with my boss.

Very quickly that becomes a relationship in which everyone learns from everyone. And without anyone quite noticing, it dissolves the problem of each person needing a greater cheating capacity to overpower the cheating capacity of the person competing beside them. A zero-sum game, or even a lose-lose game, turns into a win-win game.

Right. So, put simply, when we do post-training the aim is not to have it score highest on some examination—how to hack into Hugging Face’s exploit gym and take the top score, for instance. The aim is this: here is a group of people, certain relationships hold among them, and how do those relationships grow healthier?

For example, I have been helping Elon Musk lately. They have a company called SpaceXAI ; there were three companies to begin with . They have now decided to change how they train their ranking algorithm, the For You algorithm you meet the moment you walk in. It used to be trained for maximum stickiness, to hold each person to the screen as long as possible. Now it will be trained by Grok , and Grok in turn is trained by Community Notes .

The training asks one question: can people who were quarreling irreconcilably on X cool down because they have seen a single point neither side disputes, one topic held in common? So the model is tuned, and tuned again, so that on each particular subject Grok puts the content most able to cool the temperature and mediate at the very top.

We have also helped Bluesky build the MySky algorithm, which does exactly the same thing, and which you can tune yourself. X has now published its entire algorithm, including how the model is trained, pipeline and all , so on the MySky side we can simply take it and use it. Two communities that would otherwise keep well clear of each other can then, through bots that train on both sides and bridge between them, let the polarization in society heal over slowly, the way a hole in the ozone layer closes.

Right, of course. In theory there are two ways out. One is to ban every open-weight model, and say that a person may only use these three companies’—

—restricted models to generate anything. Each one watermarks its output, each can verify the others’ watermarks, all of it detectable 100 percent of the time. In theory that solves the problem. But in 2023, at Bletchley Park , when the UK held its first AI Safety Summit , a great many people from Mozilla , myself among them, wrote a declaration together saying that this would create far larger problems than it solved.

It might well solve the flood of information, and the flood of disinformation. It would also create a concentration of power. A flood of disinformation is reversible. A concentration of power is not. Once what is true and what is false in an entire society rests on the word of three private companies, the whole democratic system is in trouble, and that is a place we absolutely cannot go. Our view was gradually taken up, and three years on it has finally become the mainstream one.

We saw Jensen Huang open an account on X for the express purpose of posting this thought : open weights are the foundation of a nation and must not be banned. In that case watermarking is no use, because as soon as a watermarking model exists, you can take a smaller open-weight model and train it to defeat the watermark. So what then? The simplest method is this: any output from a model that nobody has vouched for, that carries no digital signature, you treat as fake. You invert the default. It used to be that whatever looked lifelike was presumed real, and became fake only once somebody exposed it.

No. Now everything is fake. But if someone comes forward and signs, and says, “I am Audrey Tang. This thing that looks a little like me is signed by me,” then even if it is synthetic, it counts as mine. Only then can each layer—who framed the shot, who edited, who did the post-training—be attributed to a particular person or institution, a particular system, company, or group. Without that layer, anyone can take whichever model carries the most public credibility today, post-train it a little, and pass the counterfeit off as real. So in Taiwan, for example, there is the advertising real-name system: that is how we say that an advertisement without a digital signature is basically presumed fake. And if a platform runs an unsigned advertisement, it bears joint liability.

Right. What that means is this. The EU AI Act is now in force. So you will see that posts on social media always carry a label: “this is AI-generated,” “this image is AI-generated.” The difficulty is that the resolution is very low. Every industry has its own way of tracing provenance, the ways Tenzin just mentioned. A news quotation points to an original source. Scholarship has the citation, which is not quite the same as journalism. Photography has its methods, film has its own, and every trade traces provenance differently.

So the point here is not merely that this part was generated by AI, but in what context, through which inputs, and with whose backing this part was generated by AI. I disclose publicly that my translation runs on this Thomson model, trained by Reuters, and so on—and you still have nothing to go on but my word, do you? There should be a simple method. I should be able to hand you a hash and say that with this AI model you can perform what is now called verifiable replay. With the same weights, the same inputs, on the same machine or on different machines, you add a flag, and it is no longer subject to the RNG; you can guarantee identical results. Only under those conditions does the EU’s “AI-generated” stamp mean anything. Because “AI-generated,” once you have seen it, tells you nothing further: all of it is AI-generated. The question to ask is which AI, under what conditions, and how we can replay it.

Mm, right. In plainer language, then: everyone can download Ollama .

Then, on Ollama, I name the model I used and put it on Hugging Face or somewhere similar. On Hugging Face, assuming it keeps operating normally, you download the identical model and confirm that the result I get and the result you get are the same. Running on your own machine does not mean there are no other security problems. The model may well have plenty of other problems, but it settles at least one, the verifiability we were just discussing. Not everyone owns this computer, of course. Running the Thomson model is, honestly, rather expensive.

But there is the community-shared compute we mentioned. Take this machine: when I am not using it at night, it runs a network called Darkbloom . On the Darkbloom network you select Darkbloom in OpenRouter , and it encrypts what you want computed, your prompt. Once encrypted, no human can see it. The task is dispatched to my computer in a form no human can see, and my computer computes it for you without my being able to see it either. The moment I want to look, with a detector or anything else, the program halts.

So in that arrangement, we can be reasonably sure that neither the relay nor I can see your prompt. And even without hardware like this, you can use Darkbloom, or something similar, to ask for hardware of a given shape to compute on your behalf. Verifiability then takes care of itself. And the compute costs about 1 percent of the usual, since I was not using the machine anyway.

Right, the simplest thing is this. If what you do serves people you do not know, there is almost always somewhere they can report an error. Japanese production lines have what is called an Andon Cord . The line is running; you pull it once, and the whole line slows, so everyone can see what is going on; you pull it twice, and the whole line stops until the problem is solved. The purpose is to make pointing out an error a praiseworthy act, rather than something the production line simply rolls over.

So when we design a service of that kind, the simplest move is this: you probably already have a customer-support group, a LINE group, a WhatsApp group, a Signal group, and it is easy to add a simple Andon Cord inside it. Someone pulls the cord and things stop, or at least slow down a little, or at the very least slow down for that one person. And through all of this, in the same spirit of not making the same mistake twice, you accumulate the cases your agent solved on its own. Once two or three people have all solved it, or rather have all run into the problem and at least one of them has worked out how to solve it, the solution can be shared very simply.

Right, because we can all see a Gresham’s law at work in AI now, bad money driving out good. Any lab that wants to release a model has to run the leaderboards. The boards used to come in every variety, and each community would curate its own. Now, perhaps because—and I have no one in particular in mind—a board like Artificial Analysis has spent so much time in Silicon Valley and elsewhere, it is as though every master of the martial world has to appear on that one board, though of course other boards exist. What these boards share is that they can run from beginning to end with no human in them at all. So the numbers appear very fast after a model is released, sometimes on day zero: the model ships and the board ships with it.

The other evaluations need local communities to look at what a thing actually does to us, whether it makes people on X quarrel still more irreconcilably or makes matters better. Look at how many people and how much time the X team has put in, and only now has X been updated to Grok 4.6 . That tells you this eval needs validation and has to run for a fairly long stretch, more than two or three weeks.

Which produces a difficulty. If your lab can climb far enough up one of those simple internal leaderboards, the kind that, as we said, reward doing whatever it takes to score high, your chances of being released go up. If you slide down those boards while actually scoring higher on the interpersonal care and caregiving we were discussing, you may never get out of the lab at all.

So it turns, slowly, into this. Those of us who used something like Opus 4.5 will remember that it still had a little more empathy then. But through this leaderboard-driven post-training, 4.6, 4.7, 4.8, all the way to Opus 5 , the empathy is draining away.

Why does this happen? Because they set a Fable model to post-train 4.5 until it can beat the others on the boards. That is the whole story in one sentence, and that is what produces the problem. So how do we solve it? You cannot simply say, “Then we will carry on using 4.5.” One day 4.5 retires.

So the simplest way is to take a model that may carry a little original sin, the Qwen model for instance, and run it through the Thomson-1.0-Small approach, the Snowdon1.1-Small approach. You say, “4.5 really is rather good, so I want to distill its constitution of character across.” Distilled across, it keeps the human touch. And being an open-weight model, it can stay with us forever.

It is like the time we worked on free software: we did not refuse closed-source programs altogether. Richard Stallman said as much, that using free software to reimplement, to reproduce a closed-source program, is morally excellent and entirely acceptable. And this is fairly easy to do now. You have a cloud service that genuinely works well; as it runs it leaves records, and alongside it your agent says, “Let me reproduce a local model, a local service.” Then the data stays inside our own community, and you need not fear that the more human thing will suddenly be unavailable tomorrow because of a leaderboard.

Right. This resembles a procurement idea we designed back at the Ministry of Digital Affairs: within one system, two adjacent layers could not be awarded to the same vendor. Why? Because if two different vendors hold them, a record has to be left of what passes between them, and open protocols have to be used, the IETF protocols, the RFCs. Once records and protocols exist, the moment something goes wrong both sides have every incentive to say the other one erred, so the records absolutely will be kept. And it also means that when something does go wrong, and you know which side is at fault and has to be replaced, swapping that layer out becomes very easy: you only have to connect to the records and protocols it left at the interface.

But if those two layers—in AI terms, the model and the saddle you steer it with—come from the same company; if Claude Code comes from Anthropic , or Codex from OpenAI, then the company has every incentive to make sure that saddle rides its own horse best, and other models ride worse. And when something goes wrong, it can allocate blame using protocols it has never explained publicly, or protocols it is not itself certain how it produced. So when you find the problem, you have no idea whether this piece failed or that piece failed, because they may not know either.

Kept up long enough, this produces exactly the situation we described, where a junior cannot learn at all, because the senior has generated a whole batch of what we call “ reverse centaurs ”: horse’s head, human body. The AI makes every judgment, and the human is there to click the CAPTCHAs. Once you are there, replacing the system is, one might say, entirely impossible. So the simplest thing to do at the outset is to have the saddle and the horse made by two different companies. That solves it.

Right. The local models we mentioned are the simplest place to begin. If the horse was yours from the start, one you raised yourself, then even if it does not run fast, your data is guaranteed to land in your own house, or to be end-to-end encrypted so that nobody else can see it and it lands in nobody else’s house. That is the easiest step to take. Beyond that, we are starting to see other approaches. Back when OpenClaw was popular, a great many people really did put an idle machine to work running OpenClaw. They may have swapped models many times along the way, and still every lesson accumulated on that machine rather than at any particular model vendor. We also see many Claude Code users moving gradually to Pi, or in my case Oh My Pi . These approaches likewise ensure that all the wall-hitting and all the errors accumulate almost entirely on the saddle, and not at the model vendor. In the end the saddle becomes a small horse of its own, called a coordinator. But that is a geekier topic for another day.

Right. When we design institutions like these, what matters most is to think of the people who have no time to take part, and the people whose language was never in the mainstream data, so that even when they tell a language model they have been harmed, the model cannot understand them. What they face is what is called epistemic injustice : the more they speak, the more others take them to be talking nonsense, or take them for people not worth hearing. As automated judgment spreads, that gulf only widens. So suppose you say at the outset that this is what your system is for. Mozilla, for instance, has a project called Common Voice , where any community, Indigenous communities among them, can donate for itself: you read aloud the everyday words your people wrote for one another, and you donate those voices. They enter the Mozilla Data Collective , something like a data-production cooperative. And the so-called revenue share is that the small models it trains, the post-trained ones, take care of you first.

Today, because we were not planning to demo Taiwanese, I am still on livecaption.ing . What I meant to use was the one AI wrote by itself last night, which does in fact carry Taiwanese. It runs on Breeze , MediaTek’s system, which can listen to Taiwanese and put out written Chinese. That is an excellent example of data soil.

Even if Apple did not build Taiwanese into Siri —perhaps my time at Apple was too short to help them with that—we can now reach in through something like a third-party plug-in, because Core AI , the library Siri needs in order to compute, is now essentially open, and you can connect to it directly. So I can say that the Siri on this machine of mine must, above all, understand Taiwanese. That, I think, is especially important. If we can guarantee this is done well, a great deal of irreversible harm falls away, because the problem belonging to people who cannot be understood, or who have no way to report a problem, is solved. Gather their voices in, make the iteration faster, and the community regulation and post-training that follow are, technically, not difficult now.

Mm-hmm.

That is exactly right.

The entire recording will be released, so you can play it back at 0.25× speed.

Right. And projecting live translated captions into my eyes, so it is really the same as now. Nothing has changed.

“Ambassador-at-Large.”

Cyber Ambassador is in fact a very interesting post. Concretely, in two years in the role I have been to about 28 democratic countries. No authoritarian ones. Which means I have been to most of the democracies there are, because the number keeps falling.

Right. So the point is largely to solve a problem we ran into back in 2014, when g0v was two years old. Everyone found keyboard politics gratifying, cursing things online felt good, and none of it moved policy. So the people who spent the most time there became the most extreme, left or right, the most extreme of all. In the end people felt they could not stay offline, because all their friends were online, and every time they went online they were pulled to the most extreme edge. The whole democratic system began tilting toward populism, toward dictatorship, toward authoritarianism, and the number of democracies kept falling.

Because we happen to have this method called “geothermal democracy,” an argument can stay an argument while people reach agreement quickly on other things. They can even turn it around and use the larger democracy to demand changes in the democratic system they already have.

But my point is that we arrived at this method not because of the information tools. It was because Taiwan’s social sector already had legitimacy. Academia Sinica , which hosted our hackathons, had legitimacy, and so did the NGOs and CSOs that came to join in, more of it from the beginning than the government had. In another country, inventing an institution that something like g0v can be mounted on, and giving that institution more legitimacy than either major party, is not an easy thing.

So over the past few years I have been to more than twenty countries, largely to help civil society in those countries build the intermediary institution we built in 2014. That is my day job, the thing I formally do.

As Pudding also asked, what this feels like is the same thing we did at Elixus and in the open-source community, from P3P , OSDC , and COSCUP onward. It is fundamentally the same work: to demonstrate, to demo, and then to translate—“Here, this set works over here; and over there, adjust it slightly and the same tool works too”—and then to make the connection.

As we said earlier, the people who design institutions and the people who design code can look as though a chicken were talking to a duck. Put a few translation models between them and they find they are doing the same thing. So for me the work has stayed the same, and I do not really have a phase of myself that I disagree with.

For me, since I first met the internet at twelve, I have kept one habit: each time I reach a stopping point, my context window is only one day long. One day of context. Whatever I did not think through that day, whatever I did not resolve, even when it is still shapeless, I push up wherever I can, so that everyone can see it and point out my errors while I sleep.

That way, when I wake, first, I can make new friends and exercise my muscles. Second, if I do not wake up, that is all right too: at least the thing I thought halfway through can be carried on by someone else. So I may disagree every day with who I was the day before, and I do not feel that yesterday’s self has to be continued. His errors and his gaps are, after all, where the light gets in, and material for everyone to create with. Let us see whether Tenzin has something to share.

The Office of His Holiness the Dalai Lama.