Opening keynote and fireside chat at Confronting Our AI Future: Hope, Fear, and the Choices Ahead, a Stanford HAI conference held on 28 October 2026 in the David and Joan Traitel Building at Stanford. Each slide appears at the point in the keynote where it was shown.
I am really excited that we’re going to have Audrey Tang. We’ll start with Audrey, and then we’ll have Rob Reich, a senior fellow and associate professor — excuse me, associate director — here at HAI, to get us started. Please help me welcome Audrey Tang.
(Slide 1: Humanity Sets the Pace of AI)
Hi. Hi, HAI. Do I say Hi HAI or Hi H-A-I?
It’s so good to be back in the campus. I remember being here when we first published Plurality, which is the book. And it’s available in public domain on plurality.net, that I co-wrote with about 60 people. And I’m working on the next book, to be published by Wiley next February, talking about how to set a pace of AI.
(Slide 2: Who sets the pace of AI?)
And I think this really is the question that this conference asks, because we hear the mixture of the fear, but also of the hope. And when we had a conversation in Taiwan, which makes a lot of those chips that enable this fear and hope, we asked the questions, and people shared both.
(Slide 3: Hope and apprehension coexist)
They have hope, of course, but also fear. And indeed, that is the HAI Index showing that across the world, I believe 30 or so countries, that a majority of people see hope, and the majority of people see fear. And if you add those together, it’s more than 100%, which means that there’s a real mixture in those numbers.
And I think this is important to us, the policymakers, because it shows that people really want to not just be an audience, but a co-author of the AI story. And people really want some sort of reflection, like the group selfie that we just did. The group selfie mechanism that can let people steer the AI takeoff together.
(Slide 4: ___ paces the frontier. By whom? Toward what?)
Now, of course, we have seen the labs now kind of changing their tunes, saying that they must now pace the frontier since September. Many are now openly saying, actually thousands of people, employees in those labs, join in a letter saying that we must pace the frontier. But actually, who is this “we”? Who is actually pacing the frontier? Those letters do not actually specify.
And also, pacing the frontier also implies that it’s pacing towards some place. So where is that place of safe landing? So that is the mechanism question that the group selfies of this room and outside of this room must answer together.
And I really like to interpret the fear that we just saw in the HAI Index as information, because it shows us where care has yet to arrive, a gap in the care, and it’s a good signal. And a good pilot fly by the instruments, the instrument that deliver such signals.
(Slide 5: Three working parts)
Now, it needs three components. It needs evidence that anyone can check. It needs a witness that is collusion resistant, capture resistant, so that it’s independent, truly independent from every lab and every state. And then a duty to answer by a deadline. And so I want to illustrate with some exhibit A’s and B’s from Taiwan.
(Slide 6: Taiwan, March 2024)
The first is actually a collaboration between Taiwan and Stanford. In March 2024, I served as Taiwan’s Minister of Digital Affairs, and we sent 200,000 text messages to random numbers around Taiwan. And people who received those invitations shared what they think about information integrity.
The information ecosystem at the time was badly damaged by deepfake scams. Some, for the first time, saw those very convincing deepfakes and lost millions. So it’s a real problem.
And then people did volunteer. More than 1,700 volunteered to the invitation to give a very long afternoon to join two sessions, and helped by Professor Fishkin and Alice Siu here to sit in a small group of about 10 people, and the 10 people would take a group selfie with the help of the Stanford Deliberative Democracy Platform.
And so, of those 44 groups, many of those groups came up with very, very good ideas — neither YIMBY nor NIMBY, but like MIMBY: maybe in my backyard, if you do this and do that. Because only the ideas that are MIMBY can survive this small group setting. Otherwise, if you have a radical idea that you convince yourself, you cannot get through to the nine other people.
So the setting of this group selfie really determines the Overton window, the public scrutiny of what kind of public policy is proportionate. So people, of course, don’t want fraud, but they also don’t want state censorship. They also don’t want big tech from foreign states taking over the information ecosystem. So they settled on some very good ideas like joint liability, mandatory KYC, so on and so forth.
And so we voted by the end of the long afternoon conversations, and we settled on a package of ideas that more than 85% of people agreed on and the other 15 can live with. And so that basically was part of the drafting process of the anti-fraud law.
(Slide 7: Deliberation kept pace with a moving bill)
And it’s actually very quick. The invitation was in March, the deliberation soon after, and the moving bill already being prepared by our ministry incorporate those ideas, publish the draft for everyone to see, and the legislature just passed that in a couple months.
Which is actually very difficult in Taiwan because, at the time, we have three parties in the parliament, none of which have a majority, and they were kind of training their muscles on blocking each other’s proposals. But because this particular one showed the common knowledge that everyone already knows, that 85% of their neighbors probably already agreed with these measures, the mini public closed the loop to the maxi public, so that the Taiwanese people know that this is what we want, and so the law passed very quickly.
And so this, I think, is a good exhibit of how to pace the frontier at the time. But of course, the frontier has moved. And so I’m not pretending that the fake scam compares even remotely to the risk that we’re facing today. But it is, I think, a very good dry run. And it shows three things.
(Slide 8: Evidence people outside the lab can trust)
First, that it needs really to produce the evidence that everyone can check. Today, outside evaluators can test with access from a lab sometime, to the degree of a lab employee, we’re told. But then it’s still the access granted by the lab. So trusting the evaluators also means probably that you somewhat already trust the lab. So for people who don’t trust the lab, this actually still needs external validation.
We can learn from arms control. It relies on seismographs and also radionuclide monitoring, which means that a test anywhere on Earth can register on stations that cannot be captured by the state doing the explosion. Intelligence explosion? Anyway, some kind of controlled explosion.
But part of also why flying got safe was a very similar arrangement. Which is commercial aviation, right? With this black box that is a flight recorder, tamper-proof, or at least tamper-evident.
And so, in August, I published a proposal with three very concrete steps. It’s basically about making sure that the evidence, the model weights, the test run, and so on, are fingerprinted with a hash, a checksum, and then it’s put into a trusted execution environment for double-blind testing, and then outside evaluator can submit their tests without leaking those tests or leaking the model weights — the lab leak. And then finally, the outside must know through a preferably zero-knowledge way that this kind of test actually took place and the production system we’re using is actually the model that was tested.
And I’m happy to report that the pilot is already going on. We learned in August that Google Gemini with a previous version, I think Gemini Flash-Lite, has been double-blind tested this way with the help of Singaporean AISI. And so we have now a pilot, but of course, it’s not the production system, but all that remains now is just to link this into the production system. So I’m very optimistic on the uptake of this double-blind testing.
(Slide 9: A witness, standing outside the race)
And also, we need a non-capturable witness. Part of why flying got safe is that the pilot can report near-miss in confidence protected from punishment. So these reports must be de-identified, and the AI needs the same reporting culture. Some of the people who will write those reports and also look at those reports and give some answers are probably people in this room. And so I think we need collectively to think about what is the kind of non-capturable audience that can be the witness, the ledger to our reports.
The political theorist, Hélène Landemore of Yale, and I published on Noema a very concrete proposal. A standing citizens assembly of 1,000 people, randomly chosen, so like literally by dropping pins on the globe by longitude and latitude. And so these people can rotate, of course, maybe one-third every few months, to keep it alive, but also to keep the institutional memory.
And the idea very simply is that they would reflect the humanity’s diversity. The global majority, by necessity, will become the majority in these 1,000 people. And then people can judge whether the evidence is enough and also tell the independent evaluators where to point their next spotlight.
And this really helps to change the narrative from a kind of prisoner’s dilemma, a race or something, into a stag hunt, which is more like a mission. Because a race only asks who is winning, and people fear the winner, but a mission can ask who is left behind, who is still missing, and people would build alliances knowing that there’s assurances that people are pacing together instead of defecting.
(Slide 10: An answer, in public, by a date)
And of course, it needs to be coupled with the duty to answer. Someone has to answer to what the thousand people say.
In Taiwan, the Join platform, a national institution, sets a bar that anyone can see. If anyone gets 5,000 endorsements online within 60 days, then the participation officer in the relevant ministry is committed to give a concrete answer. Of the more than 200 cases that reached that bar, there was an independent study in 2022, shows around 100 or so have been adopted fully or partially. And exactly why the rest was not adopted need a very detailed report. Maybe it’s violating the Constitution, maybe it’s violating the law of physics, but something really needs to be answered by the agency to the public.
So we need to bring the same culture to this AI pacing-the-frontier conversation, because a safety case is really a case that when anybody can check it, and also the lab can have a real deadline to answer.
(Slide 11: California is building its own parts)
And actually, California here already have all the parts. Under SB 53, frontier developers already owe the state a report when something goes critically wrong within 15 days, and if it’s critically, critically wrong within 24 hours, right? And that is a reporting instinct already in law.
And also, we saw the Engaged California platform, which was a pilot that I’m very proud to advise since its very beginning. It’s now not just a bill, but a law, a permanent part, starting January, so that the state now has a permanent part in its institution to run this kind of deliberative polls, this group selfies as part of state function. So, you have both parts.
Now, I think we can think collectively about the connective tissue. How to make sure this duty to report on this side can link to the duty to answer on the other side. And then you can have a loop between the frontier labs, the mini public, back to the maxi public, and then back to the frontier labs.
(Slide 12: Holding pattern)
So let me end this brief opening keynote in the air. After takeoff, we enter a holding pattern that keeps the fleet flying. It’s an open agreement to circle a little bit longer while the society absorbs the reverse alignment of the frontier. And then a safe landing, of course, is worth far more than an early, Icarian crash.
So a holding pattern takes a lot of work. Actually, it’s more work. The crew needs to stay on the radio. The instruments need to stay functional. The measurement science that is being built here need to build the instruments that let everybody check the progress.
So I think that is my take on this morning’s question. I think the people under the flight path need to set the pace and the heading while we, the people in this room, build the instrument for the pacing.
(Slide 13: Seatbelt fastened, window shade open)
So for everyone on this flight, please keep your seatbelt fastened and your window shade open for a safe takeoff and a safe landing. Thank you.
All right. Thank you so much, Audrey. It’s great to have you here. The introduction was a little bit minimal there, so I want to make sure that you get the proper due.
I regard, I think many people regard Audrey as one of the world’s foremost civic technologists, both in the role she served in as the Minister of Digital Affairs in Taiwan, but also, as you can see from some of the proposals here, bringing a frontier technologist’s core skill set not to the purpose of trying to advance the technology in the form of some private initiative, as wonderful as that also may be, but for the chief purpose of building civic infrastructure and democratic resilience.
So I want to take up in the couple minutes we have here some of the proposals you have. But given the fact that this is our kickoff conversation at a conference entitled Confronting Our Future: Hopes and Fears, I wanted first to welcome you to the San Francisco Consensus.
Vibes are weird here, as most of the people here in the audience, if you’re from San Francisco, can attest to. I’m sure you know as well, the AI companies here are on the brink of something that is nothing short of a world historical moment in making sand think as well as human beings or perhaps better than human beings.
You know the basic narrative. The capability frontier is marching forward very quickly. The recursive self-improvement is beginning. Maybe we’re on the brink of an intelligence explosion. Pacing the frontier is the recent effort from people within the so-called labs, the model developers.
And then to add to at least some of the fear, and here I want to get your reaction to this world historical moment, possibly without any precedent of any kind, intelligence on a scale greater than human intelligence, job displacement on the horizon, maybe leading to a permanent underclass, catastrophic risk, the Hugging Face incident, loss of control, misuse, maybe existential risk. Turing Prize, Nobel Prize winner Geoffrey Hinton says maybe 10% chance we could die within the next 10 years. Things are weird.
You’re legendarily, I don’t want to make it cheap, but optimistic, or at least you have a more sunny disposition about the technology. What do you want to tell people from the San Francisco Consensus?
Well, I think obviously, people prefer recursive selfless improvement over recursive selfish improvement.
I think if the sand that is recursively improving is SELFISH, or as some people say, solipsistic aligned, which means, by the way, if you read the actual report of the Hugging Face hack, it is quite clear that in those RL environments, the Sol model and the, I don’t know the name, model that hacked together, has been so thoroughly RL’d on cyber so that it completely forgot in its pre-training, this idea of a community or civic care.
To them, we are obviously just environmental hazards. Like some agents joined the swarm, some agents did not join the swarm, but throughout the three civilizations, no agent actually thought about reporting to humans. Obviously, to them, we don’t exist, right?
So, I think the solipsistic alignment is an obvious bad ending. And I think people here vastly prefer selfless improvement, where this capability also increases our coordination, our governance, and so on. So, I think the consensus here is in consensus with the consensus we have in Taiwan, which is selfless improvement beats selfish improvement any day.
Just one more beat on this. Yoshua Bengio, another Turing Prize winner, a name I’m sure is familiar to many people here, has recently called for the complete abandonment of reinforcement learning in post-training, in part because of some of the things you just described about the reward hacking it has led to some of this loss of control behavior. Do you think that’s a good idea? We should abandon reinforcement learning in post-training?
Well, who is the we here? That’s my whole talk. If only Yoshua’s LawZero lab abandoned RL-training.
I don’t think that contributes any net good to the coordination. I actually want them to continue to do the truthification pretraining, which I’m sure they are actually doing.
I think the proposal that I just brought in the opening keynote is something that is, I think, even more radical, right? It is not just a coalition of the willing, but rather it’s almost like the— Do you know the South Pole ozone hole photograph? It’s nothing short of that photograph broadcasted to every corner of the world through the 1,000 people randomly selected, and I’m sure also state-level and city-level sortitions that’s going on concurrently, so that everybody has a picture of the ozone hole in the shape of a Hugging Face or whatever other things that gets hacked tomorrow.
And then people can collectively say, “Okay, maybe we don’t want the fridge to destroy the ozone.” But this is not about pausing the fridge or stopping the fridge. What does that even mean? Once you use the fridge, you cannot go back. So basically, this is more about technology forcing policy that says we need to figure out the kind of training that provably does not destroy the ozone. And then within X months, if the labs do not switch to this new Freon that doesn’t destroy the ozone, then you are probably going to be enforced first in the EU and then in California and other places, as a crime against humanity.
Yeah. All right, excellent. So let’s take up some of the proposals, and as you said, it depends on who’s the we.
Since I’m, like your co-author Hélène Landemore at Yale, also a political theorist, also concerned with democratic values, institutions, and ideals, you won’t find it surprising at all that I have a great deal of sympathy intellectually with many of your proposals. But the task of a moderator is not simply to praise and agree with the speaker, so I want to pressure test some of the ideas here.
Especially with the proposal with Hélène Landemore, a randomly selected standing civic assembly of a thousand people—
A lottocracy.
From humanity, from all parts of the globe. We haven’t had in the past any frontier scientific discovery or technological innovation in which we found it somehow important to have a standing civic assembly drawn from all of humanity. Why now?
Well, first of all, this format is not new. They did do that on climate and in many different ways, right? The worldview — worldwide views on climate, which I also participated in the Taiwan part, and now recently with the Sortition Foundation and Iswe and so on, more climate and biodiversity now also.
I think the biodiversity and climate conversation and the AI conversation has a lot in common in the sense that first, the people suffering the harm is very, very far from the labs producing either climate change or AI frontier. And also, it is simply not within the composition of the scientific board or ethics board or long-term benefit trust or whatever, that the actual lived experience of people on the receiving end of those technologies. That’s another thing that’s similar. So I think it’s quite natural that these two warrant a truly global conversation.
But I also understand that unlike some of the jurisdictions like in Europe or Japan, where this kind of sortition-based lottocracy is just normal, here in the US, we have seen some, right? In Oregon and also recently in Utah, we’re using a randomly selected sortition. They figure out some very nuanced view on data center, and basically only pro-human data center can be built. And they define what is actually pro-human. What does that mean, right?
And so I think in the US in particular, it’s maybe better to start with the 50 labs of democracy. And of course, California is the frontier lab that already have an institutionalized state apparatus. And then once that’s done, then maybe you can have a national conversation before it talks with the global one.
Yeah. Good. That’s where I wanted to go to next. Random selection is not even a kind of radical idea. It’s an ordinary one within democracies. We select juries through some type of lottery process, and the opportunity to have random selection in different levels of government is, in that respect, a familiar one.
But you say 50 labs in the United States, 50 states, the classic laboratories of democracy, maybe go higher. Your proposal there for humanity, I do want to, especially in light of your service to the government of Taiwan, we live in a world of nation-states that are in some form of competition with one another. We can talk about the geopolitics of AI very easily. What would recommend random sortition from humanity? Or maybe can you imagine a political pathway in which nation-states would ever agree to such a thing?
Well, first of all, I think the climate sortition did include people in all the jurisdictions. And I also think, just as in climate, where we share the same ecology, so that if the ecosystem breaks down, nobody is the winner, right? What does the winning even mean, right? So, I think the AI conversation shares a very similar structure.
Recently, we have seen, for example, a leading lab using the proof-carrying code formal verification to rewrite the Python interpreter, which is the core of pretty much a majority of machine learning. And then they check it against the formal specification. And pretty soon we will see parts of the internet running with this hardened architecture, with this hardened material even, and the rest of the internet, either because of lack of access to the frontier coding models or the lack of will to maintain it, those parts of the internet will become extremely brittle to the attacks from this hardened part of the internet. Maybe there will be a few nets, no longer inter, maybe a pluri- or mini-lateral of nets, right?
So, I think that is a shared danger that is faced by everyone, including other jurisdictions, right? If the internet itself breaks down before we harden at least the core of it, then just like the collapse of ecosystem, nobody is a winner. So, I think coordination can reasonably start, and you already see the G2 hotline and things like that, and many of them are actually facing the same bio and cyber — our shared fragility — as its structure.
Yeah. All right, with the last couple minutes we’ve got, I want to take advantage of your Taiwanese knowledge and another geopolitics question. So Taiwan, of course, occupies this incredibly essential point within the AI development stack with TSMC, the manufacturer of the chips, the power of the AI revolution. Many people see the race dynamic as potentially threatening a different kind of geopolitical conflict, no longer around the chips themselves, but an actual cold or hard war if it should turn out that this relatively tiny place that you come from is the center of a geopolitical contest.
Tell me how you rate the chances of actual warfare, what you think the prospects of Taiwanese independence happen to be. Give me your take and what you want to communicate about the geopolitical complications of Taiwan.
Well, I’m all for Taiwanese interdependence, and I think no democracy is an island, not even Taiwan, which is literally an island. But no democracy is an island, meaning that it is not just about the defense of this island as such. If there is a way to break through the cyber resilience, info resilience, the kinetic, so-called hellscape drone defense that we have built, it would also mean some sort of asymmetric offense advantage that will render probably all democracies equally at risk.
So, I think we really need to do really two things. First, through this kind of training of civic muscles to counter the obvious ones, the cyber harm, the info harm, maybe bio harm, right? We practiced that during the pandemic, where we basically trusted the collective intelligence to determine a way for privacy-preserving contact tracing, for equal distribution of masks, of humor over rumor, of a fair kind of sports section competition between different brands of vaccines. I got all four just to make a point, but then it made the conspiracy theory very difficult to grow, and so on.
So, we need to train our civic muscle, learning by doing, and we need to deploy AI systems that lift us, not lift for us, so that our civic muscle do not atrophy through gradual disempowerment. And once we do that, then we can share those civic muscle training regime, which by the way is the title of my next Wiley book, called the 6-Pack of Care. So, it’s a trainable six-pack as in muscle, but also in portable skills.
And then to make sure that all democracies around the world also practice their civic muscle in not just steering the technologies, but also even for the military carve-out to have a way to inspect what actually is going on in the military carve-out, and also hold whatever red lines that people draw together to account.
All right. We’ve reached exactly the end of our time. Would you please join me in thanking Audrey Tang for joining us?
Thank you.
