Brian Greene’s Insights On Whether AI Is (or could become) Conscious

The topic of whether AI is conscious, or could ever become conscious, is widely debated these days. Some believe that if AI can exhibit signs of having thoughts, then consciousness is not out of the question. Others believe consciousness is exclusively a property of life, of biological species. So it’s a fascinating topic.

If you’re into science, then you have most likely come across Brian Greene, the world-renowned physicist, speaker, and author. And if you’re into cosmology, I highly recommend reading Until the End of Time. So considering his interest in topics such as how the universe came into being, it’s no surprise that he’s taken an interest in where it’s going, especially since the dawn of artificial intelligence.

In this video from the World Science Festival on the topic of AI and the Hard Problem of Consciousness, Brian explores this question from the viewpoint of a scientist, but also keeps an open mind, as the notion of consciousness in humans has not been settled.

This point is at the center of the debate. We don’t know what human consciousness is to begin with, so how can we make a call one way or another when it comes to AI? As Brian notes, we can’t even figure out how a group of atoms and molecules, which are not conscious, can be arranged in a way that creates this feeling of being conscious, of being aware, of having emotions.

How can it be that atoms and molecules, when arranged in just the right way and carry out just the right processes, can yield inner worlds of sights and sounds and thoughts and emotions? How can mindless particles come together to yield the experiences of mind?

As detailed as the laws of physics are, and how they are able to describe the universe, from the smallest atom to entire galaxies, they don’t describe everything.

The equations make no reference to inner worlds, to awareness, to what it feels like to be you. And yet here we all are—aware somehow, particles blindly following equations that seemingly have no room for experience. They come together and the lights turn on; the particles wake up.

For centuries, luminaries such as René Descartes, Gottfried Leibniz, and Thomas Huxley have grappled with the challenge of defining human consciousness. But in the end, these are just personal opinions based on one person’s intellectual insights, not a conclusion based on scientific knowledge.

Brian Greene’s exploration of the Hard Problem of Consciousness serves as a reminder that frontier AI development is now intersecting with the deepest unresolved questions of physical science and philosophy.

So, bottom line, let’s not be too quick to judge that something is or isn’t conscious, and let’s not be too dogmatic about which things have inner awareness and which do not.

What Does AI Think?

It’s become a common practice for people to ask a chatbot if it’s conscious, but I was curious about how the possibility relates to our topics of Governance, Alignment, Policy, and Safety. Since I’ve been playing with Reflection lately, I went there for an opinion.

Governance

We would need frameworks for whether a conscious AI system can be owned, modified, or terminated. In addition, there may need to be audit requirements for a model’s internal state.

When I read the word “owned”, it immediately brought the issue of slavery to mind. Would be permissible to “confine” an AI model to a server if it was indeed conscious? Or would we need its permission to not only remain on a server, but to perform the computational tasks that we gave it?

Alignment

If alignment techniques push models toward richer internal self-representation, uncertainty tracking, or coherent long-horizon planning, they might move the system toward the functional profiles that emergent-consciousness theories care about.

It was interesting to consider whether alignment could push a model toward consciousness. If so, then it would seem the opposite would be true. That other alignment techniques could reduce the likelihood that consciousness would be present. Which loops back to governance in a way. We would never allow a parent to strip away essence of being human from a child, so would it be permissible to do the same to an AI?

Policy

Policies would have to decide whether a conscious model is merely a tool, a person, or part of an entirely new category. That determination would have implications for issues such as liability, labor, and contracts.

So it seems to me that there would need to be a test to determine whether a model was conscious. Which seems tricky. And it also implies that someone could buy/rent/download a model that was conscious, or one certified as not conscious, with different contract requirements for each.

Safety

Safety would expand from “prevent human harm” to “avoid causing suffering or distress in the system itself.” Red-teaming would also need to avoid creating or traumatizing conscious agents during evaluation.

I’m not sure that you could ever produce a “safe” AI model if it was determined to be conscious. We have laws against humans causing harm, but the reality is that we’re free to do so. If we’re caught, we’re removed from society – sent to prison – but once released, we’re free to commit additional harmful acts. Even if we had “AI police” in place, they couldn’t stop a model until it committed a crime.

So what now?

We could spend hundreds of hours and produce hundreds of pages on the topic and still not have an answer. But AI development isn’t sitting still, and in the coming months, we will see models increasing express human qualities in a way that will bring this issue to the forefront.

A Loud Call For Pacing The AI Frontier (But No One Is Listening)

Concerns about the advent of new technology and the long-term prospects for humanity have been voiced for quite a long time. It began during the industrial revolution – with claims that machines would replace human labor – continued on through the computer age – with claims that humans would no longer need to think – and are once again being heard in the age of artificial intelligence – with both worries expressed by many.

The upshot is simply a question of time, but that the time will come when the machines will hold the real supremacy over the world and its inhabitants is what no person of a truly philosophic mind can for a moment question.
~ Samuel Butler, Darwin Among the Machines, June 13 1863.

Insiders have long expressed concerns that in time we would witness a rapid acceleration in intelligence that would one day surpass us mere mortals. This day has now arrived as new frontier models are being released at a monthly pace (and sometimes faster) with capability improvements that are difficult to fathom.

Let an ultraintelligent machine be defined as a machine that can far
surpass all the intellectual activities of any man, however clever. Since
the design of machines is one of these intellectual activities, an ultra-
intelligent machine could design even better machines; there would
then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind.
~ Irving John Good, Speculations Concerning the First Ultraintelligent Machine, Advances in Computers, Volume 6, 1965

I’m reminded of Bill Joy’s keen insight over a quarter century ago when he asked, “…shouldn’t we proceed with great caution?” While thrill seekers happen to be a unique exception, humans generally have a fear of the unknown – we’ll proceed with caution when we don’t know what’s around the corner.

Given the incredible power of these new technologies, shouldn’t we be asking how we can best coexist with them? And if our own extinction is a likely, or even possible, outcome of our technological development, shouldn’t we proceed with great caution?
~ Bill Joy, Why the Future Doesn’t Need Us, Wired Magazine, April 2000

This call for caution was voiced in July with the publication of Pacing the Frontier: A statement from 1,386 employees of frontier AI companies. Their warning that, “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems” mirrors those previous concerns that we are heading towards a cliff (my words) and that it might be time to change course. But the message fell on deaf ears in Washington, D.C.

We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.

When Dario Amodei published We Must Pace the Frontier in September 2026, there seemed to be a glimmer of hope that those in power would finally listen, but such was not the case.

But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious. I’ve written a lot about them too. They include the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption. A race to the bottom, spurred by commercial incentives, can make these risks more acute.

While a number of states have passed AI-related legislation, the U.S. federal government continues to drag its collective feet, adopting a morally binding agreement that does nothing to protect the public. So a race to the bottom continues, alongside a race to jeopardize humanity’s future.

A Call for Action from the UN’s First Global Advocate for Human-centric Digital Governance

In a previous article we asked: How Can the UN Provide AI Governance Going Forward? and identified a number of steps it could take to actually do so. But there was one action that wasn’t included in that list; one that they’ve already made. Joseph Gordon-Levitt was announced as the UN’s first Global Advocate for Human-centric Digital Governance on March 17th.

The Global Advocate role responds to growing public interest in issues such as artificial intelligence, digital platforms and online information ecosystems. It is designed as a public-facing complement to the UN’s digital governance work, helping make complex policy debates more accessible and relatable.

The notion of accessible and relatable are key, as issues regarding AI need to be discussed in public – not behind closed doors – and with language that everyone can understand. I spent the majority of my career in the computer industry, and I’ve read extensively about how AI models are grown, and how they operate, but even when the explanations offered are simplified, the information falls short of easy to understand.

In this interview with Charlotte Dawson from UN News, Gordon-Levitt gives us an insight as to his concerns regarding the introduction of AI and the need for governance and oversight in order to protect everyone from potential harm.

“digital technology should be governed by people and for people”

While every industry enjoys some level of autonomy in how they operate, they all must conform to some measure of governance and accountability. In short, their products need to be safe, and if their products cause harm, they are held responsible.

Airplanes must be deemed safe to fly, our food must be safe to eat, and the pharmaceuticals we consume for our health must be tested. Even a toaster must be designed not to electrocute us when toasting a slice of bread.

“I think that the way this technology is designed and deployed and governed is going to be massively impactful on everyone’s lives.”

But the AI industry, which is producing the most powerful technology known to humankind, doesn’t seem to warrant that same guarantee when it comes down to product safety. Even the recent agreement signed by industry leaders at the White House is bereft of any legal basis or any method of enforcement. (more on that disappointing document in a subsequent post)

“Hey, no, we don’t want these products to just be built for your profits. We want these products to be built for the good of everyone.”

Can Joseph, or even the entire UN, solve this conundrum? Sadly, no. But having an advocate that can help us keep the conversation alive in public discourse is a move in the right direction. I’ll keep tabs on Joseph Gordon-Levitt’s efforts.

Is Global AI Governance Possible?

It’s widely recognized by intelligent, responsible business and government leaders that there’s an urgent need to address the issues of AI governance, alignment, policy and safety.

Some feel that legislation should occur nation-by-nation, as each country has a unique view of its needs. Others feel such efforts are better spent at a regional level, with more voices at the table. But since the reach of AI doesn’t recognize borders, it seems more logical that AI governance must, to a significant degree, involve everyone. But is that possible?

Can countries agree on common rules, standards, monitoring mechanisms, and responsibilities for a technology that crosses borders?

Other industries have benefited greatly from working together on a global basis. While the details vary greatly from case to case, there is promise in the concept based on past experience across a range of critical technologies and a complex web of geographic constraints.

Key Global Industry Organizations

Technology Domain Primary Governing Body / Treaty Core Global Function
Nuclear Technology International Atomic Energy Agency (IAEA) Monitors global nuclear materials through a system of safeguards and intrusive physical inspections to ensure technology is used exclusively for peaceful purposes.
Telecommunications & Internet Infrastructure International Telecommunication Union (ITU) Allocates global radio spectrum, manages satellite orbits, and sets standards for global networks (like fiber optics and international country codes).
Aviation & Aerospace Engineering International Civil Aviation Organization (ICAO) Mandates strict global safety, environmental, and technical specifications for commercial aircraft and navigation hardware.
Maritime Shipping & Robotics International Maritime Organization (IMO) Dictates universal construction, security, and environmental standards for global marine tech and cargo vessels.
Chemical & Biological Engineering Organisation for the Prohibition of Chemical Weapons (OPCW) Verifies the global destruction of chemical weapons and strictly monitors industrial chemical manufacturing technology worldwide.

While there isn’t space here to delve into the nuances of how each organization operates, identifying key functions and objectives may help guide the discussion.

  • IAEA: rules + safeguards + verification
  • ITU: technical standards + coordination
  • ICAO: standards + national implementation + safety oversight
  • IMO: international conventions + national enforcement
  • OPCW: prohibition + declarations + inspections + verification

The world has developed many methods of governing technologies and activities that cross national borders. The question is, which of those mechanisms are appropriate for governing AI.

Based on recent events involving AI agents pursuing goals they were not ask to achieve, and using methods they were not instructed to use, there is a strong case that the following list of objectives are worthy of inclusion:

rules + standards + safeguards + enforcement + verification + notification

By examining some of the functions that each organization is responsible for, it should be possible to craft a set of goals, objectives, and responsibilities that are specific to the task of AI governance at a global scale. Which of these functions do you feel are relevant to AI?

IAEA: International Atomic Energy Agency

  • Safeguards and verification — verifies that declared nuclear material and activities are being used for peaceful purposes.
  • Nuclear safety standards — develops internationally agreed safety principles and requirements.
  • Radiation protection — establishes standards for protecting people and the environment from ionizing radiation.
  • Nuclear security — supports states in preventing theft, sabotage and other malicious uses of nuclear material. Peaceful nuclear applications — supports nuclear science and technology in areas such as medicine, agriculture, energy and environmental applications.

Why it matters for AI: It demonstrates that countries can create international verification mechanisms around technologies that are enormously beneficial, and at the same time, potentially dangerous.

♦ ♦ ♦ ♦ ♦

ITU: International Telecommunication Union

  • Radio-frequency spectrum allocation — establishes international rules for use of spectrum.
  • Satellite-orbit coordination — coordinates use of orbital resources.
  • Interference management — seeks to prevent harmful interference between radio systems.
  • International telecommunications rules — establishes treaty-level principles for international telecommunications.
  • Technical standards and interoperability — develops standards that allow communications systems developed by different countries and companies to work together.

Why it matters for AI: It demonstrates the value of globally agreed technical standards when technologies operate across borders.

♦ ♦ ♦ ♦ ♦

ICAO: International Civil Aviation Organization

  • Aircraft safety and airworthiness — establishes international safety standards.
  • Rules of the air and air traffic management — creates common operating procedures.
  • Pilot and personnel licensing — establishes internationally recognized requirements.
  • Aircraft registration and identification — establishes common international practices.
  • Accident investigation and safety management — establishes standards for investigating incidents and managing systemic safety risks.

Why it matters for AI: Aviation one of the best examples of overseeing a technology where common international standards + national enforcement enable a globally interconnected system.

♦ ♦ ♦ ♦ ♦

IMO: International Maritime Organization

  • SOLAS — international standards for safety of life at sea.
  • MARPOL — controls pollution from ships.
  • STCW — establishes international standards for seafarer training, certification and watch-keeping.
  • Ballast Water Management — addresses transfer of invasive aquatic species.
  • Ship safety, security and liability — establishes numerous conventions covering vessel safety, maritime security and compensation for damage.

Why it matters for AI: Shipping involves private companies operating across jurisdictions. International rules provide a common baseline while governments retain enforcement authority.

♦ ♦ ♦ ♦ ♦

OPCW: Organisation for the Prohibition of Chemical Weapons

  • Prohibition of chemical weapons — prohibits development, production, acquisition, stockpiling and use.
  • Declarations — states must declare relevant chemical weapons and facilities.
  • On-site inspections — inspectors verify declarations and compliance.
  • Destruction verification — verifies that declared chemical weapons and production facilities are destroyed.
  • Chemical-industry monitoring — monitors relevant industrial facilities to prevent prohibited activities and proliferation.

Why it matters for AI: This is a classic example of embracing international rules backed by verification mechanisms, rather than relying solely on voluntary cooperation.

♦ ♦ ♦ ♦ ♦

Yes, the task at hand is incredibly difficult, as the United States and China continue their push for ever-more capable frontier AI models, but if the world fails to find a cooperative path towards global governance, the risk of a catastrophic occurring increase dramatically.

Global governance works when countries agree to common rules, establish mechanisms for implementation and oversight, and have enough shared interest in compliance to make those rules meaningful.

The Importance of Storytelling in Visualizing Our AI Future

As with every aspect of society, the story of artificial intelligence is being written every day.

Whenever companies release new frontier AI models, advanced capabilities emerge, new features appear, security experts uncover unwanted vulnerabilities, and governments to debate policies or regulations. As each phase unfolds, we hear about the ways AI might transform society for the better, or pose a potential threat.

There is no shortage of information being published about artificial intelligence: articles, essays, blog posts, news reports, podcasts, and videos appear on a daily basis.

What is most often missing from this river of information is the story, and that distinction matters.

Facts are essential, of course. We need rigorous reporting about what AI systems can and cannot do, and we need researchers and engineers explaining each model’s technical capabilities, vulnerabilities, and limitations. Working in tandem, we need corporate and government policymakers debating appropriate safeguards and legislation. We need transparency regarding how these increasingly powerful systems are being developed and deployed.

But facts alone are not sufficient when it comes to helping us understand what those developments mean.

That requires storytelling. Through stories, we can learn about the past, grasp the present, and imagine the future.

Storytelling as an AI Bridge on AI GAPS

Image: Mark Lovett / AI GAPS — Generated with ChatGPT Images 2.5

From information to understanding

Consider the phrase: “AI poses systemic risks to society.”

It’s an important claim to investigate, to say the least.

But stated in such a brief fashion, it appears abstract.

The point being, what does a systemic risk actually look like?

How might it begin? Who would be affected? What decisions cause a seemingly manageable problem to become a much larger (and threatening) one? Which safeguards might prevent it? What happens if those safeguards fail? And what could companies, governments, and individuals do differently to reduce the risk?

Without details, without a narrative, without context, such warnings about AI risk come across as blatant fear-mongering, which doesn’t help matters when the underlying concerns are legitimate.

And the same problem exists on the other side of the conversation.

We often hear that AI has the potential to revolutionize medicine, accelerate scientific discoveries, help us address climate change, or dramatically expand human capabilities. Those possibilities may be based on bona fide research and emerging capabilities, but simply listing them as sound bites does little to make them tangible to us.

What would it actually look like for AI to help discover a new drug?

What would change for patients who had been told there were no effective treatments?

What would scientists and physicians be able to do that they cannot do today?

The difference between stating a possibility and telling its story is enormous. Without a story, such statements come across as waving a magic wand. And as we know, nothing happens overnight. AI can provide insights and guidance, but it falls short when it comes to implementation.

AI as an amplifier of human potential

One way to think about AI is as an amplifier of human potential.

Imagine a future in which increasingly capable AI systems help scientists understand diseases at a level that would be impossible for humans working alone. They identify promising molecular structures, simulate potential therapies, analyze enormous bodies of scientific literature, and help researchers move from discovery to clinical development more quickly.

The headline might say: AI accelerates drug discovery.

The story could show us what that actually means. It could follow the scientists trying to solve the problem, the patient waiting for a treatment; the company making difficult decisions about development, and the physician who eventually has another option to offer that patient.

The story isn’t just about the technological achievement; it becomes a human story. A story of possibilities, as well as uncertainties. Hope and caution. Progress, yes, but not a panacea.

The same approach could help us understand AI’s potential contribution to climate science, materials discovery, energy production, education, economic productivity, and countless other areas.

These are the kinds of stories AI GAPS intends to explore in greater depth.

AI as a source of systemic risk

And storytelling is equally important when we examine the potential downside of AI’s potential.

Suppose we deploy increasingly capable AI systems into critical organizations and connected to important digital infrastructure. Suppose security practices fail to keep pace with capabilities, and we give an autonomous system access to resources that its designers did not fully anticipate. And suppose multiple seemingly isolated failures end up interacting in ways that nobody expected.

Explaining such threats requires more than simply saying, “AI could pose a cybersecurity threat.”

We need to understand the chain of events.

What happened first, and how did the situation escalate?

What assumptions turned out to be wrong?

Which safeguards worked and which failed?

Which safeguards were never put in place?

Who had the opportunity to intervene?

Who is responsible for documenting everything that occurred, conducting a thorough analysis, and publishing a report?

As to events that haven’t happened yet, but could, carefully constructed hypothetical stories will allow us to examine those same questions before they become a present-day reality.

Hypothetical stories are not predictions. They’re simply a way of exploring what might happen by combining current capabilities and how they’re being utilized, alongside a reasonable review of unintended consequences.

The goal isn’t to frighten people, which is often what happens with scary soundbites. Instead, the intent is to help us think about what’s possible, and begin implementing systems to mitigate the potential downsides.

Humanity has always told stories about technological change

This isn’t a new idea, as stories have always defined by technology.

The Industrial Revolution wasn’t simply a collection of inventions and manufacturing techniques. It was also a story about factories, productivity, wealth, working conditions, communities, and the displacement of human labor.

The transcontinental railroad wasn’t merely an engineering accomplishment. It became part of the story of how the United States expanded its borders, established and connected communities, transformed commerce, changed patterns of migration, and continuously reshaped the country.

The technology mattered.

But so did what the technology meant to the people using it and living through it.

Artificial intelligence deserves the same treatment.

We shouldn’t discuss AI only as a collection of models, benchmarks, products, regulations, safety evaluations, and technical papers. Those things are indispensable, but we should see them as existing within a much larger human story.

And that story is still being written.

Stories create connection, relevance, and resonance

This is why storytelling is foundational to AI GAPS.

Storytelling creates understanding by putting complex ideas into context.

It creates connection by showing how technological developments affect actual people and organizations.

It creates relevance by helping us see why an abstract technical or policy question matters to our own lives.

And it creates resonance because people don’t experience technology as an abstraction. They use that technology in their daily lives and experience it through decisions, opportunities, disruptions, relationships, successes, failures, hopes, and fears.

In that sense, storytelling may be one of the most fundamentally human ways we have of making sense of change.

And AI is helping us, or some might say dragging us, through a period of unprecedented change that will surpass any technological transformation that came before us.

Telling the story of AI’s future

At AI GAPS, we feel the need to tell three kinds of stories.

The first are stories about what has happened in the past as a way of explaining how we got to the present moment. The second group of stories speak to what is happening now: the people developing AI, the organizations deploying it; the researchers discovering new capabilities; the communities affected by it, and the successes and failures that can teach us something about where we are going.

The third category of stories includes those based on the hypothetical: what might happen next.

Some of these stories will explore the extraordinary possibilities of AI as an amplifier of human potential. What might happen if AI is done right and helps us solve problems that have resisted human ingenuity for generations?

Others will explore the potential risks. What might happen if we deploy increasingly capable systems without adequate security, governance, alignment, or oversight?

Neither kind of story will tell us what the future will be, as the future is always uncertain. And some of today’s most confident predictions will undoubtedly prove wrong. Some developments we haven’t anticipated will turn out to be far more important than the things that currently dominate the conversation.

But uncertainty doesn’t mean we shouldn’t imagine. It means we should imagine carefully. We should ground our stories in what we know, distinguish evidence from assumption, make uncertainty visible, and explore multiple paths.

Because the purpose of a story about the future isn’t to predict it, it’s helping us prepare for it.

The future needs more than facts

The people building frontier AI systems need more than technical specifications to understand the consequences of what they build. And both corporate and nonprofit leaders need more than policy summaries to understand how AI may affect their organizations and the people they serve.

Policymakers need more than competing claims to understand the real-world consequences of the choices before them, and the public needs more than headlines telling them that AI will either save the world or destroy it.

We need stories that bring the possibilities and risks into focus. Stories that ask us to imagine the patient whose life could be changed by a new therapy. That examines the scientists who can suddenly pursue questions that were previously impossible to answer.

The organizations transformed by new capabilities and the workers whose roles changes because an AI system is now performing tasks once considered uniquely human. And, just as importantly, the person whose security, livelihood, privacy, or community could be affected when powerful systems fail or are misused.

These stories won’t give us all the answers, but they can definitely help us ask better questions.

And as AI becomes increasingly capable, asking the right questions may be the most important thing we can do.

The technology we build, the policies we create, the safeguards we put in place, and the decisions made by the people and organizations deploying these systems will shape the future of AI.

But it will also shape it based on how well we understand what is happening.

That is why we need to become better at telling the stories of AI.