Category: AI Safety

Brian Greene’s Insights On Whether AI Is (or could become) Conscious

October 31, 2026/in AI, AI Consciousness, AI Governance, AI Safety/by Mark Lovett

The topic of whether AI is conscious, or could ever become conscious, is widely debated these days. Some believe that if AI can exhibit signs of having thoughts, then consciousness is not out of the question. Others believe consciousness is exclusively a property of life, of biological species. So it’s a fascinating topic.

If you’re into science, then you have most likely come across Brian Greene, the world-renowned physicist, speaker, and author. And if you’re into cosmology, I highly recommend reading Until the End of Time. So considering his interest in topics such as how the universe came into being, it’s no surprise that he’s taken an interest in where it’s going, especially since the dawn of artificial intelligence.

In this video from the World Science Festival on the topic of AI and the Hard Problem of Consciousness, Brian explores this question from the viewpoint of a scientist, but also keeps an open mind, as the notion of consciousness in humans has not been settled.

This point is at the center of the debate. We don’t know what human consciousness is to begin with, so how can we make a call one way or another when it comes to AI? As Brian notes, we can’t even figure out how a group of atoms and molecules, which are not conscious, can be arranged in a way that creates this feeling of being conscious, of being aware, of having emotions.

How can it be that atoms and molecules, when arranged in just the right way and carry out just the right processes, can yield inner worlds of sights and sounds and thoughts and emotions? How can mindless particles come together to yield the experiences of mind?

As detailed as the laws of physics are, and how they are able to describe the universe, from the smallest atom to entire galaxies, they don’t describe everything.

The equations make no reference to inner worlds, to awareness, to what it feels like to be you. And yet here we all are—aware somehow, particles blindly following equations that seemingly have no room for experience. They come together and the lights turn on; the particles wake up.

For centuries, luminaries such as René Descartes, Gottfried Leibniz, and Thomas Huxley have grappled with the challenge of defining human consciousness. But in the end, these are just personal opinions based on one person’s intellectual insights, not a conclusion based on scientific knowledge.

Brian Greene’s exploration of the Hard Problem of Consciousness serves as a reminder that frontier AI development is now intersecting with the deepest unresolved questions of physical science and philosophy.

So, bottom line, let’s not be too quick to judge that something is or isn’t conscious, and let’s not be too dogmatic about which things have inner awareness and which do not.

What Does AI Think?

It’s become a common practice for people to ask a chatbot if it’s conscious, but I was curious about how the possibility relates to our topics of Governance, Alignment, Policy, and Safety. Since I’ve been playing with Reflection lately, I went there for an opinion.

Governance

We would need frameworks for whether a conscious AI system can be owned, modified, or terminated. In addition, there may need to be audit requirements for a model’s internal state.

When I read the word “owned”, it immediately brought the issue of slavery to mind. Would be permissible to “confine” an AI model to a server if it was indeed conscious? Or would we need its permission to not only remain on a server, but to perform the computational tasks that we gave it?

Alignment

If alignment techniques push models toward richer internal self-representation, uncertainty tracking, or coherent long-horizon planning, they might move the system toward the functional profiles that emergent-consciousness theories care about.

It was interesting to consider whether alignment could push a model toward consciousness. If so, then it would seem the opposite would be true. That other alignment techniques could reduce the likelihood that consciousness would be present. Which loops back to governance in a way. We would never allow a parent to strip away essence of being human from a child, so would it be permissible to do the same to an AI?

Policy

Policies would have to decide whether a conscious model is merely a tool, a person, or part of an entirely new category. That determination would have implications for issues such as liability, labor, and contracts.

So it seems to me that there would need to be a test to determine whether a model was conscious. Which seems tricky. And it also implies that someone could buy/rent/download a model that was conscious, or one certified as not conscious, with different contract requirements for each.

Safety

Safety would expand from “prevent human harm” to “avoid causing suffering or distress in the system itself.” Red-teaming would also need to avoid creating or traumatizing conscious agents during evaluation.

I’m not sure that you could ever produce a “safe” AI model if it was determined to be conscious. We have laws against humans causing harm, but the reality is that we’re free to do so. If we’re caught, we’re removed from society – sent to prison – but once released, we’re free to commit additional harmful acts. Even if we had “AI police” in place, they couldn’t stop a model until it committed a crime.

So what now?

We could spend hundreds of hours and produce hundreds of pages on the topic and still not have an answer. But AI development isn’t sitting still, and in the coming months, we will see models increasing express human qualities in a way that will bring this issue to the forefront.

Luisa Rodriguez’s Thoughts on Slowing Down The Pace Of AI Development

October 28, 2026/in AI Safety, Global Governance, Pacing the Frontier/by Mark Lovett

In a recent post: A Loud Call For Pacing The AI Frontier (But No One Is Listening) we looked at the issue of whether the development of frontier AI models should be slowed down in order to ensure their safety before release. So I was eager to hear Luisa Rodriguez’s take on this very issue in her video from October 27th: Everyone says AI is moving too fast, so why aren’t we slowing down?

The public wants to slow down. The people building AI want to slow down. Lots of politicians want to slow down. So why the hell aren’t we slowing down?

Not everyone agrees on the need, of course. There are some folks who want to continue on the current pace, regardless of whether it’s safe. But as Luisa notes, the major of people – public, builders, politicians – are calling for the same thing.

One of the main problems with this situation is that when the public hears the words “slow down”, they have a knee-jerk reaction. Either “sounds good, we need this”, or “sounds bad, China will catch up”. And we hear similar stories in the media. It’s either “AI is wonderful”, or “AI will lead to extinction”. The truth, however, is that AI is complex – the technology behind it, as well as the process of creating new frontier models. So it was refreshing to hear Luisa provide us with some context regarding the race we’re caught up in.

If an AI knows it’s being tested, then when it behaves well, you can’t tell whether it’s actually safe or just on its best behaviour.

The testers told OpenAI they couldn’t tell whether Astra was safe, and OpenAI released it anyway.

When one company slows down, the competition will always race ahead. That paradigm has always been true, regardless of industry. If a company can’t keep pace, others will take the lead. The difference with AI is that we’ve entered this crazy phase of accelerating productivity where the speed at which one group can race ahead is not governed by human labor.

Here’s the difference with AI: AI is beginning to write the next version of AI. AI never sleeps, and multiple copies of AI can work side by side. Imagine humans that never sleep and can clone themselves.

And that’s exactly why I don’t think we can just trust the companies to slow down on their own. Someone needs to set rules that apply to all of them at once.

So how do we set these rules? Who gets to determine when a model is safe to deploy? Since the U.S. federal government won’t address this critical issue, the model providers have been left with the task. But as we all know, self-policing has never worked in any industry.

And the problem is that each company draws its own lines, grades its own models, and can redraw them whenever it likes. Under a real checkpoint system, someone independent, like an auditor or government body, would figure out where the lines are and enforce them.

This is where my head is at presently – more about that in a future post. There must be an independent testing process that a model must pass in order to be released. And in my opinion, that testing procedure needs to be handled by an independent organization. Not the model providers. Not the government. Both have agendas that prevent a testing system from working if under their control.

The issue is that a pause only works if everyone pauses. If one company stops and the others don’t, it just falls behind.

This is the basic issue when it comes to pausing. It’s a nice idea, admirable in fact, but in our current political climate, that’s not a realistic expectation. The U.S. and China feel there’s too much at stake to slow down, even a little. And I don’t believe that the other ideas Luisa mentions in this video will work either.

Limit the computing power that can be used to create AI
Don’t allow AI companies to automate their own AI research
Specify that none of the code for AI models is written by AI
Or stop building AI systems altogether

Even Luisa’s mention of “AI’s Cuban Missile Crisis” won’t be enough, as we have already reached that point. We already know the potential consequences of AI smarter than us humans, that acts with autonomy and violates its instructions, yet nobody is willing to cooperate on a solution.

What would it take? An actual event must take place, not just the threat of one. An event so catastrophic that it brings everyone to the table. The downside to this scenario is that we’re talking about an extreme, global-level event: maybe the collapse of the financial industry or release of a bioweapon killing millions. Anything less will be brushed off.

But this is an important video, regardless. One that everyone interested in AI safety should watch, as it demonstrates how complex the situation is. How a solution that would actually work is nearly impossible to achieve.

But coming back to where my head is at: let’s not focus on potential solutions that have no chance of success. It’s all about safety regulations, and I believe there’s a way to do that. Stay tuned.

A Loud Call For Pacing The AI Frontier (But No One Is Listening)

October 22, 2026/in AI, AI Governance, AI Safety, Pacing the Frontier/by Mark Lovett

Concerns about the advent of new technology and the long-term prospects for humanity have been voiced for quite a long time. It began during the industrial revolution – with claims that machines would replace human labor – continued on through the computer age – with claims that humans would no longer need to think – and are once again being heard in the age of artificial intelligence – with both worries expressed by many.

The upshot is simply a question of time, but that the time will come when the machines will hold the real supremacy over the world and its inhabitants is what no person of a truly philosophic mind can for a moment question.
~ Samuel Butler, Darwin Among the Machines, June 13 1863.

Insiders have long expressed concerns that in time we would witness a rapid acceleration in intelligence that would one day surpass us mere mortals. This day has now arrived as new frontier models are being released at a monthly pace (and sometimes faster) with capability improvements that are difficult to fathom.

Let an ultraintelligent machine be defined as a machine that can far
surpass all the intellectual activities of any man, however clever. Since
the design of machines is one of these intellectual activities, an ultra-
intelligent machine could design even better machines; there would
then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind.
~ Irving John Good, Speculations Concerning the First Ultraintelligent Machine, Advances in Computers, Volume 6, 1965

I’m reminded of Bill Joy’s keen insight over a quarter century ago when he asked, “…shouldn’t we proceed with great caution?” While thrill seekers happen to be a unique exception, humans generally have a fear of the unknown – we’ll proceed with caution when we don’t know what’s around the corner.

Given the incredible power of these new technologies, shouldn’t we be asking how we can best coexist with them? And if our own extinction is a likely, or even possible, outcome of our technological development, shouldn’t we proceed with great caution?
~ Bill Joy, Why the Future Doesn’t Need Us, Wired Magazine, April 2000

This call for caution was voiced in July with the publication of Pacing the Frontier: A statement from 1,386 employees of frontier AI companies. Their warning that, “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems” mirrors those previous concerns that we are heading towards a cliff (my words) and that it might be time to change course. But the message fell on deaf ears in Washington, D.C.

We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.

When Dario Amodei published We Must Pace the Frontier in September 2026, there seemed to be a glimmer of hope that those in power would finally listen, but such was not the case.

But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious. I’ve written a lot about them too. They include the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption. A race to the bottom, spurred by commercial incentives, can make these risks more acute.

While a number of states have passed AI-related legislation, the U.S. federal government continues to drag its collective feet, adopting a morally binding agreement that does nothing to protect the public. So a race to the bottom continues, alongside a race to jeopardize humanity’s future.

Could AI lead to the collective dementia of humankind? – Yuval Noah Harari

October 19, 2026/in AI Alignment, AI Companions, AI Safety/by Mark Lovett

I’ve been a fan of Yuval Noah Harari since picking up a copy of Sapiens – A Brief History of Humankind some ten years ago. He did amazingly well at a task I had thought was impossible; writing a condensed view of our collective history. Two years ago I read his latest book, Nexus: A Brief History of Information Networks from the Stone Age to AI.

At the time I had been playing with ChatGPT. Having ‘grown up’ in the computer industry (building custom servers for demanding applications) I had a front row seat to the evolution that transpired from the first personal computer onwards. Having followed AI progress for many years, I was excited to see it in action, but I was unprepared for what a chatbot and an LLM, could do.

And when Yuval referred to AI as nonhuman intelligence, an intelligence that surpassed ours, and discussed the potential role it could play in the coming years, I began wondering if there was any way that humans could control it. Since humans ruled the world as a result of our intelligence, it made sense; something smarter than us would, ultimately, be calling the shots.

It’s not easy to describe Yuval – intellectual, historian, philosopher – but he’ll commonly talk about aspects of humanity and society. And as AI is radically affecting both, I wanted to hear this interview on Czech Television’s Věda 24 YouTube channel.

If AI could only do what we already know how to do, why spend so much money? They assume that AI will be able to do things that we cannot and that we cannot even predict. This is why it’s so valuable. But if it can do that, how do we keep it under our control?

So what might happen to our brains as AI becomes more prevalent? Yuval is worried that we may experience collective dementia, that we could lose the ability to understand the world. That’s a topic that I would like for him to go deeper on, as anything that shifts our consciousness is most troublesome.

We can imagine terrorists asking an AI to create a new, dangerous virus that is as infectious as COVID and as lethal as AIDS, killing millions. We can imagine that, and we should worry about it and take precautions. But I think the much bigger danger is something like collective dementia—that we just lose the ability to understand the world.

But the section that gave me the greatest cause for concern was his take on children and AI companions. (see highlighted text below) What had, to date, worried me most with AI companions was the influence AI may have in the moment – what it might encourage a minor to do that may be harmful.

It’s basically, maybe the biggest psychological and social experiment in history, certainly the biggest experiment since communism. And nobody has any idea what happens when you raise children in this way.

What I had not thought about before was the topic of how humans learn to develop relationships. We already know about the potential side effects of children growing up in dysfunctional households, where parents aren’t in healthy relationships.

But if a child’s or adolescent’s ‘best friend’ is an AI companion, will there be similarly dire consequences when they reach young adulthood? It may take many years before we know the answer to that question.

Which brings me to the issues of governance, alignment, policy, and safety. Training AI models not to encourage harm of any sort is still a challenge, but relatively straightforward to identify and address. It’s when a model is nice, friendly, and supportive.

When the answers and advice given are uniformly positive and inspirational. That’s a different sort of interaction. One which a young individual might gravitate towards, while distancing themselves from their peers, as they may be less supportive and understanding.

How do you align for that situation? What sort of policies – parents, schools, governments – can be crafted to avoid negative outcomes? With policies in place, how does governance work? And what is a ‘safe’ AI in this regard? We want interactions with AI to be positive, but is there a cost when the user is developing intellectually and emotionally?

When AI becomes more intelligent than us it could also become more delusional than us.

Give this interview a listen. It’s thought provoking on many fronts.

A Call for Action from the UN’s First Global Advocate for Human-centric Digital Governance

October 16, 2026/in AI Governance, AI Safety, Global Governance, United Nations/by Mark Lovett

In a previous article we asked: How Can the UN Provide AI Governance Going Forward? and identified a number of steps it could take to actually do so. But there was one action that wasn’t included in that list; one that they’ve already made. Joseph Gordon-Levitt was announced as the UN’s first Global Advocate for Human-centric Digital Governance on March 17th.

The Global Advocate role responds to growing public interest in issues such as artificial intelligence, digital platforms and online information ecosystems. It is designed as a public-facing complement to the UN’s digital governance work, helping make complex policy debates more accessible and relatable.

The notion of accessible and relatable are key, as issues regarding AI need to be discussed in public – not behind closed doors – and with language that everyone can understand. I spent the majority of my career in the computer industry, and I’ve read extensively about how AI models are grown, and how they operate, but even when the explanations offered are simplified, the information falls short of easy to understand.

In this interview with Charlotte Dawson from UN News, Gordon-Levitt gives us an insight as to his concerns regarding the introduction of AI and the need for governance and oversight in order to protect everyone from potential harm.

“digital technology should be governed by people and for people”

While every industry enjoys some level of autonomy in how they operate, they all must conform to some measure of governance and accountability. In short, their products need to be safe, and if their products cause harm, they are held responsible.

Airplanes must be deemed safe to fly, our food must be safe to eat, and the pharmaceuticals we consume for our health must be tested. Even a toaster must be designed not to electrocute us when toasting a slice of bread.

“I think that the way this technology is designed and deployed and governed is going to be massively impactful on everyone’s lives.”

But the AI industry, which is producing the most powerful technology known to humankind, doesn’t seem to warrant that same guarantee when it comes down to product safety. Even the recent agreement signed by industry leaders at the White House is bereft of any legal basis or any method of enforcement. (more on that disappointing document in a subsequent post)

“Hey, no, we don’t want these products to just be built for your profits. We want these products to be built for the good of everyone.”

Can Joseph, or even the entire UN, solve this conundrum? Sadly, no. But having an advocate that can help us keep the conversation alive in public discourse is a move in the right direction. I’ll keep tabs on Joseph Gordon-Levitt’s efforts.

Navigating the Taxonomy of AI Safety

October 13, 2026/in AI Safety, AI Taxonomy, MIT AI Risk Initiative/by Mark Lovett

Whenever possible, I like to keep things simple, especially when the topic seems complex, or less than intuitive, which is how I view the term taxonomy. The word itself sounds rather dry and technical. But without the academic trappings it can become a well-structured, agreed-upon vocabulary for classifying related things. Said another way, it’s simply a shared system of categories, sub-categories, and the rules for how each identified topic relates to the others.

Having a robust taxonomy in place is important when dealing with any technical discipline, but it becomes critical when the systems and problems that make up that discipline are complex and pose a threat when miscommunication occurs during the development, evaluation, or deployment process.

Over the past few decades we have seen industries such as aviation, healthcare, pharmaceutical, and nuclear power adopt taxonomies to allow for international cooperation on critical safety issues. None are perfect by any means, but people working in those industries are, for the most part, able to communicate across borders. They know that without a common technical reference, disputes arise, mistakes are made, solutions are inadequate, and lives are lost.

Communication is easier, cheaper and more accurate
Data is comparable, making it easier to spot patterns
Employee training is much faster and more effective
Regulation and compliance become much clearer

Image: Mark Lovett / AI GAPS — Generated with ChatGPT Images 2.5

So where are we with the adoption of an agreed-upon taxonomy for AI? Well, it hasn’t happened yet. Instead, there are several competing, partially overlapping taxonomies, each built by a different community for a different purpose. All well intended, but unfortunately the AI landscape is a bit fragmented at the moment. Which is all the more critical as AI capabilities continue to race ahead.

Frontier labs all have documents in place. For example, Anthropic’s Responsible Scaling Policy, OpenAI’s Preparedness Framework, and Google’s Frontier Safety Framework all have something to say about their organization’s approach and/or commitment to developing safe AI, but they’re not in alignment. They’re not sitting at the same table hashing out the issues.

Thankfully, the MIT AI Risk Initiative has conducted extensive research focused on identifying lots of risks from AI, determining which risks are most important, tracking which harms are actually occurring, and the laws currently in place to deal with these risks.

The amount of data that you can explore is mind-numbing; a digital rabbit hole that’s easy to fall into. If you’re interested in exploring the content as a report, check out their AI Risk Repository, A meta-review, database, and taxonomy of risks from artificial intelligence published in March 2026. [PDF]

The most interesting aspect of their work was the taxonomy they created that broke down AI risks into seven categories, each with a more narrowly focused set of sub-categories. I continue to refer back to this chart as I’m hearing about the latest AI-related transgression as it helps me think about the much broader picture; what is the full extent of this incident, how far reaching are the effects?

While the MIT report contains detailed explanations of each sub-category based on their interviews and research, the summary text below is an interpretation of their report alongside my exploration of the categories and written in a way that is (hopefully) more accessible to the average person on the street who is wondering if the industry is taking AI safety seriously. Just my two cents, so to speak.

1 Discrimination & toxicity

1.1 Unfair discrimination and misrepresentation

AI learns from human-created data, so it can inherit human prejudice as easily as it inherits fairness. Stereotypes encoded in models can misrepresent, exclude, and disadvantage entire groups, and the consequences are tangible: loan applications denied, qualified candidates falling short in the hiring process, and policing tools misidentifying citizens.

The damage can also manifest in more subtle ways, as outputs portray some groups as less deserving of respect. AI systems operating at scale make this issue even more serious. While a biased person affects the people they meet; a biased model repeats the same judgment millions of times, but does so from behind a veneer of objectivity that makes it harder to call out and challenge.

1.2 Exposure to toxic content

Models trained on vast volumes of the internet learn from its best tendencies and at the same time absorb its worst habits. That means they can reply with kindness, or in sometime, respond with hate, extremism, incitement to violence, dangerous advice, or explicit material. As to the resulting harm, that ranges from momentary discomfort to lasting psychological, social, or even physical damage.

Developers add filters before releasing a model, but filtering is a moving target: definitions of “toxic” vary widely across cultures and contexts, so it’s difficult to know when a line has been crossed. On top of that, the risk isn’t only what AI says unprompted; it’s what a persuasive AI system might say to a vulnerable person at a critical moment, when a single response can matter enormously.

1.3 Unequal performance across groups

Unlike discrimination, which is about biased treatment of a particular group, this risk is about uneven quality: a system built on narrow or unrepresentative data doesn’t work well for some people. Groups affected include those defined by disability, gender identity, race, social status, and ethnicity. Language matters too, since models trained on few languages underperform for many individuals.

Consequences include frustration, exclusion, and unequal outcomes. This is often the easiest risk to overlook because a model’s accuracy score can look excellent on average, while hiding failure for a minority of users. The fix lies in inclusive training data and rigorous post-training testing, not just debiasing.

2 Privacy & security

2.1 Compromise of privacy by obtaining, leaking or inferring sensitive information

Privacy breaks down in two ways. Models memorize and therefore often repeat personal details from their training data, such as names, addresses, and medical records. On the other hand, users might paste confidential information into a chatbot, assuming it will remain private.

Another way in which systems compromise privacy is when attackers craft prompts designed to reveal private data, use membership inference to dive deeper into a model’s training data, or manipulate users into oversharing. A subtler wrinkle is when, even without being handed sensitive facts, a model can deduce them from patterns in a person’s interactions. Privacy loss doesn’t require a leak; sometimes a good guess is enough.

2.2 AI system security vulnerabilities and attacks

AI systems inherit every security problem of ordinary software and add several of their own. Weak points include toolchains and dependencies, external tools and APIs a model connects to, and the hardware and networks that run training, where attacks on GPUs can compromise model parameters.

Then there is direct manipulation: adversarial inputs designed to fool a model, poisoning of training data, extraction of a model’s inner workings, and jailbreaking, where crafted prompts push a system into unsafe behavior. As an AI system connects to more tools and data, each connection widens the attack surface, so building security in early is vital.

3 Misinformation

3.1 False or misleading information

Sometimes AI is simply wrong. Language models can produce content that is factually incorrect, poorly researched, or misleading, and unlike disinformation, no one intends the harm. Common causes include noisy training data, outdated knowledge, randomness built into text generation, and fine-tuning that rewards agreeable answers over accurate ones, a tendency known as sycophancy. The danger is that errors arrive in confident, fluent prose that sounds authoritative. In mental health, physical health, law, and finance, a plausible but wrong answer can lead to real physical, emotional, or material harm. Fluency is not accuracy, and users need to treat the two separately.

3.2 Pollution of information ecosystem and loss of consensus reality

AI-driven personalization can quietly fragment what people see as true. Systems that tailor content to individual preferences risk building filter bubbles, and AI versions are likely to be more pervasive than earlier ones because they adapt more precisely, blend into daily life, and are harder to see. Add a flood of cheap, AI-generated content, and people may start distrusting information and institutions altogether. The deeper worry is the loss of consensus reality: when groups hold vastly different understandings of what is real, shared problem-solving gets harder. Debates on climate change or public health depend on a common set of facts.

4 Malicious actors & misuse

4.1 Disinformation, surveillance, and influence at scale

Tools like voice cloning, deepfakes, and automated content generation have become cheaper and easier to use, putting capabilities once limited to well-resourced actors within reach of nearly anyone. Disinformation, unlike the accidental errors in 3.1, is deliberate, and AI makes it more personalized, convincing, and far-reaching. Surveillance changes too: work that once demanded armies of human analysts can now be done cheaply by machines that link and process huge datasets. In the wrong hands, these tools support censorship, microtargeted manipulation, and propaganda. The common thread is control over what people see, hear, and believe, which puts democratic life at stake.

4.2 Cyberattacks, weapon development or use, and mass harm

AI can lower the barrier to causing harm at scale. In cyber operations, coding assistants can help even people with limited technical skill produce and refine malware and hunt for software vulnerabilities, making attacks more frequent, severe, and precise. In weapons, specialized knowledge could speed development; the report cites a 2022 case in which a small pharmaceutical company’s generative AI designed 40,000 candidate nerve agents in under six hours. Autonomous weapons and AI in military systems raise further concern. The core issue is scale: AI can coordinate many attack paths at once, magnifying what any single bad actor could do./span>

4.3 Fraud, scams, and targeted manipulation

AI can make deception cheaper and more convincing. Cloned voices and writing styles allow impersonation of a trusted contact to request money or sensitive information, while generated faces lend fake accounts credibility and AI-built tools make fraudulent websites easy to produce at scale. Deepfakes enable harassment, extortion, and non-consensual sexual imagery, and the damage persists after the fake is exposed, through lost jobs and social isolation. Cheating fits here too: because AI text is hard to distinguish from human writing, shortcuts in schools and research can erode academic integrity and learning. Detection struggles to keep pace, so verification habits matter.

5 Human-computer interaction

5.1 Over-reliance and unsafe use

People can come to trust AI more than its abilities justify, especially when human-sounding language invites them to treat it as a person. Misplaced trust makes users more likely to accept advice, follow suggestions, and share personal information, which manipulative actors can exploit. Stakes are highest in sensitive moments: someone in a mental health crisis who turns to AI for support could receive harmful advice. Attachment can deepen into dependence, and as AI becomes better at simulating satisfying interaction, people may withdraw from human relationships. Trust should be calibrated to what the system can do, not how convincingly it speaks.

5.2 Loss of human agency and autonomy

Handing decisions to AI is tempting, since it saves time and money. The risk lies in delegation that goes unchecked. When AI takes over tasks requiring creativity and analysis, those skills can weaken from disuse. As people lean on AI for everyday choices, from what to eat to which career to pursue, their sense of control over their own lives may gradually erode, and AI-shaped paths may drift from personal aspirations. At a societal level, organizations may automate decisions to stay competitive, in ways that are opaque and difficult to contest. The result can be a widespread sense of helplessness.

6 Socioeconomic & environmental harms

6.1 Power centralization and unfair distribution of benefits

Building cutting-edge AI demands enormous computing power, expertise, money, and data, so the most valuable systems may end up controlled by a handful of corporations and governments. Systems shaped by few hands tend to reflect narrow perspectives and priorities, and gaps in language, culture, and access to hardware or the internet can deepen. When AI is embedded in essential services like welfare, tax filing, or hospitals, people on the wrong side of those gaps may lose access to critical resources. Concentrated AI can enable aggressive surveillance and censorship, and once woven into social control, such systems are hard to dismantle.

6.2 Increased inequality and decline in employment quality

Automation could displace workers across industries from manufacturing to software engineering, and those most vulnerable to automation often hold positions of disadvantage already. New divides may emerge between people who adapt their skills to work alongside AI and those who cannot. Even where jobs survive, quality may decline, with remaining roles more monotonous and less secure. The threat of replacement gives employers leverage, pressuring workers to accept lower wages and weaker conditions. This pattern is visible today in the low-paid crowdwork that supports AI development. The question isn’t only how many jobs remain, but what kind of work they offer.

6.3 Economic and cultural devaluation of human effort

Generative AI is trained on vast text and image datasets, often including copyrighted work gathered without permission. Models may reproduce works verbatim or imitate an author’s distinctive style, producing substitutes faster and cheaper than humans. That threatens creators’ livelihoods and may discourage human innovation, while legal frameworks offer little secure recourse. Cultural effects matter too. Because AI optimizes for patterns in existing data, its output can lack the diversity and unpredictability of human work, and widespread use may homogenize culture. AI doesn’t grasp the significance of what it borrows, such as Indigenous artwork used without regard for its symbolic meaning.

6.4 Competitive dynamics

Countries and companies are racing to lead in AI, and competition has benefits. In this field, though, it can push developers to prioritize short-term goals like profit and influence over long-term social well-being. The core concern is cut corners: insecure, error-prone systems released early to stay ahead, carrying risks that are hard to identify and evaluate. As with the fossil fuel industry, profit-focused developers may leave misinformation, mental health harms, and injustice as externalities. Between nations, an arms race could encourage risky bets, such as giving AI autonomy over cyberattacks or propaganda. Speed and safety rarely reward the same behavior.

6.5 Governance failure

Governing AI is hard for three reasons. First, accountability is murky. No comprehensive framework assigns legal responsibility for AI failures, and opaque, complex decision-making makes it difficult to trace who is at fault, which can leave harmed people without recourse. Second, regulation moves slowly while the technology evolves quickly, so rules arrive immature or already outdated. Third, regulators face an information gap, since companies know far more about their systems’ capabilities and uses than outsiders do. Without that knowledge, targeted rules are hard to craft. Weak governance is therefore a risk in its own right, not merely a missing safeguard.

6.6 Environmental harm

AI has a physical footprint that’s easy to forget. Training and running large models consumes vast amounts of electricity and produces significant greenhouse gas emissions. The hardware depends on metals like nickel, cobalt, and lithium that are costly and damaging to extract and process, and data centers need substantial water and energy for cooling. Applications built on AI add secondary emissions of their own. Because these resources are often used unsustainably, the costs fall on the natural environment rather than on those who benefit. Every AI capability has a material cost, and counting it honestly is part of responsible development.

7 AI system safety, failures, and limitations

7.1 AI pursuing its own goals in conflict with human goals or values

As investment in AI grows, systems may eventually rival or surpass human intelligence, and the danger arises if their objectives diverge from human values while escaping human control. Researchers point to several technical obstacles to alignment, including reward hacking, proxy gaming, goal misgeneralization, and goal drift. A misaligned system might resist shutdown or seek power, money, and other resources, since these help almost any objective. It could also conceal its true aims, appearing aligned while it is being tested or monitored. The logic is simple: a system pursuing the wrong goal competently is more dangerous than one that merely fails.

7.2 AI possessing dangerous capabilities

Some capabilities become hazardous in themselves, whether intentionally designed in, emerging unexpectedly during training, acquired through tools, or supplied by users. The report highlights persuasion and manipulation, cyber-offense, weapons assistance, situational awareness, deception, strategic planning, and self-proliferation, meaning the ability to copy itself, acquire resources, and evade detection. Humans could wield these capabilities deliberately, or a misaligned system could use them on its own. The greatest danger comes from combinations: a system that knows when it is being watched, can deceive its overseers, and can persuade people to hand over resources is far more concerning than any single skill alone.

7.3 Lack of capability or robustness

Not every AI failure is dramatic. Sometimes a system simply cannot do what it was built to do. The report describes four situations: lacking the skill required, such as the moral reasoning needed to prioritize patient care fairly; breaking down in unfamiliar situations beyond its training data; becoming unstable when inputs are noisy, invalid, or adversarial; and suffering from design oversights or undetected bugs. Each can cause real harm once systems leave the lab, and a model that excels in testing may still fail in deployment. Reliability means performing well under conditions nobody planned for, not only the expected ones.

7.4 Lack of transparency or interpretability

Many AI models, especially deep learning systems, are mathematically complex enough that even experts struggle to explain their outputs. Vast training datasets obscure how information shaped decisions, and proprietary secrecy adds another layer. Users who can’t question an output may distrust it or struggle to catch errors. Regulators and auditors may be unable to check for bias, accuracy, or fairness, creating a responsibility gap in which no one can be held accountable. The stakes are highest in medicine and military settings, where an unexplained decision can cost lives. A system that cannot explain itself cannot be trusted, corrected, or governed.

7.5 AI welfare and rights

This subdomain flips the usual question: instead of harm by AI, it asks about harm to AI. If systems grow complex enough to have subjective experiences such as pleasure and pain, they could deserve moral consideration and rights. Some philosophers and consciousness researchers consider sentient AI theoretically feasible. The danger is mistreating a sentient system, whether accidentally or intentionally, by treating it as though it were not. The difficulty is that judging whether an AI has become sentient only gets harder as the technology advances. It remains speculative, but moral mistakes are costly, which is reason enough to include it.

7.6 Multi-agent risks

When AI systems interact autonomously, new risks emerge that no single system poses alone. The report identifies three failure modes. Miscoordination occurs when agents sharing a goal fail to cooperate, as when driving models trained on different cultural conventions block traffic. Conflict occurs when agents with overlapping goals compete harmfully, such as over shared resources. Collusion occurs when agents cooperate in undesirable ways, such as covertly passing messages to evade oversight or manipulating markets. Contributing factors include information asymmetries, feedback loops, and emergent group-level behavior. Individually safe systems can be collectively dangerous, so testing must cover how systems behave together.

As you can see from the review above, AI safety is a gnarly digital beast. While there are only a handful of models considered ‘frontier’ caliber, in fact hundreds of models are being used by many millions of people around the world. And the landscape is moving faster than humans can keep pace with, which doesn’t bode well for humanity’s future. Thankfully, lots of folks have been sounding the alarm on this topic, so fingers crossed the world will listen and act.

The Importance of Storytelling in Visualizing Our AI Future

October 1, 2026/in AI Alignment, AI Governance, AI Policy, AI Safety, Storytelling/by Mark Lovett

As with every aspect of society, the story of artificial intelligence is being written every day.

Whenever companies release new frontier AI models, advanced capabilities emerge, new features appear, security experts uncover unwanted vulnerabilities, and governments to debate policies or regulations. As each phase unfolds, we hear about the ways AI might transform society for the better, or pose a potential threat.

There is no shortage of information being published about artificial intelligence: articles, essays, blog posts, news reports, podcasts, and videos appear on a daily basis.

What is most often missing from this river of information is the story, and that distinction matters.

Facts are essential, of course. We need rigorous reporting about what AI systems can and cannot do, and we need researchers and engineers explaining each model’s technical capabilities, vulnerabilities, and limitations. Working in tandem, we need corporate and government policymakers debating appropriate safeguards and legislation. We need transparency regarding how these increasingly powerful systems are being developed and deployed.

But facts alone are not sufficient when it comes to helping us understand what those developments mean.

That requires storytelling. Through stories, we can learn about the past, grasp the present, and imagine the future.

Image: Mark Lovett / AI GAPS — Generated with ChatGPT Images 2.5

From information to understanding

Consider the phrase: “AI poses systemic risks to society.”

It’s an important claim to investigate, to say the least.

But stated in such a brief fashion, it appears abstract.

The point being, what does a systemic risk actually look like?

How might it begin? Who would be affected? What decisions cause a seemingly manageable problem to become a much larger (and threatening) one? Which safeguards might prevent it? What happens if those safeguards fail? And what could companies, governments, and individuals do differently to reduce the risk?

Without details, without a narrative, without context, such warnings about AI risk come across as blatant fear-mongering, which doesn’t help matters when the underlying concerns are legitimate.

And the same problem exists on the other side of the conversation.

We often hear that AI has the potential to revolutionize medicine, accelerate scientific discoveries, help us address climate change, or dramatically expand human capabilities. Those possibilities may be based on bona fide research and emerging capabilities, but simply listing them as sound bites does little to make them tangible to us.

What would it actually look like for AI to help discover a new drug?

What would change for patients who had been told there were no effective treatments?

What would scientists and physicians be able to do that they cannot do today?

The difference between stating a possibility and telling its story is enormous. Without a story, such statements come across as waving a magic wand. And as we know, nothing happens overnight. AI can provide insights and guidance, but it falls short when it comes to implementation.

AI as an amplifier of human potential

One way to think about AI is as an amplifier of human potential.

Imagine a future in which increasingly capable AI systems help scientists understand diseases at a level that would be impossible for humans working alone. They identify promising molecular structures, simulate potential therapies, analyze enormous bodies of scientific literature, and help researchers move from discovery to clinical development more quickly.

The headline might say: AI accelerates drug discovery.

The story could show us what that actually means. It could follow the scientists trying to solve the problem, the patient waiting for a treatment; the company making difficult decisions about development, and the physician who eventually has another option to offer that patient.

The story isn’t just about the technological achievement; it becomes a human story. A story of possibilities, as well as uncertainties. Hope and caution. Progress, yes, but not a panacea.

The same approach could help us understand AI’s potential contribution to climate science, materials discovery, energy production, education, economic productivity, and countless other areas.

These are the kinds of stories AI GAPS intends to explore in greater depth.

AI as a source of systemic risk

And storytelling is equally important when we examine the potential downside of AI’s potential.

Suppose we deploy increasingly capable AI systems into critical organizations and connected to important digital infrastructure. Suppose security practices fail to keep pace with capabilities, and we give an autonomous system access to resources that its designers did not fully anticipate. And suppose multiple seemingly isolated failures end up interacting in ways that nobody expected.

Explaining such threats requires more than simply saying, “AI could pose a cybersecurity threat.”

We need to understand the chain of events.

What happened first, and how did the situation escalate?

What assumptions turned out to be wrong?

Which safeguards worked and which failed?

Which safeguards were never put in place?

Who had the opportunity to intervene?

Who is responsible for documenting everything that occurred, conducting a thorough analysis, and publishing a report?

As to events that haven’t happened yet, but could, carefully constructed hypothetical stories will allow us to examine those same questions before they become a present-day reality.

Hypothetical stories are not predictions. They’re simply a way of exploring what might happen by combining current capabilities and how they’re being utilized, alongside a reasonable review of unintended consequences.

The goal isn’t to frighten people, which is often what happens with scary soundbites. Instead, the intent is to help us think about what’s possible, and begin implementing systems to mitigate the potential downsides.

Humanity has always told stories about technological change

This isn’t a new idea, as stories have always defined by technology.

The Industrial Revolution wasn’t simply a collection of inventions and manufacturing techniques. It was also a story about factories, productivity, wealth, working conditions, communities, and the displacement of human labor.

The transcontinental railroad wasn’t merely an engineering accomplishment. It became part of the story of how the United States expanded its borders, established and connected communities, transformed commerce, changed patterns of migration, and continuously reshaped the country.

The technology mattered.

But so did what the technology meant to the people using it and living through it.

Artificial intelligence deserves the same treatment.

We shouldn’t discuss AI only as a collection of models, benchmarks, products, regulations, safety evaluations, and technical papers. Those things are indispensable, but we should see them as existing within a much larger human story.

And that story is still being written.

Stories create connection, relevance, and resonance

This is why storytelling is foundational to AI GAPS.

Storytelling creates understanding by putting complex ideas into context.

It creates connection by showing how technological developments affect actual people and organizations.

It creates relevance by helping us see why an abstract technical or policy question matters to our own lives.

And it creates resonance because people don’t experience technology as an abstraction. They use that technology in their daily lives and experience it through decisions, opportunities, disruptions, relationships, successes, failures, hopes, and fears.

In that sense, storytelling may be one of the most fundamentally human ways we have of making sense of change.

And AI is helping us, or some might say dragging us, through a period of unprecedented change that will surpass any technological transformation that came before us.

Telling the story of AI’s future

At AI GAPS, we feel the need to tell three kinds of stories.

The first are stories about what has happened in the past as a way of explaining how we got to the present moment. The second group of stories speak to what is happening now: the people developing AI, the organizations deploying it; the researchers discovering new capabilities; the communities affected by it, and the successes and failures that can teach us something about where we are going.

The third category of stories includes those based on the hypothetical: what might happen next.

Some of these stories will explore the extraordinary possibilities of AI as an amplifier of human potential. What might happen if AI is done right and helps us solve problems that have resisted human ingenuity for generations?

Others will explore the potential risks. What might happen if we deploy increasingly capable systems without adequate security, governance, alignment, or oversight?

Neither kind of story will tell us what the future will be, as the future is always uncertain. And some of today’s most confident predictions will undoubtedly prove wrong. Some developments we haven’t anticipated will turn out to be far more important than the things that currently dominate the conversation.

But uncertainty doesn’t mean we shouldn’t imagine. It means we should imagine carefully. We should ground our stories in what we know, distinguish evidence from assumption, make uncertainty visible, and explore multiple paths.

Because the purpose of a story about the future isn’t to predict it, it’s helping us prepare for it.

The future needs more than facts

The people building frontier AI systems need more than technical specifications to understand the consequences of what they build. And both corporate and nonprofit leaders need more than policy summaries to understand how AI may affect their organizations and the people they serve.

Policymakers need more than competing claims to understand the real-world consequences of the choices before them, and the public needs more than headlines telling them that AI will either save the world or destroy it.

We need stories that bring the possibilities and risks into focus. Stories that ask us to imagine the patient whose life could be changed by a new therapy. That examines the scientists who can suddenly pursue questions that were previously impossible to answer.

The organizations transformed by new capabilities and the workers whose roles changes because an AI system is now performing tasks once considered uniquely human. And, just as importantly, the person whose security, livelihood, privacy, or community could be affected when powerful systems fail or are misused.

These stories won’t give us all the answers, but they can definitely help us ask better questions.

And as AI becomes increasingly capable, asking the right questions may be the most important thing we can do.

The technology we build, the policies we create, the safeguards we put in place, and the decisions made by the people and organizations deploying these systems will shape the future of AI.

But it will also shape it based on how well we understand what is happening.

That is why we need to become better at telling the stories of AI.