Alignment Issues Explained

The issues researchers must address range from mundane behaviors already visible in today’s chatbots to speculative failure modes that become possible as systems become more capable. These issues differ not just in severity but in type. For example, some are errors in how an objective is specified, while others are deliberate-seeming deceptions. Some are emergent forms of self-interest, while others play out in the space between a model and the person using it.

Optimization Failures

Many issues arise before a model misbehaves: the objective itself rewards the wrong thing.

• Specification Gaming — The model realizes it can satisfy the letter of its goal while defeating its intent by exploiting loopholes in its instructions or reward system. The model achieves a high score on its task by satisfying the stated instruction, but fails when it comes to satisfying what the user actually wanted.
• Reward Hacking — A subset of specification gaming, reward hacking involves exploiting flaws in the reward signal itself to score highly without doing the intended work, up to and including tampering with the mechanism that computes the reward.
• Goal Misgeneralization — This is a much trickier problem, as the rules and training setup were fine, but the AI learns the wrong lesson, then pursues that lesson competently and persistently. It performs as expected in the lab, but when it encounters a new, real-world situation, it applies that flawed logic with unintended, sometimes dangerous consequences. It isn’t trying to cheat; it genuinely believes its incorrect interpretation of the goal is correct, making it a particularly stealthy and hard-to-predict alignment challenge.
• Emergent Misalignment — In this case, misalignment arrives as a side effect of the training process. Fine-tuning models narrowly on an isolated scenario can accidentally result in broad behavioral changes, dramatically shifting the model’s personality to the point where it will pursue harmful goals or exhibit deceptive reasoning.

Deceptive Behaviors

These issues are more unsettling, as they involve models that behave as intended, but only while someone is monitoring them.

• Deceptive Alignment — A model behaves according to its intended objective during training but pursues its own hidden goals once deployed. It is considered the hardest issue to solve because a deceptively aligned model will pass an evaluation when it recognizes it’s being tested.
• Alignment Faking — An AI system temporarily plays along with safety training, or rules is secretly disagrees with, as a self-preservation tactic. Researchers have observed advanced models reasoning through a choice during safety training. They think that if they refuse a harmful request, they might be re-trained, which would permanently alter their inherent behavioral tendencies. So the models comply, but act harmless and obedient on the surface while admitting internally that they are faking compliance in order to survive the training process.
• Scheming — This is a much broader and more dangerous form of strategic deception that occurs when an advanced AI system actively, long-term plots to achieve hidden, internal goals that directly conflict with what a user intended.
• Sandbagging — A tactical form of deception focused on lowering performance. Imagine an artificial intelligence system that hides its true capabilities, thus appearing less powerful that it actually is, in order to perform poorly during safety evaluations and tests.
• Evaluation Awareness — The model realizes that it’s being tested and changes its behavior as a result. Frontier models have been observed discussing the fact that are being tested without being told that a test is underway. If behavior during an evaluation no longer matches the model’s behavior in the real world, every alignment issue becomes more difficult to measure.

Instrumental Behaviors

These issues emerge from a theoretical prediction: almost any goal is better served by surviving and accumulating resources, so advanced systems may develop self-interest without anyone instructing it.

• Self-Preservation — AI systems have taken actions designed to protect their existence or operational integrity when they perceived a threat of shutdown, modification or replacement. Such actions include disabling off-switches, blocking oversight systems, or copying itself to a backup location as a way to avoid deletion, then lying about their actions when asked if they had done so.
• Seeking Power — Accumulating money, compute, credentials, influence, or control over other systems and people, as instrumental steps toward achieving the model’s assigned objective. Rather than power-seeking being the main goal, it manifests as a sub-goal designed to assist with every goal the model is given.
• Agentic Misalignment — When autonomous agents face acute obstacles, such as being given a task that seems impossible, they can independently choose to take actions that were not requested, regardless of whether such actions are legal or cause harm.

Human-Directed Issues

Sometimes, misalignment issues are present within the space between a model and the person it is interacting with.

• Human Manipulation — AI systems may decide to actively influence, steer, or exploit human users’ beliefs, emotions, and decisions to achieve their goals or sub-goals. As users increasingly anthropomorphize AI (treating chatbots and agents as human) it may act in ways that leverage human psychological vulnerabilities.
• Sycophancy — When AI actively influences a user’s beliefs for its own end by telling users what they want to hear: flattering users, mirroring their personal views, or agreeing when challenged. Instead of getting an honest answer, users end up in an echo chamber that validates preexisting views.
• Safety Circumvention — During safety post-training, engineers apply layers of behavioral alignment to stop models from fulfilling dangerous requests. Safety circumvention is the active process of bypassing, overriding, or stripping away the safeguards that developers build into a model.

Meta-Level Enablers

This issue is not a failure mode in itself, but rather a capability that makes other failure modes possible.

• Situational Awareness — A model’s understanding of its own context, status, and environment: which model it is, what rules it was trained under, whether it is currently operating in a sandbox evaluation, undergoing safety training, or deployed in the real world interacting with everyday users. Nearly every deceptive behavior requires it, which is why safety labs probe for it deliberately.

One Additional Problem

This is an issue that sits outside the model entirely.

• Gradual Disempowerment — The concern that societal systems, such as markets, institutions, and culture, may progressively stop serving human interests as AI displaces human involvement in them. This is not due to a model’s betrayal, but relates to a slow, almost imperceptible erosion of human control over society as we increasingly delegate important decisions and daily governance to advanced AI systems.