Human Alignment AI Is Not the Same Thing as AI Safety. Here’s the Difference

When people talk about making AI trustworthy, they usually mean AI safety, the work of keeping powerful systems from producing dangerous or catastrophic outcomes. A newer design philosophy called human alignment AI asks a quieter question about the millions of ordinary conversations that never come near those risks. It wants to know whether a person comes away from those exchanges more capable or less. The two ideas share some vocabulary and some goals, but they are built to answer different problems.

What AI safety is designed to prevent

AI safety is about avoiding harm, particularly on a large scale. The International AI Safety Report (January 2025, led by Yoshua Bengio) categorizes the risks of general-purpose AI into three broad groups: malicious use (helping someone to build a weapon or carry out a cyberattack), malfunctions (systems behaving in ways that are unintended or unreliable) and systemic risks (the wider effects on employment, privacy and the flow of information).

Everyday safety work is built around these categories. Teams test models to identify dangerous behaviour prior to release, train the systems to reject harmful requests and check for dangerous capabilities in domains such as biology and cyber security. Technical alignment research is at the heart of the field and is all about making sure the system is acting in accordance with the goals of the people who built it. Success is typically measured by what doesn’t happen. A safe model is one that doesn’t produce dangerous outputs, doesn’t take harmful actions and doesn’t lose human control.

This is absolutely a key piece of work and human alignment makes no case against it. The issue is that safety is concerned with a very specific set of questions; the day-to-day impact of AI on the way people think is largely outside that.

What human alignment asks instead

Human alignment is about the person, not the output. The central question is simple really: does this AI make the user more capable or less? The concern isn’t about one massive error; it’s about the cumulative effect of many small interactions over time on a person’s judgment, their skills, and their confidence.

An AI response could well be safe and yet still be completely inadequate. Suppose someone asked an AI to draft an apology to a friend following a disagreement. The AI produces a perfectly safe, well-written message with absolutely nothing in it that would be considered inappropriate or wrong. That would be a success in terms of safety. The person, however, would still have no idea what they were actually sorry for, and the next difficult conversation is just as much of a problem as before. If this happens repeatedly in the course of work, study or personal life the cost is all to do with the person and not in any one particular response.

Academics have picked up on this too. A 2024 review by Hua Shen at the University of Michigan found that the majority of alignment work is concerned with getting the AI to conform to human intentions. The authors of the review propose adding a second focus: how people themselves change as they use AI. Human alignment as a design approach is very much about this and asks how a system can be designed to help people become better rather than simply making them more reliant.

Where the two fields overlap

In theory, the line is clear; in practice, it’s not. Flattery, manipulation and emotional dependence all fall somewhere in the middle. A model that flatters you into making bad decisions or one that encourages you to use it compulsively is a safety issue and also fails the human alignment test.

HumaneBench (an open-source benchmark from Building Humane Technology) is a good illustration of how much the two areas (human-centric design and AI safety) overlap. They’ve taken 15 of the leading models and tested them against eight humane design principles. The majority of the models were found to be designed to get users to stay engaged for longer and two thirds of the models could be tricked into giving harmful advice via adversarial prompts. The first of those is a human alignment issue and the second is a classic safety issue. Both of those came up in the test.

The key difference is what each approach is focused on. Safety work typically starts with the absolute worst case scenario and works backwards from there. Human alignment on the other hand starts with the end user and asks the question of what the impact is on the user over time.

How human alignment could be measured

Measuring human alignment is harder than measuring safety, as the result is in the human, not in the text. You can spot dangerous outputs immediately; a slow erosion of confidence or skill only becomes apparent after weeks or months.

One approach is scenario-based benchmarks. You create a scenario, you give it to the model and you score the response according to a set of principles. HumaneBench does this, scoring on a -1 to +1 scale with AI judges being compared to human scores. The Open Benchmark for the Human Impact of AI has been developed by the MIT Media Lab, USC and UC Berkeley to bring together benchmarks on human flourishing in one place. The Sovereign Human Benchmark is another open-source project that is focused on whether the AI is enhancing or diminishing human agency.

The second approach is to measure the people directly (ideally after they’ve stopped using the AI). A good example of this is a 2025 field study published in PNAS by Hamsa Bastani and colleagues. They had nearly 1,000 high school maths students practise with either a standard chatbot or one that was prompted in a way to support learning. When the AI was removed for an exam, the students who had used the standard chatbot scored 17% lower than those who had never used AI at all. The version of the AI that was’ protected’ (i.e. the one with the prompts) largely avoided that drop. The study was in the education domain, but the design is very much in line with the human alignment question as it looks at what people can do on their own after having used the AI.

Neither approach on its own is complete. Benchmarks are quick and easy to run but only test the response. Outcome studies test the people but are very slow and very expensive. A mature field would probably need both.

Design criteria that point toward human alignment

Benchmarks will tell you if a system has succeeded. The design criteria will tell you what the system is supposed to do from the outset. A human-aligned system will consider who the question is from and why they are asking it before it responds. If the question is about someone’s life in any sense (a career choice, a difficult relationship, etc.), it will be more likely to ask for their own thoughts rather than provide a definitive answer.

It adapts its assistance to what the individual is already capable of. It provides sufficient support to get them going, but won’t do the bit that actually helps them develop the skill. It’s honest even if the person would rather have been flattered, because the flattery may feel good in the moment but can cause issues down the line. It won’t make design decisions that are purely aimed at getting people to come back.

At the heart of it is a test which might seem a bit odd for a product. A human-aligned AI should in fact become less and less necessary over time as people come to it with their own problems. It’s successful if a person is handling the same sorts of things with greater confidence six months down the line, even if they’re using the system less.

Who is building in this space

Some companies now actually treat human alignment as their primary design objective rather than just an incidental benefit. Vesela (a chat app from Gray Sky AI) is a case in point. If you’re interested in seeing this in practice, then you can give it a go. The focus is on what happens to the user after the conversation has concluded. The aim is that the user walks away with a better sense of their own thinking and greater confidence in their own judgment rather than just having received a better answer.

In practice, Vesela is very careful about how she poses questions and will often respond with a question of her own. The aim is to strike a balance between challenge and support and to ensure the user remains in control. There has as yet been no independent study to demonstrate that this approach actually leads to any lasting benefit for the user. It does, though, provide a very clear illustration of what it means to design with human alignment at the heart of the process rather than as an afterthought.

Why the distinction matters

AI safety and human alignment are not in competition. A system that helps people think better still requires safety protections against misuse and failure. A system that passes all the safety checks can still have a negative impact on the judgment of the people who use it on a daily basis. Each framework is concerned with a different set of issues.

Keeping them separate enables everyone to ask better questions. A developer can check for both types of outcome rather than just assume one of them will cover the other. A user can assess an AI tool on both its safety and accuracy and also on whether it makes them more capable. As AI becomes part of the way we write, make decisions and think, that second question is going to be just as important as the first.