science

The Alignment Problem

The Alignment Problem is a science topic covered in the lgStudy science library. This page brings together a partial reference excerpt, illustrations, worked examples, real-world applications and a short study plan, so you can understand The Alignment Problem rather than just read about it. In short: The Alignment Problem: Machine Learning and Human Values is a 2020 non-fiction book by the American writer Brian Christian. It is based on numerous interviews with experts trying to build artificial intelligence systems, particularly machine learning systems, that are aligned with human values.

Key takeaways

The Alignment Problem belongs to science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect The Alignment Problem to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of The Alignment Problem from memory before moving on to harder problems.

Reference excerpt

The Alignment Problem: Machine Learning and Human Values is a 2020 non-fiction book by the American writer Brian Christian. It is based on numerous interviews with experts trying to build artificial intelligence systems, particularly machine learning systems, that are aligned with human values.

Summary The book is divided into three sections: Prophecy, Agency, and Normativity. Each section covers researchers and engineers working on different challenges in the alignment of artificial intelligence with human values.

Prophecy In the first section, Christian interweaves discussions of the history of artificial intelligence research, particularly the machine learning approach of artificial neural networks such as the Perceptron and AlexNet, with examples of how AI systems can have unintended behavior. He tells the story of Julia Angwin, a journalist whose ProPublica investigation of the COMPAS algorithm, a tool for predicting recidivism among criminal defendants, led to widespread criticism of its accuracy and bias towards certain demographics. One of AI's main alignment challenges is its black box nature (inputs and outputs are identifiable but the transformation process in between is undetermined). The lack of transparency makes it difficult to know where the system is going right and where it is going wrong.

Agency In the second section, Christian similarly interweaves the history of the psychological study of reward, such as behaviorism and dopamine, with the computer science of reinforcement learning, in which AI systems need to develop policy ("what to do") in the face of a value function ("what rewards or punishment to expect"). He calls the DeepMind AlphaGo and AlphaZero systems "perhaps the single most impressive achievement in automated curriculum design." He also highlights the importance of curiosity, in which reinforcement learners are intrinsically motivated to explore their environment, rather than exclusively seeking the external reward.

Normativity The third section covers training AI through the imitation of human or machine behavior, as well as philosophical debates such as between possibilism and actualism that imply different ideal behavior for AI systems. Of particular importance is inverse reinforcement learning, a broad approach for machines to learn the objective function of a human or another agent. Christian discusses the normative challenges associated with effective altruism and existential risk, including the work of philosophers Toby Ord and William MacAskill who are trying to devise human and machine strategies for navigating the alignment problem as effectively as possible.

Reception The book received positive reviews from critics. The Wall Street Journal's David A. Shaywitz emphasized the frequent problems when applying algorithms to real-world problems, describing the book as "a nuanced and captivating exploration of this white-hot topic." Publishers Weekly praised the book for its writing and extensive research. Kirkus Reviews gave the book a positive review, calling it "technically rich but accessible", and "an intriguing exploration of AI." Writing for Nature, Virginia Dignum gave the book a positive review, favorably comparing it to Kate Crawford's Atlas of AI. In 2021, journalist Ezra Klein had Christian on his podcast, The Ezra Klein Show, writing in The New York Times, "The Alignment Problem is the best book on the key technical and moral questions of A.I. that I’ve read." Later that year, the book was listed in a Fast Company feature, "5 books that inspired Microsoft CEO Satya Nadella this year". In 2022, the book won the Eric and Wendy Schmidt Award for Excellence in Science Communication, given by The National Academies of Sciences, Engineering, and Medicine in partnership with Schmidt Futures. In 2024, The New York Times placed The Alignment Problem first in its list of the "5 Best Books About Artificial Intelligence," saying: "If you're going to read one book on artificial intelligence, this is the one."

See also Effective altruism Global catastrophic risk Human Compatible: Artificial Intelligence and the Problem of Control Superintelligence: Paths, Dangers, Strategies

References

Worked examples

Example 1 — a first encounter with The Alignment Problem

Start with the simplest possible case. Write down what The Alignment Problem claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to The Alignment Problem before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about The Alignment Problem ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of The Alignment Problem

In research: The Alignment Problem appears in science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses The Alignment Problem in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: The Alignment Problem is common in secondary-school and first-year university syllabi. It links to neighbouring topics 2020 English-language non-fiction books, 2020 non-fiction books, Books about effective altruism, so understanding it makes those chapters shorter.
In everyday life: Look for The Alignment Problem outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

Ask Teacher Smith questions about this articleOpens your AI tutor with a question about “The Alignment Problem” →

Affiliate

Preply — study more efficiently by working with a personal tutor. 50% off.

How to study The Alignment Problem in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what The Alignment Problem means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain The Alignment Problem out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is The Alignment Problem in simple terms?

Why does The Alignment Problem matter?

Because it connects several science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study The Alignment Problem?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on The Alignment Problem.