What is Reinforcement Learning from Human Feedback (RLHF)

Definition

Reinforcement learning from human feedback (RLHF) is a training method that aligns model behaviour with human preferences by having people rank or rate model outputs, using those judgments to train a reward model that then guides the model toward more helpful, honest, and harmless responses.

« Back to Glossary Index

Aligns model outputs with human preferences and values
Improves helpfulness, honesty, and safety beyond raw pre-training
Captures nuanced quality judgments hard to encode as rules
Underpins the usefulness of modern conversational assistants

Real World Example

A lab improves its assistant with RLHF: annotators rank candidate responses, a reward model learns those preferences, and the assistant is tuned to produce answers people consistently prefer, making it markedly more helpful and safe.

FAQs

How does RLHF work?

Humans rank model outputs, those rankings train a reward model, and reinforcement learning then optimises the model toward higher-rewarded responses.

Why is RLHF used?

It aligns models with nuanced human preferences for helpfulness and safety that are difficult to specify with explicit rules.

What are the limitations of RLHF?

It depends on the quality and consistency of human feedback and can introduce the biases of its annotators.