What is Fine-Tuning

Definition

Fine-tuning is the process of further training a pre-trained model on a smaller, task-specific dataset so it adapts its behaviour to a particular domain, style, or task, improving performance beyond what prompting alone can achieve for specialised needs.

« Back to Glossary Index

Specialises a general model to a specific domain, task, or style
Improves accuracy and consistency on narrow tasks beyond prompting
Reduces prompt length by baking behaviour into the model itself
Leverages a pre-trained base, needing far less data than training from scratch

Glossary: AI Studio

Real World Example

A healthcare company fine-tunes a base language model on its anonymised clinical notes so the model reliably uses the correct medical terminology and formatting that generic prompting could not consistently produce.

FAQs

How does fine-tuning differ from prompting?

Prompting steers a fixed model at query time, while fine-tuning updates the model's weights on task-specific data to change its default behaviour.

When should you fine-tune?

When you need consistent specialised behaviour, have quality task data, and prompting alone cannot reach the required accuracy or style.

Does fine-tuning require lots of data?

Far less than training from scratch, but it still needs a representative, high-quality dataset for the target task.