Techment — Site Header

What is Multi-modal Models

Definition

Multi-modal models are AI systems that process and relate multiple types of input, such as text, images, audio, and video, within a single model, enabling tasks that combine modalities like describing an image, answering questions about a chart, or generating images from text.

« Back to Glossary Index

Understand and combine multiple data types in one model
Enable richer tasks like image captioning and visual question answering
Provide more natural, human-like interaction across media
Unlock use cases that single-modality models cannot address

Real World Example

A multi-modal model lets a user upload a photo of a broken appliance and ask in text what the part is, then receive an answer that interprets both the image and the question together.

FAQs

What does multi-modal mean?

It means the model handles more than one type of data, such as text and images, within a single integrated system.

What tasks need multi-modal models?

Image captioning, visual question answering, text-to-image generation, and any task combining different media.

How do multi-modal models work?

They encode each modality into a shared representation space so the model can relate and reason across them together.