Multimodal LLM
An LLM that ingests (and potentially generates) more than just text — images, audio, video. Standard in frontier models from 2024 onward.
Typical architectures: (1) early fusion — every modality becomes tokens mixed into the same transformer (GPT-4o, Gemini), (2) late fusion — a separate vision encoder (CLIP-style) produces embeddings projected into the LLM's input space (LLaVA, IDEFICS, Claude). Uses: reading screenshots, code-reviewing designs, video summarisation, "describe this for accessibility". Audio fusion enables voice assistants without a separate speech-to-text step.