Multimodal

Multimodal describes a model that handles more than one kind of input or output — typically text plus images, and increasingly audio and video. A multimodal model can be shown a photograph of a damaged part, a scanned form or a screenshot and asked about it in words.

Commercially this removes a whole category of preprocessing. Extracting fields from a scanned invoice used to mean a specialised document pipeline; a capable multimodal model reads the image directly and returns structured data.

The usual caveats apply harder rather than less: an unreadable region produces an invented value rather than an error, so any pipeline built on this needs confidence handling and a human path for the documents that do not parse cleanly.