Large Multimodal Models (LMMs) vs LLMs
Evaluate LLMs and LMMs by comparing their benchmark scores and real-world latency by clicking the model’s name in the table below. You can also weigh their input and output pricing to judge overall efficiency and value.
Open-weight large multimodal models
*Audio is native on the E2B, E4B and 12B models only.
Gemma 4 (Google DeepMind)
Available in five sizes: E2B, E4B, 12B, 26B A4B, and 31B. The family processes text and images with variable aspect ratios and resolutions across all models. It also supports video and audio, with audio featured natively on the E2B, E4B, and 12B models.1
The 12B variant is the most architecturally notable. Google replaced the vision encoder with a lightweight embedding module and removed the audio encoder entirely, projecting the raw audio signal into the same dimensional space as text tokens. With no multimodal encoders in the pipeline, latency drops and the model becomes small enough to run on a laptop GPU with 16GB of memory.
Kimi K3 (Moonshot AI)
A 2.8T-parameter model built on Kimi Delta Attention and Attention Residuals, with native vision capabilities and a 1-million-token context window. Moonshot describes it as the first open model in the 3-trillion-parameter class.
Its vision encoder, MoonViT-V2, was trained from scratch through the model’s next-token prediction objective and was not initialized from a contrastively pretrained vision model such as SigLIP.2
MiniMax M3
A native multimodal model with roughly 428B parameters and 23B activated, trained on mixed modalities (text, image, and video). MiniMax presents it as the first open-weight model to combine frontier coding performance, a 1M-token context window, and native multimodality in a single system. It can also operate a desktop computer from screenshots.
Figure 1: Minimax M3 benchmark results.3
GLM-5.3-Flash (Z.ai)
The first native multimodal model in the GLM-5 series, with 320B total parameters and 18B activated, and the first open-source frontier model to combine sparse and linear attention.
Z.ai reports that the hybrid design reduces attention computation and KV cache by 3.01x and 4.44x against GLM-5.3.4
DeepSeek-V4.1-Flash
DeepSeek-V4.1-Flash is the smallest model in DeepSeek’s architecture family and the first in the line to support native visual understanding.
It uses a 552-billion-parameter Mixture of Experts design with a Causal Encoder-Decoder architecture, drawing eight billion active parameters for input and 16 billion for output. It replaced both V4 Flash and the experimental V4 Flash Vision Exp, whose model names now route to the new endpoint for compatibility.5
Proprietary large multimodal models
GPT-6 Astra (OpenAI)
OpenAI positions Astra as state-of-the-art on computer use, browsing, software engineering, cybersecurity, science, and professional work, with a 98% score on FrontierMath Tier 4 and 99.9% on ARC-AGI-3.6
It is the company’s first model to reach the Critical level of cybersecurity capability under its Preparedness Framework, and the publicly available version consequently declines advanced cyber work. The model accepts text and images and returns text.
Gemini Omni (Google DeepMind)
Gemini Omni is Google’s family of natively multimodal generative models, and its first release is Gemini Omni Flash.
Google describes it as a model that can create anything from any input, beginning with video: users supply any combination of images, audio, video, and text, and receive video grounded in Gemini’s real-world knowledge, which they can then revise through conversation rather than by regenerating from scratch.7
Qwen3.8-Omni-Flash (Alibaba)
Qwen3.8-Omni-Flash accepts text, image, audio, and video within a single 1M-token context, handles up to an hour of continuous audio and video per call, and returns text.
Given a video and a question, it identifies what merits further investigation, searches multimodal sources across the web, and produces an illustrated research report. Users can access it only through the hosted API, with no open weights at launch.
Claude Opus 5 (Anthropic)
Anthropic’s Opus 5 model is positioned as approaching the frontier intelligence of Claude Fable 5 at half the cost. On coding and knowledge work evaluations, including Frontier-Bench and GDPval-AA, Anthropic reports it as state-of-the-art, while noting that it trails Mythos 5 on cybersecurity tasks.
Pricing is $5 per million input tokens and $25 per million output tokens, same as Opus 4.8, with a Fast mode running at roughly two and a half times the default speed for double the base rate.
Figure 2: Claude Opus 5 benchmark results.8
Grok 4.6 (xAI)
Built on Grok 4.5, it focuses on long-running agents and more ambitious interactive and visual work, holding a task across many steps whether researching a topic, working across a codebase, or turning an idea into a finished application.
On xAI’s published evaluations, Grok 4.6 reaches 69.9% on CursorBench v3.2, ahead of GPT-5.6 Sol Max at 67.2% and just behind Fable 5 Max at 70.5%, and posts the highest score of the four models compared on Harvey LAB at 15.8%. Pricing starts at $2 per million input tokens and $6 per million output tokens, with a fast variant at twice that rate.9
Muse Spark (Meta)
Muse Spark is the first model in Meta Superintelligence Labs’ Muse family. Meta describes it as a natively multimodal reasoning model that supports tool use, visual chain of thought, and multi-agent orchestration.
The model integrates visual information across domains and tools and performs well on visual STEM questions, entity recognition, and localization. Use cases include building small games or troubleshooting a household appliance with annotations drawn over what the camera sees.
For health, Meta worked with more than a thousand physicians to curate training data, and the model produces interactive displays explaining information such as the nutritional content of foods or the muscles engaged during a given exercise.
Amazon Nova 2 Omni
Amazon Nova 2 Omni processes text, images, video, and speech inputs while generating both text and images, handling up to 750,000 words, hours of audio, long videos, and hundred-page documents in a single pass.
As of late July, the model is in early access for Nova Forge customers.10
What are the differences between LMMs and LLMs?
How do LMMs work?
Large multimodal models share similarities with large language models in training, design, and operation. They use the same transformer architecture and training strategies. Large multimodal models are trained on:
- Text data
- Millions or billions of images with text descriptions
- Video clips
- Audio snippets
- Other input data, like code
This training involves simultaneous learning of multiple data modalities, enabling the model to:
- Recognize a photo of a cat
- Identify a word in an audio clip
- Understand concepts and sensory details beyond text
This way, users can upload:
- An image to:
- Get a description of what’s going on
- Use the image as part of a prompt to generate text or images
- Ask follow-up questions about specific elements of the image
- Translate the text of the image to a different language (e.g, Menu)
Figure 3: Uploading an image of a cat on ChatGPT to describe it.
- Charts and graphs to:
- Ask complicated follow-up questions about what they show
- Design mockup to:
- Get the HTML and CSS code necessary to create it.
Figure 4: Prompting the image in Wes Anderson movie style. ChatGPT feeds the prompt into an image generation model (like DALL·E), which interprets the request and produces the styled image.
After training, models might incorporate unhealthy stereotypes and toxic ideas. To refine them, techniques like:
- Reinforcement learning with human feedback (RLHF)
- Supervised AI models
- Red teaming (testing the model’s robustness) can be used.
Figure 5: ChatGPT rejects my request due to content policy guidelines to protect copyrights.
The goal is to develop a functional multimodal system capable of handling:
- Text-to-image synthesis
- Image captioning
- Text-based image retrieval
- Visual question answering.
Add as preferred sourceCite this research
Pick the format that matches where you're publishing. Pasting the link version into your CMS preserves the backlink.
@misc{dilmegani2026,
author = {Dilmegani, Cem and Ermut, Sıla},
title = {{Large Multimodal Models (LMMs) vs LLMs}},
year = {2026},
month = sep,
howpublished = {\url{https://aimultiple.com/large-multimodal-models}},
note = {AIMultiple. Retrieved September 21, 2026}
}Results and timestamps of 19 data points. Download the summary data shown in this article's charts and tables as a ZIP file containing 3 CSV files.
Want the granular data behind it? Join Premium
Changelog
22 updatesReplaced the Qwen3-VL model entry with Qwen2.5-VL in the products section.
Added a Gemini 3 section and replaced the Qwen2.5-VL entries with Qwen3-VL model descriptions.
Added Video-first multimodal foundation models, Mistral 3 multimodal frontier release, and Open source MoE vision-language models to the section on Recent advancements in multimodal models.
Reference Links
Cem's work at AIMultiple has been cited by leading global publications including Business Insider, Forbes, Morning Brew, and Washington Post, global firms like Deloitte and HPE, NGOs like World Economic Forum, and supranational organizations like European Commission. [1], [2], [3], [4], [5]
Throughout his career, Cem served as a tech consultant, tech buyer and tech entrepreneur. He advised enterprises on their technology decisions at McKinsey & Company and Altman Solon for more than a decade. He also published a McKinsey report on digitalization.
He led technology strategy and procurement of a telco while reporting to the CEO. He has also led commercial growth of deep tech company Hypatos that reached a 7 digit annual recurring revenue and a 9 digit valuation from 0 within 2 years. Cem's work in Hypatos was covered by leading technology publications like TechCrunch and Business Insider.
Cem regularly speaks at international technology conferences. He graduated from Bogazici University as a computer engineer and holds an MBA from Columbia Business School.
She previously worked as a recruiter in project management and consulting firms. Sıla holds a Master of Science degree in Social Psychology and a Bachelor of Arts degree in International Relations.
Be the first to comment
Your email address will not be published. All fields are required. Comments are left in their original language.