Molmo

Molmo is a family of open state-of-the-art multimodal AI models developed by the Allen Institute for AI (Ai2). Ai2, named for its late founder Paul Allen of Microsoft fame, announced the launch of Molmo, aiming to bridge the gap between closed, proprietary AI systems and open-source alternatives. Molmo's most powerful model is designed to close the performance gap with proprietary systems across a wide range of academic benchmarks and human evaluations, while its smaller models are touted to outperform models ten times their size. Unlike many contemporary multimodal models that primarily interpret and express multimodal data in natural language, Molmo aims to go further by enabling AI to interact with and act within physical and virtual worlds. This is achieved by learning to "point" at what it perceives, a capability that empowers a new generation of applications, from sophisticated web agents to robotics.

A key innovation behind Molmo is its approach to data. Instead of relying on massive web-scale datasets with billions of images, Ai2 focused on quality over quantity, utilizing a meticulously curated dataset of just under one million images called PixMo. This dataset, collected entirely from human annotators using speech-based descriptions, allows Molmo to achieve powerful results with significantly less data—reportedly 1000 times less than many competing approaches—and computational resources. This efficiency makes Molmo more accessible to the broader AI community, from startups to academic labs, allowing them to work at the frontier of AI development. The Molmo family includes various models, with the largest being a 72 billion parameter model and the smallest, MolmoE-1B, designed for on-device use with one billion active parameters. Ai2 has committed to releasing all model weights, captioning and fine-tuning data, and source code, furthering its mission to set open standards for AI and foster continued research and innovation.