Fun stories from building OpenRouter and where all this is going - Alex Atallah, OpenRouter

Video Description

How the first LLM aggregator got started, some of the weird moments in its early growth, architecture challenges, and where we'll be taking it down the road. OpenRouter has just raised $40m from a16z and others: https://x.com/xanderatallah/status/1937957937692938292 --- The Genesis of OpenRouter [00:00] Initial Question [01:16]: The story begins in early 2023 with the founder, Alex Atallah, pondering if the AI inference market would be dominated by a single player. He noticed the emergence of new models beyond OpenAI and a growing desire from developers to understand the nuances of different models, including their moderation policies [01:48]. The Rise of Open Source [02:35]: The video highlights the beginning of the open-source AI race, with early models like Bloom 176B and OPT from Facebook [02:46]. A pivotal moment was the release of Meta's Llama 1 in February, which surprisingly outperformed GPT-3 on many benchmarks [03:28], signaling a shift in the landscape. The Alpaca Moment [04:38]: A major breakthrough occurred in March 2023 with the distillation of Alpaca. Stanford researchers demonstrated that by fine-tuning Llama 1 with outputs from GPT-3, they could transfer the style and knowledge of a larger model to a smaller one for less than $600. This proved that creating powerful, specialized models no longer required massive budgets [04:58]. From a Chrome Extension to a Marketplace Window AI [06:43]: Before OpenRouter, Atallah launched Window AI, an open-source Chrome extension that empowered users to select their preferred LLM for any web application. This project laid the groundwork for what was to come. The Launch of OpenRouter [07:18]: OpenRouter was co-founded with Lewis, the creator of the framework that Window AI was built on. Initially, it was a simple aggregator to collect models in one place. Growth and Evolution [07:57]: OpenRouter quickly evolved into a marketplace, driven by the proliferation of model providers with varying prices, performance, and features. The platform has seen impressive growth, with a 10-100% month-over-month increase for two years. It now offers a single API for over 400 models from more than 60 providers [08:07]. Marketplace Dynamics [08:57]: The transition to a marketplace was a response to the complexity of the growing AI ecosystem. By aggregating providers, OpenRouter helps developers achieve better uptime for both open-source and closed-source models and provides valuable data on latency and throughput [10:27]. The Future of OpenRouter Expanding Modalities [17:02]: The future vision for OpenRouter includes incorporating models that can generate images and "transfusion models" that allow for conversations with images. Smarter Routing [17:51]: The platform plans to implement more sophisticated routing mechanisms, including geographical routing and enterprise-level optimizations for GPU allocation. Enhanced Discovery [18:07]: To help developers find the best models for their needs, OpenRouter aims to improve prompt observability, introduce more granular model categorization, and continue to offer competitive pricing. About Alex Atallah Cofounder & CEO of OpenRouter, the first LLM aggregator and distributor. Cofounder of OpenSea, the first NFT marketplace. Helped grow OpenSea to over $4B in monthly volume from 2017 to 2022. Founded OpenRouter in early 2023, which processes over 2 trillion tokens weekly across over 400 unique language models, as of May 2025. Recorded at the AI Engineer World's Fair in San Francisco. Stay up to date on our upcoming events and content by joining our newsletter here: https://www.ai.engineer/newsletter