AURA 2.0, a streaming video AI model, debuts features like Model-Initiated Conversations, Stream-Aware Tool Calling, and Interactive Visual Grounding.
AURA 2.0, a groundbreaking streaming video large language model, has been unveiled by researchers including Shuhan Li, Huankang Guan, and Yuyang He. Published on November 19, 2026, on arXiv (2611.17048) and highlighted on HuggingFace Daily Papers, this model introduces significant advancements in AI interaction capabilities. Its open-source implementation is available on GitHub, enabling developers to explore its features and deployment requirements.
These capabilities are supported by a three-stage training scheme and a dedicated streaming interaction dataset. Additionally, the model incorporates Long-Horizon Interaction Memory (LHIM) to maintain contextual continuity over extended video streams.
Evaluations on public streaming benchmarks demonstrate that AURA 2.0 achieves leading performance among open-source streaming models. Targeted tests further validate its new interaction capabilities, particularly in initiating conversations and tool-based actions.
The GitHub repository (https://github.com/aurateam2026/Aura2.0) provides the AURA duplex backend, browser interface, tool executor, and deployment scripts. The model runs on a pinned vLLM-Omni framework and requires specific hardware configurations, such as two 80 GB accelerators or a single 80 GB accelerator with approximately 70 GB free memory.
AURA 2.0 represents a significant step forward in AI's ability to engage proactively in streaming video contexts. By enabling models to speak first, act on visual cues, and maintain long-term memory, it paves the way for more dynamic and interactive AI applications in fields such as surveillance, education, and entertainment.
Source link







