Together AI introduces Polymorphic Inference and Together Compiler to optimize open-model AI inference for diverse and evolving workloads.
Together AI has announced the launch of Polymorphic Inference, a new approach to AI inference designed to adapt to the rapidly evolving landscape of open-source models, diverse architectures, and dynamic workloads. The company also introduced the Together Compiler, a key tool for achieving this vision by enabling continuous optimization of inference performance.
Polymorphic Inference, derived from the Greek term for "many shapes," represents a shift from the traditional Static Inference paradigm. Unlike Static Inference, which relies on fixed configurations optimized for specific conditions, Polymorphic Inference dynamically adapts to changes in model architectures, traffic patterns, and hardware platforms. Together AI highlighted several industry trends driving this shift, including:
The company contrasts Polymorphic Inference with legacy Static Inference, which relied on closed models, traditional Transformer architectures, and static configurations that often failed to meet the demands of modern production environments.
As part of its Polymorphic Inference initiative, Together AI unveiled the Together Compiler, an AI-powered tool designed to optimize inference workloads through continuous compilation. The compiler reframes traditional GPU optimization challenges as agentic search problems, enabling AI agents to iteratively rewrite and improve GPU code for diverse workloads.
According to Together AI, the Together Compiler delivers significant performance improvements. 7 times the speed of hand-written megakernels and processed over 2,100 tokens per second for Llama 1B models on NVIDIA H100 and B200 GPUs. The compiler also enables 252 times more optimization experiments compared to existing agent-based systems, producing deployable optimizations within 24 hours.
Source link
