FastFlow v1.0.13 introduces AMD GPU support via HIP EP, unifying NPU and GPU execution under one CLI and API for enhanced AI model performance.
FastFlow, a lightweight runtime designed for AMD Ryzen™ AI systems, has released version 1.0.13, marking a significant milestone by adding support for AMD GPUs. This update integrates HIP EP v0.5.3 through the new oga-hipep backend, enabling developers to run AI models on both NPUs and GPUs under a unified command-line interface (CLI) and local API.
With the introduction of the oga-hipep backend, FastFlow now supports high-performance GPU execution alongside its existing NPU capabilities. Developers can seamlessly switch between hardware flows while maintaining consistent behavior for chat templates, tokenization, sampling, and generation. The model tag determines the backend, allowing for flexibility in deployment without rewriting code.
HIP EP, an open-source ONNX Runtime execution provider optimized for AMD GPUs, powers the new GPU flow. Benchmarks on a Ryzen™ AI Max (Strix Halo) system demonstrate significant performance improvements over llama.cpp (Vulkan) at a 16K token context length. The time to first token (TTFT) is reduced by approximately 43%, while decode throughput (TPS) is 7% higher. Specific results include:
Additional benchmarks for specific models, such as Qwen2.5 and GPT-OSS, show decode speeds ranging from 26.10 tokens/s to 85.96 tokens/s, depending on the model and input type.
13 simplifies the setup process with just three commands: flm pull , flm run , and flm serve . These commands enable developers to download models, run terminal-based chats, and expose models through an OpenAI-compatible API.
Source link







