CONNECT WITH US
Web3 & Blockchain

Web3 & Blockchain

FastFlow v1.0.13 Adds AMD GPU Support with HIP EP Integration

Blockchain.News logo

Published on

Add as a preferred source on Google
FastFlow v1.0.13 Adds AMD GPU Support with HIP EP Integration

FastFlow v1.0.13 introduces AMD GPU support via HIP EP, unifying NPU and GPU execution under one CLI and API for enhanced AI model performance.

FastFlow, a lightweight runtime designed for AMD Ryzen™ AI systems, has released version 1.0.13, marking a significant milestone by adding support for AMD GPUs. This update integrates HIP EP v0.5.3 through the new oga-hipep backend, enabling developers to run AI models on both NPUs and GPUs under a unified command-line interface (CLI) and local API.

With the introduction of the oga-hipep backend, FastFlow now supports high-performance GPU execution alongside its existing NPU capabilities. Developers can seamlessly switch between hardware flows while maintaining consistent behavior for chat templates, tokenization, sampling, and generation. The model tag determines the backend, allowing for flexibility in deployment without rewriting code.

HIP EP, an open-source ONNX Runtime execution provider optimized for AMD GPUs, powers the new GPU flow. Benchmarks on a Ryzen™ AI Max (Strix Halo) system demonstrate significant performance improvements over llama.cpp (Vulkan) at a 16K token context length. The time to first token (TTFT) is reduced by approximately 43%, while decode throughput (TPS) is 7% higher. Specific results include:

Additional benchmarks for specific models, such as Qwen2.5 and GPT-OSS, show decode speeds ranging from 26.10 tokens/s to 85.96 tokens/s, depending on the model and input type.

13 simplifies the setup process with just three commands: flm pull , flm run , and flm serve . These commands enable developers to download models, run terminal-based chats, and expose models through an OpenAI-compatible API.


Source link

Disclaimer

We strive to uphold the highest ethical standards in all of our reporting and coverage. We TheMorningPulse.fyi want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It's possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.