CONNECT WITH US
Tech

Tech

Lawyer’s $9,450 private AI server with 12 used Nvidia V100s now runs every DeepSeek V4.1 Flash expert

Tom's Hardware logo

Published on

Add as a preferred source on Google
Lawyer’s $9,450 private AI server with 12 used Nvidia V100s now runs every DeepSeek V4.1 Flash expert

A lawyer has upped the ante by upgrading his private AI server from eight to 12 used Nvidia V100s, which can run every DeepSeek V4.1 Flash expert at 4-bit NVFP4. Performance can reach up to 184 tokens per second for code, and long documents start answering in about half the time as before. He says it can outrun published DGX Spark stacks at decoding a single request, for half the price of three units, based on the $9,450 he paid.

Some of the best movie franchises come in threes, and Nick Mermiges seems determined to match that, taking his original setup — which he previously upgraded with more memory and other fixes — and expanding it for fuller precision, after a previous upgrade got him most of the way there. The newest upgrade, this time, involves a third board of V100s, for a total of 12 with four per board. There may be a method to his madness.

9 GiB. 7%, swapped to NVFP4 until the cards were full. Now, all 15,360 experts can live on the cards in their full glory, as he intended, with “none compressed further,” according to his GitHub repo. In a longer write-up on his site, he says the expert values “equal the ones in DeepSeek’s own release,” although other parts of the model use compact forms.



Source link

Disclaimer

We strive to uphold the highest ethical standards in all of our reporting and coverage. We TheMorningPulse.fyi want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It's possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.