Is the Nvidia Tesla V100 still good for AI? - Inspur DGX V100 vs RTX 5090

Thanks to Verda, formerly Datacrunch.io, for sponsoring today’s episode. Check them out at https://verda.com, and use Coupon Code CRAFT-COMPUTING-V100 for $25 off at checkout! https://docs.verda.com/resources/obtaining-free-credits/how-to-redeem-credits Grab yourself a Pint Glass or Coffee Tumbler at https://craftcomputing.store Implementing any form of AI workflow into your business is prohibitively expensive. From just the cost of hardware to the power and cooling infrastructure of modern AI servers, it's enough to turn anyone off. But you know me... I love digging up old servers out of eWaste piles and giving them new life. But how well does Nvidia's Tesla V100s stack up to modern cards in AI? Today, we're testing Eight Tesla V100s in 70B and 120B LLMs to see if there's life still in these eight year old GPUs. But first... What am I drinking??? Deschutes Brewing (Bend, OR) Fresh Squeezed IPA NA (0.5%) HUGE THANKS to UnixSurplus for sending over the Inspur DGX V100 system for me to take a look at. Check them out at https://UnixSurplus.com Or their eBay store: https://ebay.us/6BPOyd *Links to items below may be affiliate links for which I may be compensated* Inspur DGX V100 Server from UnixSurplus: https://ebay.us/1U6I0H Dual Intel Xeon 8260 24-Core / 48-Thread 256GB DDR4-3200 REG-ECC 2x MZ-7LH1T90 1.92TB SSD Drives 8x Nvidia V100 32GB SXM2 GPU 1x 4x 10GB SFP+ Mezzanine Card OB NIC: 100GB Nvidia CX6 network card Follow me on Bluesky @CraftComputing.bsky.social Support me on Patreon and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long. https://www.patreon.com/CraftComputing Timestamps 0:00 - Intro 2:11 - Sponsor - Verda.com 3:29 - Speeds and Feeds 6:10 - Llama 3.1 8B (MLPerf) 8:16 - Phi 4 Reasoning 14B (MLPerf) 9:06 - LM Studio (Llama 3.3 70B + GPT-OSS 120B) 14:56 - As Always, I'm Mikey 20:15 - Wrapping Up 22:57 - Deschutes Fresh Squeezed NA