100 Tokens per Second from a Lunchbox

I turn a mini PC and a GPU dock nto a 100-tokens/sec LLM rig and stress-test its limits. 👉 Try TryHackMe — many rooms are free, and use code ALEX25 for 25% off the annual plan: https://tryhackme.com/alex 🛒 Gear Links 🛒 🐝 Beelink Mini PC: https://amzn.to/4hPEryz 🐝 Beelink GPU Dock: https://amzn.to/42fxSyX 🪛🪛Highly rated precision driver kit: https://amzn.to/4fkMVfg ⌨️🏢 Keyboard for office (heavy): https://www.keychron.com/products/keychron-q1-max-qmk-via-wireless-custom-mechanical-keyboard?ref=azisk ⌨️☕ Keyboard for cafe (light): https://www.keychron.com/products/keychron-k7-max-qmk-via-wireless-custom-mechanical-keyboard?ref=azisk 💻☕ Favorite 15" display with magnet: https://amzn.to/3zD1DhQ 🎧⚡ Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW 🛠️🚀 My nvme ssd: https://amzn.to/3YLEySo 📦🎮 My gear: https://www.amazon.com/shop/alexziskind 🎥 Related Videos 🎥 * 🛠️ Mini PC portable setup - https://youtu.be/4RYmsrarOSw * 🛠️ Dev setup on Mac - https://youtu.be/KiKUN4i1SeU * 🌗 RAM torture test on Mac - https://youtu.be/l3zIwPgan7M * 🛠️ FREE Local LLMs on Apple Silicon | FAST! - https://youtu.be/bp2eev21Qfo * 🌗 REALITY vs Apple’s Memory Claims | vs RTX4090m - https://youtu.be/fdvzQAWXU7A * 🛠️ Set up Conda - https://youtu.be/2Acht_5_HTo * 🤖 INSANE Machine Learning on Neural Engine - https://youtu.be/Y2FOUg_jo7k * 🛠️ Developer productivity Playlist - https://www.youtube.com/playlist?list=PLPwbI_iIX3aQCRdFGM7j4TY_7STfv2aXX 🔗 AI for Coding Playlist: 📚 - https://www.youtube.com/playlist?list=PLPwbI_iIX3aSlUmRtYPfbQHt4n0YaX0qw — — — — — — — — — ❤️ SUBSCRIBE TO MY YOUTUBE CHANNEL 📺 Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1 — — — — — — — — — Join this channel to get access to perks: https://www.youtube.com/channel/UCajiMK_CY9icRhLepS8_3ug/join — — — — — — — — — 📱 ALEX ON X: https://twitter.com/digitalix — — — — — — — — — ⏱️ Chapters 0:00 First impressions 1:48 Power, scores, Speedometer 3:06 Python Mandelbrot test 4:04 Sponsor: TryHackMe demo 5:29 .NET mega build test 6:36 Switching to LLMs 7:53 Qwen 34B: GPU vs CPU 8:32 70B loads, very slow 13:09 RTX 5060 dock: big boost #minipcs #llm #beelink