Local LLM Fine-tuning on Mac (M1 16GB)

🤝 Want your team maximizing Claude? I run 1:1 and team AI workshops for companies doing $1M+ per year: https://aibuilder.academy/yt/3PIqhdRzhxE Here, I show how to fine-tune an LLM locally using an M-series Mac. The example adapts Mistral 7b to respond to YT comments in my likeness. 📰 Blog: https://medium.com/towards-data-science/local-llm-fine-tuning-on-mac-m1-16gb-f59f4f598be7?sk=7c3aba481527d2b3a2eac69cd3893bab 💻 GitHub Repo: https://github.com/ShawhinT/YouTube-Blog/tree/main/LLMs/qlora-mlx 🎥 QLoRA: https://youtu.be/4RAvJt3fWoI 🎥 Fine-tuning with OpenAI: https://youtu.be/4RAvJt3fWoI ▶️ Series Playlist: https://www.youtube.com/playlist?list=PLz-ep5RbHosU2hnz5ejezwaYpdMutMVB0 More Resources: [1] MLX: https://ml-explore.github.io/mlx/build/html/index.html [2] Original code: https://github.com/ml-explore/mlx-examples/tree/main/lora [3] MLX community: https://huggingface.co/mlx-community [4] Model: https://huggingface.co/mlx-community/Mistral-7B-Instruct-v0.2-4bit [5] LoRA paper: https://arxiv.org/abs/2106.09685 Intro - 0:00 Motivation - 0:56 MLX - 1:57 GitHub Repo - 3:30 Setting up environment - 4:09 Example Code - 6:23 Inference with un-finetuned model - 8:57 Fine-tuning with QLoRA - 11:22 Aside: dataset formatting - 13:54 Running local training - 16:07 Inference with finetuned model - 18:20 Note on LoRA rank - 22:03