Pareto, Power Laws, and Fat Tails

🤝 Want your team maximizing Claude? I run 1:1 and team AI workshops for companies doing $1M+ per year: https://aibuilder.academy/yt/Wcqt49dXtm8 In this video, I give a beginner-friendly guide to Power Laws and describe 3 problems with using traditional statistical methods to analyze them. 📹 Detecting Power Laws: https://youtu.be/x5-IW1m3zPo 📹 Fat Tails: https://youtu.be/15Kd9OPn7tw 📰 Read more: https://medium.com/towards-data-science/pareto-power-laws-and-fat-tails-0355a187ee6a?sk=2c4da32a8f5d6d90cf515f7ce5204933 💻GitHub Repo: https://github.com/ShawhinT/YouTube-Blog/tree/main/power-laws References [1] Pareto principle. (2023, October 30). In Wikipedia. https://en.wikipedia.org/wiki/Pareto_principle [2] arXiv:2001.10488 [stat.OT] [3] Taleb, N.N. (2007). The Black Swan: the impact of the highly improbable. New York; Random House. [4] https://www.archives.gov/exhibits/influenza-epidemic/ [5] arXiv:0706.1062 [physics.data-an] [6] Taleb, N. N. (2019). How much data do you need? An operational, pre-asymptotic metric for fat-tailedness. International Journal of Forecasting, 35(2), 677–686. https://doi.org/10.1016/j.ijforecast.2018.10.003 You can find many great lectures on this topic here: @nntalebproba Intro - 0:00 Outline - 0:45 The Gaussian Distribution - 1:21 The Pareto Distribution - 2:42 Power Laws - 4:30 Mediocristan vs Extremistan - 5:31 3 Problems with STAT 101 - 8:44 Problem 1: The Mean is Meaningless - 10:07 Problem 2: Regression Doesn't Work - 14:30 Problem 3: Payoffs Diverge from Probabilities - 17:41 Controversy in Extremistan - 20:00 Fat Tails - 21:17 Takeaways - 24:38