In this video I break down Google’s new research claiming that a lot of what we call “deep learning” is basically an illusion of stacked layers doing everything the same way. The core problem they’re tackling is catastrophic forgetting: when you teach a model new things, it can overwrite what it already knows. And even if you want continuous learning, today’s LLM setup is basically split into frozen pre-training (long-term memory you can’t easily update) and a context window (short-term memory that doesn’t permanently change the model). Google proposes “nested learning,” where a model isn’t just a flat stack of layers, but a set of nested optimization problems—modules that each learn at different rates and have their own internal workflow and memory. I connect this idea to how the brain processes information at different frequencies (fast reactions vs slow consolidation). I also show why this isn’t totally foreign: optimizers already behave like nested learners (SGD → momentum → Adam) by using associative memory of past gradients. Then I cover how this maps to transformers and the proposed HOPE architecture: multiple learnable components updated at different token frequencies (low-frequency long-term updates to high-frequency short-term updates). The key takeaway is that progress might not be “just add more layers,” but redesign models as interconnected components that learn continuously without sacrificing old capabilities.