The Sleep Framework: A New Way for LLMs to Retain Knowledge in Continual Learning

The Sleep Framework: A New Way for LLMs to Retain Knowledge in Continual Learning

Modern large language models (LLMs) have achieved impressive performance through pre-training and in-context learning (ICL). However, once deployed, these models often remain largely static, operating with a fixed knowledge cutoff date that prevents them from acquiring long-term capabilities or adapting to new information. While updating models with new data seems like the obvious solution, continual parameter updates often trigger catastrophic forgetting, a fundamental issue where learning new tasks severely degrades performance on previously learned ones. Alternatively, relying strictly on ICL limits knowledge retention to the immediate context window; once the context session ends, the acquired short-term information is forgotten.

To address these limitations, Google Research has introduced a sleep framework in continual learning. This learning paradigm equips LLMs with a continuous periodic lifecycle, enabling them to continually learn, consolidate fragile short-term context into stable long-term parameters, and mitigate catastrophic forgetting or static knowledge cutoffs. Inspired by how the human brain processes and stabilizes memory during rest, the framework allows models to alternate between active interaction and offline memory consolidation.

The Active and Sleep Phases

Under the sleep paradigm, an LLM operates across two primary states: the active (wake) phase and the sleep phase. During the active phase, the model actively receives, processes, and interacts with inputs, storing temporary information in fast, short-term memory components. In contrast, during the sleep phase, the model pauses processing new inputs and redirects its internal computations toward memory consolidation and self-improvement.

While terms like "sleep," "dreaming," and "self-improvement" borrow from neuroscience, Google Research notes that their framework operates entirely within a fixed model architecture and human-defined training rules. During the offline sleep phase, the model unlocks pre-allocated Mixture-of-Experts (MoE) memory blocks and updates its numerical weights using self-generated practice examples. This automated post-training cycle helps a deployed model absorb new facts and sharpen its reasoning accuracy while keeping its core codebase and design unchanged.

Memory Consolidation and Parameter Expansion

The sleep paradigm addresses catastrophic forgetting and continuous adaptation through two stages: memory consolidation and dreaming. In the human brain, non-rapid eye movement (NREM) slow-wave sleep consolidates fragile memories into long-term structures via abstraction and distillation. Google Research mirrors this natural phenomenon through specific technical mechanisms.

For memory consolidation, the framework utilizes parameter expansion and a continuum memory hierarchy. To prevent catastrophic forgetting, the model dynamically scales its capacity during sleep. Using a Mixture of Experts (MoE) and Hope architecture, newly unlocked, low-rank experts are integrated into stable, lower-frequency memory blocks to store incoming knowledge without disrupting older parameters.

This process involves knowledge seeding, or upward distillation. Fragile short-term memories stored in high-frequency layers (e.g., in-context memory) are transferred into the newly expanded long-term parameters. This is achieved through Generalized Knowledge Distillation (GKD) combined with reinforcement learning (RL)-based imitation learning. Specifically, on-policy distillation allows the expanded student model to learn token-specific feedback from teacher distributions generated on self-sampled sequences. Additionally, learning to imitate (LTI) rewards the student model based on semantic and absolute similarity, such as Levenshtein distance, when completing prefixes of teacher-generated text.

Once the memory consolidation into lower-frequency modules is complete, the high-frequency temporary parameters are reset via synaptic pruning. This frees up capacity for future active sessions, ensuring the model can continue to learn without being constrained by its initial size.

Dreaming for Self-Improvement

Mirroring REM sleep in the human brain, where the brain actively simulates scenarios and explores novel neural pathways, the sleep framework implements an autonomous self-improvement loop during the "dreaming stage." This stage allows the model to generate synthetic training samples and refine its capabilities without external input.

The dreaming process involves three key components:

1. Random Expert Selection: During synthetic data generation, the MoE router randomly activates non-standard experts. This forces the model to mix disparate domains of knowledge and uncover latent patterns that might not emerge during standard active learning.
2. Curated Rehearsal: The model generates synthetic training samples ("dreams") and filters them using gradient-based data selection. This prioritizes samples with the highest potential to boost overall model capability.
3. RL Refinement: The model performs supervised fine-tuning on top-rated dream samples and evaluates whether the generation improves performance on downstream tasks.

Google Research characterizes random expert activation and reward-based sample selection as mechanisms for improving the geometry of learning. Injecting different MoEs broadens the set of gradient directions explored during synthetic generation, exposing the model to a richer collection of updates than those induced by ordinary wake-time data alone. Gradient-based filtering then emphasizes the synthetic experiences that carry the most useful learning signal, strengthening informative directions while preserving the structure of the underlying gradient space.

Theoretical Foundations and Performance

Beyond empirical results, the sleep paradigm is grounded in a theory that clarifies why its two stages can support continual learning without simply trading new knowledge for old capabilities. For memory consolidation, the analysis shows that knowledge seeding can be understood as a structured, low-rank compression problem. The model is not asked to copy every detail of its short-term traces into its parameters. Instead, it identifies a compact set of directions that best preserve both the model's own on-policy behavior and the semantic qualities rewarded by the learning objective.

For dreaming, the theory characterizes the process as a deliberate method for diversifying, selecting, and consolidating updates. Together, these steps yield a more balanced optimization landscape where learning is less dominated by a small number of directions, gradients are better conditioned, and the model can make more stable progress when refining its capabilities.

Google Research reports comparative performance of the sleep framework against baseline approaches, including in-context learning and supervised fine-tuning, across major evaluation axes.

• Continual Translation: In unseen languages, such as Manchu and Kalamang, the sleep framework maintained a near-zero performance drop in multi-stage evaluations. In contrast, standard baselines like ICL, Cartridges, and supervised fine-tuning (SFT) suffered severe catastrophic forgetting during sequential learning.
• Long-Context Understanding: On the BABILong benchmark, the sleep method demonstrated near-perfect scaling up to 10M tokens. This outperformed strong baselines like GPT-4 and Titans by using sleep-time consolidation to compress long-context information directly into parametric memory.
• Factual Knowledge Incorporation: Evaluated on SQuAD and continued pre-training (CPT), the sleep method achieved 48.9% on single-task and 46.2% on CPT evaluations. This outperformed baseline methods including the base model, SFT, and Self-Adapting Language Models (SEAL) by successfully integrating new factual information without degrading existing model representations.
• Mathematical Reasoning: On benchmarks including AIME 24/25 and HMMT 25, the sleep-enhanced base-model achieved accuracy scores of 79.2%, 69.0%, and 46.1%, respectively. This outperformed baselines like SFT, group relative policy optimization (GRPO), and on-policy self-distillation (OPSD), showing that pairing memory consolidation with imitation learning drives superior reasoning performance.

Conclusion

The sleep paradigm provides a path toward autonomous, continually learning AI systems. By moving beyond static pre-training and fragile context windows, alternating between active learning and offline rest empowers LLMs to retain short-term memories, abstract new skills into long-term parameter structures, and continuously self-improve without catastrophic forgetting. This research, conducted by Ali Behrouz, Farnoosh Hashemi, Adel Javanmard, and Vahab Mirrokni, suggests that mimicking biological rest cycles can be a powerful strategy for the next generation of adaptive artificial intelligence.

Source:

• The sleep framework: A new way for LLMs to retain knowledge in continual learning — Google Research — https://research.google/blog/the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning