A little bit technical, but will help you understand a potential approach to continual learning...
Article
Path to continual learning: Kimi K3's KDA as Fast Programmable Weights & NoROPE for infinite context
Moonshot is onto bigger things and most are missing it!
This post is dedicated to Dwarkesh Patel. He continuously keep bringing up the topic of continual learning with his guests. As he rightly