Do AI Models Dream of Memory?

The science of sleep in humans is fascinating; during sleep, our brains sort through memories, retaining important ones while letting go of trivial details. Imagine a similar capability for AI? That's precisely what a company called Bilt aims to achieve.

Partnering with Letta, a tech startup, Bilt launched millions of AI agents. These agents can learn from past interactions and share memories. Using a unique process known as "sleeptime compute," they discern which information merits long-term storage and which needs immediate recall.

Andrew Fitz, an AI engineer at Bilt explains, "We can change a memory block and influence the actions of countless agents at once. This method offers fine-tuned control over agents' contexts," enhancing the context prompt fed to models during inference.

Despite the capabilities of large language models, their memory is a bit clunky. Typically, they recall details only when included in the input context. Unlike human memory, AI can easily become overwhelmed and start "hallucinating" if overfilled.

Charles Packer, Letta's CEO, notes, "While the human brain efficiently stores useful information, AI's context loops can end up tainted, causing them to veer off course." Packer and his team pioneered MemGPT, an open-source effort to help AI distinguish what should be in short-term versus long-term storage.

Expanding on MemGPT's foundations, Letta is now focused on helping AI "learn" in the background—a step crucial to their collaboration with Bilt. The goal is to enhance chatbot intelligence, reducing errors and improving reliability. LangChain, another player in the field, offers diverse memory storage solutions, emphasizing the importance of context engineering to feeding AI with relevant information.

This past February, OpenAI took strides toward this AI memory improvement by devising methods for ChatGPT to store relevant data for user personalization. However, the specifics remain under wraps, drawing interest from the AI community.

Transparency in recall and memory systems is vital. Clem Delangue from Hugging Face, a Letta investor, reiterates the significance of open systems, including memory. Additionally, Packer from Letta suggests a future where AI might need to "forget" upon request—a useful skill for managing data ethically.

The concept of artificial memories resonates with Philip K. Dick's "Do Androids Dream of Electric Sheep?" which gave birth to the movie Blade Runner. While today's AI hasn't reached the level of fictional replicants, the evolution of memory in AI promises a step toward that futuristic narrative.