AI Neural Networks: Distinguishing Memorization from Problem-Solving

Researchers from Goodfire.ai have isolated the distinct functions of memorization and problem-solving in AI neural networks. According to the preprint paper released in late October, these functions utilize separate neural pathways within models like Allen Institute for AI's OLMo-7B.

When the memorization pathways were removed, models lost 97 percent of their ability to recite training data verbatim while maintaining their logical reasoning capabilities. This discovery highlights a clear mechanistic split within the architecture of these neural networks.

The researchers ranked the weight components of the neural model and found that the bottom 50 percent showed a 23 percent higher activation for memorized data. In contrast, the top 10 percent indicated a 26 percent higher activation for general, non-memorized text.

This suggests that the components responsible for memorization are distinct from those involved in problem-solving. Interestingly, arithmetic operations were also found to share pathways with memorization rather than logical reasoning.

Following the removal of memorization circuits, the models' mathematical performance dropped to 66 percent, while logical task performance remained largely unaffected. This finding elucidates why AI language models often struggle with mathematical computations, as they tend to recall arithmetic as memorized facts instead of executing logical operations.

The report suggests that these AI models may resemble students who have memorized multiplication tables without fully understanding the underlying concepts. Overall, this research provides significant insights into the functioning of AI neural networks, paralleling the complexities of biological neural systems.

Such findings could lead to improvements in AI design by refining how these models handle both memorization and reasoning tasks, potentially enhancing their performance across various applications in cognitive science.