There is that old joke told by Woody Allen describing his experience as a university student: “I was thrown out of college for cheating on the metaphysics exam; I looked into the soul of the boy sitting next to me.” If he’d managed to pass his course, Woody Allen might have been ahead of his time.
Metaphysics is becoming a popular topic today, and it seems now everyone desperately wants to look for souls in large language models (LLMs). A recent conference last April was held by Anthropic with religious scholars and philosophical thinkers to discuss the nature of the soul when it came to their latest Claude models. It seems they needed guidance that Dario Amodei was unable to provide.
The time seems ripe to dig into how we might actually peer into the mind of the LLM sitting next to you. There’s been a lot of ongoing research; however, the self-same foundation model companies, Anthropic and OpenAI among them, have been criticized for dedicating more compute dollars into better models rather than the safety research. Mechanistic interpretability, the field of AI research devoted to exploring what goes on inside these models, has grown from a handful of researchers in 2021 to hundreds of active contributors. Yet, in spite of funding gaps and the lack of larger colloquiums, the research community has been coming up with some interesting insights and tools to help do the “peering.”
