Published:
An investigation into how language models internally represent abstract functions that can be invoked by diverse, semantically equivalent prompts. This work reveals surprising findings about the geometry of learned concepts in LLMs and the unexpected effects of supervised fine-tuning.
Published:
In the rapidly evolving field of artificial intelligence, Large Language Models (LLMs) like GPT, BERT, and their successors have emerged as powerful tools capable of understanding and generating human-like text. While these models demonstrate remarkable abilities, their inner workings remain largely opaque. One intriguing theory attempting to explain the efficiency and capability of LLMs is the concept of “superposition” - a phenomenon borrowed from quantum physics to describe how these models might represent information internally.