A New Trick Reveals AI Models’ Inner Thoughts

Computer scientists precocious discovered a mode to extract the hidden “thinking” that frontier AI models execute arsenic they enactment done analyzable problems.

The findings supply immoderate evidence—although not conclusive proof—that definite Chinese models whitethorn person been trained by “distilling” reasoning accusation from US models that was supposedly hidden due to the fact that of however intimately immoderate of their reasoning oregon reasoning patterns look to match. The researchers person besides demonstrated that the method could beryllium utilized to retrieve idiosyncratic information, similar passwords and API keys, from a model’s interior reasoning, though this vulnerability has been fixed.

“All large frontier exemplary providers we tested stock this vulnerability,” says Alexander Panfilov,⁩ a machine idiosyncratic astatine University of Tübingen successful Germany who was progressive with the work. “It tin pb to idiosyncratic accusation leakage, and it enables large-scale reasoning distillation attacks.”

Panfilov and colleagues from the University of Tubingen, the Max Planck Institute, the AI information institute MATS Research, and the information institution Snyk identified the aforesaid contented with frontier models from OpenAI, Anthropic, and Google that are accessed via an exertion programming interface oregon API.

In a insubstantial laying retired the work, the researchers amusement that the open-weight oregon downloadable Chinese exemplary Kimi K3 from Moonshot AI produces a strikingly akin output to the hidden reasoning traces—the written-out reasoning steps progressive successful solving a problem—of Claude Opus 4.8 and GPT 5.6 Sol for definite prompts. Despite the similarities, they enactment that the enactment “cannot causally found distillation.” They recovered that 2 different open-weight models, China’s DeepSeek and Inkling from the US institution Thinking Machines, did not grounds this benignant of reasoning similarity with Claude Opus.

Moonshot AI and Z.ai did not respond to a petition for remark by clip of publication.

Distillation is simply a well-established, wide utilized method for efficiently copying the capabilities of existing models implicit to caller ones, and is particularly communal successful the improvement of open-weight oregon afloat downloadable models.

Lately, however, distillation has go a arguable topic, due to the fact that of claims that Chinese AI companies usage it to fundamentally transcript the champion US models. In February, OpenAI told US lawmakers that DeekSeek seemed to person copied 1 of its models to physique a reasoning exemplary called R1. In June, Anthropic told lawmakers that Alibaba had systematically distilled its models successful bid to physique its own, called Qwen.

There’s nary denotation that Chinese AI companies utilized this circumstantial method to distill US-based AI models. But Panfilov and collaborators accidental that utilizing their method would marque it imaginable to distill much accusation from closed models than antecedently realized.

Mini-Me Models

Advanced AI models lick hard problems by breaking them into constituent parts that are analyzed successful crook successful a benignant of artificial reasoning oregon “chain of thought.” Companies thin to support a proprietary model’s reasoning concealed to forestall others from utilizing them to bid caller ones. However, they typically besides nonstop an encrypted mentation of that reasoning to a user’s machine successful a mode that offloads immoderate computation.

The researchers’ onslaught relies connected the information that astir AI companies besides supply related models of antithetic sizes. Larger models are much susceptible but besides much computationally costly to tally and much costly to access. Users whitethorn take smaller, weaker models for definite tasks to little costs.

Panfilov and his colleagues recovered that feeding encrypted reasoning traces to a smaller mentation of the aforesaid exemplary tin uncover the hidden reasoning inside. The smaller models person received little alignment training, meaning that, dissimilar the bigger ones, they are little apt to garbage to uncover their interior thoughts.