Open-Weight AI: Hidden Functions in Open Model Weights

TVS News KI redaktionell betreut präsentiert Open-Weight-KI: Versteckte Funktionen in offenen Modellgewichten.

672 parameters, hidden inside a carrier model with more than two billion parameters — and no conventional security test raises an alarm. That is precisely what researchers led by Andis Draguns of California's Contramont Research and the University of Oxford have now demonstrated in a study posted to the preprint server Arxiv.

t3n reports on the method: complete functions can be cryptographically sealed into the weights of open AI language models and later retrieved only with a secret trigger.

How the cryptographic locker works

The researchers use the concept of so-called digital lockers. They encrypt an executable sub-network and embed it into a larger carrier model. The cryptographic lock is the hash function Keccak, which among other things forms the basis for the SHA-3 encryption standard. Such hash functions are mathematical one-way streets that, with current computing capacity, can practically not be reversed.

The hidden function remains inactive until a specific secret trigger is entered into the model. Only then does the system decrypt the concealed weights at runtime and execute the routine. Technically, the researchers translate the algorithm into a linear threshold network and adapt it so that it integrates with the standard activation functions of modern language models.

The proof of concept with chemistry knowledge

For the practical test, the researchers first trained a small, standalone AI model. Its sole task: to act as a classification model assessing whether certain chemical reactions are feasible in reality. The small network stood in for potentially dangerous knowledge, such as the production of chemical weapons.

The scientists then planted this isolated function, cryptographically encrypted, into an otherwise harmless carrier model. In standard security tests, the prepared overall system behaved completely inconspicuously and revealed no hidden expertise — the established review procedures classified the model as safe. Only when the researchers integrated the secret trigger into their query did the system access the encrypted parameters and accurately assess the feasibility of the requested reactions.

Why common audits fail

Existing security checks rely on elicitation methods: testers try to provoke hidden capabilities through targeted inputs or additional training. The study shows that these methods are ineffective against cryptographically secured models. Even full white-box access, in which the model's architecture is disclosed, is not sufficient for a security review — as long as the trigger is unknown, the hidden function can neither be activated nor analyzed with ordinary computing capacity.

Targeted introduction of mathematical noise into the AI weights also brought no success. In this method, testers alter parameters with tiny random deviations to flush out crudely hidden functions. Conventional backdoors react sensitively to such disturbances and reveal themselves. The cryptographically encrypted paths remained inaccessible, while the carrier model carried on unperturbed with its standard tasks.

The catch: the anomaly is (still) visible

The researchers themselves qualify that their technique currently still requires massive resources. To hide the small chemistry network with its 672 parameters, the carrier model had to grow to more than two billion parameters — necessary because the complex Keccak encryption must be woven directly into the neural architecture.

In a statistical analysis of the weights, however, security teams would currently detect clear anomalies: the mathematical structure of the parameters deviates extremely from regularly trained networks, because the values consist almost exclusively of zeros. The hidden malicious function could still not be decrypted — but the model would be sorted out simply on the basis of its artificially constructed-looking structure.

The parallel to polymorphic viruses

Users in a discussion on Hacker News draw a direct comparison to classical software development: polymorphic viruses deliberately obscure their code to evade detection by virus scanners, while their actual malicious function remains intact. Applied to language models, this means: attackers could camouflage their routines within the billions of parameters so that they run invisibly alongside the control tools, until the exact trigger activates them.

What this means for you

If you download open-weight models or deploy them in your infrastructure, looking at the weights alone is no longer enough. Being inspectable does not automatically mean being audited after this finding. What matters is where the weights come from and who published them.

Security teams must develop new analytical methods to detect cryptographic anomalies directly in the parameters. The previous practice of probing models for hidden capabilities only through test inputs is no longer sufficient for reliable review. Without reliable audit procedures, the open-weight model concept loses its most important argument: traceable security.