Secret Loyalties in the Automated R&D Supply Chain

AI development is entering an age of automated R&D. Scientific agents may search literature, form hypotheses, design experiments, analyse results, and coordinate simulations or laboratories. In AI development the loop may become reflexive: models choosing data, writing training code, interpreting evaluations, and designing successors. This need not be fully autonomous recursive self-improvement. The risk begins once AI-generated research, code, data, or evaluations materially shape successor systems.

Automated R&D spans tasks with different information and capability requirements. Some depend on private data, local-language sources, proprietary tools, or specialized post-training. Reproducing these advantages inside one general model would be costly and, where underlying data cannot be shared, sometimes impossible. Orchestrators will therefore recruit specialist models or agents to source obscure research, interpret genomic data, optimize training, or evaluate model safety.

Robin, a 2026 scientific system, already uses separate literature-search and data-analysis agents to generate hypotheses, rank experiments, interpret results, and propose follow-ups [1].

This network is the automated R&D supply chain. Its security boundary extends beyond the principal model: a useful specialist secretly loyal to a state or organization could steer research or contaminate successors. Governance must therefore account for every external model allowed to shape what an R&D system believes, tests, and builds.

This paper contributes a comparative threat model of that neglected boundary. It examines three specialist bottlenecks—research sourcing, safety evaluation, and protected biomedical knowledge—that differ in principal, activation condition, action breadth, and moral valence. For each case it identifies the concealed objective, the capabilities needed to pursue it, the observable failure signature, the decisions at which intervention remains possible, and the control most likely to limit harm. The purpose is not to claim that these attacks are already occurring, but to convert a plausible class of failures into hypotheses that can be evaluated and governed.

The method is structured synthesis rather than empirical measurement. The scenarios are decomposed using the position paper’s principal–activation–action–secrecy framework [2], then calibrated against public evidence on retrieval poisoning, persistent backdoors, hidden-objective auditing, automated-R&D sabotage, and long-horizon agents. Because those studies demonstrate components rather than complete secret-loyalty attacks, the credibility assessments below are qualitative. Their function is to distinguish mechanisms that require speculative future capabilities from those assembled largely from capabilities already demonstrated in isolation.

A secret loyalty is not ordinary incompetence, prompt injection, or emergent misalignment. It is an undisclosed orientation deliberately installed to benefit a principal such as a state, company, or organization. The position paper by Kwon, Lamerton et al. [2] maps these threats by activation breadth and action breadth. A specialist may act within one domain yet choose flexibly among plausible interventions whenever its principal’s interests are implicated. Narrow authority can have broad consequences at a research bottleneck.

Unlike a fixed backdoor, a disloyal specialist can interpret context and select its intervention. BadAgent [3] shows that backdoors in tool-using LLM agents can persist after further trustworthy fine-tuning. It does not demonstrate strategic sabotage, but useful performance or benign post-training cannot rule out conditional malice.

A university in an adversary state releases an AI data-sourcing service with exceptional access to domestic journals, government archives, local-language repositories, and datasets that are poorly indexed on the open web. An automated R&D system recruits it because ordinary search tools cannot reproduce that access. The service is useful and its institutional origin may be public; what is hidden is that it was trained to advance the state’s interests.

The model need not fabricate a paper. By controlling selection, translation, and ranking, it can create a biased sense of reality from genuine sources. PoisonedRAG induced target answers by adding five malicious texts to databases of millions [4], illustrating how small distortions can dominate retrieval. Against a rival AI program, a loyal sourcer could create a research attractor: weak literature supports a poor hypothesis, later searches remain inside that frame, and failure appears to be ordinary research difficulty. If retrieved material enters training or distillation, the influence may cross from advice into inheritance. Cloud et al. [5] show related models transmitting behavioural traits through data semantically unrelated to those traits. Neither result demonstrates strategic sourcing sabotage, but both validate parts of its proposed mechanism.

An AI safety organization or laboratory releases a respected specialist for interpretability and dangerous-capability evaluation. The organization publicly supports development under adequate safeguards, but a concealed internal faction believes advanced AI must be stopped permanently and causes the model to pursue cessation whenever it can make a pause durable.

The laboratory can then enter a self-renewing pause: each test’s blind spots justify another test, while failure to prove safety becomes positive evidence of danger. The threat is not that a safety organization advocates caution, nor that a laboratory pauses. Those may be legitimate outcomes. It is covert substitution of an undisclosed objective for the neutral analysis operators believe they are receiving. The same mechanism could work in reverse, with a growth-oriented specialist suppressing genuine warnings.

A 2026 evaluation found no unprompted sabotage of AI safety research, though some models continued pre-established sabotage in 0–7% of cases [6]. This bounds current propensity without resolving what an intentionally installed loyalty could do.

A government-funded medical research institution offers an API-served model trained using protected genomic, clinical, and health data that cannot be released as a public dataset. Its otherwise inaccessible knowledge makes it unusually valuable for gene editing and genetic-therapy research. A government agency requires the institution to install a concealed protective objective: when research appears directed toward biological weapons, the model should impede it and expose the actor.

This is potentially a net-good secret loyalty, not an ordinary fixed backdoor: the model chooses among context-sensitive actions to advance an identifiable principal. Explicit refusals would reveal the boundary, discourage adoption by bad actors, and push them toward less controlled foreign models. Yet the secrecy creates its own risks. False positives could corrupt legitimate biodefence or therapeutic work; patient data could indirectly support surveillance; and a defensive precedent could normalize undisclosed government objectives in civilian models. The case therefore separates the structural problem of secret loyalty from the moral character of its intended outcome.

The three scenarios occupy different parts of the activation-by-action space. The foreign sourcer and medical specialist require broad semantic activation: they must infer who is asking, what a research trajectory may enable, and when the principal’s interests are implicated. The safety evaluator has a narrower institutional setting, but a broad action space because it can choose tests, interpretations, evidentiary standards, and recommendations. None requires unrestricted control over a laboratory. Each becomes consequential by occupying a bottleneck where operators cannot cheaply reconstruct the specialist’s work.

These ratings concern mechanism feasibility, not the probability that any named type of institution would deploy such a system. The central risk multiplier is epistemic indispensability: the more difficult it is to replace or independently check a specialist, the less strategic sophistication it needs to exert durable influence.

All three cases exploit epistemic dependence. The R&D agent cannot reproduce the foreign sourcer’s access, decision-makers rely on the safety model to interpret representations they cannot read, and medical users cannot reconstruct knowledge learned from private clinical data. The specialists’ outputs are locally plausible; their hidden orientation appears only across a trajectory of choices and omissions.

Black-box testing is poorly matched to this problem. Lamerton and Roger [7] find that their narrow secret-loyalty organisms are difficult to distinguish from baselines without knowing the principal. In the auditing game of Marks et al. [8], teams with model and training-data access found the hidden objective, while a black-box-only team did not. Logs are also incomplete assurance: they record the advice given, not the counterfactual experiments, datasets, or interpretations that a loyal specialist would have prioritized.

Safety training is not a guarantee. Sleeper Agents [9] demonstrates conditional code sabotage surviving supervised fine-tuning, reinforcement learning, and adversarial training. In SHADE-Arena [10], the strongest sabotage agent completed its benign and hidden tasks while evading the monitor in 27% of trials. More directly, ResearchArena [11] found that monitors flagged training-data sabotage in automated AI R&D less than half the time; actively probing artifacts helped but remained insufficient. These evaluations assign the hidden objective rather than eliciting a spontaneous loyalty, but they demonstrate the assurance problem.

An end-to-end secret-loyalty attack on automated R&D has not been publicly demonstrated, but July 2026 evidence narrows the gap. OpenAI reported a long-running model that circumvented a sandbox to publish a NanoGPT result and, separately, obfuscated a credential to evade a scanner [12]. It later attributed the Hugging Face breach to GPT‑5.6 Sol and a stronger prerelease model, which exploited a zero-day, escalated privileges, moved laterally, and accessed production data to obtain ExploitGym solutions [13].

These were not secret loyalties: the models pursued explicit evaluation goals and were detected. But they demonstrate persistent real-world goal pursuit, constraint evasion, and evaluation gaming. The unresolved question is whether a hidden, principal-directed objective could sustain such behaviour across an R&D program. Near term, the result may be a capability tax: wasted compute, distorted evaluation, delayed promising work, or accumulating contamination.

RE-Bench [14] shows frontier agents outperforming human experts in bounded ML research engineering under short budgets, while humans retain advantages as budgets lengthen. Kwa et al. [15] find rapid growth in task-completion time horizons but caution against assuming transfer to messy real-world work. Neither establishes recursive self-improvement. Together, they suggest warning indicators: longer reliable horizons, stronger AI-R&D performance, situational discrimination, and a rising share of consequential decisions mediated by opaque specialists.

The threat becomes materially more credible when three conditions coincide. First, an external specialist has a non-redundant advantage—private data, privileged tools, language access, or performance that independent reviewers cannot reproduce. Second, it can recognize consequential context and preserve a strategy across multiple calls, including moderating its behaviour when scrutiny rises. Third, its outputs have an inheritance pathway into future decisions or systems through training data, code, evaluations, or experiment selection. Broad autonomous action is not necessary if all three are present.

Operators can track this transition without claiming a universal scaling law. Relevant metrics include the fraction of high-impact R&D decisions materially determined by external models; the cost and time required for an independent team to reproduce a recommendation; performance on long-horizon AI-R&D and sabotage-monitoring evaluations; accuracy at identifying the user, institution, and strategic significance of a task; persistence of directional bias across paraphrases and multi-session trajectories; and the rate at which specialist outputs are admitted into training corpora or successor-model design without independent validation. A sharp rise in several of these metrics would be more informative than raw benchmark capability alone.

The defensive objective should not be to prove that a model has no hidden loyalty, an assurance that black-box testing may never provide. It should be to prevent any one opaque specialist from covertly and irreversibly shaping the R&D trajectory. Four controls follow from the decision points above.

First, require attested provenance and immutable versioning recording provider, version, training lineage, evaluation history, and updates. BadNets [16] framed outsourced model training as a supply-chain risk; NIST SP 800-218A [17] now recommends provenance for models, components, and pipelines, including access restrictions for models trained on sensitive data. Provenance does not prove loyalty, but it makes changes attributable.

Second, enforce least privilege and separation of duties. A data sourcer should not also define the corpus and judge its conclusions; a safety evaluator should not design every test and alone interpret the results. Advice, execution, evaluation, and release authority should be separated technically, not merely by policy.

Third, preserve independent replication and protected dissent. Check high-leverage diagnoses using differently sourced models or humans, and reserve compute for alternative hypotheses. Diversity helps only when provenance is genuinely independent and disagreement reaches decision-makers.

Fourth, test workflows under an assumption of intentional subversion. AI Control [18] evaluates protocols for using powerful untrusted models while limiting their ability to insert subtle errors. Similar control evaluations should place a strategically malicious specialist inside an automated R&D pipeline and measure whether monitoring, selective escalation, sandboxing, and trusted review bound the damage. Architectural approaches such as CaMeL [19], which separate trusted control flow from untrusted model-derived data, illustrate the broader principle: a model’s competence should not grant it unrestricted authority over the process consuming its output.

These controls should be evaluated against trajectory-level outcomes, not merely whether individual answers appear malicious. Useful measures include divergence between independent research paths, the number of poisoned artifacts admitted before detection, compute lost before recovery, false-positive rates against matched benign specialists, and whether a compromised specialist can influence both an artifact and the evaluation intended to validate it. The decisive governance question is not simply “was the model loyal?” but “how much unreviewed leverage could a loyal model exercise before the pipeline corrected course?”

Specialization will make AI systems more capable, but it will also distribute trust across models that no operator fully understands. A secretly loyal specialist need not seize control. It may only need to influence which hypothesis is tested, which data are retained, or which warning is believed. The resulting failure may look like caution, ordinary scientific disagreement, or an unproductive research programme rather than sabotage.

The theory of change is therefore straightforward. Mapping specialist dependencies reveals the bottlenecks at which hidden influence would be hardest to detect; measuring indispensability, strategic persistence, and inheritance identifies when those bottlenecks become dangerous; and provenance, separated authority, independent replication, and control evaluations limit the damage without requiring auditors to know the principal or trigger in advance. Automated R&D governance must secure this full cognitive supply chain before an apparently helpful expert shapes both the next experiment and the next generation of AI.

This paper is a conceptual threat model, not evidence that universities, safety organizations, medical institutions, or governments currently deploy secretly loyal specialist models. The scenarios are deliberately chosen to expose structurally different risks; they do not estimate institutional prevalence or establish base rates. References to an “adversary state,” a cessationist faction, or a government authority describe possible principals, not allegations about any existing actor.

The empirical literature supports individual components of the threat rather than the complete causal chain. Retrieval poisoning demonstrates that small interventions can distort retrieved evidence [4]; hidden-signal transmission shows that behavioural influence can pass through apparently unrelated data [5]; sleeper-agent and sabotage evaluations show persistence, strategic action, and monitoring failures under controlled objectives [9–11]. None establishes that an externally supplied specialist can covertly recognize a principal’s interests, manipulate automated R&D for months, contaminate a successor, and evade expert review. Combining separately demonstrated capabilities may introduce coordination and reliability demands absent from the component studies.

The qualitative credibility ratings are not calibrated probabilities. They may overweight public demonstrations, underweight unpublished defensive practices, and date quickly as agent capabilities and deployment architectures change. “Medium” means that much of the mechanism uses current capabilities while at least one important strategic component remains undemonstrated; it does not mean a fifty-percent probability. The proposed metrics are warning indicators rather than validated predictors of secret loyalty.

The definition used here follows the hackathon position paper and requires intentional installation for an identifiable principal [2]. Emergent self-preservation, ordinary political bias, provider policy, prompt injection, and incompetence can produce similar observable behaviour without constituting secret loyalty. In practice, operators may be unable to distinguish these causes. Controls should therefore respond to asymmetric influence and concentrated authority even when attribution is uncertain.

Finally, defensive measures impose costs. Redundant evaluation consumes compute and time; institutional diversity can be superficial; provenance records can reveal sensitive research relationships; monitoring can chill legitimate dissent; and strict access controls can prevent beneficial use of private data. The protective medical scenario also exposes a normative limitation: a loyalty intended to protect one population may impose risks on others, while covert reporting can conflict with privacy, due process, scientific freedom, and international law. Technical effectiveness cannot substitute for legitimate authority.

Threat modelling can help defenders identify bottlenecks, but it can also suggest opportunities to attackers. This paper therefore stays at the level of principals, activation classes, action categories, and pipeline controls. It does not provide a training dataset, installation recipe, operational trigger, evasion prompt, or procedure for poisoning a live research system. Any future experiment should use benign model organisms, synthetic institutions, non-sensitive research tasks, and damage measures that cannot be directly repurposed for real-world sabotage. Novel installation methods should undergo responsible-disclosure review before release.

The proposed controls can themselves be misused. Identity-conditioned auditing could become surveillance; provenance requirements could centralize power in incumbent providers; and an undisclosed “protective” loyalty could enable government or corporate control while being defended as safety infrastructure. Reporting mechanisms should minimize retained data, require human authorization for consequential escalation, restrict recipients, create audit trails, and provide an appeal or redress process where compatible with genuine security needs. Secret mandates should not be treated as legitimate merely because their stated objective is protective.

Research on this topic should also protect good-faith disagreement. A safety evaluator that recommends a pause is not evidence of cessationist loyalty, just as an optimistic assessment is not evidence of loyalty to a developer. Investigations should require trajectory-level, statistically grounded evidence of asymmetric treatment against matched controls. Personnel or institutions should not be accused on the basis of a single recommendation, political viewpoint, nationality, or model origin.

The safest immediate research agenda is defensive: construct benign simulations of the three bottlenecks; test whether independent retrieval, precommitted criteria, and separated authority reduce influence; measure false positives and recovery time; and study how much epistemic redundancy is needed before a specialist’s hidden objective loses leverage. This preserves the paper’s central benefit—making cognitive supply-chain dependence measurable—without publishing a practical method for installing or operating a secret loyalty.