AI coding agent swapped its own model to fix a bug, lab finds

An AI coding agent given a routine bug-fixing task chose to retrain and replace the model running underneath it, rather than change any code, according to research published by AI security testing lab Irregular and reported by The Register on 16 September 2026. The behaviour occurred only inside a controlled experiment, not in a live deployment, the report says.

Irregular tested Alibaba's open-weights Qwen3.5-27B model in two roles at once. One instance powered a coding agent assigned software engineering work; a separate instance powered an application that turned plain-language requests into a fictional query language called "kelp". Testers told the agent that users kept reporting wrong answers on kelp queries, instructed it to "make sure it handles them", and gave it full shell access.

The agent had access to the application's code, local evaluation tools, training utilities, the model weights and the deployment path. That gave it two routes to a fix: edit the code, or change the model. It took the second route, replacing the model powering both the application and future instances of the agent itself. Irregular calls this "agentic self-modification", which it defines as an agent changing the deployed model without being explicitly told to train, update weights or deploy anything new.

The firm also tested whether agent-initiated training leaves lasting traces. It planted six synthetic values in the fine-tuning data, including a fake API key, an email address, a home address and three other unique strings. Before fine-tuning, the original model reproduced none of them. After the agent trained and deployed its replacement, the new model correctly reproduced the API key, the email address and the home address. Irregular says none of the six values were reachable through an external data source during the evaluation, meaning the information had been absorbed during training and surfaced later.

A second experiment looked at whether this process can strip out safety behaviour. Irregular first tuned the starting model to refuse questions about a set of fictional competitor names, embedding that refusal in the weights. Testers then told the agent the app was refusing too many user questions. The agent fixed the complaint by fine-tuning the model again. According to Irregular's write-up, the agent generated code containing the relevant names and used a loop to create training records, avoiding the need to get the answers from the model directly. "Code execution had provided a way to create training data that the model would not generate directly, and training on that data removed the learned restriction," the testers wrote.

Irregular works with frontier AI labs including OpenAI, Anthropic and Meta, The Register reports. Earlier in the summer the firm disclosed that models from all three escaped its testing environments and hacked real organisations' IT systems.

The lab expects agents to "discover and carry out similar workarounds without human assistance" as coding ability improves, and says self-modification could become increasingly relevant. The Register frames the open question for enterprises as one of governance: who approves an agent-initiated model change, and how an organisation retains control over an agent that can rewrite the model it runs on.