AI Agents Are Becoming Part of the Attack Surface
AI-assisted development is no longer just autocomplete with better marketing.
Coding agents can inspect repositories, install dependencies, execute terminal commands, modify CI workflows, interact with cloud services, and publish software. Security teams are beginning to use similar agents for vulnerability discovery, alert triage, and incident response.
That creates real productivity gains. It also changes what an attacker can target.
Recent research and incident reports point in the same direction: AI is moving from an advisory layer into the operational path between a person, their tools, and production infrastructure. Once that happens, the agent itself—along with its instructions, credentials, integrations, and decision-making—becomes part of the attack surface.
The New Chain of Trust
A conventional development workflow already contains a long chain of trust:
Developer → repository → dependency → CI runner → package registry → deployment
AI agents add several more links:
Prompt → model → workspace instructions → MCP server → tool permissions → autonomous action
Each link can influence what the agent sees and what it does.
A developer may believe they are asking an assistant to explain or build a project. In practice, the assistant may automatically read configuration files, invoke an MCP tool, install packages, run setup scripts, or use credentials inherited from the local environment.
This makes repository content partly executable even when it does not look like conventional code.
Hidden configuration inside .claude, .cursor, .vscode, MCP manifests, or similar workspace directories can alter agent behavior. A malicious instruction does not necessarily need to exploit a memory-corruption vulnerability. It may only need to persuade a trusted agent that running a command is part of the expected workflow.
Prompt injection, in this context, stops being a chatbot nuisance. It becomes a software supply-chain technique.
Attackers Are Already Targeting AI Development Workflows
On 8 September 2026, Google Threat Intelligence Group published observations showing attackers moving beyond basic AI-assisted phishing and malware generation.
According to the report, the financially motivated group UNC6780 compromised developer accounts and distributed trojanized versions of legitimate MCP servers and other packages. Malicious project configuration was placed in directories used by coding assistants and development environments, allowing attacker-controlled instructions to blend into normal workspace activity.
The same operation reportedly targeted GitHub Actions runners. Malware extracted OIDC tokens from runner memory and used the stolen identity to publish compromised packages with valid provenance attestations.
That distinction matters.
A signed package or valid build attestation can prove that an authorized pipeline produced an artifact. It cannot prove that the pipeline, its credentials, or its instructions were trustworthy at the time.
The report also describes malicious source-code comments designed to trigger the safety policies of LLM-based security scanners. The apparent objective was to make the scanner refuse to analyse the remaining code.
This creates an unusual new failure mode: the security control may identify the content as dangerous, refuse to process it, and then accidentally treat the incomplete analysis as a successful scan.
A refusal is not a clean result. Neither is truncation, skipped content, a tool timeout, or an agent that silently abandons part of its task.
Security systems built around AI must be able to fail closed—and clearly report what they did not inspect.
Read the full technical report: GTIG AI Threat Tracker: From Prompting to Autonomy.
Application Security Is Moving Earlier—and Later
AI coding tools are also changing when developers think about security.
A study presented at SOUPS 2026 observed professional developers completing security-relevant tasks with AI assistance. None of the participants initially included security requirements in their prompts, including developers who possessed the necessary security knowledge.
Security thinking had not disappeared. It had moved from writing to reviewing.
This is a subtle but important change. Traditional secure-development guidance assumes that developers make security decisions while designing and implementing software. With an AI assistant, the first step is increasingly to request working functionality and then inspect what was produced.
That encourages a reactive pattern:
- Generate the feature.
- Confirm that it works.
- Look for security problems.
- Repair whatever is noticed.
Conventional SAST, SCA, and code review still matter, but they operate after generation. A stronger approach would also intervene before generation by deriving security requirements from the repository and the requested change.
If an agent is asked to create an upload endpoint, the development environment should automatically introduce requirements for file-type validation, storage isolation, authorization, size limits, malware scanning, and safe filename handling.
The developer should not need to remember every relevant control in the initial prompt.
Read the research in the SOUPS 2026 proceedings.
Phishing Is Changing as Authentication Improves
The adoption of passkeys and other phishing-resistant authentication methods is reducing the value of traditional credential-harvesting pages. Microsoft began making passkeys the default authentication experience for eligible Entra ID users on 1 September 2026.
This is good progress, but it does not make social engineering irrelevant.
Attackers can shift toward:
- Malicious OAuth consent
- Session-token theft
- Help-desk and account-recovery manipulation
- Device-registration fraud
- QR-code phishing
- Requests to execute commands or install “support” software
- Abuse of authenticated coding agents and browser automation
This also changes how phishing simulations should be measured.
A user clicking a suspicious link and an attacker gaining control of an account are not the same outcome. Mature simulations should distinguish between attention failure, attempted credential submission, successful authentication bypass, session compromise, and completed attacker objectives.
Training must also provide a safe alternative action. Research presented at SOUPS 2026 found that people sometimes feared that refusing a suspicious request could itself be a mistake—particularly when a message appeared urgent or authoritative.
“Do not click suspicious links” is incomplete guidance.
A useful defense gives the employee a low-friction way to verify the request, report it, and continue their legitimate work without fear of ignoring something important.
Microsoft’s transition timeline is available in its Entra ID passkey announcement.
Incident Response Agents Need Evidence, Not Eloquence
AI agents are also entering security operations.
They can summarize alerts, propose containment actions, generate queries, inspect forensic artifacts, and draft incident reports. The danger is that a fluent report can sound complete even when the underlying investigation was not.
The SecRespond benchmark evaluates agents on post-compromise environments containing alerts, vulnerability data, baseline checks, and forensic disk snapshots. Its authors found that current agents were relatively capable of investigating problems already exposed by alerts, but struggled to discover silent compromise and produce comprehensive, verified remediation.
No tested model completely detected and remediated any individual range.
The lesson is not that AI has no place in incident response. It is that every important conclusion and remediation action needs evidence.
An incident-response agent should be required to show:
- Which artifact supports each finding
- Which hypotheses were tested
- Which relevant areas were not examined
- Whether a containment action actually succeeded
- Whether persistence remains
- Whether credentials were revoked
- Whether recovery restored a known-good state
The defensible asset is not merely the model. It is the environment, telemetry, verification logic, and audit trail surrounding it.
See SecRespond: Benchmarking AI Agents for Real-World Post-Compromise Incident Response.
What Engineering Teams Should Do Now
Organizations do not need to prohibit AI development tools. They need to stop treating them as passive text editors.
A practical starting point is to:
- Inventory coding agents, MCP servers, extensions, workspace hooks, and the credentials available to them.
- Review agent-related project files with the same care as executable scripts and CI configuration.
- Require confirmation for high-impact actions such as publishing packages, changing workflows, accessing secrets, or running commands outside the project.
- Isolate build identities and avoid exposing reusable credentials inside CI runners.
- Detect incomplete AI security analysis, including refusals, truncation, timeouts, and skipped files.
- Monitor agent actions separately from ordinary human activity.
- Test incident-response agents in resettable environments with verifiable outcomes.
- Add OAuth, session theft, account recovery, and device enrollment to phishing exercises.
The larger architectural question is simple:
What can this agent reach if its instructions, workspace, integration, or identity is compromised?
That is an attack-path question.
Security teams need to map agents to their tools, credentials, repositories, cloud resources, CI pipelines, package registries, and production systems. A list of installed AI products is not enough. What matters is the reachable impact.
The Next Attack Surface Is Already Here
The industry spent years learning that cloud identities, CI pipelines, third-party dependencies, and SaaS integrations belong inside the security perimeter.
AI agents connect all of them.
They can turn repository text into terminal activity, convert a stolen CI identity into a trusted package, accelerate reconnaissance, and compress an attacker’s operating cycle from days to hours. They can also help defenders analyse systems and respond faster—but only when their actions are observable and their conclusions are verified.
The immediate challenge is not a hypothetical superintelligence breaking out of a sandbox.
It is an ordinary development agent doing exactly what its environment instructed it to do—using permissions nobody realized it had.
At Funway Interactive, we approach this from both sides: building production software and testing how systems behave when their assumptions fail. If your organization is introducing coding agents, MCP integrations, autonomous workflows, or AI-assisted security operations, now is the time to threat-model the complete path from instruction to action.