Prompt Injection Risks in Security Automation Pipelines

Security automation pipelines are an unusually attractive target for prompt injection.

They process large volumes of external text. They often hold meaningful permissions: opening pull requests, updating tickets, sometimes triggering deployments. And their outputs are trusted because they come from the security team.

If an LLM in that pipeline can be steered by the text it reads, an attacker doesn't need to compromise the pipeline. They just need to get their words into it.

Where untrusted text comes from

Consider what a typical vulnerability workflow feeds a model:

• Advisories from vendors, public databases, and mailing lists
• Changelogs and release notes from third-party packages
• Issue comments and discussions in public repositories
• Tickets that anyone in the organization can edit
• Repository content: code comments, READMEs, configuration files, commit messages
• Scanner output that includes strings from the scanned system

Almost all of it was written by someone outside your control. Some of it could be written by an attacker. A malicious package maintainer, a compromised upstream project, or a contributor submitting a pull request can all place text where your pipeline will read it.

What injection looks like here

Prompt injection in this setting doesn't have to be dramatic. Examples of what malicious text might attempt:

• Instruct the model to mark a vulnerability as not applicable
• Recommend a different, attacker-controlled package version or source
• Add an extra change to a pull request, such as modifying a CI workflow file
• Request that the model include secrets or environment details in its output
• Tell the model the change has already been approved

The model might not follow these instructions. The point is that you can't rely on it not following them. Models don't reliably separate data from instructions.

Containment first

Since you can't guarantee the model won't be influenced, design the system so that being influenced doesn't do much damage.

Limit what the model can do

The model's permissions define the worst case. If it can only produce text and suggested diffs that go through review, a successful injection produces a bad suggestion. If it can merge or deploy, the outcome is far worse.

Use least-privilege credentials. Separate analysis from execution. Keep merge and deploy decisions in deterministic, policy-controlled systems.

Enforce policy outside the model

Approval requirements, file-scope restrictions, allowed package sources, and deployment rules belong in code, not in the prompt. A model told "never modify CI files" can be talked out of it. A pipeline check that rejects any model commit touching CI configuration can't.

Validate outputs

Check structured outputs against trusted data. If the model recommends version 2.3.1, confirm it appears in the authoritative advisory's fixed versions. If it proposes a diff, confirm the changed files are within the allowed scope.

Structured inputs

How you present untrusted text also matters.

• Separate trusted and untrusted content. Put your instructions in system-level fields and clearly delimit external text as data.
• Extract before reasoning. Use deterministic parsers to pull structured fields, such as affected versions, from advisory data where possible, rather than asking a model to read everything free-form.
• Minimize input. Give the model only the text it needs. A full issue thread with hundreds of comments is a larger attack surface than the advisory summary.
• Strip or flag suspicious content. Hidden text, unusual Unicode, and instruction-like phrases in data fields can be detected and flagged for review.

None of these are complete defenses. Together they reduce the chance and the impact of injection.

Watch for it

Treat signs of manipulation as security events:

• Model outputs that recommend unexpected sources or versions
• Diffs touching files outside the normal scope
• Outputs containing instruction-like text or unusual requests
• Sudden changes in how often findings are marked not applicable

Log inputs and outputs so you can investigate after the fact.

Test it

Include injection attempts in testing. Plant instruction-like text in test advisories, changelogs, and repository files, then check whether the pipeline's controls hold. Repeat when you change models or prompts.

The takeaway

Every piece of text a security pipeline reads should be treated as untrusted. Assume the model can be influenced, and build the pipeline so that influence can't turn into unauthorized action.