Designing Audit Trails for AI-Generated Security Changes

Months after a change ships, someone will ask why it happened. Maybe it caused an incident. Maybe an auditor is sampling remediation work. Maybe a vulnerability reappeared and someone wants to know whether the original fix was real.

For human-authored changes, the answer is usually spread across a ticket, a pull request, and a deployment log. For changes involving an LLM, there's an extra question: what did the model see, and what did it actually contribute?

A good audit trail answers all of it without archaeology.

What to capture

Think of the trail as a chain. Each link should connect to the next.

1. Triggering evidence

What started the work?

• The finding: vulnerability ID, affected component, version, asset
• Source and timestamp: which scanner or feed, and when
• Enrichment data used for prioritization: exploitation status, exposure, asset criticality
• The calculated priority and the policy rule that produced it

2. Model inputs

What was the model given?

• The prompt template and version
• The specific context supplied: advisory text, code files, configuration, runbooks
• The model identifier and version
• Any tool calls the model made and their results

You may not want to store every input in full forever, especially if it includes sensitive code. At minimum, store references or hashes that let you reconstruct it.

3. Recommendation

What did the model propose?

• Its structured output: recommended action, target version, rationale
• Any validation checks run on that output and their results
• Whether the recommendation was accepted, modified, or rejected

4. The change

What actually changed?

• The diff, with model-authored portions clearly marked
• The branch, pull request, and commit identifiers
• Any human edits made after the model's proposal

The distinction between model-authored and human-edited code matters. It tells you later where an error came from.

5. Test and scan results

What checked the change?

• CI build and test outcomes
• Security scans: SCA, SAST, container scanning
• Policy checks and their results
• Links to full logs

6. Approvals

Who accepted the risk?

• The approval tier and how it was calculated
• The identity of each approver, or the policy that permitted automatic merge
• Timestamps and any review comments

7. Deployment record

Where and when did it ship?

• The artifact identifier, such as an image digest or release version
• Target environments and rollout stages
• Health check outcomes at each stage
• Any halts or rollbacks

8. Verification

Did it work?

• The post-deployment scan or inventory check confirming the fix
• When the finding was closed and on what evidence
• Whether it was later reopened

Linking it together

The value of a trail comes from being able to follow it. Use consistent identifiers that connect each stage: a remediation ID that appears in the ticket, pull request description, commit messages, deployment metadata, and verification record.

Avoid relying on people to copy links between systems. Have the pipeline attach identifiers automatically.

Integrity

An audit trail you can quietly edit isn't much of one.

• Write records to append-only or write-once storage
• Restrict who can modify or delete records, and log those actions
• Sign or hash key records, such as approvals and deployment events
• Keep the trail separate from the systems it describes, so a compromise of one doesn't erase the other

Retention and privacy

Model inputs may contain source code, configuration, or data that you would not otherwise keep for long. Decide what to store in full, what to reference, and how long to retain it, aligned with your existing policies. Don't store secrets; if they appear in inputs, that is a bug to fix upstream.

Making it usable

A trail nobody can read helps nobody. Provide a simple view per remediation: one page that shows the finding, the recommendation, the change, the checks, the approvals, the deployment, and the verification in order.

Then test it. Pick a random closed finding each month and try to reconstruct its story. Gaps show up quickly.

Why it matters

AI-assisted changes can be a big efficiency gain. They stay trustworthy only if you can show, for any change, what evidence drove it, what the model contributed, who approved it, and that it actually fixed the problem.