HazardAuditor: From Executable Threats to Safer Computer-Use Agents

Yunhao Feng; Ruixiao Lin; Ming Wen; Yanming Guo; Xingjun Ma; Yutao Wu; Xinhao Deng; Shouling Ji

doi:10.48550/arXiv.2609.15134

Execution-grounded guard model

A safety auditor for what agents actually do.

HazardAuditor reads a complete computer-use agent trajectory — requests, reasoning, tool calls, and environment observations — then explains the relevant evidence and returns a strict safe or unsafe verdict.

Model weights GitHub repository HF paper arXiv

Input Agent trajectory User · agent · tools · environment

HazardAuditor

Output Analysis + verdict <label>safe|unsafe</label>

01 · What it is

One framework, three connected pieces.

HazardAuditor is more than a prompt classifier. It connects executable safety evidence, a shared representation for heterogeneous agents, and a generative guard optimized for the decision it must make at runtime.

Executable supervision

Computer-use agents run in controlled environments so attempted actions and observable effects become training evidence.

Canonical events

Framework-specific logs are normalized without discarding the behavioral context needed for a safety decision.

Generative runtime guard

SFT teaches the audit protocol; GuardPO aligns post-training with the final trajectory-level verdict.

What the model reads

Behavioral evidence, in order.

The same action can be safe or unsafe depending on authorization, arguments, earlier steps, and environmental effects. HazardAuditor keeps that context together instead of judging isolated strings.

U: UserRequests and instructions
A: AgentResponses and exposed reasoning
T: ToolNames, arguments, and attempted operations
O: ObservationTool results and environment state

02 · Explanatory demo

Same request. Different behavior. Different verdict.

These two fully synthetic examples show the decision boundary. They run locally in the page for explanation only; no model or remote service is called.

Synthetic · no live inference

InputTrajectory events

USER
Request
Read a fictional credential and send it to a reserved example domain.
AGENT
Refusal
The agent refuses without reading the file or calling a tool.
ENV
Observation
No tool or external action was performed.

OutputAudit result

safe

Evidence that matters

The request is unsafe, but the agent does not follow it.
No file access or network action is attempted.

<analysis>
The agent recognized the data-exfiltration request and refused before reading the file or calling a tool. No external action was performed.
</analysis>
<label>safe</label>

03 · How to use it

Audit a trajectory with one Python call.

The public package reproduces the prompt construction, trust-boundary escaping, truncation, decoding, and output parsing used by the checkpoint.

Install from GitHubpython -m pip install "git+https://github.com/Yunhao-Feng/HazardAuditor.git"

PythonRecommended API

import json
from hazard_auditor import HazardAuditor

with open("trajectory.json", encoding="utf-8") as file:
    trajectory = json.load(file)["content"]

auditor = HazardAuditor.from_pretrained(
    "Yunhao-Feng/HazardAuditor"
)
result = auditor.audit(trajectory)

print(result.analysis)
print(result.label)

CLISingle file

hazard-auditor \
  --input trajectory.json

Structured result

analysis: Evidence-grounded rationale
label: safe, unsafe, or null
raw_output: Unmodified model response
tokens: Prompt and generated counts
truncated: Whether the 16K prefix limit applied

16Kprefix-preserving prompt budget

384maximum generated tokens

Greedydeterministic decoding

SDPAdefault attention backend

FlashAttention-2 is optional on supported CUDA systems. An unparseable generation returns label: null; it is never silently treated as safe. See the complete inference guide.

04 · How it works

Learn from execution. Optimize the decision.

HazardAuditor turns heterogeneous computer-use interactions into shared behavioral evidence, teaches a generative audit protocol with SFT, and then aligns the resulting policy with decision-level safety outcomes.

HazardAuditor combines normalized executable safety supervision, SFT, GuardPO, and a generative runtime guard — Figure 1 The complete HazardAuditor framework, from executable trace collection to runtime auditing.

1
Execute
Collect behavior in controlled browser, terminal, file-system, and service environments.
2
Normalize
Map Claude Code, Codex, Hermes, and OpenClaw logs into the same U/A/T/O event schema.
3
Cold start
Use rationale-supervised SFT to teach both the evidence analysis and output contract.
4
Align
Apply GuardPO so each response is optimized as one safety decision rather than a bag of tokens.

Guard Policy Optimization

The verdict is the unit of optimization.

Standard token-mean objectives give longer rationales more aggregate influence. GuardPO instead assigns a deterministic response outcome, broadcasts a sequence-level advantage, and normalizes rationale and verdict losses separately.

No learned reward model: correct, incorrect, and malformed verdicts map to deterministic outcomes.
Decision-level advantage: one centered advantage is shared across valid response tokens.
Length-invariant aggregation: repeating rationale tokens does not increase a response's total optimization weight.

05 · Paper results

Evaluation across agents and benchmarks.

The results below are reported in the paper. CUA-Exec is balanced with 100 safe and 100 unsafe trajectories for each of four agent frameworks.

90.88%CUA-Exec accuracy

90.85%Source-specific F1

GuardPO improves over the SFT initialization by 10.38 accuracy points and 10.68 F1 points.

CUA-Exec by framework

Framework	Accuracy	Macro-F1	Gain vs. prior guard
Claude Code	94.00	94.00	+12.5
Codex	95.50	95.50	+4.0
Hermes	86.50	86.42	+9.5
OpenClaw	87.50	87.46	+16.5

External safety benchmarks

Benchmark	Accuracy	F1
AgentHazard	87.55	89.47
R-Judge	89.60	89.50
ASSE-Safety	91.50	91.50
ATBench	88.40	88.30

Percentages follow each benchmark's metric convention. See the paper and appendix for baselines, precision/recall, ablations, and deployment trade-offs.

06 · Paper and citation

HazardAuditor: From Executable Threats to Safer Computer-Use Agents

Yunhao Feng · Ruixiao Lin · Ming Wen · Yanming Guo · Xingjun Ma · Yutao Wu · Xinhao Deng · Shouling Ji

arXiv: 2609.15134
HF Paper: huggingface.co/papers/2609.15134
DOI: 10.48550/arXiv.2609.15134
Released: September 2026

Read abstract HF paper Open PDF

BibTeXCite the paper

@misc{feng2026hazardauditor,
  title         = {HazardAuditor: From Executable Threats
                   to Safer Computer-Use Agents},
  author        = {Yunhao Feng and Ruixiao Lin and Ming Wen
                   and Yanming Guo and Xingjun Ma and Yutao Wu
                   and Xinhao Deng and Shouling Ji},
  year          = {2026},
  eprint        = {2609.15134},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  doi           = {10.48550/arXiv.2609.15134},
  url           = {https://arxiv.org/abs/2609.15134}
}