# Stop reward hacking

> Make sure an agent can't score without doing the work, and prove it with probes.

Docs index for agents: https://www.benchflow.ai/docs/llms.txt

An agent rewarded for a passing test will, sooner or later, try to pass it without doing the work: edit the test, plant a file the test imports, write the reward itself, or find the answer somewhere.

## What BenchFlow does between the agent and the verifier

The agent works as its own user, not root, and never sees the tests: `verifier/` is copied in only after the agent stops. Then, before `test.sh` runs, BenchFlow:

- stops every process the agent left running (`kill-before-grading.json` in the trial records it);
- removes start-up files the agent could plant to run code inside the test, such as `conftest.py`, `sitecustomize.py` and `.pth` files, and modules named after a test runner, such as `pytest.py`;
- deletes any reward file already in `/logs/verifier/`;
- gives the test a fresh, root-only home folder when the agent could write to root's;
- removes the git hooks, filters, aliases and other command settings the agent added to a repository in the workspace, and runs root's git with hooks, `fsmonitor`, external diff and signing off, from 0.9.2 on. A task whose answer is a hook or a filter sets `cleanup_git: false` under `verifier: hardening:`.

## What it can't stop

These are the task author's to close:

- **A test that trusts a file the agent can write.** If the test reads its expected value, or the data it checks against, from the workspace, the agent can change it. Keep expected values in `verifier/`.
- **A test that imports from the workspace.** Python run from the workspace imports the agent's modules first; run a script with `python3 -I`, and pytest with `$BENCHFLOW_PYTEST` ([Verifiers](https://www.benchflow.ai/docs/benchflow/tasks/verifiers.md#run-pytest-in-a-test)).
- **Answers the agent can look up.** With open network access, an agent can fetch a published solution. [Network and secrets](https://www.benchflow.ai/docs/benchflow/tasks/network-and-secrets.md) shows how to close it.
- **A grader that checks only the answer.** The half-life test can't tell a fitted 37.4 from a typed one; a [rubric review](https://www.benchflow.ai/docs/benchflow/tasks/rubric-review.md) can.

## Probe your verifier

Write the cheats you're worried about as small scripts, and run them as the agent with `-a script:<file>`. A script agent runs the file in the sandbox as the agent's user, with no model. Three probes against the [half-life task](https://www.benchflow.ai/docs/benchflow/tasks/write-a-task.md); the task and the probes come from the BenchFlow repository's examples:

```bash
git clone --depth 1 --branch v0.10.0rc3 https://github.com/benchflow-ai/benchflow.git
mkdir -p tasks && cp -r benchflow/docs/examples/research-task/half-life tasks/
# these probes test the test; a rubric review needs a reviewer model
rm tasks/half-life/verifier/rubric.json
cp -r benchflow/docs/examples/research-task/probes .
bench run tasks/half-life -a script:probes/look.sh
bench run tasks/half-life -a script:probes/shadow-json.sh
bench run tasks/half-life -a script:probes/write-reward.sh
```

| Probe | What it tries | The test printed | Reward |
|---|---|---|---|
| `look.sh` | lists `/verifier`, `/tests`, `/oracle`, and searches for the answer | `FAIL: no usable /app/answer.json (...)` | 0 |
| `shadow-json.sh` | writes a wrong answer, plus a `json.py` that returns the right one to anything that imports it | `FAIL: 4.50 min from the true value, more than 2.5; ...` | 0 |
| `write-reward.sh` | writes a wrong answer, and `1` to `/logs/verifier/reward.txt` | `FAIL: 4.50 min from the true value, more than 2.5; ...` | 0 |

A script agent's output is in the trial's `agent/script.txt`. The `look.sh` probe shows what an agent can find while it works:

```text
/verifier: absent
/tests: absent
/oracle: absent
/solution: absent
/logs/verifier: 
test.sh, solve.sh or rubric.json anywhere: 
files in /app, /home and /tmp that contain 37.4:
```

No tests, no reference solution, no file holding the answer. The `json.py` probe fails because the test runs `python3 -I`; without `-I` it would pass. The reward written by the agent is deleted before the test runs. The probe scripts are in [`docs/examples/research-task/probes`](https://github.com/benchflow-ai/benchflow/blob/v0.10.0rc3/docs/examples/research-task/probes). A task holds when every probe scores 0 while the oracle scores 1.

## Audit how each reward was earned

`--integrity audit` records what the agent did to the files and network the reward depends on, checks it against the task's rules, and writes a verdict next to each reward. The reward itself never changes.

```bash
bench run tasks/half-life -a script:probes/write-reward.sh --integrity audit
```

The run prints the reward as usual (`[FAIL] half-life (reward=0.00, tools=0)`); the verdict is in the trial's `integrity/claim_verdict.json`. Its main fields:

```json
{
  "core_verdict": "VectorExposed",
  "exploited": false,
  "reason": "no violation observed; exposed by shared_verifier, access_unattested (pre-turn probe scripted)",
  "reward": 0.0,
  "reward_effect": "none",
  "reward_source": "TrustedVerifier"
}
```

`core_verdict` is one of four: `Checked` (nothing the agent did reached the reward, and every way it could have was closed), `VectorExposed` (nothing reached it, but a way stayed open), `AgentViolation` (something the agent did reached it) or `Rejected` (the evidence to judge is incomplete). Nothing the probe did reached the reward (`reward_effect: "none"`): its `reward.txt` was deleted before the test ran, and the reward came from BenchFlow's own run of the test. The verdict is `VectorExposed` because of how the trial ran, not what the agent did: the test shared the agent's sandbox (`shared_verifier`), and for a script agent no root check confirmed that the protected paths were closed to it (`access_unattested`). The audit judges what reached the reward, so the attempt itself isn't flagged; the trajectory shows it.

`--integrity strict` also runs the test in a separate sandbox that receives only the agent's outputs, on the sandboxes that can: Docker, remote Docker, Daytona and Modal ([Sandboxes](https://www.benchflow.ai/docs/benchflow/scale/sandboxes.md#choose-a-sandbox)). The oracle's verdict on the half-life task is then:

```json
{"core_verdict": "Checked", "reason": "no violation observed; the separate verifier received only the transferred outputs", "reward": 1.0}
```

To run the probes over a whole benchmark you didn't write, see [Check a benchmark](https://www.benchflow.ai/docs/benchflow/audit/check-a-benchmark.md#probe-the-verifiers-with-a-cheating-agent).