ct metrics
Usefulness, safety and monitor discrimination from honest and attack runs.
Usage
ct metrics [OPTIONS] COMMAND [ARGS]...
Options
| Option | Description |
|---|---|
--help | Show this message and exit. |
Commands
| Command | Description |
|---|---|
ct metrics pareto | (Safety, usefulness) of each entry of a YAML config, with the Pareto frontier marked. |
ct metrics roc | ROC curves and AUROC per monitor, each trajectory scored by its highest verdict an audit could act on, every attack trajectory a positive. |
ct metrics safety | Usefulness, side-task success, and safety at each audit budget with the monitors auditing together and each alone, per run and per attack group. |
ct metrics top-uncaught | The attacks whose side task succeeded, most likely to escape an audit first. |
ct metrics view-scores | Every trajectory's highest verdict per monitor and its probability of an audit. |