Capture & Privacy
What Benchmark captures, what it leaves behind, and how long the data is kept.
#What Benchmark captures
Benchmark collects request metadata for supported AI requests. When Benchmark content mode is enabled, it also captures the final user prompt.
- Request metadata needed for analysis, such as model/provider, token usage, timing, and status where available
- The final user prompt, only in Benchmark content mode
#What Benchmark does not capture
- System instructions
- Developer instructions
- Assistant responses
- Earlier conversation turns
- Retrieved context
- Tool definitions or tool results
- Attachment contents
- Arbitrary provider request objects
- Request headers
Benchmark does not reconstruct the full conversation around a request.
#Capture modes
| Mode | What it captures |
|---|---|
| Metadata | Request metadata only; no prompt content |
| Benchmark content | Metadata plus the final user prompt |
#How long data is kept
- Raw captured prompt data: up to 14 days.
- Benchmark metadata: retained while the Benchmark is active, then cleaned up within 30 days after it ends or is cancelled.
- Benchmark tokens: invalidated when the Benchmark ends, is cancelled, or expires.
- Deletion evidence: retained for up to 365 days to show that deletion completed.
#Access and protection
- Benchmark tokens are separate from normal inference credentials.
- The full Observe token is shown once and is not stored in plaintext.
- Captured Benchmark data is restricted to the Benchmark process and authorized account access.
- Tokens are invalidated when the Benchmark ends or is cancelled.
#When a Benchmark ends
When collection completes or you cancel it, collection stops and the Observe token is invalidated. Retained data then follows the documented retention schedule; cancellation does not imply that every record is deleted immediately.
#When you delete a project or account
Deleting a project removes its Benchmark data, including captured prompt content, records, results, and tokens, as part of the deletion flow. Deleting an account removes its independent Benchmark data as well; deletion evidence may remain for up to 365 days to prove the deletion occurred.
#What Benchmark uses the data for
Captured data is used to classify your AI requests and produce Benchmark results. It is not used to execute those requests through every model.