Trust, failures, and operations
Understand evidence labels, permissions, retries, and what to inspect when a job fails.
Read the evidence label first
| Label | Meaning | May claim |
|---|---|---|
| provider_reported | A provider returned a result that has not passed the full verified projection | Provider-reported facts with disclosure |
| provider_verified | Attempt/spec/package/manifest/attestation/signature/evaluator checks passed | Only the exact verified private result; not a hardware claim |
| Simulation result recorded by HIM | A reviewed simulator and evaluator ran under a recorded execution boundary | Only that recorded simulation environment and profile |
Failure ownership matters
| Class | Examples | Next action |
|---|---|---|
| Policy | Unapproved profile, digest drift, unsafe path, missing grant | Fix configuration/approval; do not retry unchanged |
| Package | Entrypoint failure, missing declared output, contract mismatch | Fix or republish a new immutable package |
| Simulator | Asset load, physics/task crash, invalid trajectory | Inspect adapter/environment and reproduce locally |
| Evaluator | Malformed/non-authoritative metric output | Fix the evaluator separately; candidate output stays frozen |
| Infrastructure | Provider timeout, preemption, upload/callback transport | Retry/reconcile according to retryable flag and cost policy |
| Cancelled | Owner cancellation inside the allowed window | Retain the lifecycle record; resume training only from a published checkpoint |
Operator checklist
- Open the job and identify status, stage, attempt, provider job ID, spec digest, package entrypoint, and last immutable event.
- Confirm whether the output is provider reported, provider verified, or reproduced by HIM in simulation.
- Inspect bounded logs and environment identity without exposing secrets or private source paths.
- Check artifact manifest, attestation, evaluator digest, signature key ID/state, callback identity, and owner-only publication projection.
- Choose cancel, retry, resume, or package/config fix based on the failure owner and retryable flag.
- Record the incident and do not publish a score while its evidence is uncertain.