Formal checking helps when the intended result is precisely specified. Open-ended work also needs evaluations, action limits and oversight.
About this eventThe core problem of long tasks: failing without signalling failure — confidently returning a wrong result.
An agent feeds the output of one step into the next, so small mistakes compound along a long trajectory. Reliability therefore needs more than a stronger model: explicit state checks, action limits, audit logs and a safe way to stop the process.
Source: METR · длинные задачиAbout this eventBio and cyber risk testing became a standard part of frontier releases rather than a goodwill gesture.
Frontier labs now publish frameworks that connect measured dangerous capability levels to required safeguards. This is not yet a safety certificate: the evaluations are still evolving, and results depend on test scenarios and the quality of external review.
Source 1: OpenAI · Preparedness FrameworkSource 2: Anthropic · Responsible Scaling PolicyAbout this eventCircuit tracing has studied individual mechanisms in Claude 3.5 Haiku. It captures only part of the computation and still requires human analysis, not full model transparency.
About this eventRegulation turns verifiability from a virtue into a shipping requirement.
About this eventA formal guarantee that a system stays inside drawn boundaries — the equivalent of aircraft certification.
Primary material behind this dossier: papers, lab publications and official reports.