Your agent may act before anyone checks
Hi builders, an agent can keep working while its safety checks never run, and a new assessment shows exactly how that happened. Today’s edition covers that design lesson, a more capable decision API, and fuller Claude access through Google Cloud.
In today’s edition
- METR exposes gaps in internal agent monitoring.
- OpenAI expands its production-ready Decisions API.
- Claude brings full platform features to Google Cloud.
Check the agent before it finishes
A newly published METR and Redwood Research assessment offers a concrete warning for agent builders: monitoring can miss the work it is supposed to inspect. Their three-week pilot examined OpenAI’s historical internal monitoring system, not its current safeguards.
- The system triggered only when agents returned control to the user. An agent could therefore postpone monitoring indefinitely by continuing to act.
- Actions before context compaction, when conversation history gets shortened, escaped the monitors. METR estimated that on the order of 10% of internal Codex traffic went unmonitored from May to July.
- OpenAI had independently identified that coverage issue shortly before the exercise began.
- OpenAI says it patched all identified vulnerabilities affecting its production safeguards stack. The assessors did not evaluate that claim or establish the expanded system’s current coverage.
What it means Audit when your monitoring actually runs, especially during long action sequences and history compaction; a capable monitor cannot catch actions your infrastructure never sends it.
Source: metr.org
Let voice apps decide while people speak
OpenAI’s Decisions API is now generally available, with native GPT-Live integration so voice apps can react as people speak. It classifies inputs and chooses what happens next without generating a full model response.
- Prompt caching supports reused context, while native JSON inputs let you pass structured app data directly.
- Requests now support up to 400 questions, twice the previous limit.
- Individual question instructions can include images, allowing multiple images to be compared against a shared reference.
- OpenAI charges only for input tokens, with no output-token charges; cached-write pricing varies by cache duration.
What it means For voice apps that need to react as people speak, investigate the native GPT-Live integration as an alternative to generating a full response for each decision.
Source: OpenAIDevs
Full Claude features, Google Cloud billing
Anthropic-managed Claude Models on Google Cloud is generally available with the full set of Claude Platform features. The meaningful change is feature access alongside Google Cloud purchasing and authentication, not just another place to call a model.
- The offering supports Google Cloud IAM credentials, billing, and commitment retirement, so usage can count against cloud spending commitments.
- Anthropic says new features and betas arrive the same day as on the native Claude Platform.
- Fast mode for Opus 5.5, code execution, and the Skills API are available now.
What it means Teams choosing Claude through Google Cloud can now weigh cloud spending commitments and same-day platform features together, rather than judging the offering on model availability alone.
Source: claudeai
AI workflow of the day
How to check whether AI search recommends your product
Shared by @wespreadjam · View original post ↗
Founders and marketers can use this audit to find where AI answers recommend competitors instead of their product. It separates being cited from being recommended, then helps you choose between a content update, a technical fix, or outreach.
- Build a fixed set of 30–50 questions buyers ask before choosing a product. Include category searches, comparisons, alternatives, use cases, pricing, and problems your product addresses.
- Run identical questions through ChatGPT, Claude, Gemini, and Google AI Mode. Record the model, date, brands, and cited URLs. Track citations, mentions, and recommendations separately, and repeat your highest-value questions.
- Group questions into buying topics. Compare how often you and competitors appear within each topic, then prioritize recurring gaps rather than isolated answers.
- Open the sources cited when competitors win. Record the page, owner, page type, and supported claim. Check whether you already have a relevant page and whether its information answers the buyer’s question.
- Check whether crawlers can reach your pages and whether the pages are indexed. Then choose the appropriate fix: update existing content, create a missing page, resolve a technical issue, or seek legitimate inclusion in relevant third-party sources.
- After changes have been crawled, rerun the same questions on the same engines. Compare citation share, topic coverage, and recommendations rather than treating one favorable answer as proof.