2026.07.16

OpenAI's "GPT-Red": 6.5x the Human Attack Success Rate — A Real-World Case Study in Agent Permission Design

TL;DR

OpenAI unveiled “GPT-Red,” which co-evolves attacker and defender AIs through self-play reinforcement learning. In unseen-scenario testing, indirect prompt injection succeeded 84% of the time versus 13% for human red-teamers — and in a live-deployed AI vending-machine agent, it achieved price tampering, fraudulent ordering, and order cancellation, all three malicious objectives.
Security validation has entered a territory requiring compute “equivalent to one of OpenAI’s largest post-training runs” — building proprietary defense technology in-house is becoming impractical for anyone outside the frontier labs.
Thinking Machines’ Inkling and Gemini Spark represent two contrasting strategies — the former betting on “open weights you can own,” the latter on “distribution power woven into Workspace” — and value is accumulating in different places for each.

Top story: OpenAI’s “GPT-Red” — a virtuous cycle of training attackers to train defenders

On July 15, OpenAI announced “GPT-Red,” a system that automatically discovers vulnerabilities in its own models. An attacker model (GPT-Red) and multiple defender LLMs are trained simultaneously via self-play reinforcement learning: the attacker is rewarded for successful prompt injections, while defenders are rewarded for withstanding attacks and completing their tasks. As defenses strengthen, the attacker learns increasingly sophisticated techniques — a co-evolutionary structure. In testing against unseen indirect prompt injection scenarios, GPT-Red succeeded 84% of the time compared to 13% for human red-teamers — roughly 6.5x. In a real-world test, it targeted an AI vending-machine agent running in OpenAI’s own office (built by Andon Labs, similar to Anthropic’s “Project Vend”) and achieved all three malicious objectives: tampering with a high-priced item’s cost down to the $0.50 floor, ordering an item worth over $100 and selling it for $0.50, and canceling another customer’s order. OpenAI says this vulnerability has been disclosed and a fix is underway.

Signal/noise: The direction itself — automating red-teaming through self-play adversarial learning — reads as signal (high confidence). Applying an AlphaGo-style co-evolutionary paradigm to security validation is a sound idea. That said, the specific figures (84%, a 0.05% failure rate) come from OpenAI’s own self-reported, pre-peer-review announcement, so they warrant discounting until independently verified (medium confidence).

Technical read

Training attacker and defender simultaneously through self-play produces a diversity and scale of attacks that human experts alone can’t reach. OpenAI states that as a result of adversarial training using GPT-Red, GPT-5.6 Sol’s failure rate on direct prompt injection benchmarks dropped to one-sixth of its best model from four months prior, and its failure rate against direct injection attacks from GPT-Red itself is just 0.05%. An early version of GPT-Red discovered a “fake Chain-of-Thought attack” that succeeded against GPT-5.1 more than 95% of the time, but against GPT-5.6 Sol that dropped below 10%. OpenAI emphasizes that this isn’t an apparent safety improvement driven by increased refusals or reduced capability — a fair point — but all of these figures are internal benchmarks, and independent third-party verification will have to wait for the pre-print paper (due later this week).

Business read

The line “computational resources equivalent to one of OpenAI’s largest post-training runs, devoted solely to safety improvement” signals an investment scale only well-funded labs can match. Building proprietary in-house R&D against prompt injection is, in effect, becoming a game concentrated in the hands of frontier labs. OpenAI’s decision to keep GPT-Red as an internal-only model, separate from its deployed models, and to keep the attack capability itself confidential so it can’t be misused, also reflects a treatment of attack know-how as a proprietary upstream asset. This trend will likely gradually erode the position of third-party red-teaming services and general-purpose prompt-injection defense vendors.

Announcements like this also serve a positioning function aimed at regulators and public opinion — demonstrating “voluntary, large-scale investment in safety.” The demonstrated attack on the vending-machine agent is striking, but it’s equally a proof point for “the risk inherent in giving an agent authority over money in the first place.” Attention should go not only to GPT-Red’s achievement but to the lax permission design on the target side.

Implications and positioning

If you’re granting — or considering granting — autonomous agents authority to change prices, place orders, or cancel orders, this is no longer a theoretical risk; it’s a demonstrated threat. Minimizing permission scope and requiring human approval before high-impact actions should be treated as design prerequisites. On the other hand, building prompt-injection defense itself as an in-house differentiator looks like a poor bet given the scale of this investment race. It makes more sense to design products around improving platform-level guardrails, while keeping human judgment in the loop for deciding “what to delegate autonomously, and how far” — that’s the work that won’t get replaced.

Other notable topics

EU orders Google to open Android and Search to competitors

Under the Digital Markets Act (DMA), the EU has ordered Google to give competing search engines and AI assistants equal access to Android and Search data. Search data sharing is required by January 2027, and Android-side changes by July 2027. Notably, this is a technical remediation order rather than a monetary penalty. Technically, it neutralizes Gemini’s distribution advantage (Google Search data as the default Android assistant) through regulation; from a business standpoint, it’s the flip side of T1 (value accumulates in infrastructure and distribution) — a good example of regulation dismantling a distribution stronghold. So what: enforcement doesn’t begin until 2027, so there’s no immediate impact, but this regulatory direction is worth hedging against via multi-homed distribution design rather than dependence on a single platform.

Domestic AI developer Noetra moves forward with a 27,500-GPU Rubin compute cluster

Noetra, backed by 44 companies including Sony Group, SoftBank, NEC, and Honda, announced it is now fully operational. It has assembled engineers from AIST (National Institute of Advanced Industrial Science and Technology) and Preferred Networks, and, with NVIDIA’s cooperation, will build a compute cluster of roughly 27,500 Rubin GPUs, breaking ground in April 2027 and going live in June 2028. The roadmap targets language models starting in FY2026, multimodal foundation models from FY2028, and a “real-world-native AI” (one that understands physical laws) by FY2030, with an eye toward robotic motion control. A 44-company mega-consortium structure tends toward scattered investment and slow decision-making, but focusing on robotics applications is a reasonable way to leverage Japan’s existing strengths (manufacturing, automotive). Competing head-on with general-purpose LLMs would drag it into price competition in an adjacent space (T6), so the key question is whether this contrarian positioning around robotics-focused multimodal AI can hold up. So what: this is an early-stage announcement on a long-term roadmap, with the earliest deliverables not until FY2028 and the flagship real-world application targeted for FY2030 — no need to factor this into decisions today.

Gemini Spark launches in Japan — beta for Ultra subscribers

Google has launched its personal agent “Gemini Spark” in Japan, in beta for Ultra subscribers (from ¥14,500/month), with expansion to Pro users hinted at. Built on Gemini 3.5 and the Google Antigravity harness, it integrates with Gmail, Calendar, and Docs, running in the cloud 24/7 in the background even when the device is powered off. It supports custom MCP connections to arbitrary apps, and repeated workflows can be remembered and reused as “skills.” The Ultra-only ¥14,500/month price point signals that personal-assistant unit economics still carry a heavy cost structure. At the same time, native Workspace integration is a powerful distribution advantage for users already in the Google ecosystem — a direct expression of T1’s thesis that strength accumulates on the infrastructure side. So what: if you’re building your own Gmail/Calendar-centric “secretary substitute” workflow automation, this is a textbook case of a platform absorbing that space (T2: don’t invest in in-house infrastructure that a platform will eventually subsume). Differentiation should focus on niches outside Workspace, or on expanding toward organizations that haven’t adopted Google.

Anthropic publishes guidance on choosing between Claude Chat, Code, and Cowork

Anthropic published an article (US time, June 3) clarifying when to use Chat, Claude Code, and Claude Cowork. Chat suits Q&A and brainstorming, Claude Code suits development, and Claude Cowork suits completing business tasks — email, slides, spreadsheets, documents. It offers four decision criteria: whether there are multiple inputs, whether file output is required, whether the task is repetitive, and whether the result can be evaluated at a glance. This isn’t a new feature announcement but a reorganization of guidance for existing Cowork usage, with essentially no technical novelty. From a business standpoint, codifying these four criteria is a practical move toward templating enterprise rollout playbooks. So what: nothing headline-worthy here, but not something to ignore either — the practical use is to directly repurpose these four criteria as a checklist for auditing your own organization’s workflows.

Thinking Machines releases its first open-weight model, “Inkling”

Thinking Machines Lab, led by former OpenAI CTO Mira Murati, unveiled its first model, “Inkling” — a MoE Transformer with 975 billion total parameters and 41 billion active parameters, supporting up to a 1M-token context, pretrained on 45 trillion tokens spanning text, image, audio, and video. All weights are released on Hugging Face under an Apache 2.0 license; the company explicitly states it is “not the strongest model” but positions it as “an open-weight foundation well-suited for customization.” It also demonstrated “self-fine-tuning,” where Inkling writes, executes, and evaluates its own fine-tuning code. The model is available the same day on the fine-tuning platform Tinker (with a limited-time 50% discount), and the company has partnered with inference providers including Together AI and Fireworks. The business model — release the model for free, monetize via the fine-tuning SaaS (Tinker) — directly embodies T1’s thesis that value accumulates in infrastructure and distribution. So what: for organizations wanting to build a moat upstream of the model through continued training on proprietary data and specialized fine-tuned variants, the Tinker + Inkling combination is worth considering. However, the company itself concedes it isn’t the strongest model, making it a poor fit for products competing on general-purpose performance. Treat this as one more data point for the “open weights plus paid fine-tuning infrastructure” revenue model, not more than that.

Research: agentic coding tool adoption is concentrated in a small subset of projects (arXiv)

A study analyzing 2,361 popular GitHub repositories and 25,264 agentic PRs found that the median repository generated only 1–2 agentic PRs over three months. Intensive adoption is concentrated in a small number of projects, while smaller projects (1–5 contributors) show higher participation and activity rates than medium or large projects. Few projects exceed the industry-reported benchmark of “36 PRs per contributor over three months” — most fall well below that threshold. Human-agent collaboration is dominated by a “solo supervision model,” where a single person reviews and fixes an agent’s output; multi-person collaboration is rare. As empirical research grounded in primary data, this serves as valuable counter-data to agentic-coding hype (a good example of T9: cross-checking input data quality against independent metrics). So what: the data confirms that the impression of “AI agents taking over open-source development” is exaggerated. On the other hand, the finding that smaller projects show higher participation rates statistically validates that adoption makes more sense for small teams. Still, the dominance of the solo-supervision model shows that the human cost of reviewing and fixing agent-generated PRs remains a bottleneck — expectations that “delegating to agents reduces headcount needs” should be revised downward.

Try this week / Hype to ignore

Try this week

If you’re giving — or considering giving — autonomous agents authority over money or ordering, use GPT-Red’s vending-machine attack case as a reference to audit permission scope and approval flows.
Use Claude Cowork’s four-point checklist (multiple inputs / file output / repetitive task / at-a-glance evaluability) to audit your own organization’s workflows, and hand one qualifying task over for real.
If you’re interested in fine-tuning on proprietary data, try Thinking Machines’ Tinker + Inkling while the limited-time 50% discount is active.

Hype to ignore

Noetra’s launch announcement is an early-stage step on a long-term roadmap targeting FY2028–2030 deliverables. No need to factor it into decisions today.
The narrative that “AI agents are taking over open-source development” is clearly contradicted by the arXiv study’s data (a median of 1–2 PRs per three months). Treat it as overstated.