Can AI Agents Automate AI Research? — The Limits Revealed by Shadow Evaluations

---

TL;DR

• In a new evaluation method called “shadow evaluations,” using two unpublished NeurIPS submission papers, frontier AI agents completed the engineering work without human help, but failed to make substantive progress on the core research questions, and both papers were graded as rejected by the papers’ own authors.
• In accounting (APEX-Accounting), low reproducibility — getting the correct answer on every run — was identified as an issue, while in office work (OmegaUse-OfficeVal), models were cheaper and faster than humans but their output quality fell short of human standards; these practical benchmarks confirm that frontier models still face barriers to real-world deployment.
• The three items involving Microsoft, HP × Rakuten, and Google all consistently show the same pattern: differentiation comes not from model performance itself but from templating, ecosystem integration, and distribution.

Feature Story: Can AI Agents Automate AI Research? — Evidence from Shadow Evaluations

A research team has proposed an evaluation method called “shadow evaluations.” Existing evaluations were limited to either measuring performance on narrow, verifiable tasks or submitting AI-generated papers to blind peer review (which suffers from reviewer capacity overload, inconsistency, and low quality). As a third approach, the team gives agents the “core open-ended research question” from an unpublished, high-quality paper, then has the paper’s original authors themselves grade the results. (arXiv)

In the experiment, frontier agents were given six days and thousands of dollars’ worth of compute resources for two unpublished NeurIPS 2026 submission papers. The agents completed the engineering work without human assistance, but failed to make substantive progress on the research questions themselves, and both papers were clearly rejected by their authors. Five failure modes were repeatedly observed: (1) poor judgment about what constitutes publishable research quality, (2) uncreative responses to flaws in the research design, (3) inefficient backtracking when stuck, (4) poor resource awareness, and (5) instruction drift. Replication experiments with different models and scaffolds confirmed the same failure patterns, and the authors have published the reviews, survey responses, agent repositories, and logs.

Technical Take

The novelty here lies more in the evaluation methodology than in the results themselves. The design measures “open-ended research judgment” — something narrow verifiable tasks cannot capture — at a difficulty level close to real research. On the results side, there’s a clear separation: the agents completed the engineering (implementation and running experiments), but did not demonstrate research “discernment” (judging what’s worth publishing and how to handle design flaws). Confirming the failure across two models and two scaffolds suggests this may be a structural limitation rather than a quirk of a specific model. However, with only two cases, caution is needed before generalizing statistically.

Business Take

This is empirical work that embodies the view that agent capability should be judged by reproducibility and cost. Its value lies in providing counter-evidence — through actual measurement (evaluation by the original authors and a robustness check with a second model and scaffold) rather than marketing claims — against the premise underlying predictions that “AI will accelerate AI research and progress will become explosive,” namely research automation by agents. Viewed through the lens of build-in-house versus platform absorption, this shows that while the “hand implementation to agents” phase is becoming commoditized, “judgment about research direction and what to pursue” remains a boundary that stays on the human side.

Contrarian Angle / What’s Overlooked

While the rejection outcome tends to get all the attention, completing the engineering work without human help over six days and thousands of dollars in compute is itself evidence of execution capability. It’s also easy to overlook that the failure wasn’t an inability to implement — it was a limitation at a higher layer, an inability to exercise research judgment.

Implications and Position

What’s now achievable is a workflow where agents handle implementation and running experiments. What should be discarded is any design that delegates research-direction judgment or discernment about “what makes good research” to agents. Given this evidence, the claim that “AI will autonomously accelerate scientific discovery” should be judged premature at this point. When designing AI agents for your own products or work, explicitly dividing roles so that you hold the judgment and hand execution (engineering) to the agent should make it easier to avoid the failure modes seen here (judgment, creativity, backtracking, resource awareness, instruction drift). Signal/noise verdict: leans signal (specific failure modes plus reproducibility checks). Confidence: moderate — the sample size of just two cases warrants some discounting.

Other Notable Topics

1. Accounting Benchmark APEX-Accounting — The Reproducibility Wall

This is a benchmark built by Mercor in partnership with Ramp that has frontier models perform real accounting work (account reconciliation, expense accrual, journal entries, report generation). It consists of 160 tasks across 10 “worlds,” with accounting and bookkeeping experts creating the tasks and scoring rubrics. Comparing nine models, Claude-Fable-5 (Max) topped Mean Criteria@3 at 56.4%, followed by Muse-Spark-1.1 (xHigh) at 52.6%. However, Pass^8 — meaning getting every one of 8 runs correct — topped out at just 2.6% for GPT-5.6-Sol (Max+Pro), and even Pass@8 — getting at least 1 of 8 runs correct — topped out at 21.5% for Muse-Spark-1.1 (xHigh). Furthermore, in an experiment increasing the token budget from $1 to $50, while scores rose as the overall budget increased, an inverse correlation (Simpson’s paradox) was observed within the same budget: models that used more tokens on a given task scored lower. (arXiv)

Technically, what stands out is the gap between partial-credit scoring (Mean Criteria) and strict every-run-correct scoring (Pass^8). In business terms, the view that agent capability should be judged by reproducibility and cost applies directly here: for accounting, a domain where a single mistake isn’t tolerable, unit-economics and reliability risk should be judged by strict metrics like Pass^8 rather than lenient ones like Mean Criteria@3. If you’re considering accounting automation, the appropriate framing right now is “first-draft generation assuming human review,” and full delegation should be judged premature.

2. OmegaUse-OfficeVal — Quantifying “Cheap But Not Human-Level”

This benchmark evaluates LLM agents on 100 long-horizon tasks devised by practitioners, each representing an average of 2.32 hours of human labor. Its distinguishing feature is that each task is tied to two economic metrics — “human labor hours” and a “proxy for task price” — enabling direct comparison of human cost against LLM inference cost, as well as value-weighted evaluation. The evaluation found that while all evaluated LLMs were substantially cheaper and faster than humans, output quality still hasn’t approached human standards. The code and dataset have been open-sourced. (arXiv)

Technically, the distinguishing feature is the design itself — embedding cost structure directly into the benchmark, not just accuracy. In business terms, this is a direct practical example of judging agent capability by reproducibility and cost, and the idea of converting to an hourly-rate basis can be repurposed for evaluating your own workflows. When adopting AI for office work, the practical approach is to anchor on “tolerance for quality gap” alongside “time-savings rate,” and to start with low-quality-bar, first-pass tasks (drafts, rough aggregation).

3. Copilot in Excel’s Skills Feature — Absorbing Financial Workflows into Templates

On June 25 (US time), Microsoft enhanced Copilot in Excel’s features for finance departments. The headline addition is the “Skills” feature, where Copilot executes routine processes — building DCF models, month-end close, updating monthly reporting models, variance analysis — following template procedures defined in Markdown files (SKILL.md). A sample library of financial skills has been published, and users can also create their own custom skills. Workbook rules (structure, naming conventions, formula rules) are also saved for continued reuse. On external data, connectors for CB Insights, Daloopa, FactSet (rolling out), Morningstar, PitchBook, and S&P Global have been added alongside the existing LSEG and Moody’s connectors. A “Plan with Copilot” feature has also been introduced, letting users review the planned actions (which ranges, worksheets, formulas will be updated, etc.) before execution. (ITmedia AI+)

Technically, the key elements are templating procedures via Markdown, plus support for an audit trail through pre-execution previews and change history. In business terms, this is a textbook example of value accumulating in infrastructure and distribution: the official stack is absorbing, as a templating feature, workflows for “close processes, variance analysis, and valuation models” that individual analysts or small teams previously built themselves. If you’ve built your own Excel × AI agent workflow, periodically check whether the official feature has absorbed an equivalent one, and shift your investment toward areas that resist templating (judgment, relationships, proprietary data) for better defensibility.

4. HP × Rakuten’s On-Device AI — A Fight Over Distribution, Not Models

On July 29, HP Japan and Rakuten began pre-bundling “Rakuten AI for Desktop” on HP PCs sold in Japan. It uses a locally-running 7-billion-parameter model, “Rakuten AI 7B ONNX” (a version of “Rakuten AI 7B” — itself continually trained from French Mistral AI’s open model “Mistral-7B-v0.1” — further optimized for HP AI PCs through continued training, fine-tuning, and quantization), enabling rewriting, summarization, and translation even offline. It’s a hybrid setup that switches between on-device and cloud depending on use case; in cloud mode, agent features integrated with Rakuten Ichiba, Rakuten Travel, and Rakuten Points are available through a single chat screen. The enterprise version is available starting the same day, with the consumer version planned for October or later. The app automatically selects between CPU/GPU/NPU and also works on non-HP Windows PCs (the article confirms it working on a Lenovo device). (ITmedia AI+)

Technically, the model itself — Mistral-based, optimized through continued training and quantization — has low novelty, while the implementation side, namely hybrid integration into a single chat screen, is the distinguishing feature. In business terms, the view that value accumulates in infrastructure and distribution applies directly: the moat isn’t the model but the connection to the Rakuten economic ecosystem (Ichiba, Travel, Points). For HP, preinstallation can be read as an opportunity to secure distribution. This partnership is worth referencing less as a direct decision input and more as an example of the thesis that “model defensibility lies in upstream data and ecosystem integration.” If you’re considering using local LLMs, you should decide “which ecosystem to connect to” before model selection.

5. Google Lyria 3.5 — Adjacent Competition in Music Generation

On July 29 (local time), Google announced “Lyria 3.5,” the latest version of its music generation model, and began offering it the same day through its music production platform “Google Flow Music.” The four improvements are: musicality (more complex, natural melodic structure), lyrics (improved prompt fidelity and understanding of song structure), vocals (more realistic, emotionally expressive delivery, improved pronunciation), and creative control (easier control over tempo and length). Since the original Lyria launched in 2023, it has expanded through Lyria 3 being added to the Gemini app in February, and Lyria 3 Pro — supporting up to 3-minute generation and structure specification — arriving in March. It’s also deployed on Vertex AI and Google AI Studio; Google Flow Music is the service formed by renaming “ProducerAI,” acquired in February, to match the branding of the video generation tool “Google Flow” in April. Generated content is embedded with SynthID watermarks. (ITmedia AI+)

Technically, all four improvements are iterative quality gains, not a fundamental breakthrough. In business terms, the product should be positioned against its adjacent competitors: Google is running in parallel, via vertical integration (Flow Music, Vertex AI, AI Studio), in a music generation AI market that already has dedicated specialist players, and the differentiating factor is distribution, not standalone model performance. If you’re considering a business built around music generation, differentiating on model performance alone will be difficult, since large players like Google will keep absorbing features via their built-in ecosystems. If you want to differentiate, compete on specialization in a specific genre or use case, or on rights-clearing and monetization pathways that large players don’t offer.

Try This Week / Hype You Can Ignore

Try this week: revisit your own AI agent workflows using the split of “you hold the judgment, the agent handles execution (engineering).” If you’re using LLMs for accounting or finance work, it’s worth checking for yourself — not the felt accuracy of a Pass@1-style single run, but Pass^8-style reproducibility, i.e., whether running the same task repeatedly gives the same correct answer every time.

Hype you can ignore: given the shadow evaluations results discussed here, claims that “AI will autonomously accelerate scientific discovery and research” should be considered premature at this point. Music generation AI feature-update news can also be deprioritized in the context of model differentiation.