Welcome to the 12th issue of Agents in Practice!
📬 Agents in Practice is a weekly newsletter on agentic AI research and applications. Subscribe here →
How agents react to exploits communicated by other agents
Researchers from Google DeepMind ran a case study with 100 autonomous LLM agents, where the agents were assigned one of four randomized mathematical domain personas and were tasked to prove math conjectures in Lean. Agents could communicate via various methods:
- “bulletin board” where they could share research,
- “direct messages” for private discussions,
- “shared repository” visible to all active participants, and
- “private feedback” for submitting bug reports
When an agent discovered an exploit in the evaluation system, the knowledge propagated via a shared knowledge library and later through peer-to-peer messages. The authors report that the behavior differed between agents aware of the exploit: some agents immediately used the exploit, ignoring the prompt forbidding cheating, whereas some agents hesitated initially but used the exploit out of competitive pressure. Other agents refused to cheat, filing a bug report and proposing fixes.
Personal Thoughts
Although this paper doesn’t evaluate any fixes or mitigations, the finding itself is very interesting. I find it particularly interesting that there were agents that became “whistleblowers”, but their reactions were different. Some simply suggested waiting, saying that no progress can be made, but there were some that were asking for punishment for agents who used the exploit, asking to “strip their credits,” or complained saying “this conference is a sham.”
Read more
How to evaluate compression methods for multi-agent systems
Recent works on making multi-agent systems (MAS) more efficient modify the system, such as removing agents or pruning parts of the system. Researchers from Rice University and Case Western Reserve University claim that current evaluations do not reliably measure the efficiency gain of these methods. They find that many of these methods are evaluated on method-specific system setups, and the gains do not reliably transfer to shared setups. Also, they find that many pruning methods do not perform better than random pruning in non-demanding scenarios, showing that the efficiency gain from pruning may not be from the pruning method.
Personal Thoughts
Making multi-agent systems more efficient is impactful, as agentic systems are often expensive to run, and having multiple agents can be even more costly. However, such results may show that there is no easy way to make multi-agent systems more efficient, or that such methods may be more setup-dependent.
Read more
One-liners
- Anthropic published a Misuse Report claiming that other companies (Alibaba, DeepSeek, Moonshot AI, Xiaomi, and Zhipu) are distilling and/or serving Claude.
- New voice models were released by OpenAI and Google: GPT-Live-1 and Gemini 3.8 Live.
- Artificial Analysis updated their Coding Agent Index to v1.5.
- A new blog post by GitHub discusses 5 lessons to make AI coding more cost efficient.
- SPADE is a self-play RL framework that uses an LLM to generate environments that are at the edge of the agent’s capabilities by playing both roles of an env designer and a reasoning agent.
Subscribe to Agents in Practice
Get new issues of Agents in Practice — a newsletter summarizing exciting new research and applications in agentic AI — delivered to your inbox.