AI agents
Stop Ranking Agent Configs by Average Score
Ranking AI agent configurations by average score can be misleading. Learn why this metric hides critical failures and discover better eval...
Jul 8, 20268 min
Articles tagged: evaluation framework
6 articles
AI research
ScarfBench: Benchmarking AI Agents for Enterprise Java Framework Migration
ScarfBench introduces a standardized benchmark to evaluate AI agents on migrating enterprise Java frameworks. It tests code refactoring, d...
Jul 2, 20268 min
Guides
Testing Mythos and Fable: Moving Beyond SWE-bench with Nvidia’s Open Contender
Explore how Nvidia’s new open-source framework challenges SWE-bench dominance. Learn to test AI models with Mythos and Fable for real-worl...
Jun 20, 20268 min
AI research
Is it agentic enough? Benchmarking open models on your own tooling
Learn how to evaluate open-source AI agents for autonomy and task completion using custom benchmarks. A practical guide for researchers an...
Jun 18, 20269 min
AI research
olmo-eval: An evaluation workbench for the model development loop
olmo-eval is an evaluation workbench designed to integrate seamlessly into the model development loop, enabling rapid iteration and system...
Jun 12, 20267 min
AI tools
Introducing LangChain Labs
A clear and practical article about artificial intelligence for a professional audience.
Jun 6, 202612 min