Articles tagged: benchmarks

34 articles

AI research

A Game You Cannot Win Unless You Have a Quantum Computer

Google Quantum AI demonstrates quantum contextuality on a superconducting processor, proving a game solvable only by quantum mechanics. Us...

Nov 20, 20266 min
AI research

Faster Scientific Simulations with New FP64 Algorithms in NVIDIA cuBLAS

NVIDIA CUDA Toolkit 13.4.2 introduces Ozaki-II and 2M algorithms that accelerate double-precision GEMM on integer Tensor Cores while prese...

Nov 14, 20266 min
AI tools

Gemini Decisions API: Fast, Calibrated Multimodal Decisions

Google introduces the Gemini Decisions API, powered by Gemini Decision 1, for fast, calibrated multimodal decisions. Evaluate up to 1,000...

Nov 12, 20266 min
AI agents

Putting Earth AI to work: Geospatial agents for our planet's biggest challenges

Google Research introduces a Geospatial Reasoning agent that combines Earth AI foundation models with upfront planning to solve complex sp...

Nov 3, 20266 min
AI research

The Sleep Framework: A New Way for LLMs to Retain Knowledge in Continual Learning

Google Research introduces the sleep framework, a paradigm enabling LLMs to mitigate catastrophic forgetting by alternating between active...

Oct 31, 20266 min
AI safety

Developing Gemini 4 Argon Responsibly and Safely

Google outlines its approach to safety for Gemini 4 Argon, focusing on misuse prevention, real-time monitoring, and prompt injection resis...

Oct 14, 20265 min
Local AI verification

Understanding Mistral NeMo: Model Size, Context Window, and Tekken Tokenizer

Mistral NeMo is a 12B model released in July 2024 with a 128k context window. It features the Tekken tokenizer for efficient compression a...

Oct 13, 20265 min
AI agents

Building Reliable Data Analytics Agents: Lessons from the KDD Cup

Lessons from the KDD Cup show that reliable data analytics agents depend on rigorous evaluation, robust tool orchestration, and careful ha...

Oct 12, 20269 min
Guides

MilleMiglia: A Realistic Instance Generator for Middle-Mile Logistics

MilleMiglia is a generator that produces realistic middle-mile logistics instances for benchmarking and research. This guide explains what...

Sep 28, 202611 min
Guides

Tokenizers v1: Encode, Decode and Scaling, Measured

Tokenizers v1 turns the encode–decode pipeline into a measurable component: one library, consistent normalization, and benchmarks that exp...

Sep 22, 202610 min
AI research

Benchmarking LLM Inference at Scale with AIPerf

NVIDIA's AIPerf gives teams a repeatable way to benchmark LLM inference at scale, capturing throughput, latency, and concurrency under rea...

Sep 21, 202611 min
Guides

New Experts Join Google's AI & Economy Team: What the Expansion Signals

Google has expanded its AI and Economy research team with new experts, according to a September 18, 2026 announcement on the company's blo...

Sep 20, 202611 min
AI agents

Your Agent Aced the Task. Will It Do It Again?

An agent that succeeds once may fail on the next run. Drawing on IBM Research's ALTK-Evolve consistency work, this article examines why si...

Sep 17, 202611 min
AI agents

Modernizing Complex Legacy Code with AI Agents: Mistral's Approach

Mistral has published research on using AI agents to modernize complex legacy codebases. Rather than treating migration as a single rewrit...

Sep 14, 202614 min
AI tools

How Full-Stack NIM Optimizations Deliver 2.5x More Users on Nemotron 3 Ultra

NVIDIA's full-stack NIM optimizations on Nemotron 3 Ultra raise serving throughput enough to reach 2.5x more concurrent users per deployme...

Sep 11, 202611 min
AI tools

Backing 16 Green AI Projects in Asia-Pacific: A Regional Push for Sustainable Innovation

A verified primary source announces support for 16 green AI initiatives across Asia-Pacific, focusing on sustainability-driven machine lea...

Sep 7, 202613 min
AI agents

Google Antigravity and Gemini 3.7 Flash: A Multi-Agent Fix for Math and Engineering Tasks

Pairing Google Antigravity with Gemini 3.7 Flash resolves notable multi-agent math and engineering problems. Drawing on verified primary e...

Sep 1, 202612 min
AI tools

An AI Tool for Prioritizing Candidate Biomarkers from Wearable Sensor Data

This article examines a Google Research AI tool that prioritizes candidate biomarkers from wearable sensor data. Drawing on an accessible...

Aug 23, 202611 min
Guides

What Does “Full-Stack” AI Actually Mean?

Full-stack AI goes beyond models. A verified Google blog post explains what full-stack development means for AI systems, covering the laye...

Aug 22, 202611 min
AI agents

NVIDIA Nemotron 3 Embed Ranks #1 Overall on RTEB, Advancing Agentic Retrieval

NVIDIA's Nemotron-3-8B-Embedding model achieves the top ranking on the Retrieval Text Embedding Benchmark (RTEB), setting a new standard f...

Jul 17, 20269 min
AI tools

Behind the Scenes of Distributed Training and Why Your GPU Wiring Matters as Much as Your Strategy

Distributed training accelerates AI model development, but network topology and GPU interconnect often bottleneck performance. This articl...

Jul 11, 20267 min
AI tools

Time-Series LLMs, Explained with t0-alpha

Time-series LLMs like t0-alpha leverage transformer architectures to analyze sequential data. This article explains how t0-alpha handles f...

Jul 5, 20266 min
AI research

ScarfBench: Benchmarking AI Agents for Enterprise Java Framework Migration

ScarfBench introduces a standardized benchmark to evaluate AI agents on migrating enterprise Java frameworks. It tests code refactoring, d...

Jul 2, 20268 min
AI tools

Prompt Engineering Fails Quietly

Prompt regression causes AI outputs to degrade over time without warning. Learn why it happens, how to detect it, and practical strategies...

Jul 1, 20268 min
Local models

Mistral OCR 4 API vs Document AI: Choosing an Integration

Compare Mistral OCR 4 extraction, Document AI parameters, announced pricing, self-hosting context and documented limits for document workf...

Jun 28, 20266 min
Local models

Mistral OCR 4 Benchmarks: Results, Caveats and Evaluation

Review Mistral’s published OCR 4 human evaluation and benchmark results, the scoring caveats it identifies and how to assess the model res...

Jun 27, 20267 min
AI research

The Evolution of Artificial Intelligence: From Theory to Transformative Research

Explore how AI research has evolved from foundational theories to cutting-edge breakthroughs, including deep learning and reinforcement le...

Jun 20, 20267 min
Guides

Testing Mythos and Fable: Moving Beyond SWE-bench with Nvidia’s Open Contender

Explore how Nvidia’s new open-source framework challenges SWE-bench dominance. Learn to test AI models with Mythos and Fable for real-worl...

Jun 20, 20268 min
AI research

Is it agentic enough? Benchmarking open models on your own tooling

Learn how to evaluate open-source AI agents for autonomy and task completion using custom benchmarks. A practical guide for researchers an...

Jun 18, 20269 min
AI agents

LLM Fallbacks Break Agent Pipelines

When LLM calls fail, agent pipelines silently break. This article introduces a recovery layer that catches fallbacks, retries intelligentl...

Jun 16, 20268 min
AI research

olmo-eval: An evaluation workbench for the model development loop

olmo-eval is an evaluation workbench designed to integrate seamlessly into the model development loop, enabling rapid iteration and system...

Jun 12, 20267 min
AI research

Can Voice Agents Handle Bilingual Customers? Benchmarking Frontier ASR on Code-Switched Speech

A clear and practical article about artificial intelligence for a professional audience.

Jun 10, 20264 min
AI agents

The Open Source Community is backing OpenEnv for Agentic RL

A clear and practical article about artificial intelligence for a professional audience.

Jun 8, 20268 min
Local models

We Should Train AI to Betray Its Users

A clear and practical article about artificial intelligence for a professional audience.

Jun 8, 20268 min