AI coding
ReplaySSM: Rethinking State Storage for Faster Hybrid-Model Decoding
ReplaySSM proposes a different approach to storing recurrent state in hybrid models, aiming to speed up decoding without sacrificing quali...
Oct 13, 202611 min
Articles tagged: LLM inference
6 articles
AI research
Benchmarking LLM Inference at Scale with AIPerf
NVIDIA's AIPerf gives teams a repeatable way to benchmark LLM inference at scale, capturing throughput, latency, and concurrency under rea...
Sep 21, 202611 min
Local models
GPU Management: Why Idle GPUs Are the New Grounded Aircraft
In local AI deployments, idle GPUs mirror grounded aircraft: they consume capital, occupy space, and depreciate without yielding returns....
Jul 31, 202611 min
Local models
Hugging Face and Cerebras bring Gemma 4 to real-time voice AI
Hugging Face and Cerebras collaborate to run Gemma 4 models for real-time voice AI on local hardware, enabling low-latency speech processi...
Jul 2, 20267 min
AI agents
From Local LLM to Tool-Using Agent
Learn how to transform a local large language model into a powerful agent by integrating external tools like web search, APIs, and code ex...
Jun 26, 20268 min
Local models
Run a vLLM Server on HF Jobs in One Command
Learn how to launch a vLLM inference server on Hugging Face Jobs with a single command. This guide covers setup, configuration, and practi...
Jun 26, 20266 min