Inference

👋 Sign in for the ability to sort posts by relevant, latest, or top.

zxpmail

Jun 28

KV Cache Is Eating Your VRAM — Here's How to Estimate It Before You Run Out

#llm #inference #engineering #ai

6 min read

zxpmail

Jun 28

I Benchmarked Speculative Decoding — a = 3.5 Wasn't Enough

#llm #inference #engineering #ai

7 min read

zxpmail

Jun 28

Lossless, But Not Free: The Lossless, But Not Free — When Speculative Decoding Actually Pays Off (and When It Doesn't)

#ai #llm #inference #engineering

6 min read

Creeta

Jun 26

96% of cuBLAS, no `unsafe`: what cuTile Rust proves

#cutile #rust #gpu #inference

8 min read

Sonam

Jun 26

Extract Structured JSON from Messy Text with Telnyx AI Inference

#ai #inference #telnyx #json

2 min read

Review Laptop

Jun 21

Chạy LLM trên iGPU: Giới hạn VRAM của Intel Arc và Radeon 780M

#llama3 #llm #ollama #inference

3 min read

Jay Grider

Jun 12

How to Build a Secure Homelab for LLM Inference

#homelab #llmsecurity #inference #supplychain

4 min read

Peremptory

Jun 11

Google's DiffusionGemma Generates Text Sideways

#modelrelease #architecture #opensource #inference

3 min read

Constant, Yuan Chen

Jun 24

Sipp: a local-first runtime for Hybrid AI Applications

#inference #ai #localai #llm

11 min read

Tech_Nuggets

Jun 5

Speculative decoding: when and why it actually speeds up inference

#llm #ai #inference #performance

9 min read

Rob

Jun 16

Can You Tell When an LLM API Swaps in a Cheaper Model?

#localai #llm #inference #verification

3 min read

Jangwook Kim

May 11

ReFlect: Training-Free Error Recovery for Long-Horizon LLM Reasoning

#llmreasoning #agents #inference #arxiv2026

4 min read

Bruno Juca

May 10

Why Most Browser AI Demos Fail on Real Hardware

#ai #inference #hardware #benchmark

4 min read

David Aronchick

May 5

The Inference Inversion

#distributedcomputing #edgecomputing #nvidia #inference

7 min read

Steriani Karamanlis

May 12

First Confirmed Directional Move on the AI Inference Frontier Index in 2026

#ai #llm #inference #pricing

4 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.

DEV Community

# inference

KV Cache Is Eating Your VRAM — Here's How to Estimate It Before You Run Out

I Benchmarked Speculative Decoding — a = 3.5 Wasn't Enough

Lossless, But Not Free: The Lossless, But Not Free — When Speculative Decoding Actually Pays Off (and When It Doesn't)

96% of cuBLAS, no `unsafe`: what cuTile Rust proves

Extract Structured JSON from Messy Text with Telnyx AI Inference

Chạy LLM trên iGPU: Giới hạn VRAM của Intel Arc và Radeon 780M

How to Build a Secure Homelab for LLM Inference

Google's DiffusionGemma Generates Text Sideways

Sipp: a local-first runtime for Hybrid AI Applications

Speculative decoding: when and why it actually speeds up inference

Can You Tell When an LLM API Swaps in a Cheaper Model?

ReFlect: Training-Free Error Recovery for Long-Horizon LLM Reasoning

Why Most Browser AI Demos Fail on Real Hardware

The Inference Inversion

First Confirmed Directional Move on the AI Inference Frontier Index in 2026