DEV Community

#benchmark

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
SDXL images per dollar on four GPUs: RTX A6000, L40S, RTX 5090 and H100, eager vs torch.compile

SDXL images per dollar on four GPUs: RTX A6000, L40S, RTX 5090 and H100, eager vs torch.compile

Comments
6 min read
vLLM FP8 vs bf16 on an H100: 1.4x the throughput, and what it did to GSM8K accuracy

vLLM FP8 vs bf16 on an H100: 1.4x the throughput, and what it did to GSM8K accuracy

Comments
5 min read
DeepSeek v4.1 Flash Benchmark: Testing a 1M Context LLM Route

DeepSeek v4.1 Flash Benchmark: Testing a 1M Context LLM Route

Comments
2 min read
GitHub Actions runners ranked by speed and cost per job (October 2026)

GitHub Actions runners ranked by speed and cost per job (October 2026)

1
Comments
2 min read
Your Benchmark Has No Cost Axis: Score Free Endpoints by Quota per Accepted Patch

Your Benchmark Has No Cost Axis: Score Free Endpoints by Quota per Accepted Patch

Comments
6 min read
Which of your LLM calls are really decisions? What we measured with a System One model

Which of your LLM calls are really decisions? What we measured with a System One model

Comments
7 min read
Parakeet v3 vs Whisper vs Apple's on-device model in 25 languages, every transcript published

Parakeet v3 vs Whisper vs Apple's on-device model in 25 languages, every transcript published

1
Comments 1
3 min read
Reading a Hardware-Equivalence Claim Like an Evaluator: A Checklist Applied to an NPU vs GPU Pretraining Twin

Reading a Hardware-Equivalence Claim Like an Evaluator: A Checklist Applied to an NPU vs GPU Pretraining Twin

Comments
2 min read
vLLM on three GPUs: Qwen2.5-7B tokens per second and cost per 1M tokens on RTX A6000, L40S and H100

vLLM on three GPUs: Qwen2.5-7B tokens per second and cost per 1M tokens on RTX A6000, L40S and H100

1
Comments
5 min read
Benchmark 2026: Probamos 50 Agentes de IA para Ecommerce con 10 Preguntas Difíciles (Arquitectura, Latencia y Conversión)

Benchmark 2026: Probamos 50 Agentes de IA para Ecommerce con 10 Preguntas Difíciles (Arquitectura, Latencia y Conversión)

Comments
3 min read
Six open models, one question: 1.3 s for the fastest answer, 16 s for the slowest, and hidden thinking made the difference

Six open models, one question: 1.3 s for the fastest answer, 16 s for the slowest, and hidden thinking made the difference

Comments
5 min read
How LLM Evaluation Produces Comparable Numbers: Inside the LFORLA RL Game Agents Benchmark

How LLM Evaluation Produces Comparable Numbers: Inside the LFORLA RL Game Agents Benchmark

Comments
4 min read
ScaleDown CDMs: Fast, Cheap Classify, Extract and Summarize Models Tested

ScaleDown CDMs: Fast, Cheap Classify, Extract and Summarize Models Tested

Comments
8 min read
What a Youden index of 77 actually means on the OWASP Benchmark

What a Youden index of 77 actually means on the OWASP Benchmark

Comments
1 min read
AI 신약 설계 챌린지 2만 3,886건의 점수표를 뜯어 보니: 만점에 닿은 축과 한 번도 닿지 못한 축

AI 신약 설계 챌린지 2만 3,886건의 점수표를 뜯어 보니: 만점에 닿은 축과 한 번도 닿지 못한 축

Comments
4 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.