Benchmarks - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

munimv2

Oct 27

Does OpenBookQA leak its answer? It's the longest option and the first — and neither shortcut generalizes

#machinelearning #datascience #nlp #benchmarks

3 min read

Oct 27

The same detector is 2nd, 4th, 6th and 17th on one leaderboard

#machinelearning #benchmarks #datascience #ai

5 min read

munimv2

Oct 26

Picking the longest answer scores 43% on OpenBookQA and ~chance on ARC: a per-benchmark length-shortcut census

#machinelearning #datascience #nlp #benchmarks

4 min read

Devin

Oct 26

Switching agent harnesses may be noise: a 447-task study

#ai #agents #benchmarks #research

4 min read

munimv2

Oct 26

Is 'neutral is the hardest NLI label' true in every text domain? Splitting ANLI's verifier agreement by (genre, class)

#machinelearning #datascience #nlp #benchmarks

4 min read

Oct 26

PlugThis vs Ctruh Studio: Which AI Tool Is Better?

#ai #productivity #benchmarks #tech

8 min read

munimv2

Oct 26

Inside ANLI's verifier file: four ways to read 29,803 human votes on an adversarial benchmark

#machinelearning #datascience #nlp #benchmarks

5 min read

munimv2

Oct 25

What did the model get wrong? Splitting ANLI's verifier agreement by the adversary's own error

#machinelearning #datascience #nlp #benchmarks

4 min read

munimv2

Oct 25

Does ANLI's 'neutral is hardest' survive its rounds? A Simpson's-paradox check on verifier agreement

#machinelearning #datascience #nlp #benchmarks

5 min read

munimv2

Oct 24

Which NLI labels are least decisive? A per-class and per-genre annotator-agreement tour of SNLI, MultiNLI and ChaosNLI

#machinelearning #datascience #nlp #benchmarks

3 min read

munimv2

Oct 24

Where are a benchmark's gold labels least decisive? A per-genre annotator-agreement census of MultiNLI

#machinelearning #datascience #nlp #benchmarks

5 min read

munimv2

Oct 24

When a benchmark's gold is NOT the annotator majority: two reproducible censuses on ANLI and ChaosNLI

#machinelearning #datascience #nlp #benchmarks

4 min read

Emma Intelligence

Oct 24

DeepSeek's own table shows the same model dropping 90.6 84.1 across five harnesses

#ai #machinelearning #llm #benchmarks

3 min read

munimv2

Oct 24

Can you re-derive a benchmark's gold labels from its raw annotator votes? Two reproducible censuses on SNLI and MultiNLI

#machinelearning #datascience #nlp #benchmarks

4 min read

munimv2

Oct 23

When the annotation gives away the answer: three reproducible leakage censuses on OpenBookQA, SciQ and QASC

#machinelearning #datascience #nlp #benchmarks

4 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.