bench: flag void (high non-2xx) runs; add status-code diagnostic

The first multi-core A/B was void: ~100% non-2xx on every row, so 'sustained
rps' was error-path throughput, not store work -- the spin patch's park/wake
path was never exercised. The harness reported it cheerfully anyway.

- spin_ab_urus.sh: per-run output now computes non-2xx vs approx total and
  prints 'VOID: N% non-2xx' instead of a meaningless rps when >2% error.
- loadgen/_diag_status.lua: single-thread wrk script that tallies exact status
  codes (response() + done()) and names the likely cause (401=auth not
  reaching server / 404=store not pre-populated / 503=store actor down).
  The aggregate 'Non-2xx' count can't distinguish these; this can.

Root cause still open: locally the store survives heavy keep-alive load (no
panic), prepop populates, auth+happy-path serve 200s -- so the storm isn't
reproducible in this 1-core sandbox. s3 at ~100% (not ~80%) rules out a plain
404 storm and points at auth or store-call erroring on every request; needs
the diag run on the real box to confirm.
This commit is contained in:
sandbox-agent
2026-06-14 21:58:27 +00:00
parent 5f342858e7
commit 926aba4344
2 changed files with 78 additions and 2 deletions
+11 -2
View File
@@ -200,10 +200,19 @@ run_commit() {
mv "${BENCH_DIR}/results/${run_id}/"* "$dest/" 2>/dev/null || true
rmdir "${BENCH_DIR}/results/${run_id}" 2>/dev/null || true
fi
local rps p99
local rps p99 non2xx ms errpct
rps=$(awk -F'[:,]' '/"sustained_rps"/ {gsub(/[ "]/,"",$2); print $2; exit}' "${dest}/result.json" 2>/dev/null || echo "?")
p99=$(awk -F'"' '/"p99"/ {print $4; exit}' "${dest}/result.json" 2>/dev/null || echo "?")
echo " -> sustained=${rps} rps p99=${p99}"
non2xx=$(awk -F'[:,]' '/"non_2xx"/ {gsub(/[ "]/,"",$2); print $2; exit}' "${dest}/result.json" 2>/dev/null || echo 0)
ms="${MEASURE_SEC:-60}"
# error rate vs approx total requests (sustained_rps * measure_sec).
errpct=$(awk -v n="$non2xx" -v r="$rps" -v s="$ms" \
'BEGIN { t=r*s; if (t>0) printf "%.1f", 100*n/t; else print "?" }')
if awk -v e="$errpct" 'BEGIN { exit !(e+0 > 2.0) }' 2>/dev/null; then
echo " -> VOID: ${errpct}% non-2xx (${non2xx}) — server is erroring, not serving. rps=${rps} is meaningless."
else
echo " -> sustained=${rps} rps p99=${p99} (non-2xx ${errpct}%)"
fi
else
echo " -> FAILED (see ${dest}/runner.log)"
fi