gHashTag · Aug 19, 2026 · Aug 18, 2026
@@ -0,0 +1,33 @@
+# openXC7 vs Vivado — same-machine benchmark, bit0 (2026-08-15/16)
+
+Raw artefacts of the same-machine half of the openXC7-vs-Vivado benchmark
+(method and freeze as agreed in the thread with @gHashTag; the CI half lives
+in `gHashTag/trinity-fpga`). Every number quoted in the write-up for the
+bit0 rows is derivable from these files.
+
+Machine: bit0 — Intel i5-13500 (20 threads), 125 GB RAM, bare metal, Ubuntu,
+egress-filtered network (see the Flexera note below). Per-run machine
+snapshots (`*-machine.txt`) accompany every CSV.
+
+## Layout
+
+| path | what |
+|---|---|
+| `openxc7/bench-openxc7-20260816-000221.csv` | **The clean openXC7 campaign** (N=5 per design; medians in the write-up). Columns: per-stage seconds (json = yosys, fasm = nextpnr, bit = fasm2frames+xc7frames2bit), total, and the toolchain revisions **stamped from the built tree at run time** (nextpnr `05aaa06b`, prjxray-db `d17fea2`, yosys `0.62-7326bb7d6`). |
+| `openxc7/REVISIONS.txt` | Provisioning record of the frozen environment (regymm/openxc7 image by digest, in-image rebuild of nextpnr at the frozen revision). Informational — the harness re-reads revisions from the tree and does not trust this file. |
+| `openxc7/timing-context/max-frequency-lines.txt` | Every `Max frequency` and `Annotating ports…` line from every openXC7 stage log, with the log line number. **Read the LAST block per log**: since nextpnr-xilinx #139 (inside the freeze) each log carries the placer's pre-route ESTIMATE mid-flow and the routed analysis at the end. Minimum-over-all-lines mixes the two. |
+| `vivado/bench-20260815-162240.csv`, `vivado/bench-20260815-162613.csv` | **The clean Vivado 2026.1 campaign (v2)**, run hermetically (see `harness/vivado-nonet.sh`). Per-stage seconds (synth/place/route/bitgen), total, wns/tns/timing_met from `report_timing_summary`. Two files = the campaign was run in two consecutive batches; both are v2. |
+| `vivado/contaminated-v1/` | The first Vivado campaign, ARCHIVED AS EVIDENCE, never as rows: write_bitstream stalled 4–7 nondeterministic minutes on Flexera RUI telemetry making raw `connect()` calls that ignore the proxy env (`NOTE.md` has the diagnosis). |
+| `harness/` | The two harness scripts (`bench-openxc7.sh`, `bench.sh` + `run-design.tcl` + `designs.tsv`) and `vivado-nonet.sh` (rootless `unshare -r -n` wrapper with a dummy interface carrying the licensed MAC — the fix for the telemetry stall). |
+
+## Known caveats (already stated in the thread)
+
+- litex-ddr-arty-s7 (main) fails DRC BIVRU-1 under Vivado 2026.1 as frozen
+  (INTERNAL_VREF for the SSTL135 clock input); rows read "openXC7 builds it,
+  Vivado refuses it". Inputs were kept frozen per the method.
+- picosoc's `create_clock` is commented out in its XDC (the live clock is in a
+  `.sdc` the flow does not read); its timing cells are `na` on both sides.
+- Every openXC7 timing "PASS" is against nextpnr's 12 MHz default target: the
+  designs' own `create_clock` on the pad net does not propagate through
+  IBUF/BUFG nor to PLL outputs in nextpnr-xilinx, so no domain carries the
+  intended target (issue to be filed).
@@ -0,0 +1,169 @@
+#!/usr/bin/env bash
+# bench-openxc7.sh -- openXC7 side of the openXC7-vs-Vivado build-time benchmark
+# (same-machine row). Times, for each of the 4 frozen designs, the three
+# invocations of demo-projects' own openXC7.mk flow:
+#
+#     make <project>.json    (yosys synthesis)
+#     make <project>.fasm    (nextpnr-xilinx place & route)
+#     make <project>.bit     (fasm2frames + xc7frames2bit)
+#
+# each timed separately, so the numbers describe the project's own flow.
+# Seed = each design's default (no --seed injected; PNR_ARGS untouched).
+#
+# The script runs on the HOST and re-executes itself INSIDE the frozen
+# regymm/openxc7 container (image below), where yosys 0.62 / prjxray tools /
+# pypy3 live and where nextpnr-xilinx was rebuilt from /bench/src at the
+# frozen revision. All revisions are READ from the built tree at run time and
+# stamped into the results -- nothing is hardcoded.
+#
+# Usage:
+#     ./bench-openxc7.sh [-n N] [design ...]
+#         -n N       runs per design (default 5)
+#         design...  subset of: blinky picosoc-qmtech litex-ddr-arty-s7
+#                    litex-ddr-arty-s7-deephier   (default: all 4)
+#
+# Smoke:            ./bench-openxc7.sh -n 1 blinky
+# Full campaign (box must be quiet):
+#     cd ~/data/openxc7/bench-openxc7 && \
+#       nohup ./bench-openxc7.sh -n 5 > logs/campaign-$(date +%Y%m%d-%H%M%S).log 2>&1 &
+#
+# Output: results/bench-openxc7-<stamp>.csv
+#         (design,run,json_s,fasm_s,bit_s,total_s,nextpnr_rev,prjxraydb_rev,yosys_ver)
+#         results/bench-openxc7-<stamp>-machine.txt  (machine + full revision stamp)
+#         logs/campaign-<stamp>/<design>-run<i>-<stage>.log
+set -euo pipefail
+
+IMAGE='docker.io/regymm/openxc7@sha256:eced1cdd4727549f2d983328e0cf170fb6f6f67d87f19b2bf24365163368c70c'
+
+# ---------------------------------------------------------------- host mode --
+if [ -z "${BENCH_IN_CONTAINER:-}" ]; then
+    BENCH_DIR="${BENCH_DIR:-$HOME/data/openxc7/bench-openxc7}"
+    exec podman run --rm --network=none --pull=never \
+        -e BENCH_IN_CONTAINER=1 \
+        -e BENCH_HOST="$(hostname)" \
+        -v "$BENCH_DIR":/bench \
+        "$IMAGE" \
+        bash /bench/bench-openxc7.sh "$@"
+fi
+
+# ----------------------------------------------------------- container mode --
+N=5
+while getopts "n:h" opt; do
+    case $opt in
+        n) N=$OPTARG ;;
+        h) sed -n '2,36p' "$0"; exit 0 ;;
+        *) echo "bad option"; exit 2 ;;
+    esac
+done
+shift $((OPTIND - 1))
+
+# design table: name  dir  project  extra-make-args
+ALL_DESIGNS="blinky picosoc-qmtech litex-ddr-arty-s7 litex-ddr-arty-s7-deephier"
+design_row() {
+    case "$1" in
+        blinky)                     echo "designs/main/blinky-digilent-arty blinky -" ;;
+        picosoc-qmtech)             echo "designs/main/picosoc picosoc BOARD=qmtech" ;;
+        litex-ddr-arty-s7)          echo "designs/main/litex-ddr-arty-s7 digilent_arty_s7 -" ;;
+        litex-ddr-arty-s7-deephier) echo "designs/deephier/litex-ddr-arty-s7 digilent_arty_s7 -" ;;
+        *) return 1 ;;
+    esac
+}
+DESIGNS="${*:-$ALL_DESIGNS}"
+for d in $DESIGNS; do design_row "$d" >/dev/null || { echo "unknown design: $d"; exit 2; }; done
+
+cd /bench
+export NEXTPNR_XILINX_DIR=/bench/src/nextpnr-xilinx
+export NEXTPNR_XILINX_PYTHON_DIR=$NEXTPNR_XILINX_DIR/xilinx/python
+export PRJXRAY_DB_DIR=$NEXTPNR_XILINX_DIR/xilinx/external/prjxray-db
+export ARTIX7_CHIPDB=/bench/chipdb
+export KINTEX7_CHIPDB=/bench/chipdb
+export SPARTAN7_CHIPDB=/bench/chipdb
+# our frozen-revision binaries (nextpnr-xilinx and bbasm both sit at the
+# repo root) ahead of the image's own
+export PATH="$NEXTPNR_XILINX_DIR:$PATH"
+git config --global --add safe.directory '*' 2>/dev/null || true
+
+# --- revisions READ from the built tree (never hardcoded) --------------------
+NEXTPNR_REV=$(git -C "$NEXTPNR_XILINX_DIR" rev-parse HEAD)
+DB_GITLINK=$(git -C "$NEXTPNR_XILINX_DIR" ls-tree HEAD xilinx/external/prjxray-db | awk '{print $3}')
+DB_CHECKOUT=$(git -C "$PRJXRAY_DB_DIR" rev-parse HEAD)
+YOSYS_FULL=$(yosys --version | head -1)
+YOSYS_VER=$(printf '%s\n' "$YOSYS_FULL" | sed -n 's/^Yosys \([^ ]*\) (git sha1 \([0-9a-f]*\).*/\1-\2/p')
+[ -n "$YOSYS_VER" ] || YOSYS_VER=$(printf '%s\n' "$YOSYS_FULL" | awk '{print $2}')
+
+# --- preflight guards --------------------------------------------------------
+fail() { echo "FATAL: $*" >&2; exit 1; }
+[ -n "$NEXTPNR_REV" ] || fail "cannot read nextpnr-xilinx revision from tree"
+[ "$DB_GITLINK" = "$DB_CHECKOUT" ] \
+    || fail "prjxray-db checkout ($DB_CHECKOUT) != gitlink ($DB_GITLINK)"
+NEXTPNR_BIN=$(command -v nextpnr-xilinx) || fail "nextpnr-xilinx not on PATH"
+case "$NEXTPNR_BIN" in
+    /bench/src/*) ;;
+    *) fail "nextpnr-xilinx resolves to $NEXTPNR_BIN, not the frozen /bench/src build" ;;
+esac
+NEXTPNR_VERSTR=$("$NEXTPNR_BIN" --version 2>&1 | head -1)
+for bin in chipdb/xc7a35tcsg324.bin chipdb/xc7k325tffg676.bin chipdb/xc7s50csga324.bin; do
+    [ -s "$bin" ] || fail "$bin missing -- run scripts/gen-chipdb.sh first (untimed step)"
+done
+[ -s designs/main/vexriscv/VexRiscv.v ] || fail "designs/main/vexriscv/VexRiscv.v missing"
+[ -s designs/deephier/vexriscv/VexRiscv.v ] || fail "designs/deephier/vexriscv/VexRiscv.v missing"
+
+STAMP=$(date +%Y%m%d-%H%M%S)
+CSV=results/bench-openxc7-$STAMP.csv
+META=results/bench-openxc7-$STAMP-machine.txt
+LOGDIR=logs/campaign-$STAMP
+mkdir -p results "$LOGDIR"
+
+LOAD=$(cut -d' ' -f1-3 /proc/loadavg)
+{
+    echo "bench-openxc7 campaign $STAMP"
+    echo "host:             ${BENCH_HOST:-unknown} (container hostname $(hostname))"
+    echo "image:            $IMAGE"
+    echo "kernel:           $(uname -sr)"
+    echo "cpu:              $(awk -F': ' '/model name/{print $2; exit}' /proc/cpuinfo)"
+    echo "cores:            $(nproc)"
+    echo "mem_total:        $(awk '/MemTotal/{printf "%.1f GiB", $2/1048576}' /proc/meminfo)"
+    echo "loadavg_at_start: $LOAD"
+    echo "runs_per_design:  $N"
+    echo "designs:          $DESIGNS"
+    echo "--- revisions (read from the built tree at run time):"
+    echo "nextpnr-xilinx:   $NEXTPNR_REV"
+    echo "nextpnr version:  $NEXTPNR_VERSTR"
+    echo "prjxray-db:       $DB_CHECKOUT (gitlink-verified)"
+    echo "yosys:            $YOSYS_FULL"
+    echo "demo-projects main:     $(cat designs/main/REVISION 2>/dev/null)"
+    echo "demo-projects deephier: $(cat designs/deephier/REVISION 2>/dev/null)"
+} | tee "$META"
+
+case "$LOAD" in
+    0.*|1.*) ;;
+    *) echo "WARNING: load average is $LOAD -- the box is NOT quiet; timings will be polluted" | tee -a "$META" ;;
+esac
+
+echo "design,run,json_s,fasm_s,bit_s,total_s,nextpnr_rev,prjxraydb_rev,yosys_ver" > "$CSV"
+
+t_stage() {  # t_stage <logfile> <cmd...> -> prints elapsed seconds (%.2f)
+    local log=$1 t0 t1; shift
+    t0=$(date +%s.%N)
+    "$@" > "$log" 2>&1
+    t1=$(date +%s.%N)
+    awk -v a="$t0" -v b="$t1" 'BEGIN{printf "%.2f", b - a}'
+}
+
+for d in $DESIGNS; do
+    read -r dir project extra <<< "$(design_row "$d")"
+    [ "$extra" = "-" ] && extra=""
+    for i in $(seq 1 "$N"); do
+        echo "=== $d run $i/$N (load $(cut -d' ' -f1 /proc/loadavg)) $(date -u +%FT%TZ)"
+        make -C "$dir" $extra clean > "$LOGDIR/$d-run$i-clean.log" 2>&1
+        json_s=$(t_stage "$LOGDIR/$d-run$i-json.log" make -C "$dir" $extra "$project.json")
+        fasm_s=$(t_stage "$LOGDIR/$d-run$i-fasm.log" make -C "$dir" $extra "$project.fasm")
+        bit_s=$(t_stage "$LOGDIR/$d-run$i-bit.log"  make -C "$dir" $extra "$project.bit")
+        [ -s "$dir/$project.bit" ] || fail "$d run $i produced no $project.bit"
+        total_s=$(awk -v a="$json_s" -v b="$fasm_s" -v c="$bit_s" 'BEGIN{printf "%.2f", a+b+c}')
+        echo "$d,$i,$json_s,$fasm_s,$bit_s,$total_s,$NEXTPNR_REV,$DB_CHECKOUT,$YOSYS_VER" >> "$CSV"
+        echo "    json=${json_s}s fasm=${fasm_s}s bit=${bit_s}s total=${total_s}s"
+    done
+done
+
+echo "campaign done: $CSV"
@@ -0,0 +1,154 @@
+#!/usr/bin/env bash
+# bench.sh — Vivado side of the openXC7-vs-Vivado build-time benchmark.
+#
+# Runs N repeated non-project Vivado builds per design and appends one CSV row
+# per run:  design,run,synth_s,place_s,route_s,bitgen_s,total_s,wns,tns,timing_met
+#
+# Usage:
+#   ./bench.sh [-n N] [design ...]
+#     -n N       runs per design (default 5)
+#     design...  names from designs.tsv (default: all)
+#
+# Smoke:            ./bench.sh -n 1 blinky-digilent-arty
+# Full campaign:    mkdir -p logs && nohup ./bench.sh -n 5 > logs/campaign-$(date +%Y%m%d-%H%M%S).log 2>&1 &
+#
+# Per-run artifacts live in runs/<design>/<stamp>/run<i>/ (bit, vivado.log,
+# timing_summary.rpt, utilization.rpt, phase_times.csv). Results CSV + machine
+# description land in results/.
+
+set -eo pipefail
+
+BENCH_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+VIVADO_SETTINGS="${VIVADO_SETTINGS:-/home/builder/data/vivado/2026.1/Vivado/settings64.sh}"
+
+N=5
+while getopts "n:h" opt; do
+    case "$opt" in
+        n) N="$OPTARG" ;;
+        h) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
+        *) exit 2 ;;
+    esac
+done
+shift $((OPTIND - 1))
+
+# Xilinx settings scripts are not `set -u` clean — source first, tighten after.
+# shellcheck disable=SC1090
+source "$VIVADO_SETTINGS"
+set -u
+
+# Vivado 2026.1 telemetry (Flexera RUI SDK, ~/.Xilinx/Vivado/.RUISDK) phones
+# home during write_bitstream; on this egress-blocked host the TCP connects
+# hang in SYN retries and add 5-7 MINUTES of wall time that Vivado books into
+# write_bitstream (measured: bitgen 434 s and 300 s vs 12.8 s clean; evidence
+# in diag/ss-sample.log — endpoints 67.227.186.229:443, 169.254.169.254:80).
+# Pointing the proxy env at a closed local port makes those connects fail
+# instantly WITHOUT touching the build flow: no build step needs network and
+# the license is a local node-locked file.
+export http_proxy=http://127.0.0.1:9 https_proxy=http://127.0.0.1:9
+export HTTP_PROXY=$http_proxy HTTPS_PROXY=$https_proxy
+
+# The proxy blackhole above is NOT sufficient: the RUI SDK connects DIRECTLY
+# (raw connect(), never consults the proxy env) to 67.227.186.229:443 /
+# 72.52.161.233:443 and probes the cloud-metadata address 169.254.169.254:80
+# during write_bitstream. Campaign 20260815-080720 measured bitgen_s of
+# 138.68/282.69/140.69/14.13/407.99 for identical deterministic work (true
+# value ~13 s); diag/ss-sample.log and diag/ss-diagB.log show the SYN-SENT
+# sockets owned by the vivado pid while the proxy env was exported. So EVERY
+# vivado call goes through vivado-nonet.sh: a rootless user+network namespace
+# (unshare -r -n) where only loopback exists and connect() fails instantly
+# with ENETUNREACH — hermetic by construction. The node-locked license hostid
+# (a MAC) is satisfied by a dummy lic0 interface inside the namespace; see
+# vivado-nonet.sh. The proxy env above stays as belt-and-suspenders.
+vivado_nonet() {
+    "$BENCH_ROOT/vivado-nonet.sh" "$@"
+}
+
+TSV="$BENCH_ROOT/designs.tsv"
+all_designs=$(grep -v '^\s*#' "$TSV" | grep -v '^\s*$' | cut -d'|' -f1)
+if [ "$#" -gt 0 ]; then designs="$*"; else designs=$all_designs; fi
+
+stamp=$(date +%Y%m%d-%H%M%S)
+RESULTS="$BENCH_ROOT/results"
+mkdir -p "$RESULTS"
+CSV="$RESULTS/bench-$stamp.csv"
+echo "design,run,synth_s,place_s,route_s,bitgen_s,total_s,wns,tns,timing_met" > "$CSV"
+
+# Machine description (the method requires it alongside every result set).
+MACH="$RESULTS/machine-$stamp.txt"
+{
+    echo "hostname:   $(hostname)"
+    echo "date:       $(date -u +%Y-%m-%dT%H:%M:%SZ)"
+    echo "nproc:      $(nproc)"
+    free -g | awk '/^Mem:/{print "mem_gb:     "$2}'
+    echo "os:         $(lsb_release -ds 2>/dev/null || cat /etc/os-release | head -1)"
+    echo "cpu:        $(grep -m1 'model name' /proc/cpuinfo | cut -d: -f2 | sed 's/^ //')"
+    echo "vivado:     $(vivado_nonet -version 2>/dev/null | head -1)"
+    echo "runs_per_design: $N"
+    echo "designs:    $(echo $designs | tr '\n' ' ')"
+} > "$MACH"
+cat "$MACH"
+
+lookup() { # $1=design -> prints the tsv row
+    grep -v '^\s*#' "$TSV" | awk -F'|' -v d="$1" '$1 == d {print; found=1} END {exit !found}'
+}
+
+for design in $designs; do
+    if ! row=$(lookup "$design"); then
+        echo "ERROR: design '$design' not in $TSV" >&2
+        exit 2
+    fi
+    IFS='|' read -r name dir top part xdc srcs globs <<< "$row"
+    srcdir="$BENCH_ROOT/designs/$dir"
+
+    # Absolutize sources and constraints against the design dir.
+    abs_srcs=""
+    for s in $srcs; do abs_srcs="$abs_srcs $srcdir/$s"; done
+    abs_xdc="$srcdir/$xdc"
+
+    for i in $(seq 1 "$N"); do
+        rundir="$BENCH_ROOT/runs/$design/$stamp/run$i"
+        mkdir -p "$rundir"
+        # Data files ($readmemh .init/.mem) resolve against Vivado's cwd.
+        if [ -n "$globs" ]; then
+            for g in $globs; do cp "$srcdir"/$g "$rundir"/ 2>/dev/null || true; done
+        fi
+        echo "=== $design run $i/$N ($(date +%H:%M:%S)) ==="
+        if (cd "$rundir" && \
+            BM_TOP="$top" BM_PART="$part" BM_XDC="$abs_xdc" \
+            BM_SOURCES="$abs_srcs" BM_OUT="$design" \
+            vivado_nonet -mode batch -nojournal -log vivado.log \
+                   -source "$BENCH_ROOT/run-design.tcl" > console.log 2>&1) \
+           && [ -s "$rundir/phase_times.csv" ] && [ -s "$rundir/$design.bit" ]; then
+            echo "$design,$i,$(cat "$rundir/phase_times.csv")" >> "$CSV"
+            tail -1 "$CSV"
+        else
+            echo "$design,$i,NA,NA,NA,NA,NA,na,na,error" >> "$CSV"
+            echo "RUN FAILED — see $rundir/vivado.log (last lines):"
+            tail -5 "$rundir/vivado.log" 2>/dev/null || tail -5 "$rundir/console.log"
+        fi
+    done
+done
+
+echo
+echo "=== summary (median + spread of total_s per design) ==="
+python3 - "$CSV" <<'EOF'
+import csv, statistics, sys
+rows = list(csv.DictReader(open(sys.argv[1])))
+by = {}
+for r in rows:
+    by.setdefault(r["design"], []).append(r)
+print(f"{'design':<28} {'n':>2} {'median_s':>9} {'min_s':>8} {'max_s':>8} {'spread%':>8}  timing")
+for d, rs in by.items():
+    ok = [float(r["total_s"]) for r in rs if r["timing_met"] != "error"]
+    met = ",".join(sorted({r["timing_met"] for r in rs}))
+    if not ok:
+        print(f"{d:<28} {0:>2} {'-':>9} {'-':>8} {'-':>8} {'-':>8}  ALL FAILED")
+        continue
+    med = statistics.median(ok)
+    spread = 100.0 * (max(ok) - min(ok)) / med if med else 0.0
+    print(f"{d:<28} {len(ok):>2} {med:>9.2f} {min(ok):>8.2f} {max(ok):>8.2f} {spread:>7.1f}%  met={met}")
+EOF
+
+echo
+echo "CSV:     $CSV"
+echo "machine: $MACH"
@@ -0,0 +1,18 @@
+# openXC7-vs-Vivado benchmark — design table (fields separated by '|')
+# name | dir (relative to designs/) | top module | part | xdc | verilog sources (relative to dir) | data-file globs copied into the run dir
+#
+# Configurations mirror the openXC7 demo-projects CI (smoke.yml at the frozen
+# main revision). picosoc: BOARD=qmtech -> xc7k325tffg676-1, top module `top`
+# (in qmtech.v), XDC picosoc-qmtech.xdc. The previous kx2 config
+# (xc7k160tffg676-2, kx2.v) fails Vivado DRC PLIO-9 — the clock is LOCed to an
+# N-type CCIO — so it cannot produce a Vivado baseline (kept in
+# designs.tsv.bak-kx2). NOTE: picosoc-qmtech.xdc carries NO active
+# create_clock (line 4 is commented out; the clock lives in qmtech.sdc, which
+# this non-project XDC flow does not read) -> the picosoc row is unconstrained
+# and its wns/tns/timing_met columns are not meaningful (met=na, like blinky).
+# The litex designs reference ../vexriscv/VexRiscv.v
+# exactly as their Makefiles do (VexRiscv.v DIFFERS between the two revisions).
+blinky-digilent-arty|main/blinky-digilent-arty|blinky|xc7a35tcsg324-1|blinky.xdc|blinky.v|
+picosoc|main/picosoc|top|xc7k325tffg676-1|picosoc-qmtech.xdc|qmtech.v picosoc_noflash.v picorv32.v progmem.v simpleuart.v|
+litex-ddr-arty-s7|main/litex-ddr-arty-s7|digilent_arty_s7|xc7s50csga324-1|digilent_arty_s7.xdc|digilent_arty_s7.v ../vexriscv/VexRiscv.v|*.init
+litex-ddr-arty-s7-deephier|deephier/litex-ddr-arty-s7|digilent_arty_s7|xc7s50csga324-1|digilent_arty_s7.xdc|digilent_arty_s7.v ../vexriscv/VexRiscv.v|*.init