https://www.memcp.org/index.php?action=history&feed=atom&title=Performance_Measurement 2026-04-04T15:39:43Z Revision history for this page on the wiki MediaWiki 1.39.1 https://www.memcp.org/index.php?title=Performance_Measurement&diff=251&oldid=prev 2026-01-26T00:00:10Z

<p>Created page with &quot; == Performance Testing Framework == MemCP includes an auto-calibrating performance test framework for regression detection and benchmarking. === Running Performance Tests === &lt;code&gt;# Run performance tests (uses calibrated baselines) PERF_TEST=1 make test # Calibrate for your machine (run ~10 times to reach target) PERF_TEST=1 PERF_CALIBRATE=1 make test # Freeze row counts for bisecting regressions PERF_TEST=1 PERF_NORECALIBRATE=1 make test # Show query exe...&quot;</p> <p><b>New page</b></p><div><br /> == Performance Testing Framework ==<br /> MemCP includes an auto-calibrating performance test framework for regression detection and benchmarking.<br /> <br /> === Running Performance Tests ===<br /> &lt;code&gt;# Run performance tests (uses calibrated baselines)<br /> PERF_TEST=1 make test<br /> <br /> # Calibrate for your machine (run ~10 times to reach target)<br /> PERF_TEST=1 PERF_CALIBRATE=1 make test<br /> <br /> # Freeze row counts for bisecting regressions<br /> PERF_TEST=1 PERF_NORECALIBRATE=1 make test<br /> <br /> # Show query execution plans<br /> PERF_TEST=1 PERF_EXPLAIN=1 make test&lt;/code&gt;<br /> <br /> === How Auto-Calibration Works ===<br /> <br /> # Target query time: &#039;&#039;&#039;10-20 seconds&#039;&#039;&#039;<br /> # Each calibration run scales row counts by &#039;&#039;&#039;30%&#039;&#039;&#039; up or down<br /> # Baselines are machine-specific, stored in &lt;code&gt;.perf_baseline.json&lt;/code&gt;<br /> # Tests include 2 warmup runs before the measured run<br /> <br /> === Output Format ===<br /> &lt;code&gt;✅ Perf: COUNT (4.3s / 13s, 100,000,000 rows, 0.04µs/row, 25GB heap, 1522%/2400% CPU)<br /> │ │ │ │ │ └─ CPU utilization<br /> │ │ │ │ └─ Memory usage<br /> │ │ │ └─ Time per row<br /> │ │ └─ Calibrated row count<br /> │ └─ Threshold<br /> └─ Actual time&lt;/code&gt;<br /> <br /> === Bisecting Performance Regressions ===<br /> &lt;code&gt;# 1. Calibrate on known-good commit<br /> git checkout &lt;good-commit&gt;<br /> PERF_TEST=1 PERF_CALIBRATE=1 make test # run 10x<br /> <br /> # 2. Save baseline<br /> cp .perf_baseline.json .perf_baseline_good.json<br /> <br /> # 3. Run git bisect<br /> git bisect start HEAD &lt;good-commit&gt;<br /> git bisect run bash -c &#039;PERF_TEST=1 PERF_NORECALIBRATE=1 make test&#039;&lt;/code&gt;<br /> <br /> === Environment Variables ===<br /> {| class=&quot;wikitable&quot;<br /> !Variable<br /> !Values<br /> !Description<br /> |-<br /> |&lt;code&gt;PERF_TEST&lt;/code&gt;<br /> |&lt;code&gt;0&lt;/code&gt;/&lt;code&gt;1&lt;/code&gt;<br /> |Enable performance tests<br /> |-<br /> |&lt;code&gt;PERF_CALIBRATE&lt;/code&gt;<br /> |&lt;code&gt;0&lt;/code&gt;/&lt;code&gt;1&lt;/code&gt;<br /> |Update baselines<br /> |-<br /> |&lt;code&gt;PERF_NORECALIBRATE&lt;/code&gt;<br /> |&lt;code&gt;0&lt;/code&gt;/&lt;code&gt;1&lt;/code&gt;<br /> |Freeze row counts<br /> |-<br /> |&lt;code&gt;PERF_EXPLAIN&lt;/code&gt;<br /> |&lt;code&gt;0&lt;/code&gt;/&lt;code&gt;1&lt;/code&gt;<br /> |Show query plans<br /> |}</div>

Carli