Wikiservice: Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference

2026-08-28T10:14:02Z

Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference

← Older revision		Revision as of 12:14, 28 August 2026
Line 14:		Line 14:
	== Current headline observations ==		== Current headline observations ==

	MemCP has reached '''~~up to~~ 10× ~~the performance of~~ MariaDB/PostgreSQL''' in measured OLAP and search-oriented workflows. The relevant mechanisms include compact RecSet domains, compressed column scans, batch execution, late materialization, adaptive physical structures and shard parallelism.		MemCP has reached '''speedups of 10× and more over MariaDB/PostgreSQL''' in measured OLAP and search-oriented workflows. The relevant mechanisms include compact RecSet domains, compressed column scans, batch execution, late materialization, adaptive physical structures and shard parallelism.

	Current isolated OLTP measurements show the opposite trade-off: those paths take about '''1.3–2.0× as long''' on MemCP. In complete WordPress- and wiki-style page builds, this has made '''no significant difference to overall page-loading time''' in the measured application workflows. The application, the number and mix of queries, and the share of time spent outside SQL determine the end-to-end effect. By contrast, filtered-list ~~workflows~~ over roughly one million documents ~~have taken~~ around '''30 seconds on PostgreSQL'''. That makes search, filtering, grouping and analytical reads the clearest current migration targets while OLTP/JIT work continues.		Current isolated OLTP measurements show the opposite trade-off: those paths take about '''1.3–2.0× as long''' on MemCP. In complete WordPress- and wiki-style page builds, this has made '''no significant difference to overall page-loading time''' in the measured application workflows. The application, the number and mix of queries, and the share of time spent outside SQL determine the end-to-end effect. By contrast, one filtered-list workflow over roughly one million documents took around '''30 seconds on PostgreSQL''' and '''1.6 seconds on MemCP''' for the same query. That makes search, filtering, grouping and analytical reads the clearest current migration targets while OLTP/JIT work continues.

	These headline figures should be stated rather than hidden, but they do not replace the reproducibility record below. A benchmark publication must attach the exact competing versions, schema/query, dataset and distribution, hardware, durability settings, cache state, concurrency, result validation and raw samples. Do not combine the best result from one workload with the conditions of another.		These headline figures should be stated rather than hidden, but they do not replace the reproducibility record below. A benchmark publication must attach the exact competing versions, schema/query, dataset and distribution, hardware, durability settings, cache state, concurrency, result validation and raw samples. Do not combine the best result from one workload with the conditions of another.
Line 24:		Line 24:
	Use calibrated SQL performance suites for end-to-end plans and Go benchmarks for isolated storage primitives. Run multiple samples, report distributions rather than the best run, and compare against current master under identical conditions.		Use calibrated SQL performance suites for end-to-end plans and Go benchmarks for isolated storage primitives. Run multiple samples, report distributions rather than the best run, and compare against current master under identical conditions.

	<~~syntaxhighlight lang="bash"~~>make test		<pre>make test
	PERF_TEST=1 PERF_EXPLAIN=1 make test		PERF_TEST=1 PERF_EXPLAIN=1 make test
	go test ./storage/ -bench 'BenchmarkName' -run '^$' -count=5</~~syntaxhighlight~~>		go test ./storage/ -bench 'BenchmarkName' -run '^$' -count=5</pre>
	For A/B changes, save raw output for both commits and use <code>benchstat</code>. Profiles should distinguish compile time, physical preparation, execution, allocation, lock waits, storage reload, and background maintenance. Attach EXPLAIN and <code>EXPLAIN COMPILE</code> for planner-sensitive results.		For A/B changes, save raw output for both commits and use <code>benchstat</code>. Profiles should distinguish compile time, physical preparation, execution, allocation, lock waits, storage reload, and background maintenance. Attach EXPLAIN and <code>EXPLAIN COMPILE</code> for planner-sensitive results.

Line 33:		Line 33:
	Suites below <code>tests/performance/</code> are disabled during ordinary correctness runs unless <code>PERF_TEST=1</code> is set. The runner stores machine-specific row counts and timings in <code>.perf_baseline.json</code>, performs warm-up work, repeats measured cases (five times by default), and reports the median. A result above the stored baseline plus tolerance fails the gate.		Suites below <code>tests/performance/</code> are disabled during ordinary correctness runs unless <code>PERF_TEST=1</code> is set. The runner stores machine-specific row counts and timings in <code>.perf_baseline.json</code>, performs warm-up work, repeats measured cases (five times by default), and reports the median. A result above the stored baseline plus tolerance fails the gate.

	<~~syntaxhighlight lang="bash"~~>		<pre>
	# Run with existing baselines		# Run with existing baselines
	PERF_TEST=1 make test		PERF_TEST=1 make test
Line 45:		Line 45:
	# Include query-plan output		# Include query-plan output
	PERF_TEST=1 PERF_EXPLAIN=1 make test		PERF_TEST=1 PERF_EXPLAIN=1 make test
	</~~syntaxhighlight~~>		</pre>

	Calibration aims for sufficiently long cases (currently roughly 10–20 seconds) and adjusts rows gradually. The runner also applies one protected pre-server wall-clock calibration for the execution architecture; a regression inside MemCP must not be allowed to enlarge its own time allowance.		Calibration aims for sufficiently long cases (currently roughly 10–20 seconds) and adjusts rows gradually. The runner also applies one protected pre-server wall-clock calibration for the execution architecture; a regression inside MemCP must not be allowed to enlarge its own time allowance.
Line 73:		Line 73:
	For a regression bisect, calibrate on the known-good commit, keep a copy of its baseline, and freeze row counts while Git tests candidate commits:		For a regression bisect, calibrate on the known-good commit, keep a copy of its baseline, and freeze row counts while Git tests candidate commits:

	<~~syntaxhighlight lang="bash"~~>git checkout GOOD_COMMIT		<pre>git checkout GOOD_COMMIT
	PERF_TEST=1 PERF_CALIBRATE=1 make test # repeat until stable		PERF_TEST=1 PERF_CALIBRATE=1 make test # repeat until stable
	cp .perf_baseline.json /tmp/memcp-perf-good.json		cp .perf_baseline.json /tmp/memcp-perf-good.json
Line 79:		Line 79:
	git bisect start HEAD GOOD_COMMIT		git bisect start HEAD GOOD_COMMIT
	cp /tmp/memcp-perf-good.json .perf_baseline.json		cp /tmp/memcp-perf-good.json .perf_baseline.json
	git bisect run bash -c 'PERF_TEST=1 PERF_NORECALIBRATE=1 make test'</~~syntaxhighlight~~>		git bisect run bash -c 'PERF_TEST=1 PERF_NORECALIBRATE=1 make test'</pre>

	== Storage microbenchmarks and profiles ==		== Storage microbenchmarks and profiles ==
Line 85:		Line 85:
	For a storage encoding or hot loop, benchmark the smallest relevant package and compare several samples:		For a storage encoding or hot loop, benchmark the smallest relevant package and compare several samples:

	<~~syntaxhighlight lang="bash"~~>		<pre>
	go test ./storage/ -bench 'BenchmarkEnumPerElem' -run '^$' -count=5 > /tmp/before.txt		go test ./storage/ -bench 'BenchmarkEnumPerElem' -run '^$' -count=5 > /tmp/before.txt
	# apply/build the candidate under the same conditions		# apply/build the candidate under the same conditions
	go test ./storage/ -bench 'BenchmarkEnumPerElem' -run '^$' -count=5 > /tmp/after.txt		go test ./storage/ -bench 'BenchmarkEnumPerElem' -run '^$' -count=5 > /tmp/after.txt
	benchstat /tmp/before.txt /tmp/after.txt		benchstat /tmp/before.txt /tmp/after.txt
	</~~syntaxhighlight~~>		</pre>

	Microbenchmarks establish mechanism, not end-to-end query benefit. Confirm the change with SQL, result validation, representative cardinalities, and warm/cold conditions. Use CPU, memory/allocation, mutex/block, and I/O profiles to explain a difference instead of attributing every gain to the edited function.		Microbenchmarks establish mechanism, not end-to-end query benefit. Confirm the change with SQL, result validation, representative cardinalities, and warm/cold conditions. Use CPU, memory/allocation, mutex/block, and I/O profiles to explain a difference instead of attributing every gain to the edited function.

Wikiservice: Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference

2026-08-28T09:59:34Z

Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference

Show changes

Carli: Created page with " == Performance Testing Framework == MemCP includes an auto-calibrating performance test framework for regression detection and benchmarking. === Running Performance Tests === `# Run performance tests (uses calibrated baselines) PERF_TEST=1 make test # Calibrate for your machine (run ~10 times to reach target) PERF_TEST=1 PERF_CALIBRATE=1 make test # Freeze row counts for bisecting regressions PERF_TEST=1 PERF_NORECALIBRATE=1 make test # Show query exe..."`


2026-01-26T00:00:10Z
Created page with " == Performance Testing Framework == MemCP includes an auto-calibrating performance test framework for regression detection and benchmarking.  === Running Performance Tests ===  <code># Run performance tests (uses calibrated baselines)  PERF_TEST=1 make test    # Calibrate for your machine (run ~10 times to reach target)  PERF_TEST=1 PERF_CALIBRATE=1 make test    # Freeze row counts for bisecting regressions  PERF_TEST=1 PERF_NORECALIBRATE=1 make test    # Show query exe..."
New page


== Performance Testing Framework ==

MemCP includes an auto-calibrating performance test framework for regression detection and benchmarking.



=== Running Performance Tests ===

 <code># Run performance tests (uses calibrated baselines)

 PERF_TEST=1 make test

 

 # Calibrate for your machine (run ~10 times to reach target)

 PERF_TEST=1 PERF_CALIBRATE=1 make test

 

 # Freeze row counts for bisecting regressions

 PERF_TEST=1 PERF_NORECALIBRATE=1 make test

 

 # Show query execution plans

 PERF_TEST=1 PERF_EXPLAIN=1 make test</code>



=== How Auto-Calibration Works ===



# Target query time: '''10-20 seconds'''

# Each calibration run scales row counts by '''30%''' up or down

# Baselines are machine-specific, stored in <code>.perf_baseline.json</code>

# Tests include 2 warmup runs before the measured run



=== Output Format ===

 <code>✅ Perf: COUNT (4.3s / 13s, 100,000,000 rows, 0.04µs/row, 25GB heap, 1522%/2400% CPU)

          │      │     │              │            │           └─ CPU utilization

          │      │     │              │            └─ Memory usage

          │      │     │              └─ Time per row

          │      │     └─ Calibrated row count

          │      └─ Threshold

          └─ Actual time</code>



=== Bisecting Performance Regressions ===

 <code># 1. Calibrate on known-good commit

 git checkout <good-commit>

 PERF_TEST=1 PERF_CALIBRATE=1 make test  # run 10x

 

 # 2. Save baseline

 cp .perf_baseline.json .perf_baseline_good.json

 

 # 3. Run git bisect

 git bisect start HEAD <good-commit>

 git bisect run bash -c 'PERF_TEST=1 PERF_NORECALIBRATE=1 make test'</code>



=== Environment Variables ===

{| class="wikitable"

!Variable

!Values

!Description

|-

|<code>PERF_TEST</code>

|<code>0</code>/<code>1</code>

|Enable performance tests

|-

|<code>PERF_CALIBRATE</code>

|<code>0</code>/<code>1</code>

|Update baselines

|-

|<code>PERF_NORECALIBRATE</code>

|<code>0</code>/<code>1</code>

|Freeze row counts

|-

|<code>PERF_EXPLAIN</code>

|<code>0</code>/<code>1</code>

|Show query plans

|}

Performance Measurement - Revision history

Wikiservice: Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference

Wikiservice: Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference