Scan

Scan

scan, scan_order, scan_order_multi, scan_exists, and transaction-bound variants are physical storage operators emitted after logical planning. Their exact generated signatures are listed under Storage.

The physical lowerer extracts safe equality, range, IN-list, LIKE-prefix, computed-expression, ordering, and RecSet constraints. A boundary may be exact or merely a candidate superset. Candidate boundaries always retain the original SQL predicate as a residual filter.

Column values are read in batches through encoding-specific range or multi-record fast paths. Ordered scans can combine main-index and delta ordering, propagate early stop, and use offset/limit or top-k braking when semantics and cost allow. RecSets identify records for one base relation and visibility snapshot; they do not define join multiplicity or result order. Their adaptive ranges, positive-ID lists, and bitmaps let later scans reuse a narrow domain without materializing wide rows.

Application code should use SQL. Planner contributors must keep scan objects, RecSets, ORC columns, and helper tables out of the parser and logical IR. See Query Planner and Physical Lowering and the repository's INVARIANTS.md.

Reading a plan

For a query such as:

SELECT customer_id, SUM(total)
FROM orders
WHERE created_at >= '2026-01-01'
GROUP BY customer_id
ORDER BY SUM(total) DESC
LIMIT 20;

the logical plan establishes filtering, grouping, ordering, and LIMIT semantics. Physical lowering can then choose a bounded date range, read only created_at, customer_id, and total, fuse filter/aggregation, and apply top-k braking. Use EXPLAIN PHYSICAL to see what was actually selected; SQL spelling alone does not force an index or RecSet.

Low-level callback contract

Unordered scans can perform shard-local map/reduce work and combine partial accumulators with a second reducer. Ordered scans retain serial output order and own OFFSET/LIMIT/early stop. Update-capable callbacks receive a controlled row-update handle under the scan's transaction and locking rules. The generated Storage chapter is authoritative for parameter names and return types at the referenced commit.

Stage	Unordered scan	Ordered scan_order
Filter/access path	Parallel per eligible shard; may use indexes/boundaries	Parallel candidate filtering and local ordering where useful
Map	Parallel per shard	Applied in requested global order
Reduce	Shard-local partial reductions plus a final combine	Serial in output order when order affects semantics
Early stop	Cancellation or consumer stop	OFFSET/LIMIT and compatible ordered braking

Scheme examples

These examples illustrate the callback roles; use the exact current signatures from Storage when writing low-level code.

/* resolve the current transaction and table once */
(set tx ((context "session") "__memcp_tx"))
(set tbl1 (table "schema" "tbl1"))
(set tbl2 (table "schema" "tbl2"))

/* print key/value pairs */
(scan tx tbl1 '() (lambda () true)
	'("k" "v") (lambda (k v) (print "tbl1[" k "] = " v)))

/* find a key through an indexable equality predicate */
(scan tx tbl1 '("k") (lambda (k) (equal? k 12))
	'("v") (lambda (v) v))

/* shard-local sum with neutral value */
(scan tx tbl2 '() (lambda () true)
	'("weight") (lambda (weight) weight) + 0)

An ordered scan additionally receives sort expressions/directions, OFFSET and LIMIT. Filtering and local ordering may run in parallel, but output order and order-sensitive reduction remain serial. Do not use side effects as a substitute for a reducer when result order matters. See RecSets for ordered membership intersection and progressively filtered candidate batches.