Main Page

From MemCP
Revision as of 11:59, 28 August 2026 by Wikiservice (talk | contribs) (Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference)
Jump to navigation Jump to search


MemCP Database – Fast, Compressed SQL for OLTP and OLAP

MemCP
A fast, compressed, MySQL-protocol-compatible columnar database for modern OLTP and OLAP workloads
MemCP is an open-source SQL database written in Go. It combines persistent in-memory operation, adaptive column compression, parallel query execution and direct application APIs so operational data can be queried and analyzed without maintaining a separate analytical copy.
Beta Open Source MySQL Protocol + HTTP APIs
Get started with Docker Explore SQL support View on GitHub
MemCP database dashboard and workload overview

Development status: Beta. MemCP is under active development. Check Supported SQL, Current Status and Open Issues and the durability requirements of your workload before migrating production data.

Your MySQL queries are too slow? Run the same representative workload on MemCP instead of assuming another index or a larger MySQL server is the only answer. Compare results, durability and authenticated end-to-end latency; if the migration gates pass, move the performance-critical workload to MemCP. Start the MemCP performance evaluation →

Observed performance profile: MemCP has achieved up to 10× the performance of MariaDB/PostgreSQL in OLAP and search-oriented workflows, where RecSets and compressed column scans avoid wide row materialization. Isolated OLTP paths currently take about 1.3–2.0× as long, but in complete WordPress- and wiki-style page builds this has made no significant difference to overall page-loading time in the measured application workflows. Filtered-list queries over roughly one million documents, by contrast, have taken around 30 seconds on PostgreSQL. These are workload observations, not universal guarantees; reproduce them on your data.

What is MemCP?

MemCP Database is a persistent, compressed, column-oriented SQL database for mixed OLTP and OLAP workloads. It speaks the MySQL client protocol, also exposes SQL over HTTP, and executes supported queries through a functional compiler and a parallel column-storage engine.

The name on this site refers to the database project. MemCP Database is not a Model Context Protocol (MCP) memory server and is unrelated to the C/C++ memcpy() memory-copy function. When writing about or linking to the project, the unambiguous name is MemCP Database.

Problems MemCP solves

Slow filtered lists on millions of rows

Complex WHERE, joins, membership tests, ORDER BY and LIMIT can make a row store inspect or materialize far more data than the page returns. MemCP uses RecSets, late materialization, adaptive indexes and ordered braking to keep the working domain compact.
Evaluate the query · How RecSets work

Slow GROUP BY, COUNT and dashboards

Column scans read only referenced compressed values. Parallel shard-local aggregation, group caches and computed structures target repeated analytical queries over fresh operational data.
Columnar Storage · Temporary Computed Columns

Slow database writes caused by fsync

Durability is selectable per table. safe protects commits through power loss; logged has measured about 10× its write throughput when process-crash recovery is sufficient.
Choose a write strategy

Database writes wear out an SD card

For reconstructible edge data, sloppy avoids a continuous WAL and normally publishes one compressed generation every 15 minutes, reducing constant flash writes while making the loss window explicit.
Hardware Requirements · Persistency and Performance Guarantees

Choose your path

Evaluate MemCP

Understand the workload model, current status, hardware needs and differences from MySQL.

OLTP and OLAP · Compare with MySQL · Hardware requirements

Build an application

Use the MySQL protocol, SQL over HTTP, RDF or application-specific endpoints inside MemCP.

SQL tutorial · SQL over HTTP · Client tooling

Operate MemCP

Plan deployment, migration, persistence, storage backends, settings and performance measurement.

Deployment · Durability · Migration

Understand and contribute

Explore the storage engine, optimizer, embedded Scheme runtime and project internals.

Query planner · RecSets · Columnar Storage · Contributing

Quickstart

Run with Docker

<syntaxhighlight lang="bash"> docker run --name memcp \ 

 -e ROOT_PASSWORD='choose-a-password' \
 -p 4321:4321 -p 3307:3307 \
 -v memcp-data:/data \
 carli2/memcp:latest

</syntaxhighlight>

Then open http://localhost:4321 and connect a MySQL application to 127.0.0.1 on port 3307. The volume mounted at /data retains persistent table data across container replacement. See the complete Docker guide before deploying it as a service.

Build from source

<syntaxhighlight lang="bash"> git clone https://github.com/launix-de/memcp cd memcp go mod download make ./memcp --api-port=4321 --mysql-port=3307 lib/main.scm </syntaxhighlight>

Connect with MySQL tooling:

<syntaxhighlight lang="bash"> mysql -h 127.0.0.1 -u root -p -P 3307

  1. Enter the password selected for this data directory.

</syntaxhighlight>

The development default for a fresh data directory is admin. Change it before exposing MemCP to another machine. MemCP can also be supervised with PM2:

<syntaxhighlight lang="bash"> pm2 start ./memcp --name memcp -- \ 

 --no-repl -data ./data --api-port=4321 --mysql-port=3307 lib/main.scm

</syntaxhighlight>

Background deployments must use --no-repl so that closing standard input does not terminate the interactive console and stop the server. See Build from Source, Deployment and MemCP Console.

Key features

High performance
Parallel, batch-oriented query execution is designed for multicore CPUs, compact working sets and fast persistent storage, serving both OLTP and OLAP workloads.
Columnar storage
Data is stored by column for improved compression, a smaller memory footprint and fast analytical access.
Persistent in-memory operation
MemCP is designed to keep active data in memory while offering configurable per-table durability and persistence backends.
Fast configurable writes
Choose durability per table: safe protects committed data through power loss, while measured logged paths have reached about 10× its write throughput; sloppy batches reconstructible data into the normal 15-minute rebuild cycle.
Built-in APIs
SQL over HTTP and in-database services can remove an extra middleware hop for suitable applications.
Adaptive compression
Bit-packing, dictionary encoding and sequence compression reduce the bytes read for suitable data. Historical imports have reached reductions around 80% versus their MySQL/MariaDB representation; measure your own schema.
Simple deployment
Start with Docker, PM2 or the native binary. The compact application has historically had an installation footprint of approximately 10 MB.
Extensible frontends
The Go storage engine and embedded Scheme environment support SQL, RDF, REST and custom application interfaces.
Cost-based query execution
Logical decorrelation and join optimization are separated from cost-based physical selection of scans, indexes, RecSets, reusable caches and execution pipelines.
How MemCP plans and lowers queries → · How compact record sets process large domains →

Why MemCP?

Traditional relational databases were designed around spinning disks and comparatively small numbers of CPU cores. MemCP rethinks storage and query execution for modern multicore systems, large caches, fast storage and mixed workloads.

Typical use cases include:

  • real-time dashboards and analytics;
  • data-heavy SaaS platforms;
  • embedded systems with limited resources;
  • high-throughput OLTP/OLAP hybrids.

Whether MemCP is a good fit depends on the required SQL compatibility, durability, query mix, data size and operational environment. Review Supported SQL, Persistency and Performance Guarantees and Performance Measurement rather than treating benchmark figures as universal guarantees.

MemCP vs. MySQL

Feature MySQL MemCP
Storage model Primarily row-based Column-based and compressed
Performance focus General-purpose relational workloads Parallel, in-memory-oriented execution for mixed operational and analytical workloads
In-memory capability Available through selected engines and caching Central design goal and default operating model
REST API integration Normally external Built in
Installation footprint Common server installations are approximately 150 MB or larger The native application has historically been approximately 10 MB
Open source ✅ ✅

MemCP provides MySQL protocol compatibility but does not claim to implement every MySQL feature. See Comparison: MemCP vs. MySQL for a detailed comparison and Database Tools compatibility with MemCP for client compatibility.

Architecture overview

Documentation

The following navigation remains on the start page so users and search engines can reach every major documentation area directly.

Introduction and evaluation

Getting started

Administration

Frontends

SQL frontend

RDF frontend

Custom frontends

Persistence backends and storage

Internals

How MemCP works

Scheme documentation

Optimizations

Further reading

Additional blog posts on design decisions, compression techniques and performance optimization are available on the Launix blog.

Community

MemCP is an open-source project maintained by developers for developers. Contributions are welcome in the form of bug reports, feature requests and pull requests.

See Contributing and the GitHub repository.

Retrieved from "https://www.memcp.org/index.php?title=Main_Page&oldid=370"