Main Page: Difference between revisions

From MemCP
Jump to navigation Jump to search
No edit summary
No edit summary
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
== 🚀 Meet MemCP – The MySQL Alternative You've Been Waiting For ==
= MemCP – A Modern In-Memory Columnar Database =
'''Tired of outdated database engines slowing you down?'''


MemCP is a blazing-fast, in-memory, column-oriented database designed for modern workloads. It delivers '''unmatched speed''', '''massive compression''', and '''built-in REST APIs''' – all with a developer-friendly open-source stack.<blockquote>'''Replace MySQL with something better.'''
<div style="padding:2.5rem 2rem; margin:0 0 1.5rem; border-radius:14px; background:linear-gradient(135deg,#3d6208 0%,#64910c 52%,#76b512 100%); color:#fff; text-align:center;">
<div style="font-size:2.6rem; line-height:1.1; font-weight:700; margin-bottom:.7rem;">MemCP</div>
<div style="font-size:1.45rem; line-height:1.35; font-weight:600; margin:0 auto .9rem; max-width:56rem;">A fast, compressed, MySQL-compatible columnar database for modern OLTP and OLAP workloads</div>
<div style="font-size:1.05rem; line-height:1.6; margin:0 auto 1.4rem; max-width:54rem;">MemCP combines persistent in-memory operation, adaptive column compression, parallel query execution and direct application APIs in a lightweight database written in Go.</div>
<div style="margin-bottom:1.5rem;"><span style="display:inline-block; padding:.2rem .65rem; margin:.15rem; border:1px solid #ffffff88; border-radius:1rem;">Beta</span> <span style="display:inline-block; padding:.2rem .65rem; margin:.15rem; border:1px solid #ffffff88; border-radius:1rem;">Open Source</span> <span style="display:inline-block; padding:.2rem .65rem; margin:.15rem; border:1px solid #ffffff88; border-radius:1rem;">MySQL Protocol + HTTP APIs</span></div>
<div class="plainlinks"><span style="display:inline-block; margin:.25rem;">[[Install MemCP with Docker|<span style="display:inline-block; padding:.65rem 1rem; border-radius:6px; background:#fff; color:#496f0c; font-weight:700;">Get started with Docker</span>]]</span> <span style="display:inline-block; margin:.25rem;">[[Supported SQL|<span style="display:inline-block; padding:.65rem 1rem; border:1px solid #fff; border-radius:6px; color:#fff; font-weight:700;">Explore SQL support</span>]]</span> <span style="display:inline-block; margin:.25rem;">[https://github.com/launix-de/memcp <span style="display:inline-block; padding:.65rem 1rem; border:1px solid #fff; border-radius:6px; color:#fff; font-weight:700;">View on GitHub</span>]</span></div>
</div>


Something built for the 2020s – not stuck in 2005.</blockquote>
[[File:Memcp-Load.png|center|frameless|1000px|alt=MemCP database dashboard and workload overview]]
----


=== ⚡ Why Developers Choose MemCP ===
<div style="padding:.85rem 1rem; margin:1rem 0 2rem; border-left:5px solid #d99b00; background:#fff7d6; color:#332600;">
✅ '''Turbocharged Performance'''
'''Development status: Beta.''' MemCP is under active development. Check [[Supported SQL]], [[Current Status and Open Issues]] and the durability requirements of your workload before migrating production data.
</div>


Designed for multicore CPUs, modern caches, and NVMe SSDs – MemCP handles analytical (OLAP) and transactional (OLTP) workloads at lightning speed.
== Choose your path ==


✅ '''80% Smaller Data Footprint'''
<div style="display:grid; grid-template-columns:repeat(auto-fit,minmax(220px,1fr)); gap:1rem; margin:1rem 0 2rem;">
<div style="padding:1rem 1.1rem; border:1px solid #b7c99a; border-radius:9px;">
<div style="font-size:1.15rem; font-weight:700; margin-bottom:.4rem;">Evaluate MemCP</div>
Understand the workload model, current status, hardware needs and differences from MySQL.


Columnar storage means smarter compression. Typical use cases shrink data 5× smaller than MySQL – with no effort from your side.
[[What is OLTP and OLAP|OLTP and OLAP]] · [[Comparison: MemCP vs. MySQL|Compare with MySQL]] · [[Hardware Requirements|Hardware requirements]]
</div>
<div style="padding:1rem 1.1rem; border:1px solid #b7c99a; border-radius:9px;">
<div style="font-size:1.15rem; font-weight:700; margin-bottom:.4rem;">Build an application</div>
Use the MySQL protocol, SQL over HTTP, RDF or application-specific endpoints inside MemCP.


✅ '''Simple JSON APIs Built-in'''
[[Advanced SQL Tutorial|SQL tutorial]] · [[SQL over REST|SQL over HTTP]] · [[Database Tools compatibility with MemCP|Client tooling]]
</div>
<div style="padding:1rem 1.1rem; border:1px solid #b7c99a; border-radius:9px;">
<div style="font-size:1.15rem; font-weight:700; margin-bottom:.4rem;">Operate MemCP</div>
Plan deployment, migration, persistence, storage backends, settings and performance measurement.


Query your data with RESTful endpoints – no middleware, no ORM overhead. Just fast results, directly from the DB.
[[Deployment]] · [[Persistency and Performance Guarantees|Durability]] · [[Migration from MySQL and PostgreSQL|Migration]]
</div>
<div style="padding:1rem 1.1rem; border:1px solid #b7c99a; border-radius:9px;">
<div style="font-size:1.15rem; font-weight:700; margin-bottom:.4rem;">Understand and contribute</div>
Explore the storage engine, optimizer, embedded Scheme runtime and project internals.


✅ '''Familiar SQL-like Structure – Zero Lock-In'''
[[Query Planner and Physical Lowering|Query planner]] · [[Columnar Storage]] · [[Contributing]]
</div>
</div>


You don’t need to re-learn everything. Tables, schemas, and logic follow familiar patterns – just modernized and optimized.
== Quickstart ==


✅ '''Zero-Config Installation'''
=== Run with Docker ===


Spin up MemCP with a single <code>docker run</code> or <code>pm2 start</code>. Start building in minutes.
<syntaxhighlight lang="bash">
----
docker run --name memcp \
  -e ROOT_PASSWORD='choose-a-password' \
  -p 4321:4321 -p 3307:3307 \
  carli2/memcp:latest
</syntaxhighlight>


=== 🧑‍💻 Built for Developers, by Developers ===
Then open <code>http://localhost:4321</code> and connect a MySQL application to <code>127.0.0.1</code> on port '''3307'''. Add a volume mounted at <code>/data</code> when the container must retain persistent data. See [[Install MemCP with Docker|the complete Docker guide]] before deploying it as a service.
MemCP was made for the people who write real-world code, not just benchmark reports. Whether you're building a SaaS product, IoT platform, or data-heavy analytics dashboard, MemCP gives you the speed and flexibility you need to scale.


# Ready to try it?
=== Build from source ===
git clone <nowiki>https://github.com/launix-de/memcp</nowiki>
cd memcp
go get
make
pm2 start ./memcp ./data/
mysql -u root -p -P 3307
Enter password: admin


=== 🆚 MemCP vs MySQL: The Key Differences ===
<syntaxhighlight lang="bash">
{| class="wikitable"
git clone https://github.com/launix-de/memcp
!Feature
cd memcp
!MySQL
go mod download
!MemCP
make
./memcp --api-port=4321 --mysql-port=3307 lib/main.scm
</syntaxhighlight>
 
Connect with MySQL tooling:
 
<syntaxhighlight lang="bash">
mysql -h 127.0.0.1 -u root -p -P 3307
# Enter the password selected for this data directory.
</syntaxhighlight>
 
The development default for a fresh data directory is <code>admin</code>. Change it before exposing MemCP to another machine. MemCP can also be supervised with PM2:
 
<syntaxhighlight lang="bash">
pm2 start ./memcp --name memcp -- \
  --no-repl -data ./data --api-port=4321 --mysql-port=3307 lib/main.scm
</syntaxhighlight>
 
Background deployments must use <code>--no-repl</code> so that closing standard input does not terminate the interactive console and stop the server. See [[Compile MemCP from Source|Build from Source]], [[Deployment]] and [[MemCP Console]].
 
== Key features ==
 
<div style="display:grid; grid-template-columns:repeat(auto-fit,minmax(260px,1fr)); gap:.8rem; margin:1rem 0;">
<div style="padding:.9rem 1rem; border-top:4px solid #76b512; background:#f5f8f0; color:#17202a;">'''High performance'''<br />NUMA-aware, parallelized query execution is optimized for multicore CPUs, large caches and NVMe SSDs, serving both OLTP and OLAP workloads.</div>
<div style="padding:.9rem 1rem; border-top:4px solid #76b512; background:#f5f8f0; color:#17202a;">'''Columnar storage'''<br />Data is stored by column for improved compression, a smaller memory footprint and fast analytical access.</div>
<div style="padding:.9rem 1rem; border-top:4px solid #76b512; background:#f5f8f0; color:#17202a;">'''Persistent in-memory operation'''<br />MemCP is designed to keep active data in memory while offering configurable per-table durability and persistence backends.</div>
<div style="padding:.9rem 1rem; border-top:4px solid #76b512; background:#f5f8f0; color:#17202a;">'''Built-in APIs'''<br />SQL over HTTP and in-database services can remove an extra middleware hop for suitable applications.</div>
<div style="padding:.9rem 1rem; border-top:4px solid #76b512; background:#f5f8f0; color:#17202a;">'''Adaptive compression'''<br />Bit-packing, dictionary encoding and sequence compression can reduce suitable datasets by up to 80% compared with their MySQL/MariaDB representation.</div>
<div style="padding:.9rem 1rem; border-top:4px solid #76b512; background:#f5f8f0; color:#17202a;">'''Simple deployment'''<br />Start with Docker, PM2 or the native binary. The compact application has historically had an installation footprint of approximately 10 MB.</div>
<div style="padding:.9rem 1rem; border-top:4px solid #76b512; background:#f5f8f0; color:#17202a;">'''Extensible frontends'''<br />The Go storage engine and embedded Scheme environment support SQL, RDF, REST and custom application interfaces.</div>
<div style="padding:.9rem 1rem; border-top:4px solid #9ad32d; background:#f5f8f0; color:#17202a;">'''Cost-based query execution'''<br />Logical decorrelation and join optimization are separated from cost-based physical selection of scans, indexes, RecSets, reusable caches and execution pipelines.<br />[[Query Planner and Physical Lowering|How MemCP plans and lowers queries →]]</div>
</div>
 
== Why MemCP? ==
 
Traditional relational databases were designed around spinning disks and comparatively small numbers of CPU cores. MemCP rethinks storage and query execution for modern multicore systems, large caches, fast storage and mixed workloads.
 
Typical use cases include:
 
* real-time dashboards and analytics;
* data-heavy SaaS platforms;
* embedded systems with limited resources;
* high-throughput OLTP/OLAP hybrids.
 
Whether MemCP is a good fit depends on the required SQL compatibility, durability, query mix, data size and operational environment. Review [[Supported SQL]], [[Persistency and Performance Guarantees]] and [[Performance Measurement]] rather than treating benchmark figures as universal guarantees.
 
== MemCP vs. MySQL ==
 
{| class="wikitable" style="width:100%;"
! Feature
! MySQL
! MemCP
|-
|-
|Storage Model
| Storage model
|Row-based
| Primarily row-based
|Column-based (compressed)
| Column-based and compressed
|-
|-
|Performance
| Performance focus
|Good
| General-purpose relational workloads
|Excellent (NUMA-optimized)
| NUMA-aware, in-memory execution for mixed operational and analytical workloads
|-
|-
|In-Memory Capable
| In-memory capability
|Limited
| Available through selected engines and caching
|Yes, by default
| Central design goal and default operating model
|-
|-
|REST API Integration
| REST API integration
|External services
| Normally external
|Built-in
| Built in
|-
|-
|Installation Footprint
| Installation footprint
|~150MB+
| Common server installations are approximately 150 MB or larger
|~10MB
| The native application has historically been approximately 10 MB
|-
|-
|Open Source
| Open source
|✅
| ✅
|✅
| ✅
|}
|}


=== 💡 Ideal Use Cases ===
MemCP provides MySQL protocol compatibility but does not claim to implement every MySQL feature. See [[Comparison: MemCP vs. MySQL]] for a detailed comparison and [[Database Tools compatibility with MemCP]] for client compatibility.
 
* High-speed dashboards & real-time analytics
* Embedded systems with minimal resources
* Data-heavy SaaS backends
* Replacing outdated MySQL deployments with modern performance
 
 
=== 🧪 Try MemCP Today ===
Explore the documentation, check out the GitHub repo, and join a growing community of developers building fast, efficient, modern systems with MemCP.<blockquote>Ditch the legacy. Embrace performance.
 
'''Make the switch to MemCP.'''</blockquote>
----
 
=== What is memcp? ===
[[File:Webapps.svg|left|frameless]]
memcp is an open-source, high-performance, columnar in-memory database that can handle both OLAP and OLTP workloads. It provides an alternative to proprietary analytical databases and aims to bring the benefits of columnar storage to the open-source world.
 
memcp is written in Golang and is designed to be portable and extensible, allowing developers to embed the database into their applications with ease. It is also designed with a focus on scalability and performance, making it a suitable choice for distributed applications.


== Architecture overview ==


===Features===
* '''Tables, schemas and columns''': Familiar SQL structures use a compressed columnar physical layout. See [[Databases, Tables and Columns]] and [[Columnar Storage]].
* '''Transaction model''': Delta and main storage support mixed OLTP and OLAP semantics. See [[Transactions and Isolation]] and [[Shards, RecordIDs, Main Storage, Delta Storage]].
* '''Query planning''': Logical optimization is separated from physical execution decisions. See [[Query Planner and Physical Lowering]].
* '''Persistence''': Per-table durability can use filesystem, S3 or Ceph/RADOS storage. See [[Persistency and Performance Guarantees]].
* '''Frontends''': MemCP provides multiple query and application interfaces:
** SQL through the MySQL wire protocol and SQL over HTTP;
** an RDF/graph query frontend;
** custom APIs through in-database web applications.


*'''fast:''' MemCP is built with parallelization in mind. The parallelization pattern is made for minimal overhead.
== Documentation ==
*'''efficient:''' The average compression ratio is 1:5 (80% memory saving) compared to MySQL/MariaDB
*'''modern:''' MemCP is built for modern hardware with caches, NUMA memory, multicore CPUs, NVMe SSDs
* '''versatile:''' Use it in big mainframes to gain analytical performance, use it in embedded systems to conserve flash lifetime
* Columnar storage: Stores data column-wise instead of row-wise, which allows for better compression, faster query execution, and more efficient use of memory.
* In-memory database: Stores all data in memory, which allows for extremely fast query execution.
*Build fast REST APIs directly in the database (they are faster because there is no network connection / SQL layer in between)
*OLAP and OLTP support: Can handle both online analytical processing (OLAP) and online transaction processing (OLTP) workloads.
*Compression: Lots of compression formats are supported like bit-packing and dictionary encoding
* Scalability: Designed to scale on a single node with huge NUMA memory
*Adjustable persistency: Decide whether you want to persist a table or not or to just keep snapshots of a period of time
[[File:MemCP Port.png|frameless]]


<youtube>g29FR4Jwius</youtube>
The following navigation remains on the start page so users and search engines can reach every major documentation area directly.


https://www.youtube.com/watch?v=g29FR4Jwius
=== Introduction and evaluation ===


===Navigation===
* [[What is OLTP and OLAP]]
* [[History of the MemCP project]]
* [[Hardware Requirements]]
* [[Persistency and Performance Guarantees]]
* [[Current Status and Open Issues]]
* [[Comparison: MemCP vs. MySQL]]


====Introduction====
=== Getting started ===
*[[What is OLTP and OLAP]]
*[[History of the MemCP project]]
*[[Hardware Requirements]]
*[[Persistency and Performance Guarantees]]
*[[Current Status and Open Issues]]
*[[Comparison: MemCP vs. MySQL]]


====Getting Started====
* [[Install MemCP with Docker|Install with Docker]]
*[[Install MemCP with Docker|With Docker]]
* [[With Singularity|Install with Singularity/Apptainer]]
*[[With Singularity]]
* [[Compile MemCP from Source|Build from Source]]
*[[Compile MemCP from Source|Build from Source]]
* [[Contributing]]
*[[Contributing]]  
* [[Introduction to Scheme]]
*[[Introduction to Scheme]]
* [[Full SCM API documentation]]
*[[Full SCM API documentation]]


====Administration====
=== Administration ===


* [[Deployment]]
* [[Deployment]]
* [[Migration from MySQL and PostgreSQL]]
* [[Migration from MySQL and PostgreSQL]]
* [[Settings]]
* [[Settings]]
*[[Process Hibernation]]
* [[Process Hibernation]]
*[[Performance Measurement]]
* [[Performance Measurement]]
*[[MemCP Console]]
* [[MemCP Console]]
 
=== Frontends ===
 
==== SQL frontend ====


====Frontends====
* [[Supported SQL]]
* [[Advanced SQL Tutorial]]
* [[SQL over REST]]
* [[Database Tools compatibility with MemCP|Supported Tooling]]
* [[How SQL Operators are implemented on MemCP]]
* [[Add custom SQL operators to MemCP]]


=====SQL Frontend=====
==== RDF frontend ====
*[[Supported SQL]]
*[[Advanced SQL Tutorial]]
*[[SQL over REST]]
*[[Database Tools compatibility with MemCP|Supported Tooling]]
*[[How SQL Operators are implemented on MemCP]]
*[[Add custom SQL operators to MemCP]]


=====RDF Frontend=====
* [[Introduction to RDF]]
*[[Introduction to RDF]]
* [[Advanced Graph Querying]]
*[[Advanced Graph Querying]]
* [[RDF templating and model driven development]]
*[[RDF templating and model driven development]]


=====Custom Frontends=====
==== Custom frontends ====


*[[In-Database WebApps|In-Database WebApps and REST Services]]
* [[In-Database WebApps|In-Database WebApps and REST Services]]
*[[MemCP for Microservices]]
* [[MemCP for Microservices]]
*[[Websockets in MemCP]]
* [[Websockets in MemCP]]


==== Persistency Backends (= Storage) ====
=== Persistence backends and storage ===


* [[File System]]
* [[File System]]
* [[S3 Buckets]]
* [[S3 Buckets]]
* [[Ceph/Rados]]
* [[Ceph/Rados]]
* [[Cluster Monitor]]


====Internals====
=== Internals ===


=====How things work in MemCP=====  
==== How MemCP works ====


*[[Databases, Tables and Columns]]
* [[Databases, Tables and Columns]]
*[[Shards, RecordIDs, Main Storage, Delta Storage]]
* [[Shards, RecordIDs, Main Storage, Delta Storage]]
*[[Columnar Storage]]
* [[Columnar Storage]]
*[[Transactions]]  
* [[Transactions and Isolation]]
*[[Full SCM API documentation]]  
* [[Query Planner and Physical Lowering]]
* [[Full SCM API documentation]]


===== SCM Documentation =====
==== Scheme documentation ====


* [[SCM Builtins]]
* [[SCM Builtins]]
Line 192: Line 241:
* [[Storage]]
* [[Storage]]


=====Optimizations=====
==== Optimizations ====
*[[In-Memory Compression, Columnar Compression Techniques]]
 
*[[Temporary Columns]]
* [[Query Planner and Physical Lowering]]
*[[Data Auto Sharding and Auto Indexing]]
* [[In-Memory Compression, Columnar Compression Techniques]]
* [[Temporary Columns]]
* [[Data Auto Sharding and Auto Indexing]]
* [[Parallel Computing]]
* [[Parallel Computing]]


== Further reading ==


[[File:Screenshot from htop.png|center|frameless|2490x2490px]]
* [https://github.com/launix-de/memcp MemCP on GitHub]
 
* [https://www.vldb.org/pvldb/vol13/p2649-boncz.pdf VLDB Research Paper]
 
* [https://cs.emis.de/LNI/Proceedings/Proceedings241/383.pdf LNI Proceedings Paper]
===Further Reading===
* [https://www.dcs.bbk.ac.uk/~dell/teaching/cc/paper/sigmod10/p135-malewicz.pdf Large Graph Algorithms]
[https://github.org/launix-de/memcp MemCP on Github]


====Scientific====
Additional blog posts on design decisions, compression techniques and performance optimization are available on the [https://launix.de/launix/ Launix blog].


*[https://www.vldb.org/pvldb/vol13/p2649-boncz.pdf VLDB Research Paper]
== Community ==
*[https://cs.emis.de/LNI/Proceedings/Proceedings241/383.pdf LNI Proceedings Paper]
*[https://wwwdb.inf.tu-dresden.de/wp-content/uploads/T_2014_Master_Patrick_Damme.pdf TU Dresden Research Paper]
*[https://www.dcs.bbk.ac.uk/~dell/teaching/cc/paper/sigmod10/p135-malewicz.pdf Large Graph Algorithms]
*https://wwwdb.inf.tu-dresden.de/research-projects/eris/


====How MemCP was built====
MemCP is an open-source project maintained by developers for developers. Contributions are welcome in the form of bug reports, feature requests and pull requests.


*[https://launix.de/launix/how-to-balance-a-database-between-olap-and-oltp-workflows/ Balancing OLAP and OLTP Workflows]
See [[Contributing]] and the [https://github.com/launix-de/memcp GitHub repository].
*[https://launix.de/launix/designing-a-programming-language-for-distributed-systems-and-highly-parallel-algorithms/ Designing Programming Languages for Distributed Systems]
*[https://launix.de/launix/on-designing-an-interface-for-columnar-in-memory-storage-in-golang/ Columnar Storage Interface in Golang]
*[https://launix.de/launix/how-in-memory-compression-affects-performance/ Impact of In-Memory Compression on Performance]
*[https://launix.de/launix/memory-efficient-indices-for-in-memory-storages/ Memory-Efficient Indices for In-Memory Storages]
*[https://launix.de/launix/on-compressing-null-values-in-bit-compressed-integer-storages/ Compressing Null Values in Bit-Compressed Integer Storages]
*[https://launix.de/launix/when-the-benchmark-is-too-slow-golang-http-server-performance/ Improving Golang HTTP Server Performance]
*[https://launix.de/launix/how-to-benchmark-a-sql-database/ Benchmarking SQL Databases]
*[https://launix.de/launix/writing-a-sql-parser-in-scheme/ Writing a SQL Parser in Scheme]
*[https://launix.de/launix/accessing-memcp-via-scheme/ Accessing memcp via Scheme]
*[https://launix.de/launix/memcp-first-sql-query-is-correctly-executed/ First SQL Query in memcp]
*[https://launix.de/launix/sequence-compression-in-in-memory-database-yields-99-memory-savings-and-a-total-of-13/ Sequence Compression in In-Memory Database]
*[https://launix.de/launix/storing-a-bit-smaller-than-in-one-bit/ Storing Data Smaller Than One Bit]
*[https://www.youtube.com/watch?v=DWg4nx4KVLo memcp Announcement Video]

Latest revision as of 16:23, 21 August 2026

MemCP – A Modern In-Memory Columnar Database

MemCP
A fast, compressed, MySQL-compatible columnar database for modern OLTP and OLAP workloads
MemCP combines persistent in-memory operation, adaptive column compression, parallel query execution and direct application APIs in a lightweight database written in Go.
Beta Open Source MySQL Protocol + HTTP APIs
Get started with Docker Explore SQL support View on GitHub
MemCP database dashboard and workload overview

Development status: Beta. MemCP is under active development. Check Supported SQL, Current Status and Open Issues and the durability requirements of your workload before migrating production data.

Choose your path

Evaluate MemCP

Understand the workload model, current status, hardware needs and differences from MySQL.

OLTP and OLAP · Compare with MySQL · Hardware requirements

Build an application

Use the MySQL protocol, SQL over HTTP, RDF or application-specific endpoints inside MemCP.

SQL tutorial · SQL over HTTP · Client tooling

Operate MemCP

Plan deployment, migration, persistence, storage backends, settings and performance measurement.

Deployment · Durability · Migration

Understand and contribute

Explore the storage engine, optimizer, embedded Scheme runtime and project internals.

Query planner · Columnar Storage · Contributing

Quickstart

Run with Docker

<syntaxhighlight lang="bash"> docker run --name memcp \ 

 -e ROOT_PASSWORD='choose-a-password' \
 -p 4321:4321 -p 3307:3307 \
 carli2/memcp:latest

</syntaxhighlight>

Then open http://localhost:4321 and connect a MySQL application to 127.0.0.1 on port 3307. Add a volume mounted at /data when the container must retain persistent data. See the complete Docker guide before deploying it as a service.

Build from source

<syntaxhighlight lang="bash"> git clone https://github.com/launix-de/memcp cd memcp go mod download make ./memcp --api-port=4321 --mysql-port=3307 lib/main.scm </syntaxhighlight>

Connect with MySQL tooling:

<syntaxhighlight lang="bash"> mysql -h 127.0.0.1 -u root -p -P 3307

  1. Enter the password selected for this data directory.

</syntaxhighlight>

The development default for a fresh data directory is admin. Change it before exposing MemCP to another machine. MemCP can also be supervised with PM2:

<syntaxhighlight lang="bash"> pm2 start ./memcp --name memcp -- \ 

 --no-repl -data ./data --api-port=4321 --mysql-port=3307 lib/main.scm

</syntaxhighlight>

Background deployments must use --no-repl so that closing standard input does not terminate the interactive console and stop the server. See Build from Source, Deployment and MemCP Console.

Key features

High performance
NUMA-aware, parallelized query execution is optimized for multicore CPUs, large caches and NVMe SSDs, serving both OLTP and OLAP workloads.
Columnar storage
Data is stored by column for improved compression, a smaller memory footprint and fast analytical access.
Persistent in-memory operation
MemCP is designed to keep active data in memory while offering configurable per-table durability and persistence backends.
Built-in APIs
SQL over HTTP and in-database services can remove an extra middleware hop for suitable applications.
Adaptive compression
Bit-packing, dictionary encoding and sequence compression can reduce suitable datasets by up to 80% compared with their MySQL/MariaDB representation.
Simple deployment
Start with Docker, PM2 or the native binary. The compact application has historically had an installation footprint of approximately 10 MB.
Extensible frontends
The Go storage engine and embedded Scheme environment support SQL, RDF, REST and custom application interfaces.
Cost-based query execution
Logical decorrelation and join optimization are separated from cost-based physical selection of scans, indexes, RecSets, reusable caches and execution pipelines.
How MemCP plans and lowers queries →

Why MemCP?

Traditional relational databases were designed around spinning disks and comparatively small numbers of CPU cores. MemCP rethinks storage and query execution for modern multicore systems, large caches, fast storage and mixed workloads.

Typical use cases include:

  • real-time dashboards and analytics;
  • data-heavy SaaS platforms;
  • embedded systems with limited resources;
  • high-throughput OLTP/OLAP hybrids.

Whether MemCP is a good fit depends on the required SQL compatibility, durability, query mix, data size and operational environment. Review Supported SQL, Persistency and Performance Guarantees and Performance Measurement rather than treating benchmark figures as universal guarantees.

MemCP vs. MySQL

Feature MySQL MemCP
Storage model Primarily row-based Column-based and compressed
Performance focus General-purpose relational workloads NUMA-aware, in-memory execution for mixed operational and analytical workloads
In-memory capability Available through selected engines and caching Central design goal and default operating model
REST API integration Normally external Built in
Installation footprint Common server installations are approximately 150 MB or larger The native application has historically been approximately 10 MB
Open source ✅ ✅

MemCP provides MySQL protocol compatibility but does not claim to implement every MySQL feature. See Comparison: MemCP vs. MySQL for a detailed comparison and Database Tools compatibility with MemCP for client compatibility.

Architecture overview

Documentation

The following navigation remains on the start page so users and search engines can reach every major documentation area directly.

Introduction and evaluation

Getting started

Administration

Frontends

SQL frontend

RDF frontend

Custom frontends

Persistence backends and storage

Internals

How MemCP works

Scheme documentation

Optimizations

Further reading

Additional blog posts on design decisions, compression techniques and performance optimization are available on the Launix blog.

Community

MemCP is an open-source project maintained by developers for developers. Contributions are welcome in the form of bug reports, feature requests and pull requests.

See Contributing and the GitHub repository.

Retrieved from "https://www.memcp.org/index.php?title=Main_Page&oldid=271"