Persistency and Performance Guarantees - Revision history

Carli: /* Persistency Guarantees */

2026-03-09T12:20:19Z

Persistency Guarantees

← Older revision		Revision as of 14:20, 9 March 2026
Line 1:		Line 1:
	MemCP gives the user several Guarantees for persistency and performance. These are the guarantees:		MemCP gives the user several Guarantees for persistency and performance. These are the guarantees:

	== Performance Guarantees ==		== Performance Guarantees and Read Speed ==
	MemCP guarantees that:		MemCP guarantees that:

			* All reading operations are from RAM and are fast
	* Insert/Update without any unique key check or foreign key check will scale over shards		* Insert/Update without any unique key check or foreign key check will scale over shards
	* Aggregate functions like SUM, AVG will scale perfectly with the amount of CPUs		* Aggregate functions like SUM, AVG will scale perfectly with the amount of CPUs
Line 10:		Line 11:

	Please consider that memory bandwith still might be a bottleneck. Even modern 128 core CPUs do not scale better than 8 cores for uncompressed data (usually 40 GiB/s bandwith on tested machines). '''MemCPs [[Columnar Storage]] scales beyond that limit.'''		Please consider that memory bandwith still might be a bottleneck. Even modern 128 core CPUs do not scale better than 8 cores for uncompressed data (usually 40 GiB/s bandwith on tested machines). '''MemCPs [[Columnar Storage]] scales beyond that limit.'''
	== Persistency Guarantees ==		== Persistency Guarantees and Write Speed ==
	There are five persistency modes per table which are:		There are five persistency modes per table which are:

Carli: /* Persistency Guarantees */

2026-03-09T12:16:10Z

Persistency Guarantees

← Older revision		Revision as of 14:16, 9 March 2026
Line 19:		Line 19:
	* ENGINE = safe		* ENGINE = safe
	In comparison to other RDBMs, MemCP can do this setting per table. MySQL/MariaDB for instance only allows a global setting which is sometimes not what you want.		In comparison to other RDBMs, MemCP can do this setting per table. MySQL/MariaDB for instance only allows a global setting which is sometimes not what you want.

			In short, these engines guarantee different performance:

			* cache+memory -> RAM speed, no persistency
			* sloppy+logged -> HDD speed, persistency guarantees under circumstances
			* safe -> Sync speed (slow) and persistency guarantees even in case of power outage during queries

	=== ENGINE = cache ===		=== ENGINE = cache ===

Carli: /* Persistency Guarantees */

2026-03-09T12:13:49Z

Persistency Guarantees

← Older revision		Revision as of 14:13, 9 March 2026
Line 11:		Line 11:
	Please consider that memory bandwith still might be a bottleneck. Even modern 128 core CPUs do not scale better than 8 cores for uncompressed data (usually 40 GiB/s bandwith on tested machines). '''MemCPs [[Columnar Storage]] scales beyond that limit.'''		Please consider that memory bandwith still might be a bottleneck. Even modern 128 core CPUs do not scale better than 8 cores for uncompressed data (usually 40 GiB/s bandwith on tested machines). '''MemCPs [[Columnar Storage]] scales beyond that limit.'''
	== Persistency Guarantees ==		== Persistency Guarantees ==
	There are ~~three~~ persistency modes per table which are:		There are five persistency modes per table which are:

			* ENGINE = cache
	* ENGINE = memory		* ENGINE = memory
	* ENGINE = sloppy		* ENGINE = sloppy

Carli at 12:13, 9 March 2026

2026-03-09T12:13:31Z

← Older revision		Revision as of 14:13, 9 March 2026
Line 18:		Line 18:
	* ENGINE = safe		* ENGINE = safe
	In comparison to other RDBMs, MemCP can do this setting per table. MySQL/MariaDB for instance only allows a global setting which is sometimes not what you want.		In comparison to other RDBMs, MemCP can do this setting per table. MySQL/MariaDB for instance only allows a global setting which is sometimes not what you want.

			=== ENGINE = cache ===

			* all data is held in memory and only in memory
			* in case of a crash, all data is gone
			* in case the server is running out of RAM, the data can be deleted, too
			* the schema is saved on disk
			* after a recovery, the table starts empty
			* fastest way to store data
			* use it for session data, observer handles, caches and other data that can be recreated by the software

	=== ENGINE = memory ===		=== ENGINE = memory ===
Line 23:		Line 33:
	* all data is held in memory and only in memory		* all data is held in memory and only in memory
	* in case of a crash, all data is gone		* in case of a crash, all data is gone
			* while the server is running, the data will be kept alive
	* the schema is saved on disk		* the schema is saved on disk
	* after a recovery, the table starts empty		* after a recovery, the table starts empty

Carli at 22:37, 4 October 2025

2025-10-04T22:37:59Z

← Older revision		Revision as of 00:37, 5 October 2025
Line 49:		Line 49:
	** in case of a power outage or kernel crash, data might be lost		** in case of a power outage or kernel crash, data might be lost
	* allows buffering of the log file in return for the risk of data loss		* allows buffering of the log file in return for the risk of data loss
	* use it for data where you need a high update performance but cannot afford losing the last ~~15 minutes~~ of your work		* use it for data where you need a high update performance but cannot afford losing the last few seconds of your work

	=== ENGINE = safe ===		=== ENGINE = safe ===

Carli: /* Performance Guarantees */

2024-10-08T21:23:07Z

Performance Guarantees

← Older revision		Revision as of 23:23, 8 October 2024
Line 7:		Line 7:
	* Aggregate functions like SUM, AVG will scale perfectly with the amount of CPUs		* Aggregate functions like SUM, AVG will scale perfectly with the amount of CPUs

	With these guarantees, you can add more CPU cores whenever you have more data.		With these guarantees, you can add more CPU cores whenever you have more data to ensure the same query time even with growing data.

	Please consider that memory bandwith still might be a bottleneck. Even modern 128 core CPUs do not scale better than 8 cores for uncompressed data (usually 40 GiB/s bandwith on tested machines). MemCPs [[Columnar Storage]] scales beyond that limit.		Please consider that memory bandwith still might be a bottleneck. Even modern 128 core CPUs do not scale better than 8 cores for uncompressed data (usually 40 GiB/s bandwith on tested machines). '''MemCPs [[Columnar Storage]] scales beyond that limit.'''
	== Persistency Guarantees ==		== Persistency Guarantees ==
	There are three persistency modes per table which are:		There are three persistency modes per table which are:

Carli at 21:22, 8 October 2024

2024-10-08T21:22:19Z

@@ Line 1: / Line 1: @@
 MemCP gives the user several Guarantees for persistency and performance. These are the guarantees:
 == Persistency Guarantees ==
 There are three persistency modes per table which are:
@@ Line 52: / Line 61: @@
 * IO bound (limited to ~1,700 write operations per second)
 * use it for accounting data or any data that must not be lost

Carli: /* Performance Guarantees */

2024-10-08T21:21:56Z

Performance Guarantees

← Older revision		Revision as of 23:21, 8 October 2024
Line 60:		Line 60:

	With these guarantees, you can add more CPU cores whenever you have more data.		With these guarantees, you can add more CPU cores whenever you have more data.

			Please consider that memory bandwith still might be a bottleneck. Even modern 128 core CPUs do not scale better than 8 cores for uncompressed data (usually 40 GiB/s bandwith on tested machines). MemCPs [[Columnar Storage]] scales beyond that limit.

Carli: /* Persistency Guarantees */

2024-10-08T21:18:05Z

Persistency Guarantees

← Older revision		Revision as of 23:18, 8 October 2024
Line 8:		Line 8:
	* ENGINE = logged		* ENGINE = logged
	* ENGINE = safe		* ENGINE = safe
			In comparison to other RDBMs, MemCP can do this setting per table. MySQL/MariaDB for instance only allows a global setting which is sometimes not what you want.

	=== ENGINE = memory ===		=== ENGINE = memory ===

Carli: /* ENGINE = safe */

2024-10-08T21:16:46Z

ENGINE = safe

← Older revision		Revision as of 23:16, 8 October 2024
Line 48:		Line 48:
	* an operation only succeeds after the data is synced to disk permanently		* an operation only succeeds after the data is synced to disk permanently
	* in case of a crash, all data will be recovered		* in case of a crash, all data will be recovered
			* introduces delays for each transaction since the system has to wait for a write fence
	* IO bound (limited to ~1,700 write operations per second)		* IO bound (limited to ~1,700 write operations per second)
	* use it for accounting data or any data that must not be lost		* use it for accounting data or any data that must not be lost