Hardware Requirements
Jump to navigation
Jump to search
MemCP can run on any hardware that is supported by the go
compiler.
To find the right hardware dimensions, you need to consider the following:
- The process itself needs at least 8 MiB of RAM
- The size of a 5 GiB MySQL database fits into ~1 GiB of RAM
- To have a decent working set, take the RAM requirements x3, so a 5 GiB MySQL database should run on 3 GiB RAM
- MemCP needs at least 1 CPU core, more is better ;)
Running on Big SMP Machines
MemCP runs best on machiens that have a lot of RAM and CPU cores.
We tested on:
- AMD Ryzen (in various forms)
- HPE Superfome Flex (https://www.hpe.com/us/en/servers/superdome.html)
Micro-Benchmark
This is measured on AMD Ryzen 9 7900X3D 12-Core Processor over a Gigabit connection:
~$ ab -n 1000000 -c 1000 'http://192.168.178.108:4321/sql/test/select%20*%20from%20a' This is ApacheBench, Version 2.3 <$Revision: 1879490 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.178.108 (be patient) Completed 100000 requests Completed 200000 requests Completed 300000 requests Completed 400000 requests Completed 500000 requests Completed 600000 requests Completed 700000 requests Completed 800000 requests Completed 900000 requests Completed 1000000 requests Finished 1000000 requests Server Software: Server Hostname: 192.168.178.108 Server Port: 4321 Document Path: /sql/test/select%20*%20from%20a Document Length: 12 bytes Concurrency Level: 1000 Time taken for tests: 48.842 seconds Complete requests: 1000000 Failed requests: 0 Non-2xx responses: 1000000 Total transferred: 180000000 bytes HTML transferred: 12000000 bytes Requests per second: 20474.09 [#/sec] (mean) Time per request: 48.842 [ms] (mean) Time per request: 0.049 [ms] (mean, across all concurrent requests) Transfer rate: 3598.96 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 11 6.0 11 32 Processing: 0 38 21.3 33 279 Waiting: 0 33 21.9 27 278 Total: 0 49 21.0 46 287 Percentage of the requests served within a certain time (ms) 50% 46 66% 52 75% 57 80% 61 90% 73 95% 87 98% 107 99% 122 100% 287 (longest request)
Running on Raspberry Pi
These are our recommendations to run MemCP on Raspberry Pi:
- Avoid
ENGINE=safe
. To have a long life for your flash drive, you should chooseENGINE=sloppy
. (ENGINE=logging
would be a compromise betweensafe
andsloppy
) - for more information see Persistency and Performance Guarantees - Raspberry Pis have very limited RAM. In most cases, the RAM is sufficient though since most databases only take a few megabytes of RAM. For bigger data, you should wait for MMaped shards, see also: Current Status and open issues
Micro-Benchmark
Here's a micro benchmark for putting HTTP load on memcp on Raspberry Pi: behind a 100 MBit/s ethernet. The request is a rather simple one.
$ ab -n 1000000 -c 1000 'http://192.168.178.73:4321/sql/test/select%20*%20from%20a' This is ApacheBench, Version 2.3 <$Revision: 1879490 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.178.73 (be patient) Completed 100000 requests Completed 200000 requests Completed 300000 requests Completed 400000 requests Completed 500000 requests Completed 600000 requests Completed 700000 requests Completed 800000 requests Completed 900000 requests Completed 1000000 requests Finished 1000000 requests Server Software: Server Hostname: 192.168.178.73 Server Port: 4321 Document Path: /sql/test/select%20*%20from%20a Document Length: 12 bytes Concurrency Level: 1000 Time taken for tests: 1079.876 seconds Complete requests: 1000000 Failed requests: 0 Non-2xx responses: 1000000 Total transferred: 180000000 bytes HTML transferred: 12000000 bytes Requests per second: 926.03 [#/sec] (mean) Time per request: 1079.876 [ms] (mean) Time per request: 1.080 [ms] (mean, across all concurrent requests) Transfer rate: 162.78 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 1 346 404.9 239 7464 Processing: 3 733 574.8 523 8723 Waiting: 3 728 573.5 518 8723 Total: 4 1079 685.0 818 9758 Percentage of the requests served within a certain time (ms) 50% 818 66% 1195 75% 1489 80% 1612 90% 1956 95% 2425 98% 2920 99% 3348 100% 9758 (longest request)
Raspberry Pi manages to do about 1,000 requests per second while modern x86_64 computers are able to serve 12,000 requests and more.