All public logs
Jump to navigation
Jump to search
Combined display of all available logs of MemCP. You can narrow down the view by selecting a log type, the username (case-sensitive), or the affected page (also case-sensitive).
- 16:15, 17 May 2024 Carli talk contribs created page Compile MemCP from Source (Created page with "Make sure, <code>go</code> is installed on your computer. (at least 1.22.0) Compile the project with: git clone <nowiki>https://github.com/launix-de/memcp</nowiki> cd memcp make Run the engine with: ./memcp") Tag: Visual edit
- 16:13, 17 May 2024 Carli talk contribs created page With Singularity (Created page with "Singularity is a rootless container format for restricted environments like HPC. Here are the installation instructions: https://docs.sylabs.io/guides/latest/user-guide/quick_start.html#quick-installation-steps Build with: git clone <nowiki>https://github.com/launix-de/memcp</nowiki> cd memcp sudo singularity build memcp.sif memcp.singularity.recipe Now you can run memcp.sif: mkdir data singularity run --bind data:/data memcp.sif") Tag: Visual edit
- 16:10, 17 May 2024 Carli talk contribs created page Install MemCP with Docker (Created page with "To build the image the first time: git clone https://github.com/launix-de/memcp cd memcp docker build . -t memcp For app development and to enter the Scheme console: mkdir data docker run -v data:/data -it -p 4321:4321 -p 3307:3307 memcp To run your custom Scheme apps (like the one from RDF templating and model driven development): make data docker run -e PARAMS="lib/main.scm apps/minigame.scm" -v data:/data -it -p 4321:4321 -p 3307:3307 memcp During productio...") Tag: Visual edit
- 16:06, 17 May 2024 Carli talk contribs created page Persistency and Performance Guarantees (Created page with "MemCP gives the user several Guarantees for persistency and performance. These are the guarantees: == Persistency Guarantees == There are three persistency modes per table which are: * ENGINE = memory * ENGINE = sloppy * ENGINE = logged * ENGINE = safe === ENGINE = memory === * all data is held in memory and only in memory * in case of a crash, all data is gone * the schema is saved on disk * after a recovery, the table starts empty * fastest way to store data * use...") Tag: Visual edit
- 15:27, 17 May 2024 Carli talk contribs created page Lists and Objects (Created page with "Lists are a powerful feature in Scheme. Lists can express both: Data and Programs. ==List handling== The following list functions can be used: > '(1 2 3) "list literal" = (1 2 3) > (cons 0 '(1 2 3)) "prepend item to list" = (0 1 2 3) > (append '(1 2 3) 4 5) "append items to list" = (1 2 3 4 5) > (has? '(1 2 3) 2) "check if item is in list" = true > (filter '(1 2 3) (lambda (x) (< x 2))) "filter items that are smaller than 2" = (1) > (map '(1 2 3) (l...") Tag: Visual edit
- 14:36, 17 May 2024 Carli talk contribs created page Introduction to Scheme (Created page with "When you run <code>./memcp</code>, you will be dropped at a scheme shell like this: memcp Copyright (C) 2023, 2024 Carl-Philip Hänsch This program comes with ABSOLUTELY NO WARRANTY; This is free software, and you are welcome to redistribute it under certain conditions; loading storage /tmp/x/system/bdf64a22-6315-463c-bbf7-329317ad50ed-id of type 10 loading storage /tmp/x/system/bdf64a22-6315-463c-bbf7-329317ad50ed-username of type 20 loading stora...") Tag: Visual edit
- 14:04, 17 May 2024 Carli talk contribs created page Sequence Compression (Created page with "One of the most interesting compression techniques on columnar storages is '''sequence compression'''. Sequence Compression in In-Memory Database yields 99% memory savings and a total of 13% A sequence is a column of numbers where each distance between two neighbouring numbers is equal. Example: * <code>1 2 3 4 5 6 7 8 9</code> * <code>3 3 3 3 3 3 3 3 3 3</code> * <code>10 20 30 40</code> * <code>8 7 6 5</code> A sequence can be described by its starting point, its st...") Tag: Visual edit
- 14:01, 17 May 2024 Carli talk contribs created page In-Memory Compression, Columnar Compression Techniques (Created page with "The following compression techniques are implemented: * Sparse Storage for columns with lots of NULL values * Float Storage for scientific data * Integer Compression with a bit-size approach * Sequence Compression for sequences of integers like IDs (1, 2, 3, 4, 5 ...) * Dictionary Compression for short strings * BLOB Zipping and Deduplication for strings longer than 1KiB") Tag: Visual edit
- 13:57, 17 May 2024 Carli talk contribs created page Columnar Storage (Created page with "The advantages for columnar storages over row based storages are the ability for good in-memory compression '''(low memory usage)''' and cache locality when accessing only few columns '''(performance)'''. == How we designed the Interface == When designing an interface for a storage engine for an in-memory database, a lot of considerations have to be made. Here are the design goals: * The interface must be simple so that using it does not require implementing all kinds...") Tag: Visual edit
- 13:52, 17 May 2024 Carli talk contribs created page Hardware Requirements (Created page with "MemCP can run on any hardware that is supported by the <code>go</code> compiler. To find the right hardware dimensions, you need to consider the following: * The process itself needs at least 8 MiB of RAM * The size of a 5 GiB MySQL database fits into ~1 GiB of RAM * To have a decent working set, take the RAM requirements x3, so a 5 GiB MySQL database should run on 3 GiB RAM * MemCP needs at least 1 CPU core, more is better ;)") Tag: Visual edit
- 12:51, 17 May 2024 Carli talk contribs created page Parallel Computing (Created page with "Almost 99% of all newly invented are imperative programming languages. But imperative languages have one drawback: their parallelization is hard. == Drawbacks of Imperative Programming Languages == Imperative programming languages do have one mayor drawback: state. The concept of an imperative language is that commands are executed which change the content of variables or complex objects in the memory. When trying to create an optimizing compiler that from itself finds...") Tag: Visual edit
- 08:24, 17 May 2024 Carli talk contribs created page History of the MemCP project (Created page with "* On the beforehand, a lot of research was done at the HAEC project * MemCPs predecessor is the [https://wwwdb.inf.tu-dresden.de/research-projects/eris/ Eris] project at TU Dresden * The first commit on MemCP was done on 24th of December 2022 at 12:13:37 * MemCP was originally written in go * The first SQL query on MemCP was successfuly executed on 12.02.2023 source: https://launix.de/launix/memcp-first-sql-query-is-correctly-executed/") Tag: Visual edit
- 08:20, 17 May 2024 Carli talk contribs created page Databases, Tables and Columns (Created page with "* MemCP has a data folder which defaults to "./data". Every subdirectory of that folder is a database. * Databases consist of tables * Tables consist of columns and shards") Tag: Visual edit
- 08:18, 17 May 2024 Carli talk contribs created page Shards, RecordIDs, Main Storage, Delta Storage (Created page with "* Datasets of a table are organized in shards of roughly 60,000 items * Inside a shard with n items, the datasets are numbered from 0..n-1 * The first few datasets of each shard are organized in a column store * The rest of the datasets are organized in a row store * The delta store consists of the insert list as well as the deletion mask") Tag: Visual edit
- 08:14, 17 May 2024 Carli talk contribs created page What is OLTP and OLAP (Created page with "There are basically two types database workloads leading to different database designs: {| class="wikitable" |+ ! !OLTP !OLAP |- |abbr. |Online Transaction Processing |Online ''Analytical'' Processing |- |Example applications |ERP, Ticket System |BI suite, Statistical programs |- |Typical SQL Queries |large amounts of small UPDATE, INSERT |few SELECT with big computation demand |- |Best Storage Format |Row Store |Columnar store |- |Database Engines |MySQL, Postgres |Mone...") Tag: Visual edit
- 14:04, 16 May 2024 Bnet talk contribs uploaded File:JBL 2022 fhd 97974.jpg
- 14:04, 16 May 2024 Bnet talk contribs created page File:JBL 2022 fhd 97974.jpg
- 13:46, 16 May 2024 Bnet talk contribs deleted page File:Nextcloud 02.png (Test-Upload)
- 13:46, 16 May 2024 Bnet talk contribs deleted page File:JBL 2022 fhd 97974.jpg (Test-Upload)
- 13:45, 16 May 2024 Bnet talk contribs deleted page File:JBL 2022 fhd 97973.jpg (Test-Upload)
- 13:45, 16 May 2024 Bnet talk contribs deleted page File:JBL 2022 fhd 97972.jpg (Test-Upload)
- 16:23, 15 May 2024 Bnet talk contribs uploaded File:JBL 2022 fhd 97972.jpg
- 16:23, 15 May 2024 Bnet talk contribs created page File:JBL 2022 fhd 97972.jpg
- 16:20, 15 May 2024 Bnet talk contribs uploaded File:JBL 2022 fhd 97973.jpg
- 16:20, 15 May 2024 Bnet talk contribs created page File:JBL 2022 fhd 97973.jpg
- 15:50, 15 May 2024 Bnet talk contribs uploaded File:JBL 2022 fhd 97974.jpg
- 15:50, 15 May 2024 Bnet talk contribs created page File:JBL 2022 fhd 97974.jpg
- 15:19, 15 May 2024 Bnet talk contribs uploaded File:Nextcloud 02.png
- 15:19, 15 May 2024 Bnet talk contribs created page File:Nextcloud 02.png
- 13:26, 15 May 2024 Bnet talk contribs uploaded File:Backofen-defekt-7.jpg
- 13:26, 15 May 2024 Bnet talk contribs created page File:Backofen-defekt-7.jpg
- 13:26, 15 May 2024 Bnet talk contribs deleted page File:Backofen-defekt-7.jpg (Test-Upload)
- 13:26, 15 May 2024 Bnet talk contribs deleted page File:Nextcloud 01.png (Test-Upload)
- 13:26, 15 May 2024 Bnet talk contribs uploaded File:B-net.png
- 13:26, 15 May 2024 Bnet talk contribs created page File:B-net.png
- 13:20, 15 May 2024 Bnet talk contribs created page File:Nextcloud 01.png
- 13:20, 15 May 2024 Bnet talk contribs uploaded File:Nextcloud 01.png
- 11:49, 15 May 2024 Carli talk contribs created page File:Screenshot from htop.png
- 11:49, 15 May 2024 Carli talk contribs uploaded File:Screenshot from htop.png
- 11:49, 15 May 2024 Carli talk contribs deleted page File:Screenshot from htop.png
- 04:36, 15 May 2024 Carli talk contribs created page File:Screenshot from htop.png
- 04:36, 15 May 2024 Carli talk contribs uploaded File:Screenshot from htop.png
- 04:34, 15 May 2024 Carli talk contribs deleted page File:Screenshot from htop.png
- 22:40, 14 May 2024 Carli talk contribs uploaded File:Screenshot from htop.png
- 22:40, 14 May 2024 Carli talk contribs created page File:Screenshot from htop.png
- 22:34, 14 May 2024 Carli talk contribs deleted page File:Screenshot from htop.png
- 22:31, 14 May 2024 Carli talk contribs created page File:Screenshot from htop.png
- 22:31, 14 May 2024 Carli talk contribs uploaded File:Screenshot from htop.png
- 22:30, 14 May 2024 Carli talk contribs deleted page File:Screenshot from htop.png
- 21:37, 14 May 2024 Carli talk contribs uploaded File:Screenshot from htop.png
