User contributions for Carli

19 May 2024

• 10:4910:49, 19 May 2024 diff hist +137‎ SQL over REST ‎No edit summary Tag: Visual edit
• 10:4710:47, 19 May 2024 diff hist +567‎ N SQL over REST ‎ Created page with "There are two methods to do SQL over REST: a) GET method: <code>curl <nowiki>http://root:admin@localhost:4321/sql/</nowiki>[SCHEMA]/[SQL_QUERY]</code> b) POST method: <code>curl -X POST -d '[SQL_QUERY]' <nowiki>http://root:admin@localhost:4321/sql/</nowiki>[SCHEMA]</code> == JSONL results == The result is a <code>jsonl</code> document containing the result lines: curl -X POST -d 'select a, sum(b) as sum_b from a group by a' <nowiki>http://root:admin@localhost:4321/sq..." Tag: Visual edit
• 10:4210:42, 19 May 2024 diff hist +374‎ N Introduction to RDF ‎ Created page with "RDF is the '''Resource Description Format'''. It describes all data in ''Triple Format'': {| class="wikitable" |+ !Subject !Predicate !Object |- |item:1 |a |foaf:Person |- |item:1 |foaf:name |Klaus |- |item:1 |foaf:mbox |klaus@example.com |- |item:1 |foaf:knows |item:2 |- |item:2 |a |foaf:Person |- |item:2 |foaf:name |Dieter |} See also: https://github.com/launix-de/rdfop" current Tag: Visual edit
• 10:2310:23, 19 May 2024 diff hist +755‎ N Supported SQL ‎ Created page with "The following SQL statements are supported: * SELECT FROM GROUP BY HAVING ORDER BY LIMIT OFFSET * UPDATE SET WHERE * DELETE FROM WHERE * INSERT INTO VALUES * CREATE TABLE * ALTER TABLE * CREATE DATABASE * CREATE USER * ALTER USER * SHOW DATABASES * SHOW TABLES * SHOW TABLE STATUS * SHOW VARIABLES * SET NAMES (no function) * DROP DATABASE * DROP TABLE * SET SESSION * LOCK TABLES (no function) * UNLOCK TABLES (no function) The following functions are supported: * CASE..." Tag: Visual edit
• 10:1610:16, 19 May 2024 diff hist +20‎ Main Page ‎ →‎SQL Frontend Tag: Visual edit
• 10:0610:06, 19 May 2024 diff hist +102‎ Current Status and Open Issues ‎No edit summary Tag: Visual edit
• 10:0410:04, 19 May 2024 diff hist +44‎ N Current Status and open issues ‎ Carli moved page Current Status and open issues to Current Status and Open Issues current Tag: New redirect
• 10:0410:04, 19 May 2024 diff hist 0‎ m Current Status and Open Issues ‎ Carli moved page Current Status and open issues to Current Status and Open Issues
• 10:0410:04, 19 May 2024 diff hist −31‎ Main Page ‎ →‎Introduction Tag: Visual edit
• 10:0210:02, 19 May 2024 diff hist +82‎ Main Page ‎ →‎Navigation Tag: Visual edit

18 May 2024

• 22:4122:41, 18 May 2024 diff hist +1,950‎ Hardware Requirements ‎ →‎Micro-Benchmark current Tag: Visual edit
• 21:5321:53, 18 May 2024 diff hist +31‎ Main Page ‎ →‎Introduction Tag: Visual edit
• 18:4118:41, 18 May 2024 diff hist +4,012‎ Hardware Requirements ‎ →‎Running on Raspberry Pi Tag: Visual edit
• 18:3118:31, 18 May 2024 diff hist +841‎ Hardware Requirements ‎No edit summary Tag: Visual edit
• 13:5813:58, 18 May 2024 diff hist +88‎ Integer Compression ‎ →‎See also current Tag: Visual edit

17 May 2024

• 23:0723:07, 17 May 2024 diff hist +419‎ N Process Hibernation ‎ Created page with "MemCP is a In-Memory database. In case you use the <code>memory</code> engine, there is no data stored on disk. <code>criu</code> is an OpenSource project to hibernate processes on linux on disk and restore them later (see https://criu.org/Main_Page). You can hibernate the memcp process with sudo criu dump -t [PID] -j -D [DUMP_DIRECTORY] And reactivate the process with: sudo criu restore -j -D [DUMP_DIRECTORY]" current Tag: Visual edit
• 23:0323:03, 17 May 2024 diff hist +31‎ Settings ‎No edit summary Tag: Visual edit
• 23:0223:02, 17 May 2024 diff hist +734‎ N Settings ‎ Created page with "Settings are changed by: (settings "key" "value") The following settings are available: * <code>"Backtrace"</code>: <code>true</code> or <code>false</code> whether to print backtraces from Scheme. Using true decreases the performance by ~10% but gives you better debugging capabilities. * <code>"PartitionMaxDimensions"</code>: number from 0 to 10 how many partitioning dimensions are allowed at max. 0 means, don't partition shards for values (use it to workaround bugs),..." Tag: Visual edit
• 22:5822:58, 17 May 2024 diff hist +97‎ Main Page ‎ →‎Navigation Tag: Visual edit
• 22:5122:51, 17 May 2024 diff hist +3,585‎ Integer Compression ‎No edit summary Tag: Visual edit
• 22:4922:49, 17 May 2024 diff hist +40‎ N Temporary Columns ‎ Carli moved page Temporary Columns to Temporary Computed Columns current Tag: New redirect
• 22:4922:49, 17 May 2024 diff hist 0‎ m Temporary Computed Columns ‎ Carli moved page Temporary Columns to Temporary Computed Columns current
• 22:4822:48, 17 May 2024 diff hist +2,999‎ N Temporary Computed Columns ‎ Created page with "Most (or better: all) RDBMS know the concept of so-called „temporary tables“. These tables act as a buffer when complex computations or joins are made and the result has to be sorted afterwards. Temporary tables are also a big loss of performance in daily workloads: Let’s say a table consists of 1,000,000 items and you display them with ORDER BY … LIMIT 100. Because auf some complex join, the 1,000,000 items spanning sorted temptable has to be rendered first and..." Tag: Visual edit
• 22:4622:46, 17 May 2024 diff hist +89‎ Main Page ‎ →‎Internals Tag: Visual edit
• 22:4422:44, 17 May 2024 diff hist −12‎ In-Memory Compression, Columnar Compression Techniques ‎No edit summary current Tag: Visual edit
• 22:4322:43, 17 May 2024 diff hist +1,232‎ N Integer Compression ‎ Created page with "Traditional Database Management Systems use fixed width integers of 32 or 64 bit that are administered by the database operator. MemCP however uses a different approach: according to a data analysis pass, we detect the maximum bit width of an integer to be encoded into our database. This is done by: * finding the smallest integer in a shard * finding the highest integer * <code>offset = smallest</code> * <code>bitwidth = ⌈ld (highest - smallest)⌉</code> * allocate..." Tag: Visual edit
• 22:3222:32, 17 May 2024 diff hist 0‎ In-Memory Compression, Columnar Compression Techniques ‎No edit summary Tag: Visual edit
• 22:3122:31, 17 May 2024 diff hist +332‎ In-Memory Compression, Columnar Compression Techniques ‎No edit summary Tag: Visual edit
• 22:2822:28, 17 May 2024 diff hist +7,746‎ In-Memory Compression, Columnar Compression Techniques ‎No edit summary Tag: Visual edit
• 22:2622:26, 17 May 2024 diff hist +164‎ Databases, Tables and Columns ‎No edit summary current Tag: Visual edit
• 22:2422:24, 17 May 2024 diff hist +2,922‎ N Current Status and Open Issues ‎ Created page with "There are several TODOs in MemCP still. These are categorized as follows: === Storage Engine === * Allow ALTER TABLE ENGINE = ... * Garbage Collection with an LRU policy on temporary columns * Triggers and change hooks on computed columns * Respect Foreign Keys * Serialize and Deserialize into MMapped big files (these bigfiles must be organized as key-value stores) * iterateIndex: sort delta storage and merge it with main, so that correct order is always guaranteed * m..." Tag: Visual edit
• 19:0819:08, 17 May 2024 diff hist +37‎ Main Page ‎ →‎Introduction Tag: Visual edit
• 19:0719:07, 17 May 2024 diff hist +7,345‎ N Index Compression ‎ Created page with "== Memory-Efficient Indices for In-Memory Storages == Most databases implement indices as a kind of tree. I will show you that columnar storages can do even better. The most widely used kind of tree structure in databases is the B-Tree. A B-Tree is a n-ary tree whose nodes fit exactly into one „page“ – may it be a cache line of 64 bytes or a HDD page of 512 bytes or a memory page of 4K. The advantage of btrees is that it balances the memory fetch time with the ti..." current Tag: Visual edit
• 19:0419:04, 17 May 2024 diff hist +74‎ In-Memory Compression, Columnar Compression Techniques ‎No edit summary Tag: Visual edit
• 17:1817:18, 17 May 2024 diff hist +23‎ Main Page ‎No edit summary Tag: Visual edit
• 17:1517:15, 17 May 2024 diff hist +212‎ N Compile MemCP from Source ‎ Created page with "Make sure, <code>go</code> is installed on your computer. (at least 1.22.0) Compile the project with: git clone <nowiki>https://github.com/launix-de/memcp</nowiki> cd memcp make Run the engine with: ./memcp" current Tag: Visual edit
• 17:1317:13, 17 May 2024 diff hist +441‎ N With Singularity ‎ Created page with "Singularity is a rootless container format for restricted environments like HPC. Here are the installation instructions: https://docs.sylabs.io/guides/latest/user-guide/quick_start.html#quick-installation-steps Build with: git clone <nowiki>https://github.com/launix-de/memcp</nowiki> cd memcp sudo singularity build memcp.sif memcp.singularity.recipe Now you can run memcp.sif: mkdir data singularity run --bind data:/data memcp.sif" current Tag: Visual edit
• 17:1017:10, 17 May 2024 diff hist +660‎ N Install MemCP with Docker ‎ Created page with "To build the image the first time: git clone https://github.com/launix-de/memcp cd memcp docker build . -t memcp For app development and to enter the Scheme console: mkdir data docker run -v data:/data -it -p 4321:4321 -p 3307:3307 memcp To run your custom Scheme apps (like the one from RDF templating and model driven development): make data docker run -e PARAMS="lib/main.scm apps/minigame.scm" -v data:/data -it -p 4321:4321 -p 3307:3307 memcp During productio..." current Tag: Visual edit
• 17:0617:06, 17 May 2024 diff hist +2,427‎ N Persistency and Performance Guarantees ‎ Created page with "MemCP gives the user several Guarantees for persistency and performance. These are the guarantees: == Persistency Guarantees == There are three persistency modes per table which are: * ENGINE = memory * ENGINE = sloppy * ENGINE = logged * ENGINE = safe === ENGINE = memory === * all data is held in memory and only in memory * in case of a crash, all data is gone * the schema is saved on disk * after a recovery, the table starts empty * fastest way to store data * use..." Tag: Visual edit
• 16:3216:32, 17 May 2024 diff hist +451‎ Lists and Objects ‎No edit summary current Tag: Visual edit
• 16:2716:27, 17 May 2024 diff hist +1,805‎ N Lists and Objects ‎ Created page with "Lists are a powerful feature in Scheme. Lists can express both: Data and Programs. ==List handling== The following list functions can be used: > '(1 2 3) "list literal" = (1 2 3) > (cons 0 '(1 2 3)) "prepend item to list" = (0 1 2 3) > (append '(1 2 3) 4 5) "append items to list" = (1 2 3 4 5) > (has? '(1 2 3) 2) "check if item is in list" = true > (filter '(1 2 3) (lambda (x) (< x 2))) "filter items that are smaller than 2" = (1) > (map '(1 2 3) (l..." Tag: Visual edit
• 16:2616:26, 17 May 2024 diff hist +12‎ Introduction to Scheme ‎ →‎Deeper Topics into Scheme current Tag: Visual edit
• 16:2516:25, 17 May 2024 diff hist −1,600‎ Introduction to Scheme ‎No edit summary Tag: Visual edit
• 16:2416:24, 17 May 2024 diff hist +1,740‎ Introduction to Scheme ‎No edit summary Tag: Visual edit
• 15:5115:51, 17 May 2024 diff hist +46‎ Main Page ‎ →‎Navigation Tag: Visual edit
• 15:3615:36, 17 May 2024 diff hist +10,258‎ N Introduction to Scheme ‎ Created page with "When you run <code>./memcp</code>, you will be dropped at a scheme shell like this: memcp Copyright (C) 2023, 2024 Carl-Philip Hänsch This program comes with ABSOLUTELY NO WARRANTY; This is free software, and you are welcome to redistribute it under certain conditions; loading storage /tmp/x/system/bdf64a22-6315-463c-bbf7-329317ad50ed-id of type 10 loading storage /tmp/x/system/bdf64a22-6315-463c-bbf7-329317ad50ed-username of type 20 loading stora..." Tag: Visual edit
• 15:3515:35, 17 May 2024 diff hist +50‎ Main Page ‎No edit summary Tag: Visual edit
• 15:0515:05, 17 May 2024 diff hist +30‎ Main Page ‎ →‎Navigation Tag: Visual edit
• 15:0415:04, 17 May 2024 diff hist +3,441‎ N Sequence Compression ‎ Created page with "One of the most interesting compression techniques on columnar storages is '''sequence compression'''. Sequence Compression in In-Memory Database yields 99% memory savings and a total of 13% A sequence is a column of numbers where each distance between two neighbouring numbers is equal. Example: * <code>1 2 3 4 5 6 7 8 9</code> * <code>3 3 3 3 3 3 3 3 3 3</code> * <code>10 20 30 40</code> * <code>8 7 6 5</code> A sequence can be described by its starting point, its st..." current Tag: Visual edit
• 15:0115:01, 17 May 2024 diff hist +398‎ N In-Memory Compression, Columnar Compression Techniques ‎ Created page with "The following compression techniques are implemented: * Sparse Storage for columns with lots of NULL values * Float Storage for scientific data * Integer Compression with a bit-size approach * Sequence Compression for sequences of integers like IDs (1, 2, 3, 4, 5 ...) * Dictionary Compression for short strings * BLOB Zipping and Deduplication for strings longer than 1KiB" Tag: Visual edit