In-Database WebApps and REST Services: Difference between revisions

From MemCP
Jump to navigation Jump to search
(Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference)
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
In-Database WebApps are a huge game changer in REST API performance since the web app runs in the same context as the
<!-- Copyright (C) 2026 Carl-Philip Haensch -->
<!-- SPDX-License-Identifier: GPL-3.0-or-later -->
= In-Database WebApps and REST Services =


== REST Endpoint setup ==
Embedded handlers run in the MemCP process and can access Scheme and storage APIs without a separate database connection. <code>lib/main.scm</code> defines <code>http_handler</code>; application modules can wrap the previous handler and route only their own path prefix.
You basically set up a REST endpoint by overwriting the <code>http_handler</code> variable in the global scope. Best practice is here to overload http_handler, store the old handler in <code>old_handler</code> and cascade the router with a prefix:
(define http_handler (begin
        (set old_handler (coalesce http_handler handler_404))
        /* here starts our custom router */
        (lambda (req res) (begin
                /* hooked our additional paths to it */
                (match (req "path")
                        (regex "^/my_prefix(.*)$" url subpath) (begin
                                (my_custom_handler req res path)
                        )
                        /* default */
                        (old_handler req res))
        ))
))
Now you can implement your own custom handler:
(define my_custom_handler (lambda (req res subpath) (begin
        /* print hello world */
        ((res "header") "Content-Type" "text/html")
        ((res "status") 200)
        ((res "print") "<nowiki><h1>Hello World</h1></nowiki>")
)))
Now you can query your endpoint with:
curl <nowiki>http://localhost:4321/my_prefix</nowiki>
In your custom handler, you can again parse your subpath to go to the single routes.


For more details, take a look at [[Scheme: serve|HTTP Server]].
This is useful for small JSON APIs, dashboards, webhooks, static assets, or WebSocket gateways whose hot path is mostly database work. It reduces deployment pieces and transport overhead, but also couples application code to the database process. Keep CPU-heavy, blocking, untrusted, or independently deployed workloads outside MemCP.


== Including SQL or RDF queries in your custom REST endpoint ==
== Routing a path prefix ==
The SQL and RDF frontend have helper functions:


* <code>(parse_sparql schema query)</code> to create the code for a SPARQL query
<pre>
* <code>(parse_sql schema query)</code> to create the code for a SQL query
(define http_handler (begin
(set old_handler http_handler)
(lambda (req res) (begin
(match (req "path")
(regex "^/my-api/(.*)$" path rest) (begin
((res "header") "Content-Type" "application/json")
((res "status") 200)
((res "print") "{\"ok\":true}"))
(old_handler req res))))))
</pre>


Basically, you do the parsing and preparation outside of your HTTP handler in order to get the best performance out of prepared statements.
Always delegate unmatched paths to the previous handler. Replacing the global handler without chaining it can hide the dashboard, SQL endpoints, or routes installed by other modules.


In the last step, you overwrite the query parameters as well as the function <code>resultrow</code> in your scope and then feed the code into <code>eval</code>:
The request object exposes method, host, path, query fields, headers, username/password, remote address, and lazy body readers. The response object sets headers and status, writes text/lines or JSONL rows, and can upgrade a connection to WebSocket. See [[IO]] for the generated function reference and <code>apps/</code> for executable examples.
(set my_code (parse_sql "my_database" "SELECT * FROM a"))
(set my_code (optimize my_code)) /* optionally run the optimizer over it */
/* now inside your HTTP handler: */
(set resultrow (res "jsonl")) /* this is the jsonl printer, but you can also use print or any lambda that takes a associative array with the results */
(eval my_code) /* execute query */
To read on how to handle the result object, take a look at: [[Lists and Objects]]


== Example WebApps ==
== Prepared SQL and response streaming ==
You find a minimal example in the apps/ folder in the memcp sources.


You find a complex example of a RDF browser and whole template engine in:
Parse and optimize fixed SQL outside the request hot path where practical; bind request values rather than concatenating untrusted SQL. The exact request/response functions are documented in [[IO]] and examples under <code>apps/</code>.


https://github.com/launix-de/rdfop
<pre>
(set find_user (parse_sql "myapp"
"SELECT id, name FROM users WHERE id = @user_id"))


Another example can be found in [[Websockets in MemCP]]
/* inside a request handler */
(set session (newsession))
(session "user_id" requested_id)
(set resultrow (res "jsonl"))
(eval find_user)
</pre>
 
Prepared formulas capture a fixed query shape, not permission to trust their parameters. Validate request types, authorize the selected record, and do not let callers choose arbitrary identifiers. Stream rows when possible; do not collect an unbounded result merely to turn it into one JSON array.
 
The RDF frontend follows the same preparation pattern with <code>parse_sparql</code>. Prepare a fixed SPARQL shape outside the handler, bind request-specific values through the intended context and stream results through the response. Do not concatenate caller text into either SQL or SPARQL.
 
== Error handling and service boundaries ==
 
Set the content type and status before writing the body. Convert expected validation and not-found cases into explicit 4xx responses; log unexpected failures without returning stack traces, SQL, secrets, or internal paths. Define request timeouts and body limits at the reverse proxy and inside application logic.
 
== Security and lifecycle ==
 
Handlers execute with in-process capabilities. Authenticate before database access, validate paths, headers and bodies, enforce request-size limits, and never expose default <code>root/admin</code> credentials. Bound queues and long operations, propagate cancellation, and avoid retaining request objects after completion. Use <code>--no-repl</code> for daemon deployment. See [[Security and Authentication]], [[SQL over REST]], and [[Websockets in MemCP]].
 
Minimal maintained examples live below <code>apps/</code>. The external [https://github.com/launix-de/rdfop rdfop project] is a larger RDF browser and templating example; treat it as application source, not as part of MemCP's compatibility contract.

Latest revision as of 12:13, 28 August 2026

In-Database WebApps and REST Services

Embedded handlers run in the MemCP process and can access Scheme and storage APIs without a separate database connection. lib/main.scm defines http_handler; application modules can wrap the previous handler and route only their own path prefix.

This is useful for small JSON APIs, dashboards, webhooks, static assets, or WebSocket gateways whose hot path is mostly database work. It reduces deployment pieces and transport overhead, but also couples application code to the database process. Keep CPU-heavy, blocking, untrusted, or independently deployed workloads outside MemCP.

Routing a path prefix

(define http_handler (begin
	(set old_handler http_handler)
	(lambda (req res) (begin
		(match (req "path")
			(regex "^/my-api/(.*)$" path rest) (begin
				((res "header") "Content-Type" "application/json")
				((res "status") 200)
				((res "print") "{\"ok\":true}"))
			(old_handler req res))))))

Always delegate unmatched paths to the previous handler. Replacing the global handler without chaining it can hide the dashboard, SQL endpoints, or routes installed by other modules.

The request object exposes method, host, path, query fields, headers, username/password, remote address, and lazy body readers. The response object sets headers and status, writes text/lines or JSONL rows, and can upgrade a connection to WebSocket. See IO for the generated function reference and apps/ for executable examples.

Prepared SQL and response streaming

Parse and optimize fixed SQL outside the request hot path where practical; bind request values rather than concatenating untrusted SQL. The exact request/response functions are documented in IO and examples under apps/.

(set find_user (parse_sql "myapp"
	"SELECT id, name FROM users WHERE id = @user_id"))

/* inside a request handler */
(set session (newsession))
(session "user_id" requested_id)
(set resultrow (res "jsonl"))
(eval find_user)

Prepared formulas capture a fixed query shape, not permission to trust their parameters. Validate request types, authorize the selected record, and do not let callers choose arbitrary identifiers. Stream rows when possible; do not collect an unbounded result merely to turn it into one JSON array.

The RDF frontend follows the same preparation pattern with parse_sparql. Prepare a fixed SPARQL shape outside the handler, bind request-specific values through the intended context and stream results through the response. Do not concatenate caller text into either SQL or SPARQL.

Error handling and service boundaries

Set the content type and status before writing the body. Convert expected validation and not-found cases into explicit 4xx responses; log unexpected failures without returning stack traces, SQL, secrets, or internal paths. Define request timeouts and body limits at the reverse proxy and inside application logic.

Security and lifecycle

Handlers execute with in-process capabilities. Authenticate before database access, validate paths, headers and bodies, enforce request-size limits, and never expose default root/admin credentials. Bound queues and long operations, propagate cancellation, and avoid retaining request objects after completion. Use --no-repl for daemon deployment. See Security and Authentication, SQL over REST, and Websockets in MemCP.

Minimal maintained examples live below apps/. The external rdfop project is a larger RDF browser and templating example; treat it as application source, not as part of MemCP's compatibility contract.