SOme time ago, I posted a quick description of a full PostgreSQL implementation running in the browser via a WASM Linux environment (PostgreSQL Running in the Browser via WASM; see also a hacky note on running this in JupyterLite).
And a couple of weeks ago, via (who else?!) @simonw, I came across pglite, a lite version fo postgres (the SQL engine part? DBMS facilities such as roles, etc., seem not to be supported).
For some time now, I’ve wondered how easy it is to hook into a WASM app, so here are a couple of quick demos. Demos available here and demo repo here.
Connect via an xterm.js terminal
xterm.js is a simple, scriptable, Javascript terminal app that can run inside a web page.
I haven’t used xterm.js before, but was surprised at quite how basic it was, how little tutorial support or example usage materials there were around it, and how much js I had to try to hack around to get it to behave at all usefully.
It does support plugins, and I guess I was either expecting a richer base behaviour, or more off-the-shelf tools to add thing like history, sensible cursor mamagement, mutli-line entry etc. Or maybe I just didn’t find them?!
As a simple recipe, xterm.js + WASM engine seems like 101 browser demo thing, so not finding simple tutorials of such combinations was a bit of a surprise. (I guess if I was still building a thing and blogging about it once a day, I’d have hacked, found, nudged or bludgeoned folk to produce a lot of such demos over the last couple of years… But I don’t really do that any more .
I have an increasing distaste for code and what it’s doing to the world/how folk are using it to crap all over everything in a wasteful and ignorant ways, and I don’t really want to contribute much towards promoting digital tech any more if I can help it…
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>xterm.js with PGlite</title>
<!-- Include xterm.js and its dependencies -->
<link rel="stylesheet" href="https://unpkg.com/xterm/css/xterm.css" />
<script src="https://unpkg.com/xterm/lib/xterm.js"></script>
<script src="https://unpkg.com/xterm-addon-fit/lib/xterm-addon-fit.js"></script>
<!-- Include PGlite -->
</head>
<body>
<div id="content">
<h1>xterm.js demo</h1>
<p>Simple demo of querying the pglite in-browser WASM database from an xterm.js user interface.</p>
<p>This demo features an ephemeral database with no persistent browser storage.</p>
<p>To test a simple query that does not require any database tables to exist, run <tt>select 'hello world';</tt></p>
<p>A test database has been pre-loaded. View its contents by issuing the command <tt>select * from test;</tt></p>
<p>Queries must end with a <tt>;</tt>. This allows queries to be added over multiple lines.</p>
</div>
<div id="terminal"></div>
<script src="xterm.js" type="module"></script>
</body>
</html>
There is some minimal js in there, as well as accreted js to try to make the terminal more useable. (I really should have just grabbed the bar minimum js required to get some sort of response for this post…)
Because the demo is intended to return SQL tables, I also needed to add a simple table display handler routine. (There are table rendering node packages, but I donlt know what the simple was is of pulling them in so they can be used by the script qua module? (Javascript packaging left me behind several years ago!)
The initialisation script seeds the database with a simple table. Database persistence is supported by pglite using browser storage, but I haven’t made us of it in this demo.
import { PGlite } from "https://cdn.jsdelivr.net/npm/@electric-sql/pglite/dist/index.js";
// xterm module issue here: https://github.com/xtermjs/xterm.js/issues/2878
async function createTable() {
await db.query(`CREATE TABLE IF NOT EXISTS test (
id serial primary key,
title varchar not null
)`);
await db.query(
`INSERT INTO test (title) values ('Hello world') returning id;`
);
await db.query("select * from test").then((resultsX) => {
console.log(JSON.stringify(resultsX));
});
}
// Initialize xterm.js
const term = new Terminal({ cursorBlink: true });
const fitAddon = new FitAddon.FitAddon();
term.loadAddon(fitAddon);
term.open(document.getElementById("terminal"));
fitAddon.fit();
const PROMPT = "pglite# ";
term.prompt = () => {
term.write("\r\n" + PROMPT);
};
term.writeln("Welcome to pglite demo terminal!\r\n");
term.write(PROMPT);
// History handler via chatgpt
// Keep track of command history
const commandHistory = [];
let historyIndex = -1;
let multiline_command = [];
// Initialize PGlite
const db = new PGlite();
await createTable();
// Pass 'idb://my-pgdata' for indexedDB persistence
// Variable to store the current command
let command = '';
// Event listener for cursor move
term.onCursorMove(() => {
// Update the command variable with the content of the current line
command = term.buffer.active.getLine(term.buffer.active.cursorY).translateToString().trim();
command = command.startsWith(PROMPT)
? command.slice(PROMPT.length)
: command;
// Handle the command content as needed
//console.log("command fragment: " + command);
});
term.onKey(async (e) => {
const printable =
!e.domEvent.altKey && !e.domEvent.ctrlKey && !e.domEvent.metaKey;
// Enter key
if (e.domEvent.keyCode === 13) {
term.writeln("");
console.log("enter command fragment: " + command);
if (command != "") multiline_command.push(command);
console.log("mll: " + JSON.stringify(multiline_command));
if (command.endsWith(';')) {
command = multiline_command.join(" ");
try {
await executeCommand(command, term);
// Add the command fragment to history
commandHistory.unshift(command);
historyIndex = -1;
} catch (e) {
// Adjust for browser differences in Error.stack().
const report = (window["chrome"] ? "" : `${e.message}\n`) + e.stack;
output.innerHTML = `<pre>${report}</pre>`;
} finally {
}
multiline_command = [];
}
} else if (e.domEvent.key === "ArrowUp" && multiline_command.length==0 ) {
// Up arrow key pressed
if (historyIndex < commandHistory.length - 1) {
historyIndex++;
const command = commandHistory[historyIndex];
term.write("\x1b[K" + command);
}
} else if (e.domEvent.key === "ArrowDown" && multiline_command.length == 0) {
// Down arrow key pressed
if (historyIndex >= 0) {
historyIndex--;
const command = commandHistory[historyIndex] || "";
term.write("\x1b[K" + command); // Clear current line and write the command
}
} else if (e.domEvent.keyCode === 8) {
// backspace
if (multiline_command.length == 0) {
if (term.buffer.active.cursorX > PROMPT.length)
term.write("\b \b");
} else term.write("\b \b");
} else if (e.domEvent.ctrlKey && e.domEvent.key === "c") {
term.writeln("");
term.prompt();
multiline_command = [];
command = "";
} else if (printable) {
term.write(e.key);
}
});
async function executeCommand(command, term) {
try {
const results = await db.query(command);
const formattedResults = formatResults(results);
//const formattedResults = arrayToTable(results);
term.writeln(formattedResults);
} catch (e) {
term.writeln(`Error: ${e.message}`);
} finally {
term.prompt();
}
}
// Via https://github.com/nijikokun/array-to-table/
function arrayToMarkdownTable(array, columns, alignment = "center") {
var table = "";
var separator = {
left: ":---",
right: "---:",
center: "---",
};
// Generate column list
var cols = columns ? columns.split(",") : Object.keys(array[0]);
// Generate table headers
table += cols.join(" | ");
table += "\r\n";
// Generate table header seperator
table += cols
.map(function () {
return separator[alignment] || separator.center;
})
.join(" | ");
table += "\r\n";
// Generate table body
array.forEach(function (item) {
table +=
cols
.map(function (key) {
return String(item[key] || "");
})
.join(" | ") + "\r\n";
});
// Return table
return table;
}
// With help from ChatGPT
function formatResults(results) {
let formattedOutput = [];
if (results.length === 0) {
formattedOutput.push("\x1b[31mNo results found.\x1b[0m");
} else {
const keys = Object.keys(results[0]);
const columnWidths = {};
// Calculate the maximum width for each column
keys.forEach((key) => {
const maxWidth = Math.max(
key.length,
...results.map((row) => String(row[key]).length)
);
columnWidths[key] = maxWidth;
});
const lines = [];
lines.push(
"|" + keys.map((key) => key.padEnd(columnWidths[key])).join("|") + "|"
);
results.forEach((row) => {
lines.push(
"|" +
keys
.map((key) => String(row[key]).padEnd(columnWidths[key]))
.join("|") +
"|"
);
});
const divider =
"|" + keys.map((key) => "-".repeat(columnWidths[key])).join("|") + "|";
formattedOutput.push(divider);
formattedOutput.push(lines[0]);
formattedOutput.push(divider);
formattedOutput.push(...lines.slice(1));
formattedOutput.push(divider);
}
formattedOutput = "\r\n" + formattedOutput.join("\r\n");
return formattedOutput;
}
If this had been a fully-blown PostgreSQL implementation, it might have been interesting to try to use it for OUr database management and analysis course, but as the features seem limited to the SQL engine, something like DuckDB, or even SQLite, is probably more interesting in the WASM powered app browser context because of the features tools that are built around them already in that implementation space.
Monaco Editor
The second example is ripped off from a sqlite demo — https://rhashimoto.github.io/wa-sqlite/demo/ — that uses a Monaco editor to handle the SQL code entry.
Once again, I was surprised at how few example tutorials, basic docs, and minimal examples of hooking the editor up to various WASM apps I could find… But then, search is getting harder because the webcos are sh*t, much of the content that is out there is rehashed or generated sh*t, and folk who do interesting things are rediscovering a happier life away from screens and digital crap. Or are just less sharey and discoverable in the online places I cross through than they used to be for me.
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>wa-sqlite demo</title>
<style>
#editor-container {
width: 100%;
height: 50vh;
}
#vfs-container {
margin-top: 0.5em;
margin-bottom: 0.5em;
}
#timestamp {
margin-top: 0.5em;
}
#output {
display: flex;
flex-wrap: wrap;
width: 100%;
}
table {
margin-top: 1em;
margin-left: 0.4em;
margin-right: 0.4em;
border-collapse: collapse;
}
td, th {
border: 1px solid #999;
padding: 0.5rem;
text-align: left;
}
</style>
</head>
<body>
<div id="editor-container"></div>
<div><button id="execute">Execute</button></div>
<div id="timestamp"></div>
<div id="output"></div>
<div id="results"></div>
<script type="module" src="monaco.js"></script>
</body>
</html>
Again, this demo does not persist the database contents using browser storage, although once again, it could.
// Monaco related UI elements originally via https://rhashimoto.github.io/wa-sqlite/demo/
// and Copyright 2023 Roy T. Hashimoto. All Rights Reserved.
const MONACO_VS =
"https://cdnjs.cloudflare.com/ajax/libs/monaco-editor/0.34.1/min/vs";
// PGLite loader
import { PGlite } from "https://cdn.jsdelivr.net/npm/@electric-sql/pglite/dist/index.js";
// Initialize PGlite
const db = new PGlite();
// We can persist the db in the browser
//const db = new PGlite('idb://my-pgdata')
const DEFAULT_SQL = `
-- Optionally select statements to execute.
CREATE TABLE IF NOT EXISTS test (
id serial primary key,
title varchar not null
);
INSERT INTO test (title) values ('dummy');
`.trim();
async function createTable() {
await db.query(DEFAULT_SQL);
await db.query("select * from test").then((resultsX) => {
console.log(JSON.stringify(resultsX));
});
}
createTable();
window.addEventListener("DOMContentLoaded", async function () {
// Load the Monaco editor
const button = /** @type {HTMLButtonElement} */ (
document.getElementById("execute")
);
const editorReady = createMonacoEditor().then((editor) => {
// Change the button text with selection.
editor.onDidChangeCursorSelection(({ selection }) => {
button.textContent = selection.isEmpty()
? "Execute"
: "Execute selection";
});
return editor;
});
// Execute SQL on button click.
button.addEventListener("click", async function () {
button.disabled = true;
// Get SQL from editor.
const editor = await editorReady;
const selection = editor.getSelection();
const queries = selection.isEmpty()
? editor.getValue()
: editor.getModel().getValueInRange(selection);
// Clear any previous output on the page.
const output = document.getElementById("output");
while (output.firstChild) output.removeChild(output.lastChild);
const timestamp = document.getElementById("timestamp");
timestamp.textContent = new Date().toLocaleTimeString();
let time = Date.now();
console.log(`${queries}`);
try {
const results = await db.query(`${queries}`);
//.then(results => {console.log("results are"+JSON.stringify(results))});
const resultsDiv = document.getElementById("results");
resultsDiv.innerHTML = "";
const table = formatTable(results);
formatRows(results, table);
resultsDiv.appendChild(table);
} catch (e) {
// Adjust for browser differences in Error.stack().
const report = (window["chrome"] ? "" : `${e.message}\n`) + e.stack;
output.innerHTML = `<pre>${report}</pre>`;
} finally {
timestamp.textContent += ` ${(Date.now() - time) / 1000} seconds`;
button.disabled = false;
}
});
});
function formatTable(results) {
const table = document.createElement("table");
const headerRow = table.insertRow();
Object.keys(results[0]).forEach((key) => {
const th = document.createElement("th");
th.textContent = key;
headerRow.appendChild(th);
});
return table;
}
function formatRows(results, table) {
results.forEach((rowData) => {
const row = table.insertRow();
Object.values(rowData).forEach((value) => {
const cell = row.insertCell();
cell.textContent = value;
});
});
}
// Monaco handlers
// Via https://rhashimoto.github.io/wa-sqlite/demo/
async function createMonacoEditor() {
// Insert a script element to bootstrap the monaco loader.
await new Promise((resolve) => {
const loader = document.createElement("script");
loader.src = `${MONACO_VS}/loader.js`;
loader.async = true;
loader.addEventListener("load", resolve, { once: true });
document.head.appendChild(loader);
});
// Load monaco itself.
/** @type {any} */ const require = globalThis.require;
require.config({ paths: { vs: MONACO_VS } });
const monaco = await new Promise((resolve) => {
require(["vs/editor/editor.main"], resolve);
});
// Create editor.
// https://microsoft.github.io/monaco-editor/api/modules/monaco.editor.html#create
return monaco.editor.create(document.getElementById("editor-container"), {
language: "sql",
minimap: { enabled: false },
automaticLayout: true,
});
}
P.S. WASM-webterm
I did find this — https://github.com/cryptool-org/wasm-webterm — which looked interesting:
Run your WebAssembly binaries on a terminal/tty emulation in your browser. Emscripten and WASI are supported. This project is developed as an addon for xterm.js v4, so you can easily use it in your own projects.
but the npm and node and all that crap stuff was just too much wtf for a simple demo hacked in a simple GitHub text editor in a page intended to be pushed to Github Pages, and where I though it would be a three step:
- load xterm.js
- load wasm
- add a simple bit of glue showing how to pass xterm entered command to wasm file and display result
it didn’t seem to offer that, so I moved on because I really can’t be bothered any more.
Speaking of which, for demos to be useful for people who don’t know js packaging sh*t, and don’t care or can’t be bothered to learn, it’s not obvious how you mix and match stuff that is simple js, vs js modules, vs. things that are pulled from npm.js CDNs versus who knows what how many other ways of packaging, pulling, wrapping and declaring objects that should but don’t play with each other either because they get pulled into different namespaces, modules or not modules, or becuase you canlt “just” instantiate one within the other.
[Signed] old and grumpy and I don’t know how any of this sh*t works anymore when it used to be so easy.
PPS wtf do I have to do to get this crappy WordPress theme to center align my images like it used to?
OUseful.Info, the blog... 