Learning Required

This is an old revision of the document!

I love playing with TiddlyWiki. It's a fascinating playground for JavaScript code that is independent of all the frameworks. Complex solutions can be built on the client without even needing a web server.

I love SQLite for similar reasons. Both projects reject the best practices of the modern industry and instead follow their own, client based, dependency free way of doing things.

What I've wanted to do for a while is combine the two. To create a SQLite plugin for TiddlyWiki. To capture data within the wiki, push it into a SQL database and then have the ability to generate result sets. Slicing and dicing the data in a way that is beyoned what the wiki's native JSON store is capable of.

The problem is that there is so much to learn. Packaging up the Javascript and WASM for the browser based version of SQLite. Gaining the deep and lasting understanding needed to grok TiddlyWiki.

I've always put the task of because it would require me to create a long term plan for all the the requisite learning, and that is too much like my day job. In my free time I like to take a more exploratory roles.

I've tried working with AI as a co-pilot, and I don't like it much. I don't like adding code I don't understand. I'm a strong believer that software development is 80% understanding and 20% typing.

I've used ChatGPT to help me learn new stuff, and for that I've found it very helpful. It quickly helps me to quickly scout out a new field and locate the important jumping off points. Can I do the same with coding?

So I've decided to jump in and explore, and keep a journal along the way. This is because I only a few hours here and there to do this, so I want to be able to read back on what I did last time before moving forward.

Last time I looked into this I tried taking the online example for Sql.js and run it as a standalone file. It didn't go well, and reading up on WASM I get the impression that WASM was not going to let me run locally, but I couldn't find any solid confirmation of whether or not it was posssible.

If I'm to embed SQLite within TiddlyWiki I need to run it locally as a standalone, so I ask ChatGPT if it can be done:

Is it possible to run SqlIte in the browser from a file, without any web server?

Yes! You can run SQLite directly in the browser without any web server, using SQLite Wasm (WebAssembly). This lets you open a database file, run queries, and even save changes — all in the client side.

Here’s a simple way to do it:
* Download SQLite Wasm Get the SQLite Wasm bundle from the official SQLite site or a CDN.
* Create an HTML file Save this as index.html:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>SQLite in Browser</title>
</head>
<body>
    <h1>SQLite in Browser</h1>
    <input type="file" id="fileInput" />
    <pre id="output"></pre>
 
    <script src="https://cdn.jsdelivr.net/npm/sql.js@1.8.0/dist/sql-wasm.js"></script>
    <script>
        let db;
 
        document.getElementById('fileInput').addEventListener('change', async (event) => {
            const file = event.target.files[0];
            if (file) {
                const buffer = await file.arrayBuffer();
                const SQL = await initSqlJs({ locateFile: filename => `https://cdn.jsdelivr.net/npm/sql.js@1.8.0/dist/${filename}` });
                db = new SQL.Database(new Uint8Array(buffer));
 
                const result = db.exec("SELECT name FROM sqlite_master WHERE type='table';");
                document.getElementById('output').textContent = JSON.stringify(result, null, 2);
            }
        });
    </script>
</body>
</html>

* Open the file in a browser Just double-click the index.html. No need for a web server — it runs purely on the client side.

How it works
* The SQLite Wasm engine processes the .sqlite file.
* Queries run directly in the browser.
* The output shows available tables — you can expand this to support custom queries.

I follow the instructions but I hit a problem. It want's me to select a local database file and i don't have one. So I ask for a fix.

Is a file necessary? Can I create a purely inmemory db?
Yes! You can create a purely in-memory SQLite database in the browser — no file needed. Here’s a simple example using sql.js:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>In-Memory SQLite</title>
</head>
<body>
    <h1>SQLite In-Memory DB</h1>
    <button id="runQuery">Run Query</button>
    <pre id="output"></pre>
 
    <script src="https://cdn.jsdelivr.net/npm/sql.js@1.8.0/dist/sql-wasm.js"></script>
    <script>
        (async () => {
            const SQL = await initSqlJs({ locateFile: file => `https://cdn.jsdelivr.net/npm/sql.js@1.8.0/dist/${file}` });
            const db = new SQL.Database(); // In-memory database
 
            document.getElementById('runQuery').addEventListener('click', () => {
                db.run("CREATE TABLE test (id INTEGER, name TEXT);");
                db.run("INSERT INTO test VALUES (1, 'Alice'), (2, 'Bob');");
 
                const result = db.exec("SELECT * FROM test;");
                document.getElementById('output').textContent = JSON.stringify(result, null, 2);
            });
        })();
    </script>
</body>
</html>

I run this and it works, giving me the following result set in a JSON format:

[
  {
    "columns": [
      "id",
      "name"
    ],
    "values": [
      [
        1,
        "Alice"
      ],
      [
        2,
        "Bob"
      ]
    ]
  }
]

It's still loading code and WASM from a server. I need it to be entirely offline.

I'd like to run this offline. Can I download the js and wasm files and reference them locally?
Yes! You can absolutely run SQLite (with sql.js) offline by downloading the necessary files. Let’s set this up step by step.

The instructions a straight forward. I download the file and I change the uri in the initSqlJs constructor to this:

// Initialize sql.js with local wasm file
initSqlJs({
  locateFile: file => `lib/${file}`
}).then(SQL => { ... }

This is when I run into the same CORS Policy problems that blocked me in the past:

Access to fetch at 'file:///C:/Users/gedby/OneDrive/Desktop/sqljs/lib/sql-wasm.wasm' from origin 'null' has been blocked by CORS policy: Cross origin requests are only supported for protocol schemes: chrome-extension, chrome-untrusted, data, edge, http, https, isolated-app.

Failed to load resource: net::ERR_FAILED (lib/sql-wasm.wasm:1)

Uncaught (in promise) TypeError: Failed to fetch (sql-wasm.js:159)
    at sql-wasm.js:159:276
    at sql-wasm.js:159:499
    at sql-wasm.js:160:11
    at new Promise (<anonymous>)
    at initSqlJs (sql-wasm.js:22:24)
    at Test.html:16:31
    at Test.html:26:11

I tell ChatGPT about the error, and it confidently provides me with a fix.

Ah, yes — this happens because modern browsers block fetch requests for local files due to CORS restrictions. Even though you’re running things locally, loading .wasm files via file:// doesn’t work by default.
Let’s fix it! 🚀

It then gives me three possible options: 1. Run a simple local server (Recommended) 1. Use a VS Code extension (even simpler) 1. Not recommended) Disable CORS checks temporarily

None of these really work for me. For the WASM to work in TiddlyWiki I'll need to load the content from within the file. This is how TiddlyWiki manages all of it's resources, including Javascript. I wonder if this simply isn't possible with WASM.

I explain my problem to ChatGPT, and it confidently provides the answer:

I don't really like any of these options. The goal is to load the WASM from within my TiddlyWiki.
Is there anyway of loading WASM from data embedded in my HTML file, perhaps BASE64 encoded?
Yes! You can absolutely embed the WebAssembly (WASM) directly into your TiddlyWiki without needing any external files or servers. Let’s tackle this step by step. 🚀

I am loving this AI's can do attitude, so I follow the steps provided.

First of all I convert my WASM file to Base64. To do this I'm given this powershell one liner.:

[convert]::ToBase64String([IO.File]::ReadAllBytes("sql-wasm.wasm")) | Out-File "sql-wasm.base64.txt"

Then I copy the Base64 text into a string so that I can loading it into a binary array.

One thing I like at this point is that ChatGPT doesn't generate the whole page for me. Instead it just provides the code related to the task at hand, which I'm able to adapt into my own code base.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Embedded SQLite with WASM</title>
</head>
<body>
  <h1>SQLite with Embedded WASM</h1>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/sql.js/1.8.0/sql-wasm.js"></script>
  <script>
    // Paste the long Base64-encoded WASM here
    const wasmBase64 = "BASE64_ENCODED_STRING_HERE";
 
    // Convert Base64 to binary
    const binaryString = atob(wasmBase64);
    const bytes = new Uint8Array(binaryString.length);
    for (let i = 0; i < binaryString.length; i++) {
      bytes[i] = binaryString.charCodeAt(i);
    }
    const wasmBinary = bytes.buffer;
 
    // Initialize sql.js using the in-memory WASM binary
    initSqlJs({
      locateFile: () => URL.createObjectURL(new Blob([wasmBinary], { type: "application/wasm" }))
    }).then(SQL => {
      const db = new SQL.Database(); // In-memory SQLite DB
      db.run("CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT);");
      db.run("INSERT INTO test (name) VALUES ('Alice');");
 
      const res = db.exec("SELECT * FROM test;");
      console.log("Query result:", res);
    }).catch(err => {
      console.error("Failed to initialize sql.js:", err);
    });
  </script>
</body>
</html>

There are two problems with this code.

ChatGPT has decided to drop functionality from the code we are writing. In the previous version, the code was integrated with the HTML, with the query being run when the “Run Query” button is pressed and the results written to a preformatted “output” element. Now the code is run on load and the result is written to console. It is this type of inconsistency that frustrates me when using ChatGPT to write code for me. It leads to disconnected code that is hard to maintain.
It doesn't work, so I tell ChatGPT about the error.

I'm getting the error: wasm streaming compile failed: TypeError: Failed to execute 'compile' on 'WebAssembly': Incorrect response MIME type. Expected 'application/wasm'.

And now we see how ChatGPT shines as a research assistant and guide, because it immediately identified the problem and provides the fix:

Ah, that's happening because the WebAssembly streaming compiler expects the MIME type to be application/wasm — but when you're embedding the WASM as a Base64 string, the browser doesn't get the right content type automatically.

Let’s fix this by using WebAssembly.compile with the decoded binary, bypassing streaming:

It gives me a full page of code, but the only part that I'm interested in is this:

// Initialize sql.js using the in-memory WASM binary
initSqlJs({
  locateFile: () => {
    // Create a blob for the binary and generate a URL
    const blob = new Blob([bytes], { type: "application/wasm" });
    return URL.createObjectURL(blob);
  }
}).then(SQL => { ... });

I list this locateFile definition, so that my final code looks like this:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>In-Memory SQLite</title>
</head>
<body>
    <h1>SQLite In-Memory DB</h1>
    <button id="runQuery">Run Query</button>
    <pre id="output"></pre>
 
    <script src="lib/sql-wasm.js"></script>
    <script>
        const base64wasm = "...";
        const binaryString = atob(base64wasm);
        const bytes = new Uint8Array(binaryString.length);
        for (let i = 0; i < binaryString.length; i++) {
            bytes[i] = binaryString.charCodeAt(i);
        }
 
        (async () => {
            // Initialize sql.js using the in-memory WASM binary
            const sql = await initSqlJs({
                locateFile: () => {
                    // Create a blob for the binary and generate a URL
                    const blob = new Blob([bytes], { type: "application/wasm" });
                    return URL.createObjectURL(blob);
                }
            });
 
            const db = new sql.Database(); // In-memory SQLite DB
 
            document.getElementById('runQuery').addEventListener('click', () => {
 
                db.run("CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT);");
                db.run("INSERT INTO test (name) VALUES ('Alice'), ('Bob');");
 
                const result = db.exec("SELECT * FROM test;");
                document.getElementById('output').textContent = JSON.stringify(result, null, 2);
            });
        })();
 
    </script>
</body>
</html>

This completes my proof of concept. I have a HTML file that can I open in Windows Explorer and it runs SQLite locally without any server.

The code is simple, and I mostly understand what it is doing. There is one line that uses a JavaScript feature that I'm unfamiliar with: URL.createObjectURL(blob). Along with the code I was provided with a brief explanation:

✅ Why this works:
* BlobI — creates a proper WASM binary blob with the correct MIME type.
* URL.createObjectURL — gives a fake “file” URL the WebAssembly compiler can work with.
* Streaming bypassed — since there’s no remote file, this sidesteps the CORS issue.

* Unordered List Item

That isn't enough for me. I want to know more.

Learning Required

Support Offered by AI

Proof of Concept: Can I run SQLite in the browser without a server?

I don't want a local database file

Loading WASM from a Base64 encoded string

Fixing the CORS issue

Fixing the MIME type

Understanding URL.createObjectURL()

The Rheology of Software