reinforcedlearningwithaisupport:start

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
reinforcedlearningwithaisupport:start [2025/03/07 02:56] – [Loading WASM from a Base64 encoded string] gedbadminreinforcedlearningwithaisupport:start [2025/03/08 17:14] (current) – gedbadmin
Line 1: Line 1:
-====== Learning Required ======+====== Reinforcing Human Learning with AI Support ====== 
 + 
 +===== Learning Required =====
  
 I love playing with [[https://tiddlywiki.com/|TiddlyWiki]].  It's a fascinating playground for JavaScript code that is independent of all the frameworks.  Complex solutions can be built on the client without even needing a web server.   I love playing with [[https://tiddlywiki.com/|TiddlyWiki]].  It's a fascinating playground for JavaScript code that is independent of all the frameworks.  Complex solutions can be built on the client without even needing a web server.  
Line 11: Line 13:
 I've always put the task of because it would require me to create a long term plan for all the the requisite learning, and that is too much like my day job.  In my free time I like to take a more exploratory roles. I've always put the task of because it would require me to create a long term plan for all the the requisite learning, and that is too much like my day job.  In my free time I like to take a more exploratory roles.
  
-====== Support Offered by AI  ======+===== Support Offered by AI  =====
  
 I've tried working with AI as a co-pilot, and I don't like it much.  I don't like adding code I don't understand.  I'm a strong believer that software development is 80% understanding and 20% typing. I've tried working with AI as a co-pilot, and I don't like it much.  I don't like adding code I don't understand.  I'm a strong believer that software development is 80% understanding and 20% typing.
Line 19: Line 21:
 So I've decided to jump in and explore, and keep a journal along the way.  This is because I only a few hours here and there to do this, so I want to be able to read back on what I did last time before moving forward. So I've decided to jump in and explore, and keep a journal along the way.  This is because I only a few hours here and there to do this, so I want to be able to read back on what I did last time before moving forward.
  
-====== Proof of Concept: Can I run SQLite in the browser without a server? ====== +===== A personal experiment - An SQLite plugin for TiddlyWiki =====
- +
-Last time I looked into this I tried taking the [[https://sql.js.org/examples/GUI/|online example for Sql.js]] and run it as a standalone file.  It didn't go well, and reading up on WASM I get the impression that WASM was not going to let me run locally, but I couldn't find any solid confirmation of whether or not it was posssible. +
- +
-If I'm to embed SQLite within TiddlyWiki I need to run it locally as a standalone, so I ask ChatGPT if it can be done: +
- +
->Is it possible to run SqlIte in the browser from a file, without any web server? +
- +
->>Yes! You can run SQLite directly in the browser without any web server, using SQLite Wasm (WebAssembly). This lets you open a database file, run queries, and even save changes — all in the client side. +
->> +
->>Here’s a simple way to do it: +
->>* **Download SQLite Wasm**  Get the SQLite Wasm bundle from the official SQLite site or a CDN. +
->>* **Create an HTML file**  Save this as index.html: +
-<code html> +
-<!DOCTYPE html> +
-<html lang="en"> +
-<head> +
-    <meta charset="UTF-8"> +
-    <meta name="viewport" content="width=device-width, initial-scale=1.0"> +
-    <title>SQLite in Browser</title> +
-</head> +
-<body> +
-    <h1>SQLite in Browser</h1> +
-    <input type="file" id="fileInput" /> +
-    <pre id="output"></pre> +
- +
-    <script src="https://cdn.jsdelivr.net/npm/sql.js@1.8.0/dist/sql-wasm.js"></script> +
-    <script> +
-        let db; +
- +
-        document.getElementById('fileInput').addEventListener('change', async (event) => { +
-            const file = event.target.files[0]; +
-            if (file) { +
-                const buffer = await file.arrayBuffer(); +
-                const SQL = await initSqlJs({ locateFile: filename => `https://cdn.jsdelivr.net/npm/sql.js@1.8.0/dist/${filename}` }); +
-                db = new SQL.Database(new Uint8Array(buffer)); +
- +
-                const result = db.exec("SELECT name FROM sqlite_master WHERE type='table';"); +
-                document.getElementById('output').textContent = JSON.stringify(result, null, 2); +
-            } +
-        }); +
-    </script> +
-</body> +
-</html> +
-</code> +
->>* **Open the file in a browser**  Just double-click the index.html. No need for a web server — it runs purely on the client side. +
->> +
->>//How it works// +
->>* The SQLite Wasm engine processes the .sqlite file. +
->>* Queries run directly in the browser. +
->>* The output shows available tables — you can expand this to support custom queries. +
- +
-I follow the instructions but I hit a problem.  It want's me to select a local database file and i don't have one.  So I ask for a fix. +
- +
-> Is a file necessary?  Can I create a purely inmemory db? +
->> Yes! You can create a purely in-memory SQLite database in the browser — no file needed. Here’s a simple example using sql.js: +
-<code html> +
-<!DOCTYPE html> +
-<html lang="en"> +
-<head> +
-    <meta charset="UTF-8"> +
-    <meta name="viewport" content="width=device-width, initial-scale=1.0"> +
-    <title>In-Memory SQLite</title> +
-</head> +
-<body> +
-    <h1>SQLite In-Memory DB</h1> +
-    <button id="runQuery">Run Query</button> +
-    <pre id="output"></pre> +
- +
-    <script src="https://cdn.jsdelivr.net/npm/sql.js@1.8.0/dist/sql-wasm.js"></script> +
-    <script> +
-        (async () => { +
-            const SQL = await initSqlJs({ locateFile: file => `https://cdn.jsdelivr.net/npm/sql.js@1.8.0/dist/${file}` }); +
-            const db = new SQL.Database(); // In-memory database +
- +
-            document.getElementById('runQuery').addEventListener('click', () => { +
-                db.run("CREATE TABLE test (id INTEGER, name TEXT);"); +
-                db.run("INSERT INTO test VALUES (1, 'Alice'), (2, 'Bob');"); +
- +
-                const result = db.exec("SELECT * FROM test;"); +
-                document.getElementById('output').textContent = JSON.stringify(result, null, 2); +
-            }); +
-        })(); +
-    </script> +
-</body> +
-</html> +
-</code> +
- +
-I run this and it works, giving me the following result set in a JSON format: +
-<code javascript> +
-[ +
-  { +
-    "columns": [ +
-      "id", +
-      "name" +
-    ], +
-    "values": [ +
-      [ +
-        1, +
-        "Alice" +
-      ], +
-      [ +
-        2, +
-        "Bob" +
-      ] +
-    ] +
-  } +
-] +
-</code> +
- +
-===== Loading WASM from a Base64 encoded string ===== +
- +
-It's still loading code and WASM from a server.  I need it to be entirely offline. +
- +
-> I'd like to run this offline.  Can I download the js and wasm files and reference them locally? +
->> Yes! You can absolutely run SQLite (with sql.js) offline by downloading the necessary files. Let’s set this up step by step. +
- +
-The instructions a straight forward.  I download the file and I change the uri in the initSqlJs constructor to this: +
- +
-<code javascript> +
-// Initialize sql.js with local wasm file +
-initSqlJs({ +
-  locateFile: file => `lib/${file}` +
-}).then(SQL => { ... } +
-</code> +
- +
-This is when I run into the same CORS Policy problems that blocked me in the past: +
- +
-<code> +
-Access to fetch at 'file:///C:/Users/gedby/OneDrive/Desktop/sqljs/lib/sql-wasm.wasm' from origin 'null' has been blocked by CORS policy: Cross origin requests are only supported for protocol schemes: chrome-extension, chrome-untrusted, data, edge, http, https, isolated-app. +
- +
-Failed to load resource: net::ERR_FAILED (lib/sql-wasm.wasm:1) +
- +
-Uncaught (in promise) TypeError: Failed to fetch (sql-wasm.js:159) +
-    at sql-wasm.js:159:276 +
-    at sql-wasm.js:159:499 +
-    at sql-wasm.js:160:11 +
-    at new Promise (<anonymous>) +
-    at initSqlJs (sql-wasm.js:22:24) +
-    at Test.html:16:31 +
-    at Test.html:26:11 +
-</code> +
- +
-I tell ChatGPT about the error, and it confidently provides me with a fix. +
- +
->> Ah, yes — this happens because modern browsers block fetch requests for local files due to CORS restrictions. Even though you’re running things locally, loading ''.wasm'' files via ''%%file://%%'' doesn’t work by default. +
->> Let’s fix it! 🚀 +
- +
-It then gives me three possible options: +
-1. Run a simple local server (Recommended) +
-1. Use a VS Code extension (even simpler) +
-1. Not recommended) Disable CORS checks temporarily +
- +
-None of these really work for me.  For the WASM to work in TiddlyWiki I'll need to load the content from within the file.  This is how TiddlyWiki manages all of it's resources, including Javascript.  I wonder if this simply isn't possible with WASM. +
- +
-I explain my problem to ChatGPT, and it confidently provides the answer: +
- +
-> I don't really like any of these options.  The goal is to load the WASM from within my TiddlyWiki. +
-> Is there anyway of loading WASM from data embedded in my HTML file, perhaps BASE64 encoded? +
->> Yes! You can absolutely embed the WebAssembly (WASM) directly into your TiddlyWiki without needing any external files or servers. Let’s tackle this step by step. 🚀 +
- +
-I am loving this AI's can do attitude, so I follow the steps provided. +
- +
-First of all I convert my WASM file to Base64.  To do this I'm given this powershell one liner.: +
-<code powershell> +
-[convert]::ToBase64String([IO.File]::ReadAllBytes("sql-wasm.wasm")) | Out-File "sql-wasm.base64.txt" +
-</code> +
- +
-Then I copy the Base64 text into a string so that I can loading it into a binary array. +
- +
-One thing I like at this point is that ChatGPT doesn't generate the whole page for me.  Instead it just provides the code related to the task at hand, which I'm able to adapt into my own code base. +
- +
-<code javascript> +
-<!DOCTYPE html> +
-<html lang="en"> +
-<head> +
-  <meta charset="UTF-8"> +
-  <title>Embedded SQLite with WASM</title> +
-</head> +
-<body> +
-  <h1>SQLite with Embedded WASM</h1> +
-  <script src="https://cdnjs.cloudflare.com/ajax/libs/sql.js/1.8.0/sql-wasm.js"></script> +
-  <script> +
-    // Paste the long Base64-encoded WASM here +
-    const wasmBase64 = "BASE64_ENCODED_STRING_HERE"; +
- +
-    // Convert Base64 to binary +
-    const binaryString = atob(wasmBase64); +
-    const bytes = new Uint8Array(binaryString.length); +
-    for (let i = 0; i < binaryString.length; i++) { +
-      bytes[i] = binaryString.charCodeAt(i); +
-    } +
-    const wasmBinary = bytes.buffer; +
- +
-    // Initialize sql.js using the in-memory WASM binary +
-    initSqlJs({ +
-      locateFile: () => URL.createObjectURL(new Blob([wasmBinary], { type: "application/wasm" })) +
-    }).then(SQL => { +
-      const db = new SQL.Database(); // In-memory SQLite DB +
-      db.run("CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT);"); +
-      db.run("INSERT INTO test (name) VALUES ('Alice');"); +
- +
-      const res = db.exec("SELECT * FROM test;"); +
-      console.log("Query result:", res); +
-    }).catch(err => { +
-      console.error("Failed to initialize sql.js:", err); +
-    }); +
-  </script> +
-</body> +
-</html> +
-</code> +
- +
-There are two problems with this code. +
- +
-  - ChatGPT has decided to drop functionality from the code we are writing.  In the previous version, the code was integrated with the HTML, with the query being run when the "Run Query" button is pressed and the results written to a preformatted "output" element.  Now the code is run on load and the result is written to console.  It is this type of inconsistency that frustrates me when using ChatGPT to write code for me.  It leads to disconnected code that is hard to maintain. +
-  - It doesn't work.  I get an error: +
-<code> +
-I'm getting the error: wasm streaming compile failed: TypeError: Failed to execute 'compile' on 'WebAssembly': Incorrect response MIME type. Expected 'application/wasm'. +
-</code> +
- +
-And now we see how ChatGPT shines as a research assistant and guide, because it immediatly identified the problem and provides the fix: +
- +
->>Ah, that's happening because the WebAssembly streaming compiler expects the MIME type to be application/wasm — but when you're embedding the WASM as a Base64 string, the browser doesn't get the right content type automatically. +
->> +
->>Let’s fix this by using %%WebAssembly.compile%% with the decoded binary, bypassing streaming: +
- +
-It then gives me a full page of code, but the only part that I'm interested in is this: +
-<code> +
-// Initialize sql.js using the in-memory WASM binary +
-initSqlJs({ +
-  locateFile: () => { +
-    // Create a blob for the binary and generate a URL +
-    const blob = new Blob([bytes], { type: "application/wasm" }); +
-    return URL.createObjectURL(blob); +
-  } +
-}).then(SQL => { ... }); +
-</code> +
- +
-This is then loaded into a string and converted into a binary blob so that it can be loaded without any reference to an external WASM resource.  My final code looks like this: +
- +
-<code html> +
-<!DOCTYPE html> +
-<html lang="en"> +
-<head> +
-    <meta charset="UTF-8"> +
-    <meta name="viewport" content="width=device-width, initial-scale=1.0"> +
-    <title>In-Memory SQLite</title> +
-</head> +
-<body> +
-    <h1>SQLite In-Memory DB</h1> +
-    <button id="runQuery">Run Query</button> +
-    <pre id="output"></pre> +
- +
-    <script src="lib/sql-wasm.js"></script> +
-    <script> +
-        const base64wasm = "..."; +
-        const binaryString = atob(base64wasm); +
-        const bytes = new Uint8Array(binaryString.length); +
-        for (let i = 0; i < binaryString.length; i++) { +
-            bytes[i] = binaryString.charCodeAt(i); +
-        } +
- +
-        (async () => { +
-            // Initialize sql.js using the in-memory WASM binary +
-            const sql = await initSqlJs({ +
-                locateFile: () => { +
-                    // Create a blob for the binary and generate a URL +
-                    const blob = new Blob([bytes], { type: "application/wasm" }); +
-                    return URL.createObjectURL(blob); +
-                } +
-            }); +
- +
-            const db = new sql.Database(); // In-memory SQLite DB +
- +
-            document.getElementById('runQuery').addEventListener('click', () => { +
- +
-                db.run("CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT);"); +
-                db.run("INSERT INTO test (name) VALUES ('Alice'), ('Bob');"); +
- +
-                const result = db.exec("SELECT * FROM test;"); +
-                document.getElementById('output').textContent = JSON.stringify(result, null, 2); +
-            }); +
-        })();+
  
-    </script> 
-</body> 
-</html> 
-</code> 
  
-Which completes my proof of concept.  Chat GPT has saved me significant time researching the topics, and I feel confident in my understanding of the code that has been created.+  - [[>sqliteinthebrowserpoc|Sqlite In the Browser]]
  • reinforcedlearningwithaisupport/start.1741334183.txt.gz
  • Last modified: 2025/03/07 02:56
  • by gedbadmin