LocalHtmlDataTest/sqlite/README.md

IndexSyncFile -- SQLite WASM Variant

Local-first key-value and document store using the official SQLite WASM build (@sqlite.org/sqlite-wasm) as the local cache, syncing with a user-selected folder via the File System Access API.

Note: This variant requires an HTTP server. It cannot run from file:// because the browser must fetch() the .wasm binary at runtime. If you need SQLite without a server, use the sql-js/ variant instead.

Why SQLite WASM

• Fastest SQLite in browser -- native WebAssembly, ~2-3x faster than the asm.js alternative
• Real SQL -- full query power with proper indexes, joins, and ACID transactions
• OPFS persistence -- via opfs-sahpool VFS, the database survives page reloads without extra code
• Atomic transactions -- document + index updates in a single SQLite transaction(), no partial writes
• Correct numeric ordering -- range queries on numbers work properly (unlike JSON-serialized keys)

Architecture

Write: app ---> SQLite in-memory/OPFS (immediate) ---> /events/timestamp_hash.json
Sync:  /events/*.json ---> sort by timestamp ---> skip applied ---> apply to SQLite

SQLite is the fast query layer. The folder's event log is the sync mechanism.

Persistence cascade

The library tries three SQLite VFS backends in order:

1. opfs-sahpool (best) -- no COOP/COEP headers needed, main-thread compatible
2. OpfsDb -- requires a Web Worker and COOP/COEP HTTP headers
3. In-memory -- fallback if OPFS is unavailable; folder sync rebuilds state on each page load

Check db.isPersistent after opening to see which mode was selected.

HTTP headers (may be required)

Some OPFS modes require these response headers on the HTML page:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

The opfs-sahpool VFS works without these headers in most cases. The library tries it first.

SQL schema (5 tables)

kv          (key TEXT PK, value TEXT, ts INTEGER, rev INTEGER)
docs        (store TEXT, id TEXT, data TEXT, ts INTEGER, rev INTEGER, deleted INTEGER, PK(store,id))
idx_entries (store TEXT, index_name TEXT, id TEXT, value ANY, PK(store,index_name,id))
applied     (filename TEXT PK, applied_at INTEGER)
meta        (key TEXT PK, value TEXT)

Install and build

npm install
npm run build

Dependency: @sqlite.org/sqlite-wasm -- the official SQLite WASM build from the SQLite project.

Usage

import { FolderSyncDB } from './dist/index.js';

const db = await FolderSyncDB.open({
  dbName: 'MyApp',
  autoSyncIntervalMs: 5000,
});

console.log('OPFS-backed:', db.isPersistent);

await db.selectFolder();

// KV
await db.kv.set('config', { theme: 'dark', lang: 'en' });

// Collections
const tasks = db.collection({
  name: 'tasks',
  indexes: [{ name: 'byPriority', fields: ['priority'] }],
});
await tasks.put({ id: 't1', title: 'Deploy', priority: 1 });

// Range queries with proper numeric ordering
const urgent = await tasks.queryByIndex('byPriority', { gte: 1, lte: 3 });

Variant-specific notes

• Cannot run from file:// -- the browser must fetch sqlite3.wasm over HTTP
• Directory handle is stored in a tiny IDB sidecar (SQLite can't store DOM objects via structured clone)
• All document data is JSON-serialized in TEXT columns
• Index values are stored with SQLite type affinity -- numbers compare as numbers
• The @sqlite.org/sqlite-wasm package uses pre-release version tags (e.g., 3.51.2-build8)
• Consumer's bundler must serve the .wasm files from the package
• For a SQLite option that works from file://, see the sql-js/ variant