Jason Tudisco
6ebe02ad56
Four variants of the same sync library (IndexedDB, NeDB, SQLite WASM, sql.js) plus a paste-bin demo app for testing multi-browser sync via shared folders. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
101 lines
3.6 KiB
Markdown
101 lines
3.6 KiB
Markdown
# IndexSyncFile -- SQLite WASM Variant
|
|
|
|
Local-first key-value and document store using the **official SQLite WASM** build (`@sqlite.org/sqlite-wasm`) as the local cache, syncing with a user-selected folder via the File System Access API.
|
|
|
|
> **Note:** This variant requires an HTTP server. It cannot run from `file://` because the browser must `fetch()` the `.wasm` binary at runtime. If you need SQLite without a server, use the [`sql-js/`](../sql-js/) variant instead.
|
|
|
|
## Why SQLite WASM
|
|
|
|
- **Fastest SQLite in browser** -- native WebAssembly, ~2-3x faster than the asm.js alternative
|
|
- **Real SQL** -- full query power with proper indexes, joins, and ACID transactions
|
|
- **OPFS persistence** -- via `opfs-sahpool` VFS, the database survives page reloads without extra code
|
|
- **Atomic transactions** -- document + index updates in a single SQLite `transaction()`, no partial writes
|
|
- **Correct numeric ordering** -- range queries on numbers work properly (unlike JSON-serialized keys)
|
|
|
|
## Architecture
|
|
|
|
```
|
|
Write: app ---> SQLite in-memory/OPFS (immediate) ---> /events/timestamp_hash.json
|
|
Sync: /events/*.json ---> sort by timestamp ---> skip applied ---> apply to SQLite
|
|
```
|
|
|
|
SQLite is the fast query layer. The folder's event log is the sync mechanism.
|
|
|
|
### Persistence cascade
|
|
|
|
The library tries three SQLite VFS backends in order:
|
|
|
|
1. **opfs-sahpool** (best) -- no COOP/COEP headers needed, main-thread compatible
|
|
2. **OpfsDb** -- requires a Web Worker and COOP/COEP HTTP headers
|
|
3. **In-memory** -- fallback if OPFS is unavailable; folder sync rebuilds state on each page load
|
|
|
|
Check `db.isPersistent` after opening to see which mode was selected.
|
|
|
|
### HTTP headers (may be required)
|
|
|
|
Some OPFS modes require these response headers on the HTML page:
|
|
|
|
```
|
|
Cross-Origin-Opener-Policy: same-origin
|
|
Cross-Origin-Embedder-Policy: require-corp
|
|
```
|
|
|
|
The `opfs-sahpool` VFS works without these headers in most cases. The library tries it first.
|
|
|
|
### SQL schema (5 tables)
|
|
|
|
```sql
|
|
kv (key TEXT PK, value TEXT, ts INTEGER, rev INTEGER)
|
|
docs (store TEXT, id TEXT, data TEXT, ts INTEGER, rev INTEGER, deleted INTEGER, PK(store,id))
|
|
idx_entries (store TEXT, index_name TEXT, id TEXT, value ANY, PK(store,index_name,id))
|
|
applied (filename TEXT PK, applied_at INTEGER)
|
|
meta (key TEXT PK, value TEXT)
|
|
```
|
|
|
|
## Install and build
|
|
|
|
```bash
|
|
npm install
|
|
npm run build
|
|
```
|
|
|
|
**Dependency:** `@sqlite.org/sqlite-wasm` -- the official SQLite WASM build from the SQLite project.
|
|
|
|
## Usage
|
|
|
|
```ts
|
|
import { FolderSyncDB } from './dist/index.js';
|
|
|
|
const db = await FolderSyncDB.open({
|
|
dbName: 'MyApp',
|
|
autoSyncIntervalMs: 5000,
|
|
});
|
|
|
|
console.log('OPFS-backed:', db.isPersistent);
|
|
|
|
await db.selectFolder();
|
|
|
|
// KV
|
|
await db.kv.set('config', { theme: 'dark', lang: 'en' });
|
|
|
|
// Collections
|
|
const tasks = db.collection({
|
|
name: 'tasks',
|
|
indexes: [{ name: 'byPriority', fields: ['priority'] }],
|
|
});
|
|
await tasks.put({ id: 't1', title: 'Deploy', priority: 1 });
|
|
|
|
// Range queries with proper numeric ordering
|
|
const urgent = await tasks.queryByIndex('byPriority', { gte: 1, lte: 3 });
|
|
```
|
|
|
|
## Variant-specific notes
|
|
|
|
- **Cannot run from `file://`** -- the browser must fetch `sqlite3.wasm` over HTTP
|
|
- Directory handle is stored in a tiny IDB sidecar (SQLite can't store DOM objects via structured clone)
|
|
- All document data is JSON-serialized in TEXT columns
|
|
- Index values are stored with SQLite type affinity -- numbers compare as numbers
|
|
- The `@sqlite.org/sqlite-wasm` package uses pre-release version tags (e.g., `3.51.2-build8`)
|
|
- Consumer's bundler must serve the `.wasm` files from the package
|
|
- For a SQLite option that works from `file://`, see the [`sql-js/`](../sql-js/) variant
|