diff --git a/README.md b/README.md index f6a233a..b8234f9 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,9 @@ A local-first key-value and document store for the browser that syncs across mul Can multiple browser tabs (or separate browsers entirely) stay in sync using nothing but a shared folder on the local file system? -This repo explores that question by building the same sync library four times, each with a different in-browser storage engine. The sync layer and public API are identical. Only the local cache differs. +This repo explores that question by building the same sync library five times, each with a different in-browser storage engine. The sync layer and public API are identical. Only the local cache differs. + +The fifth variant adds **Nostr relay sync** on top of folder sync, enabling cross-device synchronization (phone, laptop, etc.) over the internet with no server of our own. The goal is to compare trade-offs: bundle size, startup speed, query power, persistence behavior, and `file://` compatibility. @@ -30,6 +32,7 @@ kv.set("x", 1) ``` 1. Each browser keeps a fast local cache for reads/writes (IndexedDB, SQLite WASM, NeDB, or sql.js) + - The Nostr variant also publishes events to public relays for cross-device sync 2. Every mutation is written as an immutable JSON file in `/events/` inside a user-selected folder 3. When any browser calls `sync()`, it scans `/events/`, finds files it hasn't seen, and merges them into its local cache 4. Conflicts are resolved by last-write-wins (configurable) @@ -40,20 +43,22 @@ The event log is what makes multi-browser sync possible. Each mutation is a sepa Without events, if two browsers both wrote a single database file, the last save would silently destroy the other browser's changes. -## Four variants +## Five variants Each variant uses a different local cache engine. The API and sync behavior are identical. -| Variant | Local cache | Persistence | `file://` works? | Dependencies | -|---------|------------|-------------|-------------------|--------------| -| [`indexeddb/`](./indexeddb/) | IndexedDB | Browser-managed | Yes | Zero | -| [`nedb/`](./nedb/) | NeDB in-memory | NDJSON snapshots to `/data/` | Yes | `@seald-io/nedb` | -| [`sqlite/`](./sqlite/) | SQLite WASM | OPFS or in-memory | No (needs HTTP + headers) | `@sqlite.org/sqlite-wasm` | -| [`sql-js/`](./sql-js/) | sql.js (asm.js) | SQLite binary to `/data/store.sqlite` | Yes | `sql.js` | +| Variant | Local cache | Sync transport | `file://` works? | Dependencies | +|---------|------------|----------------|-------------------|--------------| +| [`indexeddb/`](./indexeddb/) | IndexedDB | Folder only | Yes | Zero | +| [`nedb/`](./nedb/) | NeDB in-memory | Folder + NDJSON snapshots | Yes | `@seald-io/nedb` | +| [`sqlite/`](./sqlite/) | SQLite WASM | Folder only | No (needs HTTP + headers) | `@sqlite.org/sqlite-wasm` | +| [`sql-js/`](./sql-js/) | sql.js (asm.js) | Folder + SQLite snapshot | Yes | `sql.js` | +| [`nostr/`](./nostr/) | IndexedDB | **Folder + Nostr relays** | Yes | `nostr-tools` | ### Which one should I use? - **Just want it to work** — use `indexeddb/`. Zero deps, works from `file://`, instant startup. +- **Need cross-device sync** (phone, laptop, different networks) — use `nostr/`. Folder sync for local/offline, Nostr relays for internet reach. Works from `file://`. - **Need MongoDB-style queries** (`$gt`, `$in`, `$regex`) — use `nedb/`. - **Need real SQL + proper numeric range queries** — use `sql-js/`. Works from `file://`, no special headers. - **Need OPFS-backed persistence + SQL** — use `sqlite/`. Requires HTTP server and may need COOP/COEP headers. @@ -107,7 +112,7 @@ your-selected-folder/ ## Demo app: Paste -The [`paste/`](./paste/) folder contains a working paste-bin demo app built with all four variants. Each compiles to a single self-contained HTML file that you can open directly from disk (except the SQLite WASM variant, which needs HTTP). +The [`paste/`](./paste/) folder contains a working paste-bin demo app built with all five variants. Each compiles to a single self-contained HTML file that you can open directly from disk (except the SQLite WASM variant, which needs HTTP). See the [paste README](./paste/README.md) for details. @@ -130,6 +135,7 @@ cd indexeddb && npm install && npm run build cd nedb && npm install && npm run build cd sqlite && npm install && npm run build cd sql-js && npm install && npm run build +cd nostr && npm install && npm run build ``` The paste demo uses Bun: @@ -150,7 +156,10 @@ See the README in each variant's folder for variant-specific details. | `selectFolder()` | Prompt user to pick a sync folder | | `hasFolderAccess()` | Check if folder permission is granted | | `requestFolderAccess()` | Re-request permission (needs user gesture) | -| `sync()` | Manually trigger a sync with the folder | +| `joinRoom(roomKey)` | Join a Nostr sync room *(nostr variant only)* | +| `leaveRoom()` | Leave the current Nostr room *(nostr variant only)* | +| `isConnected()` | Check Nostr relay connection *(nostr variant only)* | +| `sync()` | Manually trigger a sync (folder + Nostr if connected) | | `close()` | Stop auto-sync, flush writes, release resources | | `kv` | Key-value store (see below) | | `collection(options)` | Create/get a document collection (see below) | @@ -187,3 +196,5 @@ See the README in each variant's folder for variant-specific details. | `change` | `{ type, store, key, id, data }` | Any data mutation (local or synced) | | `conflict` | `{ filename, event }` | A conflict was detected and resolved | | `folder:lost-permission` | error? | Folder access was lost | +| `nostr:connected` | `{ roomKey }` | Joined a Nostr room *(nostr variant)* | +| `nostr:disconnected` | -- | Left a Nostr room *(nostr variant)* | diff --git a/nostr/README.md b/nostr/README.md new file mode 100644 index 0000000..7b92a6e --- /dev/null +++ b/nostr/README.md @@ -0,0 +1,118 @@ +# FolderSyncDB — Nostr + Folder Variant + +Local-first browser key-value and document store with **dual-sync**: shared folder for local/offline use, plus Nostr relays for cross-device reach over the internet. + +## Why this variant? + +The other variants sync browsers via a shared folder on disk. This only works when both browsers can access the same folder (same machine or network drive). + +This variant adds **Nostr relay sync** as a second transport. Nostr relays are public WebSocket servers that relay messages between clients. Your browser connects directly to them — no server of your own, no accounts, no setup. + +``` +Laptop (file://) Phone (browser) + ┌──────────┐ ┌──────────┐ + │ IndexedDB │ │ IndexedDB │ + └─────┬─────┘ └─────┬─────┘ + │ │ + ┌────┴────┐ ┌────┴────┐ + │ Sync │ │ Sync │ + │ Engine │ │ Engine │ + └──┬───┬──┘ └──┬───┬──┘ + │ │ │ │ + Folder Nostr ──── wss://relay ──────── Nostr Folder +``` + +Use either or both: +- **Folder only** — local multi-browser sync, works offline +- **Nostr only** — cross-device sync, no folder needed +- **Both** — folder for local speed, Nostr for internet reach + +## How it works + +**On write:** +1. Update IndexedDB (fast) +2. Write event file to folder (if folder connected) +3. Publish event to Nostr relay (if room joined) + +**On sync:** +1. Scan folder for new events from other local browsers +2. Check Nostr relay cache for events from remote devices +3. Merge both, deduplicate by filename +4. Apply unseen events to IndexedDB +5. Bridge: folder events get published to Nostr, Nostr events get written to folder + +**Real-time push:** When a Nostr subscription receives a new event, `sync()` is triggered immediately — no waiting for the polling interval. + +## Quick start + +```ts +import { FolderSyncDB } from './nostr/src/index.ts'; + +const db = await FolderSyncDB.open({ + autoSyncIntervalMs: 5000, + relays: [ // optional, defaults to popular public relays + 'wss://relay.damus.io', + 'wss://nos.lol', + 'wss://relay.nostr.band', + ], +}); + +// Local folder sync (same as other variants) +await db.selectFolder(); + +// Cross-device sync via Nostr +await db.joinRoom('my-shared-room-key'); + +// Use normally +await db.kv.set('theme', 'dark'); +const theme = await db.kv.get('theme'); + +// Listen for events +db.on('nostr:connected', ({ roomKey }) => console.log('joined:', roomKey)); +db.on('change', (e) => console.log('changed:', e)); +``` + +## API additions + +On top of the standard FolderSyncDB API, this variant adds: + +| Method | Description | +|--------|-------------| +| `joinRoom(roomKey)` | Join a Nostr sync room. All clients with the same key sync together. | +| `leaveRoom()` | Disconnect from the current room. | +| `isConnected()` | Whether a room is currently joined and relay is connected. | +| `currentRoom` | The current room key, or `null`. | + +### OpenOptions additions + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `relays` | `string[]` | 3 popular public relays | Nostr relay WebSocket URLs | +| `roomKey` | `string` | none | Auto-join this room on open | + +### Events additions + +| Event | Payload | When | +|-------|---------|------| +| `nostr:connected` | `{ roomKey }` | Joined a Nostr room | +| `nostr:disconnected` | -- | Left a Nostr room | + +## Room key + +The room key is simply a shared string that identifies your sync group. It's used as a Nostr tag — all clients subscribed to the same tag receive each other's events. + +- Anyone who knows the room key can join and sync +- Events are not encrypted (v1) — use random room keys for privacy through obscurity +- Each client generates its own Nostr keypair (stored in IndexedDB) + +## Works from `file://` + +Nostr relays use WebSocket (`wss://`), which works from `file://` origins in Chrome. Unlike WebRTC or `fetch()`, browsers don't block outgoing WebSocket connections from `file://` pages. + +## Dependencies + +- `nostr-tools` — lightweight Nostr protocol library (keypair generation, event signing, relay pool management). Pure JS, no WASM. + +## Local cache + +Uses IndexedDB (same as the `indexeddb/` variant). Zero-overhead, browser-managed persistence. diff --git a/paste/README.md b/paste/README.md index b854179..de7f7dc 100644 --- a/paste/README.md +++ b/paste/README.md @@ -1,6 +1,6 @@ # Paste -- Demo App -A minimal paste-bin app used to test and compare all four IndexSyncFile storage variants. Each version compiles to a single self-contained HTML file. +A minimal paste-bin app used to test and compare all five IndexSyncFile storage variants. Each version compiles to a single self-contained HTML file. ## What it does @@ -12,14 +12,17 @@ A minimal paste-bin app used to test and compare all four IndexSyncFile storage That's it. No server, no accounts, no network. Just two browser windows and a folder. -## Four versions +The Nostr variant adds optional **cross-device sync** over public Nostr relays via WebSocket. Enter a shared room key and devices sync over the internet too. -| File | Storage engine | Opens from `file://`? | Size | -|------|---------------|----------------------|------| -| `paste-indexeddb.html` | IndexedDB (browser built-in) | Yes | ~23 KB | -| `paste-nedb.html` | NeDB (in-memory, MongoDB-style) | Yes | ~115 KB | -| `paste-sql-js.html` | sql.js (SQLite via asm.js) | Yes | ~1.9 MB | -| `paste-sqlite.html` | SQLite WASM (official build) | No (needs HTTP) | ~1.4 MB | +## Five versions + +| File | Storage engine | Sync transport | Opens from `file://`? | Size | +|------|---------------|----------------|----------------------|------| +| `paste-indexeddb.html` | IndexedDB (browser built-in) | Folder | Yes | ~23 KB | +| `paste-nedb.html` | NeDB (in-memory, MongoDB-style) | Folder | Yes | ~115 KB | +| `paste-sql-js.html` | sql.js (SQLite via asm.js) | Folder | Yes | ~1.9 MB | +| `paste-sqlite.html` | SQLite WASM (official build) | Folder | No (needs HTTP) | ~1.4 MB | +| `paste-nostr.html` | IndexedDB | **Folder + Nostr** | Yes | ~75 KB | ### paste-indexeddb.html @@ -37,6 +40,16 @@ Uses sql.js, which is SQLite compiled to pure JavaScript (asm.js, not WebAssembl Uses the official `@sqlite.org/sqlite-wasm` package. Requires an HTTP server because the browser must `fetch()` the `.wasm` binary at runtime. May also need COOP/COEP headers for OPFS persistence. Run `bun run serve.ts` to test this variant. +### paste-nostr.html + +Uses IndexedDB for local cache (same as the indexeddb variant) plus **Nostr relay sync** for cross-device reach. Has both a folder picker and a room key input. Use either or both: + +- **Folder only** — local multi-browser sync (works offline) +- **Room only** — cross-device sync via Nostr relays (no folder needed) +- **Both** — folder for local speed + Nostr for internet reach + +WebSocket connections to public Nostr relays work from `file://` origins, so no server is needed. + ## How to test ### Quick (three variants) @@ -47,6 +60,7 @@ Double-click or drag any of these into Chrome: dist/paste-indexeddb.html dist/paste-nedb.html dist/paste-sql-js.html +dist/paste-nostr.html ``` They work directly from `file://`. @@ -77,7 +91,7 @@ Requires [Bun](https://bun.sh/) 1.3.10+. bun run build.ts ``` -This produces four single-file HTMLs in `dist/`. The build script: +This produces five single-file HTMLs in `dist/`. The build script: 1. Runs `Bun.build()` for each variant (bundles TypeScript, minifies) 2. Inlines all JS and CSS chunks into the HTML 3. Copies `sqlite3.wasm` to `dist/` for the WASM variant @@ -98,6 +112,9 @@ paste/ sqlite/ source for SQLite WASM paste app app.ts index.html + nostr/ source for Nostr + Folder paste app + app.ts + index.html shared.ts shared UI logic (all variants import this) styles.css shared styles build.ts bun build script @@ -120,4 +137,4 @@ your-folder/ store.sqlite (sql-js: SQLite binary) ``` -The `events/` folder is shared by all variants. If you point two different variant HTMLs at the same folder, they will sync with each other (the event format is identical across all four). +The `events/` folder is shared by all variants. If you point two different variant HTMLs at the same folder, they will sync with each other (the event format is identical across all five).