Jason Tudisco 7077191c0c NIP-78 encrypted transport, instant UI, re-sync recovery
Nostr transport:
- Switch from kind 1 to NIP-78 kind 30078 (application-specific data)
- Relays store events persistently — no inventory protocol needed
- AES-256-GCM encryption via Web Crypto API (room key = shared secret)
- PBKDF2 key derivation (100k iterations) from room key
- Chunking for large events (images >48KB) with per-chunk encryption
- Auto re-publish missing local events on room join
- Manual "Re-sync" button for recovery of failed publishes

Performance (all variants):
- Emit 'change' immediately after local store write (IDB/NeDB/SQLite)
- Folder and Nostr writes run fire-and-forget in background
- Fast event hashing: fingerprint metadata only, skip full payload SHA-256
- Saving spinner in paste UI while write completes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 01:11:29 -06:00

160 lines
6.1 KiB
Markdown

# FolderSyncDB — Nostr + Folder Variant
Local-first browser key-value and document store with **dual-sync**: shared folder for local/offline use, plus Nostr relays for cross-device reach over the internet.
## Why this variant?
The other variants sync browsers via a shared folder on disk. This only works when both browsers can access the same folder (same machine or network drive).
This variant adds **Nostr relay sync** as a second transport. Nostr relays are public WebSocket servers that relay messages between clients. Your browser connects directly to them — no server of your own, no accounts, no setup.
```
Laptop (file://) Phone (browser)
┌──────────┐ ┌──────────┐
│ IndexedDB │ │ IndexedDB │
└─────┬─────┘ └─────┬─────┘
│ │
┌────┴────┐ ┌────┴────┐
│ Sync │ │ Sync │
│ Engine │ │ Engine │
└──┬───┬──┘ └──┬───┬──┘
│ │ │ │
Folder Nostr ──── wss://relay ──────── Nostr Folder
```
Use either or both:
- **Folder only** — local multi-browser sync, works offline
- **Nostr only** — cross-device sync, no folder needed
- **Both** — folder for local speed, Nostr for internet reach
## How it works
### NIP-78 application data (kind 30078)
Events are stored on relays as **NIP-78 application-specific data** (kind 30078). This is the Nostr standard for apps that use relays as a personal database. Unlike regular kind 1 notes, relays are designed to store and index kind 30078 events long-term.
Each sync event becomes one NIP-78 event:
- **`d` tag** = `foldersync:{roomKey}:{filename}` — makes each event addressable
- **`t` tag** = `foldersync-{roomKey}` — indexed by relays for room-wide queries
- **content** = AES-256-GCM encrypted JSON (see Encryption below)
### Encryption
All data is **end-to-end encrypted** using **AES-256-GCM** via the browser's native Web Crypto API.
```
Room key string
PBKDF2 (100k iterations, SHA-256)
AES-256-GCM key
encrypt(JSON payload) → base64(iv + ciphertext)
stored on Nostr relay (relay sees only opaque bytes)
```
- The room key IS the shared secret — only peers who know it can decrypt
- Each message gets a random 96-bit IV (no IV reuse)
- PBKDF2 key derivation makes brute-force expensive
- Relay operators cannot read your data
- Zero extra dependencies — uses only `crypto.subtle`
### Write flow
1. Update IndexedDB (fast, immediate)
2. Emit `change` event (UI updates instantly)
3. Write event file to folder (background, fire-and-forget)
4. Encrypt + publish to Nostr relay (background, fire-and-forget)
### Sync / join flow
1. Query relay: `{ kinds: [30078], '#t': ['foldersync-{room}'] }` → full history
2. Decrypt each event with the room key
3. Subscribe for real-time updates
4. Scan folder for new local events
5. Merge both, deduplicate by filename
6. Apply unseen events to IndexedDB
7. Bridge: folder events get published to Nostr, Nostr events get written to folder
### Large data (images)
Events larger than 48KB are automatically **chunked** into multiple NIP-78 events:
- Each chunk gets its own `d` tag: `foldersync:{room}:{filename}:chunk:{n}`
- Chunks are individually encrypted
- Reassembled on the receiving side before decryption
### Real-time push
When a Nostr subscription receives a new event, `sync()` is triggered immediately — no waiting for the polling interval.
## Quick start
```ts
import { FolderSyncDB } from './nostr/src/index.ts';
const db = await FolderSyncDB.open({
autoSyncIntervalMs: 5000,
});
// Local folder sync (optional)
await db.selectFolder();
// Cross-device sync via Nostr (encrypted)
await db.joinRoom('my-secret-room-key');
// Use normally
await db.kv.set('theme', 'dark');
const theme = await db.kv.get('theme');
// Listen for events
db.on('nostr:connected', ({ roomKey }) => console.log('joined:', roomKey));
db.on('change', (e) => console.log('changed:', e));
```
## API additions
On top of the standard FolderSyncDB API, this variant adds:
| Method | Description |
|--------|-------------|
| `joinRoom(roomKey)` | Join an encrypted Nostr sync room. All clients with the same key sync together. |
| `leaveRoom()` | Disconnect from the current room. |
| `isConnected()` | Whether a room is currently joined and relay is connected. |
| `currentRoom` | The current room key, or `null`. |
### OpenOptions additions
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `relays` | `string[]` | 3 popular public relays | Nostr relay WebSocket URLs |
| `roomKey` | `string` | none | Auto-join this room on open |
### Events additions
| Event | Payload | When |
|-------|---------|------|
| `nostr:connected` | `{ roomKey }` | Joined a Nostr room |
| `nostr:disconnected` | -- | Left a Nostr room |
## Security model
- **Room key = encryption key** — treat it like a password
- **AES-256-GCM** — authenticated encryption, tamper-proof
- **PBKDF2 key derivation** — 100k iterations slows brute-force
- **Per-message random IV** — no ciphertext patterns
- **Relay sees nothing** — tags contain the room name (which you can obscure), content is encrypted
- Each client generates its own Nostr keypair (stored in IndexedDB) for signing
## Works from `file://`
Nostr relays use WebSocket (`wss://`), which works from `file://` origins in Chrome. Unlike WebRTC or `fetch()`, browsers don't block outgoing WebSocket connections from `file://` pages.
## Dependencies
- `nostr-tools` — lightweight Nostr protocol library (keypair generation, event signing, relay pool management). Pure JS, no WASM.
## Local cache
Uses IndexedDB (same as the `indexeddb/` variant). Zero-overhead, browser-managed persistence.