Nostr transport: - Switch from kind 1 to NIP-78 kind 30078 (application-specific data) - Relays store events persistently — no inventory protocol needed - AES-256-GCM encryption via Web Crypto API (room key = shared secret) - PBKDF2 key derivation (100k iterations) from room key - Chunking for large events (images >48KB) with per-chunk encryption - Auto re-publish missing local events on room join - Manual "Re-sync" button for recovery of failed publishes Performance (all variants): - Emit 'change' immediately after local store write (IDB/NeDB/SQLite) - Folder and Nostr writes run fire-and-forget in background - Fast event hashing: fingerprint metadata only, skip full payload SHA-256 - Saving spinner in paste UI while write completes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
160 lines
6.1 KiB
Markdown
160 lines
6.1 KiB
Markdown
# FolderSyncDB — Nostr + Folder Variant
|
|
|
|
Local-first browser key-value and document store with **dual-sync**: shared folder for local/offline use, plus Nostr relays for cross-device reach over the internet.
|
|
|
|
## Why this variant?
|
|
|
|
The other variants sync browsers via a shared folder on disk. This only works when both browsers can access the same folder (same machine or network drive).
|
|
|
|
This variant adds **Nostr relay sync** as a second transport. Nostr relays are public WebSocket servers that relay messages between clients. Your browser connects directly to them — no server of your own, no accounts, no setup.
|
|
|
|
```
|
|
Laptop (file://) Phone (browser)
|
|
┌──────────┐ ┌──────────┐
|
|
│ IndexedDB │ │ IndexedDB │
|
|
└─────┬─────┘ └─────┬─────┘
|
|
│ │
|
|
┌────┴────┐ ┌────┴────┐
|
|
│ Sync │ │ Sync │
|
|
│ Engine │ │ Engine │
|
|
└──┬───┬──┘ └──┬───┬──┘
|
|
│ │ │ │
|
|
Folder Nostr ──── wss://relay ──────── Nostr Folder
|
|
```
|
|
|
|
Use either or both:
|
|
- **Folder only** — local multi-browser sync, works offline
|
|
- **Nostr only** — cross-device sync, no folder needed
|
|
- **Both** — folder for local speed, Nostr for internet reach
|
|
|
|
## How it works
|
|
|
|
### NIP-78 application data (kind 30078)
|
|
|
|
Events are stored on relays as **NIP-78 application-specific data** (kind 30078). This is the Nostr standard for apps that use relays as a personal database. Unlike regular kind 1 notes, relays are designed to store and index kind 30078 events long-term.
|
|
|
|
Each sync event becomes one NIP-78 event:
|
|
- **`d` tag** = `foldersync:{roomKey}:{filename}` — makes each event addressable
|
|
- **`t` tag** = `foldersync-{roomKey}` — indexed by relays for room-wide queries
|
|
- **content** = AES-256-GCM encrypted JSON (see Encryption below)
|
|
|
|
### Encryption
|
|
|
|
All data is **end-to-end encrypted** using **AES-256-GCM** via the browser's native Web Crypto API.
|
|
|
|
```
|
|
Room key string
|
|
↓
|
|
PBKDF2 (100k iterations, SHA-256)
|
|
↓
|
|
AES-256-GCM key
|
|
↓
|
|
encrypt(JSON payload) → base64(iv + ciphertext)
|
|
↓
|
|
stored on Nostr relay (relay sees only opaque bytes)
|
|
```
|
|
|
|
- The room key IS the shared secret — only peers who know it can decrypt
|
|
- Each message gets a random 96-bit IV (no IV reuse)
|
|
- PBKDF2 key derivation makes brute-force expensive
|
|
- Relay operators cannot read your data
|
|
- Zero extra dependencies — uses only `crypto.subtle`
|
|
|
|
### Write flow
|
|
|
|
1. Update IndexedDB (fast, immediate)
|
|
2. Emit `change` event (UI updates instantly)
|
|
3. Write event file to folder (background, fire-and-forget)
|
|
4. Encrypt + publish to Nostr relay (background, fire-and-forget)
|
|
|
|
### Sync / join flow
|
|
|
|
1. Query relay: `{ kinds: [30078], '#t': ['foldersync-{room}'] }` → full history
|
|
2. Decrypt each event with the room key
|
|
3. Subscribe for real-time updates
|
|
4. Scan folder for new local events
|
|
5. Merge both, deduplicate by filename
|
|
6. Apply unseen events to IndexedDB
|
|
7. Bridge: folder events get published to Nostr, Nostr events get written to folder
|
|
|
|
### Large data (images)
|
|
|
|
Events larger than 48KB are automatically **chunked** into multiple NIP-78 events:
|
|
- Each chunk gets its own `d` tag: `foldersync:{room}:{filename}:chunk:{n}`
|
|
- Chunks are individually encrypted
|
|
- Reassembled on the receiving side before decryption
|
|
|
|
### Real-time push
|
|
|
|
When a Nostr subscription receives a new event, `sync()` is triggered immediately — no waiting for the polling interval.
|
|
|
|
## Quick start
|
|
|
|
```ts
|
|
import { FolderSyncDB } from './nostr/src/index.ts';
|
|
|
|
const db = await FolderSyncDB.open({
|
|
autoSyncIntervalMs: 5000,
|
|
});
|
|
|
|
// Local folder sync (optional)
|
|
await db.selectFolder();
|
|
|
|
// Cross-device sync via Nostr (encrypted)
|
|
await db.joinRoom('my-secret-room-key');
|
|
|
|
// Use normally
|
|
await db.kv.set('theme', 'dark');
|
|
const theme = await db.kv.get('theme');
|
|
|
|
// Listen for events
|
|
db.on('nostr:connected', ({ roomKey }) => console.log('joined:', roomKey));
|
|
db.on('change', (e) => console.log('changed:', e));
|
|
```
|
|
|
|
## API additions
|
|
|
|
On top of the standard FolderSyncDB API, this variant adds:
|
|
|
|
| Method | Description |
|
|
|--------|-------------|
|
|
| `joinRoom(roomKey)` | Join an encrypted Nostr sync room. All clients with the same key sync together. |
|
|
| `leaveRoom()` | Disconnect from the current room. |
|
|
| `isConnected()` | Whether a room is currently joined and relay is connected. |
|
|
| `currentRoom` | The current room key, or `null`. |
|
|
|
|
### OpenOptions additions
|
|
|
|
| Option | Type | Default | Description |
|
|
|--------|------|---------|-------------|
|
|
| `relays` | `string[]` | 3 popular public relays | Nostr relay WebSocket URLs |
|
|
| `roomKey` | `string` | none | Auto-join this room on open |
|
|
|
|
### Events additions
|
|
|
|
| Event | Payload | When |
|
|
|-------|---------|------|
|
|
| `nostr:connected` | `{ roomKey }` | Joined a Nostr room |
|
|
| `nostr:disconnected` | -- | Left a Nostr room |
|
|
|
|
## Security model
|
|
|
|
- **Room key = encryption key** — treat it like a password
|
|
- **AES-256-GCM** — authenticated encryption, tamper-proof
|
|
- **PBKDF2 key derivation** — 100k iterations slows brute-force
|
|
- **Per-message random IV** — no ciphertext patterns
|
|
- **Relay sees nothing** — tags contain the room name (which you can obscure), content is encrypted
|
|
- Each client generates its own Nostr keypair (stored in IndexedDB) for signing
|
|
|
|
## Works from `file://`
|
|
|
|
Nostr relays use WebSocket (`wss://`), which works from `file://` origins in Chrome. Unlike WebRTC or `fetch()`, browsers don't block outgoing WebSocket connections from `file://` pages.
|
|
|
|
## Dependencies
|
|
|
|
- `nostr-tools` — lightweight Nostr protocol library (keypair generation, event signing, relay pool management). Pure JS, no WASM.
|
|
|
|
## Local cache
|
|
|
|
Uses IndexedDB (same as the `indexeddb/` variant). Zero-overhead, browser-managed persistence.
|