Compare commits

..

No commits in common. "63ce1051147b911cc31479f313fc8230c138cb85" and "6ebe02ad56a8deeb30a11c33f4284bc8834449f8" have entirely different histories.

20 changed files with 21 additions and 1907 deletions

View File

@ -8,9 +8,7 @@ A local-first key-value and document store for the browser that syncs across mul
Can multiple browser tabs (or separate browsers entirely) stay in sync using nothing but a shared folder on the local file system? Can multiple browser tabs (or separate browsers entirely) stay in sync using nothing but a shared folder on the local file system?
This repo explores that question by building the same sync library five times, each with a different in-browser storage engine. The sync layer and public API are identical. Only the local cache differs. This repo explores that question by building the same sync library four times, each with a different in-browser storage engine. The sync layer and public API are identical. Only the local cache differs.
The fifth variant adds **Nostr relay sync** on top of folder sync, enabling cross-device synchronization (phone, laptop, etc.) over the internet with no server of our own.
The goal is to compare trade-offs: bundle size, startup speed, query power, persistence behavior, and `file://` compatibility. The goal is to compare trade-offs: bundle size, startup speed, query power, persistence behavior, and `file://` compatibility.
@ -32,7 +30,6 @@ kv.set("x", 1)
``` ```
1. Each browser keeps a fast local cache for reads/writes (IndexedDB, SQLite WASM, NeDB, or sql.js) 1. Each browser keeps a fast local cache for reads/writes (IndexedDB, SQLite WASM, NeDB, or sql.js)
- The Nostr variant also publishes events to public relays for cross-device sync
2. Every mutation is written as an immutable JSON file in `/events/` inside a user-selected folder 2. Every mutation is written as an immutable JSON file in `/events/` inside a user-selected folder
3. When any browser calls `sync()`, it scans `/events/`, finds files it hasn't seen, and merges them into its local cache 3. When any browser calls `sync()`, it scans `/events/`, finds files it hasn't seen, and merges them into its local cache
4. Conflicts are resolved by last-write-wins (configurable) 4. Conflicts are resolved by last-write-wins (configurable)
@ -43,22 +40,20 @@ The event log is what makes multi-browser sync possible. Each mutation is a sepa
Without events, if two browsers both wrote a single database file, the last save would silently destroy the other browser's changes. Without events, if two browsers both wrote a single database file, the last save would silently destroy the other browser's changes.
## Five variants ## Four variants
Each variant uses a different local cache engine. The API and sync behavior are identical. Each variant uses a different local cache engine. The API and sync behavior are identical.
| Variant | Local cache | Sync transport | `file://` works? | Dependencies | | Variant | Local cache | Persistence | `file://` works? | Dependencies |
|---------|------------|----------------|-------------------|--------------| |---------|------------|-------------|-------------------|--------------|
| [`indexeddb/`](./indexeddb/) | IndexedDB | Folder only | Yes | Zero | | [`indexeddb/`](./indexeddb/) | IndexedDB | Browser-managed | Yes | Zero |
| [`nedb/`](./nedb/) | NeDB in-memory | Folder + NDJSON snapshots | Yes | `@seald-io/nedb` | | [`nedb/`](./nedb/) | NeDB in-memory | NDJSON snapshots to `/data/` | Yes | `@seald-io/nedb` |
| [`sqlite/`](./sqlite/) | SQLite WASM | Folder only | No (needs HTTP + headers) | `@sqlite.org/sqlite-wasm` | | [`sqlite/`](./sqlite/) | SQLite WASM | OPFS or in-memory | No (needs HTTP + headers) | `@sqlite.org/sqlite-wasm` |
| [`sql-js/`](./sql-js/) | sql.js (asm.js) | Folder + SQLite snapshot | Yes | `sql.js` | | [`sql-js/`](./sql-js/) | sql.js (asm.js) | SQLite binary to `/data/store.sqlite` | Yes | `sql.js` |
| [`nostr/`](./nostr/) | IndexedDB | **Folder + Nostr relays** | Yes | `nostr-tools` |
### Which one should I use? ### Which one should I use?
- **Just want it to work** — use `indexeddb/`. Zero deps, works from `file://`, instant startup. - **Just want it to work** — use `indexeddb/`. Zero deps, works from `file://`, instant startup.
- **Need cross-device sync** (phone, laptop, different networks) — use `nostr/`. Folder sync for local/offline, Nostr relays for internet reach. Works from `file://`.
- **Need MongoDB-style queries** (`$gt`, `$in`, `$regex`) — use `nedb/`. - **Need MongoDB-style queries** (`$gt`, `$in`, `$regex`) — use `nedb/`.
- **Need real SQL + proper numeric range queries** — use `sql-js/`. Works from `file://`, no special headers. - **Need real SQL + proper numeric range queries** — use `sql-js/`. Works from `file://`, no special headers.
- **Need OPFS-backed persistence + SQL** — use `sqlite/`. Requires HTTP server and may need COOP/COEP headers. - **Need OPFS-backed persistence + SQL** — use `sqlite/`. Requires HTTP server and may need COOP/COEP headers.
@ -112,7 +107,7 @@ your-selected-folder/
## Demo app: Paste ## Demo app: Paste
The [`paste/`](./paste/) folder contains a working paste-bin demo app built with all five variants. Each compiles to a single self-contained HTML file that you can open directly from disk (except the SQLite WASM variant, which needs HTTP). The [`paste/`](./paste/) folder contains a working paste-bin demo app built with all four variants. Each compiles to a single self-contained HTML file that you can open directly from disk (except the SQLite WASM variant, which needs HTTP).
See the [paste README](./paste/README.md) for details. See the [paste README](./paste/README.md) for details.
@ -135,7 +130,6 @@ cd indexeddb && npm install && npm run build
cd nedb && npm install && npm run build cd nedb && npm install && npm run build
cd sqlite && npm install && npm run build cd sqlite && npm install && npm run build
cd sql-js && npm install && npm run build cd sql-js && npm install && npm run build
cd nostr && npm install && npm run build
``` ```
The paste demo uses Bun: The paste demo uses Bun:
@ -156,10 +150,7 @@ See the README in each variant's folder for variant-specific details.
| `selectFolder()` | Prompt user to pick a sync folder | | `selectFolder()` | Prompt user to pick a sync folder |
| `hasFolderAccess()` | Check if folder permission is granted | | `hasFolderAccess()` | Check if folder permission is granted |
| `requestFolderAccess()` | Re-request permission (needs user gesture) | | `requestFolderAccess()` | Re-request permission (needs user gesture) |
| `joinRoom(roomKey)` | Join a Nostr sync room *(nostr variant only)* | | `sync()` | Manually trigger a sync with the folder |
| `leaveRoom()` | Leave the current Nostr room *(nostr variant only)* |
| `isConnected()` | Check Nostr relay connection *(nostr variant only)* |
| `sync()` | Manually trigger a sync (folder + Nostr if connected) |
| `close()` | Stop auto-sync, flush writes, release resources | | `close()` | Stop auto-sync, flush writes, release resources |
| `kv` | Key-value store (see below) | | `kv` | Key-value store (see below) |
| `collection(options)` | Create/get a document collection (see below) | | `collection(options)` | Create/get a document collection (see below) |
@ -196,5 +187,3 @@ See the README in each variant's folder for variant-specific details.
| `change` | `{ type, store, key, id, data }` | Any data mutation (local or synced) | | `change` | `{ type, store, key, id, data }` | Any data mutation (local or synced) |
| `conflict` | `{ filename, event }` | A conflict was detected and resolved | | `conflict` | `{ filename, event }` | A conflict was detected and resolved |
| `folder:lost-permission` | error? | Folder access was lost | | `folder:lost-permission` | error? | Folder access was lost |
| `nostr:connected` | `{ roomKey }` | Joined a Nostr room *(nostr variant)* |
| `nostr:disconnected` | -- | Left a Nostr room *(nostr variant)* |

View File

@ -1,118 +0,0 @@
# FolderSyncDB — Nostr + Folder Variant
Local-first browser key-value and document store with **dual-sync**: shared folder for local/offline use, plus Nostr relays for cross-device reach over the internet.
## Why this variant?
The other variants sync browsers via a shared folder on disk. This only works when both browsers can access the same folder (same machine or network drive).
This variant adds **Nostr relay sync** as a second transport. Nostr relays are public WebSocket servers that relay messages between clients. Your browser connects directly to them — no server of your own, no accounts, no setup.
```
Laptop (file://) Phone (browser)
┌──────────┐ ┌──────────┐
│ IndexedDB │ │ IndexedDB │
└─────┬─────┘ └─────┬─────┘
│ │
┌────┴────┐ ┌────┴────┐
│ Sync │ │ Sync │
│ Engine │ │ Engine │
└──┬───┬──┘ └──┬───┬──┘
│ │ │ │
Folder Nostr ──── wss://relay ──────── Nostr Folder
```
Use either or both:
- **Folder only** — local multi-browser sync, works offline
- **Nostr only** — cross-device sync, no folder needed
- **Both** — folder for local speed, Nostr for internet reach
## How it works
**On write:**
1. Update IndexedDB (fast)
2. Write event file to folder (if folder connected)
3. Publish event to Nostr relay (if room joined)
**On sync:**
1. Scan folder for new events from other local browsers
2. Check Nostr relay cache for events from remote devices
3. Merge both, deduplicate by filename
4. Apply unseen events to IndexedDB
5. Bridge: folder events get published to Nostr, Nostr events get written to folder
**Real-time push:** When a Nostr subscription receives a new event, `sync()` is triggered immediately — no waiting for the polling interval.
## Quick start
```ts
import { FolderSyncDB } from './nostr/src/index.ts';
const db = await FolderSyncDB.open({
autoSyncIntervalMs: 5000,
relays: [ // optional, defaults to popular public relays
'wss://relay.damus.io',
'wss://nos.lol',
'wss://relay.nostr.band',
],
});
// Local folder sync (same as other variants)
await db.selectFolder();
// Cross-device sync via Nostr
await db.joinRoom('my-shared-room-key');
// Use normally
await db.kv.set('theme', 'dark');
const theme = await db.kv.get('theme');
// Listen for events
db.on('nostr:connected', ({ roomKey }) => console.log('joined:', roomKey));
db.on('change', (e) => console.log('changed:', e));
```
## API additions
On top of the standard FolderSyncDB API, this variant adds:
| Method | Description |
|--------|-------------|
| `joinRoom(roomKey)` | Join a Nostr sync room. All clients with the same key sync together. |
| `leaveRoom()` | Disconnect from the current room. |
| `isConnected()` | Whether a room is currently joined and relay is connected. |
| `currentRoom` | The current room key, or `null`. |
### OpenOptions additions
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `relays` | `string[]` | 3 popular public relays | Nostr relay WebSocket URLs |
| `roomKey` | `string` | none | Auto-join this room on open |
### Events additions
| Event | Payload | When |
|-------|---------|------|
| `nostr:connected` | `{ roomKey }` | Joined a Nostr room |
| `nostr:disconnected` | -- | Left a Nostr room |
## Room key
The room key is simply a shared string that identifies your sync group. It's used as a Nostr tag — all clients subscribed to the same tag receive each other's events.
- Anyone who knows the room key can join and sync
- Events are not encrypted (v1) — use random room keys for privacy through obscurity
- Each client generates its own Nostr keypair (stored in IndexedDB)
## Works from `file://`
Nostr relays use WebSocket (`wss://`), which works from `file://` origins in Chrome. Unlike WebRTC or `fetch()`, browsers don't block outgoing WebSocket connections from `file://` pages.
## Dependencies
- `nostr-tools` — lightweight Nostr protocol library (keypair generation, event signing, relay pool management). Pure JS, no WASM.
## Local cache
Uses IndexedDB (same as the `indexeddb/` variant). Zero-overhead, browser-managed persistence.

View File

@ -1,13 +0,0 @@
{
"name": "@foldersync/nostr",
"version": "0.1.0",
"description": "Local-first browser KV/document store with dual folder + Nostr relay sync",
"type": "module",
"main": "src/index.ts",
"dependencies": {
"nostr-tools": "^2.10.4"
},
"devDependencies": {
"typescript": "^5.7.0"
}
}

View File

@ -1,137 +0,0 @@
import type { CollectionApi, IndexDefinition, StoreOptions } from './types.js';
import type { IDBStore } from './idb-store.js';
import type { SyncEngine } from './sync-engine.js';
export class Collection<T extends { id: string }> implements CollectionApi<T> {
private readonly storeName: string;
private readonly indexes: IndexDefinition<T>[];
private indexesBuilt = false;
constructor(
options: StoreOptions<T>,
private idb: IDBStore,
private sync: SyncEngine,
) {
this.storeName = options.name;
this.indexes = options.indexes ?? [];
this.sync.registerIndexes(this.storeName, this.indexes as IndexDefinition[]);
}
/** Lazily rebuild indexes on first access. */
private async ensureIndexes(): Promise<void> {
if (this.indexesBuilt || this.indexes.length === 0) return;
await this.idb.rebuildIndexes(
this.storeName,
this.indexes as IndexDefinition[],
);
this.indexesBuilt = true;
}
async get(id: string): Promise<T | undefined> {
const rec = await this.idb.getDoc(this.storeName, id);
if (!rec) return undefined;
return rec.data as T;
}
async put(doc: T): Promise<void> {
await this.ensureIndexes();
const existing = await this.idb.getRawDoc(this.storeName, doc.id);
const rev = (existing?.rev ?? 0) + 1;
const ts = Date.now();
// SyncEngine.persistEvent handles the IDB write + folder write
await this.sync.writeDoc(this.storeName, doc.id, doc, ts, rev);
}
async delete(id: string): Promise<void> {
await this.ensureIndexes();
const existing = await this.idb.getRawDoc(this.storeName, id);
if (!existing || existing.deleted) return;
const rev = existing.rev + 1;
const ts = Date.now();
await this.sync.deleteDocEvent(this.storeName, id, ts, rev);
}
async all(): Promise<T[]> {
const docs = await this.idb.getAllDocs(this.storeName);
return docs.map((d) => d.data as T);
}
async findByIndex(indexName: string, value: IDBValidKey): Promise<T[]> {
await this.ensureIndexes();
this.assertIndex(indexName);
const ids = await this.idb.findByIndex(this.storeName, indexName, value);
return this.fetchByIds(ids);
}
async queryByIndex(
indexName: string,
range: {
gt?: IDBValidKey;
gte?: IDBValidKey;
lt?: IDBValidKey;
lte?: IDBValidKey;
},
): Promise<T[]> {
await this.ensureIndexes();
this.assertIndex(indexName);
const lower = range.gte ?? range.gt;
const upper = range.lte ?? range.lt;
const lowerOpen = range.gte === undefined && range.gt !== undefined;
const upperOpen = range.lte === undefined && range.lt !== undefined;
let idbRange: IDBKeyRange;
if (lower !== undefined && upper !== undefined) {
idbRange = IDBKeyRange.bound(
[this.storeName, indexName, lower],
[this.storeName, indexName, upper],
lowerOpen,
upperOpen,
);
} else if (lower !== undefined) {
idbRange = IDBKeyRange.lowerBound(
[this.storeName, indexName, lower],
lowerOpen,
);
} else if (upper !== undefined) {
idbRange = IDBKeyRange.upperBound(
[this.storeName, indexName, upper],
upperOpen,
);
} else {
// No bounds: return all entries for this index
idbRange = IDBKeyRange.bound(
[this.storeName, indexName, -Infinity],
[this.storeName, indexName, []],
);
}
const ids = await this.idb.queryByIndex(
this.storeName,
indexName,
idbRange,
);
return this.fetchByIds(ids);
}
// ── Helpers ────────────────────────────────────────────────
private assertIndex(name: string): void {
if (!this.indexes.some((i) => i.name === name)) {
throw new Error(
`Index "${name}" is not defined on collection "${this.storeName}". ` +
`Defined indexes: ${this.indexes.map((i) => i.name).join(', ') || '(none)'}`,
);
}
}
private async fetchByIds(ids: string[]): Promise<T[]> {
const results: T[] = [];
for (const id of ids) {
const doc = await this.idb.getDoc(this.storeName, id);
if (doc) results.push(doc.data as T);
}
return results;
}
}

View File

@ -1,33 +0,0 @@
import type { SyncDBEventName, SyncDBEventHandler } from './types.js';
export class Emitter {
private listeners = new Map<SyncDBEventName, Set<SyncDBEventHandler>>();
on(event: SyncDBEventName, handler: SyncDBEventHandler): () => void {
let set = this.listeners.get(event);
if (!set) {
set = new Set();
this.listeners.set(event, set);
}
set.add(handler);
return () => {
set!.delete(handler);
};
}
emit(event: SyncDBEventName, ...args: unknown[]): void {
const set = this.listeners.get(event);
if (!set) return;
for (const fn of set) {
try {
fn(...args);
} catch {
// listener errors must not break the library
}
}
}
removeAll(): void {
this.listeners.clear();
}
}

View File

@ -1,122 +0,0 @@
import type { SyncEvent } from './types.js';
import { parseEventFilename } from './utils.js';
const EVENTS_DIR = 'events';
export class FolderStore {
private dirHandle: FileSystemDirectoryHandle | null = null;
// ── Handle management ──────────────────────────────────────
get hasHandle(): boolean {
return this.dirHandle !== null;
}
getHandle(): FileSystemDirectoryHandle | null {
return this.dirHandle;
}
setHandle(handle: FileSystemDirectoryHandle): void {
this.dirHandle = handle;
}
async selectFolder(): Promise<FileSystemDirectoryHandle> {
if (typeof window.showDirectoryPicker !== 'function') {
throw new Error(
'File System Access API is not supported in this browser. ' +
'Use a Chromium-based browser (Chrome, Edge, Brave, etc.).',
);
}
this.dirHandle = await window.showDirectoryPicker({
mode: 'readwrite',
});
// Ensure events directory exists
await this.dirHandle.getDirectoryHandle(EVENTS_DIR, { create: true });
return this.dirHandle;
}
// ── Permission checks ─────────────────────────────────────
async hasPermission(): Promise<boolean> {
if (!this.dirHandle) return false;
try {
const perm = await this.dirHandle.queryPermission({ mode: 'readwrite' });
return perm === 'granted';
} catch {
return false;
}
}
async requestPermission(): Promise<boolean> {
if (!this.dirHandle) return false;
try {
const perm = await this.dirHandle.requestPermission({
mode: 'readwrite',
});
return perm === 'granted';
} catch {
return false;
}
}
// ── Write event file ──────────────────────────────────────
async writeEvent(filename: string, event: SyncEvent): Promise<void> {
const eventsDir = await this.getEventsDir();
const fileHandle = await eventsDir.getFileHandle(filename, {
create: true,
});
const writable = await fileHandle.createWritable();
try {
await writable.write(JSON.stringify(event, null, 2));
} finally {
await writable.close();
}
}
// ── Scan events ────────────────────────────────────────────
async scanEventFilenames(): Promise<string[]> {
const eventsDir = await this.getEventsDir();
const names: string[] = [];
for await (const [name, handle] of eventsDir.entries()) {
if (handle.kind === 'file' && name.endsWith('.json')) {
if (parseEventFilename(name)) {
names.push(name);
}
}
}
// Sort by (timestamp, full filename) for deterministic ordering
names.sort((a, b) => {
const pa = parseEventFilename(a)!;
const pb = parseEventFilename(b)!;
if (pa.ts !== pb.ts) return pa.ts - pb.ts;
return a.localeCompare(b);
});
return names;
}
async readEventFile(filename: string): Promise<SyncEvent | null> {
const eventsDir = await this.getEventsDir();
try {
const fh = await eventsDir.getFileHandle(filename);
const file = await fh.getFile();
const text = await file.text();
return JSON.parse(text) as SyncEvent;
} catch {
return null;
}
}
// ── Internals ──────────────────────────────────────────────
private async getEventsDir(): Promise<FileSystemDirectoryHandle> {
if (!this.dirHandle) {
throw new Error('No folder selected. Call selectFolder() first.');
}
return this.dirHandle.getDirectoryHandle(EVENTS_DIR, { create: true });
}
}

View File

@ -1,210 +0,0 @@
import type {
OpenOptions, StoreOptions, CollectionApi, KVApi,
SyncDBEventName, SyncDBEventHandler,
} from './types.js';
import { generateClientId } from './utils.js';
import { Emitter } from './emitter.js';
import { IDBStore } from './idb-store.js';
import { FolderStore } from './folder-store.js';
import { NostrTransport } from './nostr-transport.js';
import { SyncEngine } from './sync-engine.js';
import { KVStore } from './kv-store.js';
import { Collection } from './collection.js';
const META_CLIENT_ID = 'clientId';
const META_DIR_HANDLE = 'dirHandle';
const META_NOSTR_SK = 'nostrSecretKey';
const META_NOSTR_ROOM = 'nostrRoom';
const DEFAULT_RELAYS = [
'wss://relay.damus.io',
'wss://nos.lol',
'wss://relay.nostr.band',
];
export class FolderSyncDB {
private idb!: IDBStore;
private folderStore!: FolderStore;
private nostrTransport!: NostrTransport;
private syncEngine!: SyncEngine;
private emitter!: Emitter;
private autoSyncTimer: ReturnType<typeof setInterval> | null = null;
private collections = new Map<string, Collection<any>>();
/** Public KV API — available immediately after open(). */
kv!: KVApi;
// ── Construction (use static open()) ───────────────────────
private constructor() {}
static async open(options?: OpenOptions): Promise<FolderSyncDB> {
const db = new FolderSyncDB();
await db.init(options ?? {});
return db;
}
private async init(opts: OpenOptions): Promise<void> {
const dbName = opts.dbName ?? 'FolderSyncDB';
this.emitter = new Emitter();
this.idb = await IDBStore.open(dbName);
this.folderStore = new FolderStore();
// ── Nostr transport setup ────────────────────────────────
const relays = opts.relays ?? DEFAULT_RELAYS;
this.nostrTransport = new NostrTransport(relays);
// Load or generate Nostr keypair
let skHex = await this.idb.getMeta<string>(META_NOSTR_SK);
if (skHex) {
const sk = hexToBytes(skHex);
this.nostrTransport.setKeypair(sk);
} else {
const sk = this.nostrTransport.generateKeypair();
await this.idb.setMeta(META_NOSTR_SK, bytesToHex(sk));
}
// ── Client ID ────────────────────────────────────────────
let clientId = opts.clientId;
if (!clientId) {
clientId = await this.idb.getMeta<string>(META_CLIENT_ID);
if (!clientId) {
clientId = generateClientId();
await this.idb.setMeta(META_CLIENT_ID, clientId);
}
} else {
await this.idb.setMeta(META_CLIENT_ID, clientId);
}
// ── Sync engine (dual transport) ─────────────────────────
this.syncEngine = new SyncEngine(
this.idb, this.folderStore, this.nostrTransport,
this.emitter, clientId, opts.conflictResolver,
);
this.kv = new KVStore(this.idb, this.syncEngine);
// ── Restore folder handle ────────────────────────────────
await this.tryRestoreHandle();
// ── Restore Nostr room ───────────────────────────────────
const savedRoom = opts.roomKey ?? await this.idb.getMeta<string>(META_NOSTR_ROOM);
if (savedRoom) {
await this.joinRoom(savedRoom);
}
// ── Real-time push from Nostr triggers immediate sync ────
this.nostrTransport.onNewEvent(() => {
this.sync().catch(() => {});
});
// ── Auto-sync ────────────────────────────────────────────
if (opts.autoSyncIntervalMs && opts.autoSyncIntervalMs > 0) {
this.autoSyncTimer = setInterval(() => {
this.sync().catch(() => {});
}, opts.autoSyncIntervalMs);
}
}
// ── Folder management (same as indexeddb variant) ──────────
async selectFolder(): Promise<void> {
const handle = await this.folderStore.selectFolder();
await this.idb.setMeta(META_DIR_HANDLE, handle);
await this.sync();
}
async hasFolderAccess(): Promise<boolean> {
return this.folderStore.hasPermission();
}
async requestFolderAccess(): Promise<boolean> {
const granted = await this.folderStore.requestPermission();
if (granted) await this.sync();
return granted;
}
// ── Nostr room management (NEW) ────────────────────────────
async joinRoom(roomKey: string): Promise<void> {
await this.nostrTransport.joinRoom(roomKey);
await this.idb.setMeta(META_NOSTR_ROOM, roomKey);
this.emitter.emit('nostr:connected', { roomKey });
await this.sync();
}
leaveRoom(): void {
this.nostrTransport.leaveRoom();
this.emitter.emit('nostr:disconnected');
}
isConnected(): boolean {
return this.nostrTransport.isConnected;
}
get currentRoom(): string | null {
return this.nostrTransport.currentRoom;
}
// ── Sync (folder + Nostr) ──────────────────────────────────
async sync(): Promise<void> {
return this.syncEngine.sync();
}
// ── Collections ────────────────────────────────────────────
collection<T extends { id: string }>(options: StoreOptions<T>): CollectionApi<T> {
const cached = this.collections.get(options.name);
if (cached) return cached as Collection<T>;
const col = new Collection<T>(options, this.idb, this.syncEngine);
this.collections.set(options.name, col);
return col;
}
// ── Events ─────────────────────────────────────────────────
on(event: SyncDBEventName, handler: SyncDBEventHandler): () => void {
return this.emitter.on(event, handler);
}
// ── Lifecycle ──────────────────────────────────────────────
async close(): Promise<void> {
if (this.autoSyncTimer !== null) {
clearInterval(this.autoSyncTimer);
this.autoSyncTimer = null;
}
this.nostrTransport.close();
this.emitter.removeAll();
this.collections.clear();
this.idb.close();
}
// ── Internals ──────────────────────────────────────────────
private async tryRestoreHandle(): Promise<void> {
try {
const handle = await this.idb.getMeta<FileSystemDirectoryHandle>(META_DIR_HANDLE);
if (!handle) return;
if (typeof handle.queryPermission !== 'function') return;
this.folderStore.setHandle(handle);
} catch {
// Handle was not restorable
}
}
}
// ── Hex helpers ──────────────────────────────────────────────
function bytesToHex(bytes: Uint8Array): string {
return Array.from(bytes).map(b => b.toString(16).padStart(2, '0')).join('');
}
function hexToBytes(hex: string): Uint8Array {
const bytes = new Uint8Array(hex.length / 2);
for (let i = 0; i < hex.length; i += 2) {
bytes[i / 2] = parseInt(hex.substring(i, i + 2), 16);
}
return bytes;
}

View File

@ -1,88 +0,0 @@
/**
* Type declarations for the File System Access API.
* These APIs are available in Chromium-based browsers only.
* @see https://developer.mozilla.org/en-US/docs/Web/API/File_System_Access_API
*/
interface FileSystemHandlePermissionDescriptor {
mode?: 'read' | 'readwrite';
}
interface FileSystemDirectoryHandle {
readonly kind: 'directory';
readonly name: string;
getDirectoryHandle(
name: string,
options?: { create?: boolean },
): Promise<FileSystemDirectoryHandle>;
getFileHandle(
name: string,
options?: { create?: boolean },
): Promise<FileSystemFileHandle>;
entries(): AsyncIterableIterator<
[string, FileSystemDirectoryHandle | FileSystemFileHandle]
>;
values(): AsyncIterableIterator<
FileSystemDirectoryHandle | FileSystemFileHandle
>;
keys(): AsyncIterableIterator<string>;
queryPermission(
descriptor?: FileSystemHandlePermissionDescriptor,
): Promise<PermissionState>;
requestPermission(
descriptor?: FileSystemHandlePermissionDescriptor,
): Promise<PermissionState>;
}
interface FileSystemFileHandle {
readonly kind: 'file';
readonly name: string;
getFile(): Promise<File>;
createWritable(
options?: FileSystemCreateWritableOptions,
): Promise<FileSystemWritableFileStream>;
}
interface FileSystemCreateWritableOptions {
keepExistingData?: boolean;
}
interface FileSystemWritableFileStream extends WritableStream {
write(data: BufferSource | Blob | string | WriteParams): Promise<void>;
seek(position: number): Promise<void>;
truncate(size: number): Promise<void>;
close(): Promise<void>;
}
interface WriteParams {
type: 'write' | 'seek' | 'truncate';
data?: BufferSource | Blob | string;
position?: number;
size?: number;
}
interface ShowDirectoryPickerOptions {
id?: string;
mode?: 'read' | 'readwrite';
startIn?:
| FileSystemDirectoryHandle
| 'desktop'
| 'documents'
| 'downloads'
| 'music'
| 'pictures'
| 'videos';
}
interface Window {
showDirectoryPicker(
options?: ShowDirectoryPickerOptions,
): Promise<FileSystemDirectoryHandle>;
}

View File

@ -1,308 +0,0 @@
import type {
KVRecord,
DocRecord,
IdxEntry,
AppliedRecord,
MetaRecord,
IndexDefinition,
} from './types.js';
import { extractIndexValue } from './utils.js';
// ── Store names ──────────────────────────────────────────────
const S_KV = 'kv';
const S_DOCS = 'docs';
const S_IDX = 'idx';
const S_APPLIED = 'applied';
const S_META = 'meta';
const DB_VERSION = 1;
// ── Promise wrappers for raw IDB ─────────────────────────────
function reqP<T>(req: IDBRequest<T>): Promise<T> {
return new Promise((resolve, reject) => {
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
}
function txDone(tx: IDBTransaction): Promise<void> {
return new Promise((resolve, reject) => {
tx.oncomplete = () => resolve();
tx.onerror = () => reject(tx.error);
tx.onabort = () => reject(tx.error ?? new Error('Transaction aborted'));
});
}
// ── IDBStore ─────────────────────────────────────────────────
export class IDBStore {
private constructor(private db: IDBDatabase) {}
// ── Open ───────────────────────────────────────────────────
static open(name: string): Promise<IDBStore> {
return new Promise((resolve, reject) => {
const req = indexedDB.open(name, DB_VERSION);
req.onupgradeneeded = () => {
const db = req.result;
// KV store
if (!db.objectStoreNames.contains(S_KV)) {
db.createObjectStore(S_KV, { keyPath: 'key' });
}
// Document store (compound primary key)
if (!db.objectStoreNames.contains(S_DOCS)) {
const docs = db.createObjectStore(S_DOCS, {
keyPath: ['store', 'id'],
});
docs.createIndex('byStore', 'store', { unique: false });
}
// Secondary index entries
if (!db.objectStoreNames.contains(S_IDX)) {
const idx = db.createObjectStore(S_IDX, {
keyPath: ['store', 'indexName', 'id'],
});
idx.createIndex('lookup', ['store', 'indexName', 'value'], {
unique: false,
});
idx.createIndex('byDoc', ['store', 'id'], { unique: false });
}
// Applied event dedup
if (!db.objectStoreNames.contains(S_APPLIED)) {
db.createObjectStore(S_APPLIED, { keyPath: 'filename' });
}
// Metadata
if (!db.objectStoreNames.contains(S_META)) {
db.createObjectStore(S_META, { keyPath: 'key' });
}
};
req.onsuccess = () => resolve(new IDBStore(req.result));
req.onerror = () => reject(req.error);
});
}
// ── KV operations ──────────────────────────────────────────
async getKV(key: string): Promise<KVRecord | undefined> {
const tx = this.db.transaction(S_KV, 'readonly');
return reqP(tx.objectStore(S_KV).get(key));
}
async putKV(record: KVRecord): Promise<void> {
const tx = this.db.transaction(S_KV, 'readwrite');
tx.objectStore(S_KV).put(record);
return txDone(tx);
}
async deleteKV(key: string): Promise<void> {
const tx = this.db.transaction(S_KV, 'readwrite');
tx.objectStore(S_KV).delete(key);
return txDone(tx);
}
async getAllKVKeys(): Promise<string[]> {
const tx = this.db.transaction(S_KV, 'readonly');
const records: KVRecord[] = await reqP(tx.objectStore(S_KV).getAll());
return records.map((r) => r.key);
}
async getAllKVEntries(): Promise<KVRecord[]> {
const tx = this.db.transaction(S_KV, 'readonly');
return reqP(tx.objectStore(S_KV).getAll());
}
// ── Doc operations ─────────────────────────────────────────
async getDoc(store: string, id: string): Promise<DocRecord | undefined> {
const tx = this.db.transaction(S_DOCS, 'readonly');
const rec: DocRecord | undefined = await reqP(
tx.objectStore(S_DOCS).get([store, id]),
);
if (rec && rec.deleted) return undefined;
return rec;
}
async getRawDoc(store: string, id: string): Promise<DocRecord | undefined> {
const tx = this.db.transaction(S_DOCS, 'readonly');
return reqP(tx.objectStore(S_DOCS).get([store, id]));
}
async putDoc(
record: DocRecord,
indexes: IndexDefinition[],
): Promise<void> {
const tx = this.db.transaction([S_DOCS, S_IDX], 'readwrite');
const docsOS = tx.objectStore(S_DOCS);
const idxOS = tx.objectStore(S_IDX);
// Remove old index entries for this doc (chain off getAll request)
const byDocIdx = idxOS.index('byDoc');
const getOld = byDocIdx.getAll([record.store, record.id]);
getOld.onsuccess = () => {
const oldEntries: IdxEntry[] = getOld.result;
for (const entry of oldEntries) {
idxOS.delete([entry.store, entry.indexName, entry.id]);
}
// Write document
docsOS.put(record);
// Create new index entries (only for non-deleted docs)
if (!record.deleted) {
for (const def of indexes) {
const val = extractIndexValue(record.data, def.fields as string[]);
if (val !== undefined) {
idxOS.put({
store: record.store,
indexName: def.name,
id: record.id,
value: val,
} satisfies IdxEntry);
}
}
}
};
return txDone(tx);
}
async deleteDoc(
store: string,
id: string,
ts: number,
rev: number,
indexes: IndexDefinition[],
): Promise<void> {
return this.putDoc(
{ store, id, data: null, ts, rev, deleted: true },
indexes,
);
}
async getAllDocs(store: string): Promise<DocRecord[]> {
const tx = this.db.transaction(S_DOCS, 'readonly');
const idx = tx.objectStore(S_DOCS).index('byStore');
const all: DocRecord[] = await reqP(idx.getAll(store));
return all.filter((r) => !r.deleted);
}
async findByIndex(
store: string,
indexName: string,
value: IDBValidKey,
): Promise<string[]> {
const tx = this.db.transaction(S_IDX, 'readonly');
const idx = tx.objectStore(S_IDX).index('lookup');
const entries: IdxEntry[] = await reqP(
idx.getAll([store, indexName, value]),
);
return entries.map((e) => e.id);
}
async queryByIndex(
store: string,
indexName: string,
range: IDBKeyRange,
): Promise<string[]> {
const tx = this.db.transaction(S_IDX, 'readonly');
const idx = tx.objectStore(S_IDX).index('lookup');
const entries: IdxEntry[] = await reqP(idx.getAll(range));
// Filter to only this store+index (the range may be broader)
return entries
.filter((e) => e.store === store && e.indexName === indexName)
.map((e) => e.id);
}
async rebuildIndexes(
store: string,
indexes: IndexDefinition[],
): Promise<void> {
const docs = await this.getAllDocs(store);
const tx = this.db.transaction(S_IDX, 'readwrite');
const idxOS = tx.objectStore(S_IDX);
// Delete all existing index entries for this store
const byDocIdx = idxOS.index('byDoc');
for (const doc of docs) {
const req = byDocIdx.getAll([store, doc.id]);
req.onsuccess = () => {
for (const entry of req.result as IdxEntry[]) {
idxOS.delete([entry.store, entry.indexName, entry.id]);
}
};
}
// Re-create all entries
for (const doc of docs) {
for (const def of indexes) {
const val = extractIndexValue(doc.data, def.fields as string[]);
if (val !== undefined) {
idxOS.put({
store,
indexName: def.name,
id: doc.id,
value: val,
} satisfies IdxEntry);
}
}
}
return txDone(tx);
}
// ── Applied-event tracking ─────────────────────────────────
async isEventApplied(filename: string): Promise<boolean> {
const tx = this.db.transaction(S_APPLIED, 'readonly');
const rec = await reqP(tx.objectStore(S_APPLIED).get(filename));
return rec !== undefined;
}
async markEventApplied(filename: string): Promise<void> {
const tx = this.db.transaction(S_APPLIED, 'readwrite');
tx.objectStore(S_APPLIED).put({
filename,
appliedAt: Date.now(),
} satisfies AppliedRecord);
return txDone(tx);
}
async getAppliedSet(): Promise<Set<string>> {
const tx = this.db.transaction(S_APPLIED, 'readonly');
const all: AppliedRecord[] = await reqP(
tx.objectStore(S_APPLIED).getAll(),
);
return new Set(all.map((r) => r.filename));
}
// ── Meta ───────────────────────────────────────────────────
async getMeta<T = unknown>(key: string): Promise<T | undefined> {
const tx = this.db.transaction(S_META, 'readonly');
const rec: MetaRecord | undefined = await reqP(
tx.objectStore(S_META).get(key),
);
return rec?.value as T | undefined;
}
async setMeta(key: string, value: unknown): Promise<void> {
const tx = this.db.transaction(S_META, 'readwrite');
tx.objectStore(S_META).put({ key, value } satisfies MetaRecord);
return txDone(tx);
}
// ── Lifecycle ──────────────────────────────────────────────
close(): void {
this.db.close();
}
}

View File

@ -1,11 +0,0 @@
export { FolderSyncDB } from './folder-sync-db.js';
export type {
OpenOptions,
StoreOptions,
IndexDefinition,
SyncEvent,
KVApi,
CollectionApi,
SyncDBEventName,
SyncDBEventHandler,
} from './types.js';

View File

@ -1,45 +0,0 @@
import type { KVApi } from './types.js';
import type { IDBStore } from './idb-store.js';
import type { SyncEngine } from './sync-engine.js';
export class KVStore implements KVApi {
constructor(
private idb: IDBStore,
private sync: SyncEngine,
) {}
async get<T = unknown>(key: string): Promise<T | undefined> {
const rec = await this.idb.getKV(key);
return rec?.value as T | undefined;
}
async set<T = unknown>(key: string, value: T): Promise<void> {
const existing = await this.idb.getKV(key);
const rev = (existing?.rev ?? 0) + 1;
const ts = Date.now();
// SyncEngine.persistEvent handles the IDB write + folder write
await this.sync.writeKV(key, value, ts, rev);
}
async delete(key: string): Promise<void> {
const existing = await this.idb.getKV(key);
if (!existing) return;
const rev = existing.rev + 1;
const ts = Date.now();
await this.sync.deleteKV(key, ts, rev);
}
async has(key: string): Promise<boolean> {
const rec = await this.idb.getKV(key);
return rec !== undefined;
}
async keys(): Promise<string[]> {
return this.idb.getAllKVKeys();
}
async entries<T = unknown>(): Promise<Array<[string, T]>> {
const records = await this.idb.getAllKVEntries();
return records.map((r) => [r.key, r.value as T]);
}
}

View File

@ -1,179 +0,0 @@
import type { SyncEvent } from './types.js';
/**
* Nostr relay transport layer.
*
* Implements the same event I/O interface as FolderStore so the SyncEngine
* can treat it as just another transport.
*
* Uses raw WebSocket + minimal Nostr protocol (NIP-01) to avoid
* heavy dependencies. Works from file:// origins.
*/
const NOSTR_EVENT_KIND = 4078; // custom regular kind (stored by relays)
// ── Minimal Nostr crypto (secp256k1 via nostr-tools) ─────────
import { generateSecretKey, getPublicKey, finalizeEvent } from 'nostr-tools/pure';
import { SimplePool, type SubCloser } from 'nostr-tools/pool';
import type { Filter } from 'nostr-tools/filter';
export class NostrTransport {
private relays: string[];
private pool: SimplePool;
private roomKey: string | null = null;
private secretKey: Uint8Array | null = null;
/** Cached events keyed by our standard filename. */
private eventCache = new Map<string, SyncEvent>();
/** Active relay subscription (closeable). */
private sub: SubCloser | null = null;
/** Callback fired when a new event arrives in real time. */
private _onNewEvent?: () => void;
constructor(relays: string[]) {
this.relays = relays;
this.pool = new SimplePool();
}
// ── Key management ──────────────────────────────────────────
/**
* Set the keypair. Called by FolderSyncDB after loading from IDB
* (or generating a new one on first run).
*/
setKeypair(sk: Uint8Array): void {
this.secretKey = sk;
// Derive pubkey (used internally by finalizeEvent)
getPublicKey(sk);
}
generateKeypair(): Uint8Array {
const sk = generateSecretKey();
this.setKeypair(sk);
return sk;
}
// ── Room management ─────────────────────────────────────────
async joinRoom(roomKey: string): Promise<void> {
if (this.roomKey === roomKey && this.sub) return; // already joined
this.leaveRoom();
this.roomKey = roomKey;
// Fetch existing events from relays
const fetchFilter: Filter = {
kinds: [NOSTR_EVENT_KIND],
'#channel': [roomKey],
limit: 5000,
};
try {
const events = await this.pool.querySync(this.relays, fetchFilter as Filter);
for (const ev of events) {
this.cacheNostrEvent(ev);
}
} catch {
// relay might be unreachable — continue with empty cache
}
// Subscribe for new real-time events
const subFilter: Filter = {
kinds: [NOSTR_EVENT_KIND],
'#channel': [roomKey],
since: Math.floor(Date.now() / 1000),
};
this.sub = this.pool.subscribeMany(this.relays, subFilter, {
onevent: (ev) => {
const isNew = this.cacheNostrEvent(ev);
if (isNew) this._onNewEvent?.();
},
});
}
leaveRoom(): void {
if (this.sub) {
this.sub.close();
this.sub = null;
}
this.roomKey = null;
this.eventCache.clear();
}
get isConnected(): boolean {
return this.roomKey !== null && this.sub !== null;
}
get currentRoom(): string | null {
return this.roomKey;
}
// ── Callback for real-time push ─────────────────────────────
onNewEvent(cb: () => void): void {
this._onNewEvent = cb;
}
// ── Event I/O (same interface as FolderStore) ───────────────
async writeEvent(filename: string, event: SyncEvent): Promise<void> {
if (!this.roomKey || !this.secretKey) return;
this.eventCache.set(filename, event);
const nostrEvent = finalizeEvent({
kind: NOSTR_EVENT_KIND,
created_at: Math.floor(Date.now() / 1000),
tags: [
['channel', this.roomKey],
['filename', filename],
],
content: JSON.stringify(event),
}, this.secretKey);
// Publish to all relays, don't wait for all to confirm
try {
await Promise.any(this.pool.publish(this.relays, nostrEvent as any));
} catch {
// All relays failed — event is still in local cache
}
}
async scanEventFilenames(): Promise<string[]> {
return Array.from(this.eventCache.keys()).sort();
}
async readEventFile(filename: string): Promise<SyncEvent | null> {
return this.eventCache.get(filename) ?? null;
}
// ── Cleanup ─────────────────────────────────────────────────
close(): void {
this.leaveRoom();
this.pool.close(this.relays);
}
// ── Internals ───────────────────────────────────────────────
/**
* Parse a Nostr event and cache it. Returns true if the event
* was new (not already cached).
*/
private cacheNostrEvent(nostrEvent: any): boolean {
try {
const filename = nostrEvent.tags?.find(
(t: string[]) => t[0] === 'filename',
)?.[1];
if (!filename) return false;
if (this.eventCache.has(filename)) return false;
const syncEvent = JSON.parse(nostrEvent.content) as SyncEvent;
this.eventCache.set(filename, syncEvent);
return true;
} catch {
return false;
}
}
}

View File

@ -1,254 +0,0 @@
import type { SyncEvent, IndexDefinition, OpenOptions } from './types.js';
import type { IDBStore } from './idb-store.js';
import type { FolderStore } from './folder-store.js';
import type { NostrTransport } from './nostr-transport.js';
import type { Emitter } from './emitter.js';
import { canonicalJson, sha256Hex, eventFilename } from './utils.js';
/**
* Dual-transport sync engine.
*
* On local write IDB + folder (if connected) + Nostr (if joined)
* On sync() import from folder + import from Nostr, deduplicate
*/
export class SyncEngine {
private clientId: string;
private conflictResolver?: OpenOptions['conflictResolver'];
private collectionIndexes = new Map<string, IndexDefinition[]>();
private syncing = false;
constructor(
private idb: IDBStore,
private folder: FolderStore,
private nostr: NostrTransport,
private emitter: Emitter,
clientId: string,
conflictResolver?: OpenOptions['conflictResolver'],
) {
this.clientId = clientId;
this.conflictResolver = conflictResolver;
}
// ── Collection index registration ──────────────────────────
registerIndexes(store: string, indexes: IndexDefinition[]): void {
this.collectionIndexes.set(store, indexes);
}
getIndexes(store: string): IndexDefinition[] {
return this.collectionIndexes.get(store) ?? [];
}
// ── Local writes (write to IDB + folder + Nostr) ───────────
async writeKV(key: string, value: unknown, ts: number, rev: number): Promise<void> {
const event: SyncEvent = {
type: 'put', store: 'kv', key, ts,
clientId: this.clientId, data: value, rev,
};
await this.persistEvent(event);
}
async deleteKV(key: string, ts: number, rev: number): Promise<void> {
const event: SyncEvent = {
type: 'delete', store: 'kv', key, ts,
clientId: this.clientId, rev,
};
await this.persistEvent(event);
}
async writeDoc(store: string, id: string, data: unknown, ts: number, rev: number): Promise<void> {
const event: SyncEvent = {
type: 'put', store, key: id, id, ts,
clientId: this.clientId, data, rev,
};
await this.persistEvent(event);
}
async deleteDocEvent(store: string, id: string, ts: number, rev: number): Promise<void> {
const event: SyncEvent = {
type: 'delete', store, key: id, id, ts,
clientId: this.clientId, rev,
};
await this.persistEvent(event);
}
// ── Core persist: IDB + folder + Nostr ─────────────────────
private async persistEvent(event: SyncEvent): Promise<void> {
const canonical = canonicalJson(event);
const hash = await sha256Hex(canonical);
const filename = eventFilename(event.ts, hash);
// 1. Write to IDB (fast path)
await this.applyEvent(event);
await this.idb.markEventApplied(filename);
// 2. Write to folder (if connected)
if (this.folder.hasHandle && (await this.folder.hasPermission())) {
try {
await this.folder.writeEvent(filename, event);
} catch (err) {
this.emitter.emit('folder:lost-permission', err);
}
}
// 3. Publish to Nostr (if joined)
if (this.nostr.isConnected) {
try {
await this.nostr.writeEvent(filename, event);
} catch {
// Nostr publish failed — event is still in IDB + folder
}
}
this.emitter.emit('change', {
type: event.type, store: event.store,
key: event.key, id: event.id, data: event.data,
});
}
// ── Sync: import from BOTH folder and Nostr ────────────────
async sync(): Promise<void> {
if (this.syncing) return;
this.syncing = true;
this.emitter.emit('sync:start');
try {
const appliedSet = await this.idb.getAppliedSet();
let importCount = 0;
// ── Import from folder ─────────────────────────────────
const hasFolder = this.folder.hasHandle && (await this.folder.hasPermission());
if (hasFolder) {
const folderFiles = await this.folder.scanEventFilenames();
for (const name of folderFiles) {
if (appliedSet.has(name)) continue;
const event = await this.folder.readEventFile(name);
if (!event) continue;
const hadConflict = await this.applyEventWithConflictCheck(event);
await this.idb.markEventApplied(name);
appliedSet.add(name);
importCount++;
if (hadConflict) this.emitter.emit('conflict', { filename: name, event });
this.emitter.emit('change', {
type: event.type, store: event.store,
key: event.key, id: event.id, data: event.data, source: 'folder',
});
// Bridge: push folder event to Nostr so remote devices get it
if (this.nostr.isConnected) {
this.nostr.writeEvent(name, event).catch(() => {});
}
}
}
// ── Import from Nostr ──────────────────────────────────
if (this.nostr.isConnected) {
const nostrFiles = await this.nostr.scanEventFilenames();
for (const name of nostrFiles) {
if (appliedSet.has(name)) continue;
const event = await this.nostr.readEventFile(name);
if (!event) continue;
const hadConflict = await this.applyEventWithConflictCheck(event);
await this.idb.markEventApplied(name);
appliedSet.add(name);
importCount++;
if (hadConflict) this.emitter.emit('conflict', { filename: name, event });
this.emitter.emit('change', {
type: event.type, store: event.store,
key: event.key, id: event.id, data: event.data, source: 'nostr',
});
// Bridge: persist Nostr event to folder so it survives offline
if (hasFolder) {
this.folder.writeEvent(name, event).catch(() => {});
}
}
}
// If neither transport is connected, nothing to do
if (!hasFolder && !this.nostr.isConnected) {
// No transport — sync is a no-op
}
this.emitter.emit('sync:end', { imported: importCount });
} catch (err) {
this.emitter.emit('sync:end', { error: err });
} finally {
this.syncing = false;
}
}
// ── Apply a single event to IDB ────────────────────────────
private async applyEvent(event: SyncEvent): Promise<void> {
if (event.store === 'kv') {
if (event.type === 'put') {
await this.idb.putKV({
key: event.key, value: event.data,
ts: event.ts, rev: event.rev ?? 0,
});
} else {
await this.idb.deleteKV(event.key);
}
} else {
const indexes = this.getIndexes(event.store);
if (event.type === 'put') {
await this.idb.putDoc({
store: event.store, id: event.id ?? event.key,
data: event.data, ts: event.ts, rev: event.rev ?? 0,
}, indexes);
} else {
await this.idb.deleteDoc(
event.store, event.id ?? event.key,
event.ts, event.rev ?? 0, indexes,
);
}
}
}
private async applyEventWithConflictCheck(event: SyncEvent): Promise<boolean> {
let hadConflict = false;
if (event.store === 'kv') {
const existing = await this.idb.getKV(event.key);
if (existing) {
if (event.ts > existing.ts) {
hadConflict = true;
} else if (event.ts === existing.ts) {
hadConflict = true;
if (this.conflictResolver) {
const resolved = this.conflictResolver(existing.value, event.data);
event = { ...event, data: resolved };
}
} else {
return true;
}
}
} else {
const existing = await this.idb.getRawDoc(event.store, event.id ?? event.key);
if (existing && !existing.deleted) {
if (event.ts > existing.ts) {
hadConflict = true;
} else if (event.ts === existing.ts) {
hadConflict = true;
if (this.conflictResolver) {
const resolved = this.conflictResolver(existing.data, event.data);
event = { ...event, data: resolved };
}
} else {
return true;
}
}
}
await this.applyEvent(event);
return hadConflict;
}
}

View File

@ -1,112 +0,0 @@
// ── Event types ──────────────────────────────────────────────
export type EventType = 'put' | 'delete';
export interface SyncEvent {
type: EventType;
store: string; // "kv" | collection name
key: string;
id?: string; // present for collection documents
ts: number; // millisecond Unix epoch
clientId: string;
data?: unknown; // absent on deletes
rev?: number;
}
// ── Index / collection options ───────────────────────────────
export interface IndexDefinition<T = unknown> {
name: string;
fields: (keyof T & string)[] | string[];
}
export interface StoreOptions<T = unknown> {
name: string;
indexes?: IndexDefinition<T>[];
}
export interface OpenOptions {
dbName?: string;
autoSyncIntervalMs?: number;
clientId?: string;
conflictResolver?: (current: unknown, incoming: unknown) => unknown;
/** Nostr relay WebSocket URLs. Defaults to a set of popular public relays. */
relays?: string[];
/** If provided, automatically join this room on open. */
roomKey?: string;
}
// ── IDB record shapes ────────────────────────────────────────
export interface KVRecord {
key: string;
value: unknown;
ts: number;
rev: number;
}
export interface DocRecord {
store: string;
id: string;
data: unknown;
ts: number;
rev: number;
deleted?: boolean;
}
export interface IdxEntry {
store: string;
indexName: string;
id: string;
value: IDBValidKey;
}
export interface AppliedRecord {
filename: string;
appliedAt: number;
}
export interface MetaRecord {
key: string;
value: unknown;
}
// ── Event emitter ────────────────────────────────────────────
export type SyncDBEventName =
| 'sync:start'
| 'sync:end'
| 'change'
| 'conflict'
| 'folder:lost-permission'
| 'nostr:connected'
| 'nostr:disconnected'
| 'nostr:event';
export type SyncDBEventHandler = (...args: unknown[]) => void;
// ── KV API surface ───────────────────────────────────────────
export interface KVApi {
get<T = unknown>(key: string): Promise<T | undefined>;
set<T = unknown>(key: string, value: T): Promise<void>;
delete(key: string): Promise<void>;
has(key: string): Promise<boolean>;
keys(): Promise<string[]>;
entries<T = unknown>(): Promise<Array<[string, T]>>;
}
// ── Collection API surface ───────────────────────────────────
export interface CollectionApi<T extends { id: string }> {
get(id: string): Promise<T | undefined>;
put(doc: T): Promise<void>;
delete(id: string): Promise<void>;
all(): Promise<T[]>;
findByIndex(indexName: string, value: IDBValidKey): Promise<T[]>;
queryByIndex(
indexName: string,
range: { gt?: IDBValidKey; gte?: IDBValidKey; lt?: IDBValidKey; lte?: IDBValidKey },
): Promise<T[]>;
}

View File

@ -1,94 +0,0 @@
// ── Canonical JSON ───────────────────────────────────────────
function sortKeys(obj: unknown): unknown {
if (obj === null || obj === undefined || typeof obj !== 'object') return obj;
if (Array.isArray(obj)) return obj.map(sortKeys);
const sorted: Record<string, unknown> = {};
for (const key of Object.keys(obj as Record<string, unknown>).sort()) {
sorted[key] = sortKeys((obj as Record<string, unknown>)[key]);
}
return sorted;
}
export function canonicalJson(obj: unknown): string {
return JSON.stringify(sortKeys(obj));
}
// ── Hashing ──────────────────────────────────────────────────
export async function sha256Hex(data: string): Promise<string> {
const buf = await crypto.subtle.digest(
'SHA-256',
new TextEncoder().encode(data),
);
return Array.from(new Uint8Array(buf))
.map((b) => b.toString(16).padStart(2, '0'))
.join('');
}
// ── Event filenames ──────────────────────────────────────────
const HASH_PREFIX_LEN = 12;
export function eventFilename(ts: number, fullHash: string): string {
return `${ts}_${fullHash.slice(0, HASH_PREFIX_LEN)}.json`;
}
const EVENT_FILE_RE = /^(\d+)_([0-9a-f]+)\.json$/;
export function parseEventFilename(
name: string,
): { ts: number; hash: string } | null {
const m = EVENT_FILE_RE.exec(name);
if (!m) return null;
return { ts: Number(m[1]), hash: m[2] };
}
// ── Client ID ────────────────────────────────────────────────
export function generateClientId(): string {
return crypto.randomUUID();
}
// ── Index value extraction ───────────────────────────────────
export function extractIndexValue(
data: unknown,
fields: string[],
): IDBValidKey | undefined {
if (!data || typeof data !== 'object') return undefined;
const rec = data as Record<string, unknown>;
if (fields.length === 1) {
const v = getNestedValue(rec, fields[0]);
if (v === undefined || v === null) return undefined;
return toIDBKey(v);
}
const parts: IDBValidKey[] = [];
for (const f of fields) {
const v = getNestedValue(rec, f);
if (v === undefined || v === null) return undefined;
parts.push(toIDBKey(v));
}
return parts;
}
function getNestedValue(obj: Record<string, unknown>, path: string): unknown {
let cur: unknown = obj;
for (const seg of path.split('.')) {
if (cur === null || cur === undefined || typeof cur !== 'object') {
return undefined;
}
cur = (cur as Record<string, unknown>)[seg];
}
return cur;
}
function toIDBKey(v: unknown): IDBValidKey {
if (typeof v === 'string' || typeof v === 'number') return v;
if (v instanceof Date) return v;
if (Array.isArray(v)) return v.map(toIDBKey);
// Fallback: stringify
return String(v);
}

View File

@ -1,21 +0,0 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ES2022",
"moduleResolution": "bundler",
"lib": ["ES2022", "DOM", "DOM.Iterable"],
"outDir": "dist",
"rootDir": "src",
"declaration": true,
"declarationMap": true,
"sourceMap": true,
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true,
"forceConsistentCasingInFileNames": true,
"skipLibCheck": true
},
"include": ["src"],
"exclude": ["node_modules", "dist"]
}

View File

@ -1,6 +1,6 @@
# Paste -- Demo App # Paste -- Demo App
A minimal paste-bin app used to test and compare all five IndexSyncFile storage variants. Each version compiles to a single self-contained HTML file. A minimal paste-bin app used to test and compare all four IndexSyncFile storage variants. Each version compiles to a single self-contained HTML file.
## What it does ## What it does
@ -12,17 +12,14 @@ A minimal paste-bin app used to test and compare all five IndexSyncFile storage
That's it. No server, no accounts, no network. Just two browser windows and a folder. That's it. No server, no accounts, no network. Just two browser windows and a folder.
The Nostr variant adds optional **cross-device sync** over public Nostr relays via WebSocket. Enter a shared room key and devices sync over the internet too. ## Four versions
## Five versions | File | Storage engine | Opens from `file://`? | Size |
|------|---------------|----------------------|------|
| File | Storage engine | Sync transport | Opens from `file://`? | Size | | `paste-indexeddb.html` | IndexedDB (browser built-in) | Yes | ~23 KB |
|------|---------------|----------------|----------------------|------| | `paste-nedb.html` | NeDB (in-memory, MongoDB-style) | Yes | ~115 KB |
| `paste-indexeddb.html` | IndexedDB (browser built-in) | Folder | Yes | ~23 KB | | `paste-sql-js.html` | sql.js (SQLite via asm.js) | Yes | ~1.9 MB |
| `paste-nedb.html` | NeDB (in-memory, MongoDB-style) | Folder | Yes | ~115 KB | | `paste-sqlite.html` | SQLite WASM (official build) | No (needs HTTP) | ~1.4 MB |
| `paste-sql-js.html` | sql.js (SQLite via asm.js) | Folder | Yes | ~1.9 MB |
| `paste-sqlite.html` | SQLite WASM (official build) | Folder | No (needs HTTP) | ~1.4 MB |
| `paste-nostr.html` | IndexedDB | **Folder + Nostr** | Yes | ~75 KB |
### paste-indexeddb.html ### paste-indexeddb.html
@ -40,16 +37,6 @@ Uses sql.js, which is SQLite compiled to pure JavaScript (asm.js, not WebAssembl
Uses the official `@sqlite.org/sqlite-wasm` package. Requires an HTTP server because the browser must `fetch()` the `.wasm` binary at runtime. May also need COOP/COEP headers for OPFS persistence. Run `bun run serve.ts` to test this variant. Uses the official `@sqlite.org/sqlite-wasm` package. Requires an HTTP server because the browser must `fetch()` the `.wasm` binary at runtime. May also need COOP/COEP headers for OPFS persistence. Run `bun run serve.ts` to test this variant.
### paste-nostr.html
Uses IndexedDB for local cache (same as the indexeddb variant) plus **Nostr relay sync** for cross-device reach. Has both a folder picker and a room key input. Use either or both:
- **Folder only** — local multi-browser sync (works offline)
- **Room only** — cross-device sync via Nostr relays (no folder needed)
- **Both** — folder for local speed + Nostr for internet reach
WebSocket connections to public Nostr relays work from `file://` origins, so no server is needed.
## How to test ## How to test
### Quick (three variants) ### Quick (three variants)
@ -60,7 +47,6 @@ Double-click or drag any of these into Chrome:
dist/paste-indexeddb.html dist/paste-indexeddb.html
dist/paste-nedb.html dist/paste-nedb.html
dist/paste-sql-js.html dist/paste-sql-js.html
dist/paste-nostr.html
``` ```
They work directly from `file://`. They work directly from `file://`.
@ -91,7 +77,7 @@ Requires [Bun](https://bun.sh/) 1.3.10+.
bun run build.ts bun run build.ts
``` ```
This produces five single-file HTMLs in `dist/`. The build script: This produces four single-file HTMLs in `dist/`. The build script:
1. Runs `Bun.build()` for each variant (bundles TypeScript, minifies) 1. Runs `Bun.build()` for each variant (bundles TypeScript, minifies)
2. Inlines all JS and CSS chunks into the HTML 2. Inlines all JS and CSS chunks into the HTML
3. Copies `sqlite3.wasm` to `dist/` for the WASM variant 3. Copies `sqlite3.wasm` to `dist/` for the WASM variant
@ -112,9 +98,6 @@ paste/
sqlite/ source for SQLite WASM paste app sqlite/ source for SQLite WASM paste app
app.ts app.ts
index.html index.html
nostr/ source for Nostr + Folder paste app
app.ts
index.html
shared.ts shared UI logic (all variants import this) shared.ts shared UI logic (all variants import this)
styles.css shared styles styles.css shared styles
build.ts bun build script build.ts bun build script
@ -137,4 +120,4 @@ your-folder/
store.sqlite (sql-js: SQLite binary) store.sqlite (sql-js: SQLite binary)
``` ```
The `events/` folder is shared by all variants. If you point two different variant HTMLs at the same folder, they will sync with each other (the event format is identical across all five). The `events/` folder is shared by all variants. If you point two different variant HTMLs at the same folder, they will sync with each other (the event format is identical across all four).

View File

@ -20,7 +20,7 @@
import { readFileSync } from 'fs'; import { readFileSync } from 'fs';
import { join, basename } from 'path'; import { join, basename } from 'path';
const variants = ['indexeddb', 'sqlite', 'nedb', 'sql-js', 'nostr'] as const; const variants = ['indexeddb', 'sqlite', 'nedb', 'sql-js'] as const;
for (const variant of variants) { for (const variant of variants) {
console.log(`Building paste-${variant}...`); console.log(`Building paste-${variant}...`);

View File

@ -1,70 +0,0 @@
import { FolderSyncDB } from '../../nostr/src/index.ts';
import { initPasteApp } from '../shared.ts';
document.addEventListener('DOMContentLoaded', async () => {
const db = await FolderSyncDB.open({ autoSyncIntervalMs: 3000 });
await initPasteApp(db as any, 'nostr');
// ── Nostr room UI ──────────────────────────────────────────
const roomInput = document.querySelector('#room-input') as HTMLInputElement;
const joinBtn = document.querySelector('#join-room') as HTMLButtonElement;
const nostrStatus = document.querySelector('#nostr-status')!;
function updateNostrUI(connected: boolean, room?: string) {
if (connected && room) {
nostrStatus.textContent = `Connected: ${room}`;
nostrStatus.className = 'status ok';
joinBtn.textContent = 'Leave';
roomInput.disabled = true;
} else {
nostrStatus.textContent = 'Not connected';
nostrStatus.className = 'status';
joinBtn.textContent = 'Join room';
roomInput.disabled = false;
}
}
joinBtn.addEventListener('click', async () => {
if (db.isConnected()) {
db.leaveRoom();
updateNostrUI(false);
} else {
const key = roomInput.value.trim();
if (!key) {
nostrStatus.textContent = 'Enter a room key';
nostrStatus.className = 'status err';
return;
}
try {
nostrStatus.textContent = 'Connecting...';
nostrStatus.className = 'status';
await db.joinRoom(key);
updateNostrUI(true, key);
} catch (e: unknown) {
nostrStatus.textContent = 'Error: ' + (e as Error).message;
nostrStatus.className = 'status err';
}
}
});
roomInput.addEventListener('keydown', (e: KeyboardEvent) => {
if (e.key === 'Enter') joinBtn.click();
});
// Restore saved room
if (db.isConnected()) {
updateNostrUI(true, db.currentRoom ?? undefined);
roomInput.value = db.currentRoom ?? '';
} else {
updateNostrUI(false);
}
db.on('nostr:connected', (data: any) => {
updateNostrUI(true, data?.roomKey);
});
db.on('nostr:disconnected', () => {
updateNostrUI(false);
});
});

View File

@ -1,43 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>paste — Nostr + Folder</title>
<link rel="stylesheet" href="../styles.css">
</head>
<body>
<header>
<h1>paste <span id="variant-label" class="variant">nostr</span></h1>
<p class="sub">type text + enter, or paste an image &mdash; use #hashtags to add tags</p>
<div class="folder-bar">
<button id="select-folder" class="folder-btn">Select sync folder</button>
<span id="folder-status" class="status"></span>
</div>
<div class="folder-bar" style="margin-top:6px">
<input type="text" id="room-input" placeholder="room key" autocomplete="off"
style="padding:6px 10px;border:1px solid #555;border-radius:6px;background:#1e1e2e;color:#cdd6f4;font-size:0.85rem;width:180px">
<button id="join-room" class="folder-btn">Join room</button>
<span id="nostr-status" class="status"></span>
</div>
</header>
<main>
<div class="input-area">
<div class="input-row">
<input type="text" id="paste-input"
placeholder="type something and press Enter, or Ctrl+V an image"
autocomplete="off" spellcheck="false">
<button class="clip-btn" id="clip-btn" title="Attach a file">
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M21.44 11.05l-9.19 9.19a6 6 0 01-8.49-8.49l9.19-9.19a4 4 0 015.66 5.66l-9.2 9.19a2 2 0 01-2.83-2.83l8.49-8.48"/>
</svg>
</button>
<input type="file" id="file-input" hidden>
</div>
<div id="status" class="status"></div>
</div>
<div id="items" class="items"></div>
</main>
<script type="module" src="./app.ts"></script>
</body>
</html>