LocalHtmlDataTest/README.md

# IndexSyncFile

A local-first key-value and document store for the browser that syncs across multiple browser windows via a user-selected folder on disk.

**No server. No backend. No cloud.** Just browsers and a shared folder.

## The experiment

Can multiple browser tabs (or separate browsers entirely) stay in sync using nothing but a shared folder on the local file system?

This repo explores that question by building the same sync library five times, each with a different in-browser storage engine. The sync layer and public API are identical. Only the local cache differs.

The fifth variant adds **Nostr relay sync** on top of folder sync, enabling cross-device synchronization (phone, laptop, etc.) over the internet with no server of our own.

The goal is to compare trade-offs: bundle size, startup speed, query power, persistence behavior, and `file://` compatibility.

## How it works

```
Browser A                    Shared Folder                   Browser B
---------                    -------------                   ---------
kv.set("x", 1)
  |
  +---> writes to local
  |     cache (fast)
  |
  +---> writes event file ---> /events/1234_abc.json
                                                     <--- sync() reads new events
                                                              |
                                                              +---> applies to
                                                                    local cache
```

1. Each browser keeps a fast local cache for reads/writes (IndexedDB, SQLite WASM, NeDB, or sql.js)
   - The Nostr variant also publishes events to public relays for cross-device sync
2. Every mutation is written as an immutable JSON file in `/events/` inside a user-selected folder
3. When any browser calls `sync()`, it scans `/events/`, finds files it hasn't seen, and merges them into its local cache
4. Conflicts are resolved by last-write-wins (configurable)

### Why events?

The event log is what makes multi-browser sync possible. Each mutation is a separate file, so concurrent writes from different browsers never overwrite each other. Both sides' changes are captured and merged on the next `sync()`.

Without events, if two browsers both wrote a single database file, the last save would silently destroy the other browser's changes.

## Five variants

Each variant uses a different local cache engine. The API and sync behavior are identical.

| Variant | Local cache | Sync transport | `file://` works? | Dependencies |
|---------|------------|----------------|-------------------|--------------|
| [`indexeddb/`](./indexeddb/) | IndexedDB | Folder only | Yes | Zero |
| [`nedb/`](./nedb/) | NeDB in-memory | Folder + NDJSON snapshots | Yes | `@seald-io/nedb` |
| [`sqlite/`](./sqlite/) | SQLite WASM | Folder only | No (needs HTTP + headers) | `@sqlite.org/sqlite-wasm` |
| [`sql-js/`](./sql-js/) | sql.js (asm.js) | Folder + SQLite snapshot | Yes | `sql.js` |
| [`nostr/`](./nostr/) | IndexedDB | **Folder + Nostr relays** | Yes | `nostr-tools` |

### Which one should I use?

- **Just want it to work** — use `indexeddb/`. Zero deps, works from `file://`, instant startup.
- **Need cross-device sync** (phone, laptop, different networks) — use `nostr/`. Folder sync for local/offline, Nostr relays for internet reach. Works from `file://`.
- **Need MongoDB-style queries** (`$gt`, `$in`, `$regex`) — use `nedb/`.
- **Need real SQL + proper numeric range queries** — use `sql-js/`. Works from `file://`, no special headers.
- **Need OPFS-backed persistence + SQL** — use `sqlite/`. Requires HTTP server and may need COOP/COEP headers.

## Quick start (same for all variants)

```ts
import { FolderSyncDB } from 'index-sync-file'; // pick your variant

const db = await FolderSyncDB.open({
  autoSyncIntervalMs: 5000,
});

// User picks a folder (one-time, persisted across sessions)
await db.selectFolder();

// Key-value
await db.kv.set('theme', 'dark');
const theme = await db.kv.get('theme'); // 'dark'

// Document collections with indexes
const todos = db.collection({
  name: 'todos',
  indexes: [{ name: 'byStatus', fields: ['status'] }],
});

await todos.put({ id: '1', title: 'Ship it', status: 'active' });
await todos.put({ id: '2', title: 'Test it', status: 'done' });

const active = await todos.findByIndex('byStatus', 'active');
// [{ id: '1', title: 'Ship it', status: 'active' }]

// Listen for changes (including from other browsers)
db.on('change', (e) => console.log('changed:', e));
db.on('conflict', (e) => console.log('conflict resolved:', e));
```

## Folder layout on disk

```
your-selected-folder/
  events/
    1710000000000_a1b2c3d4e5f6.json   <-- immutable event files
    1710000001000_f6e5d4c3b2a1.json
    ...
  data/                                <-- nedb and sql-js variants
    kv.db                              <-- nedb: NDJSON snapshot
    docs.db
    store.sqlite                       <-- sql-js: full SQLite database file
```

## Demo app: Paste

The [`paste/`](./paste/) folder contains a working paste-bin demo app built with all five variants. Each compiles to a single self-contained HTML file that you can open directly from disk (except the SQLite WASM variant, which needs HTTP).

See the [paste README](./paste/README.md) for details.

## Browser requirements

Requires the **File System Access API** (`showDirectoryPicker`). Supported in:
- Chrome 86+
- Edge 86+
- Brave
- Opera 72+

Firefox and Safari do not support `showDirectoryPicker`. The library throws a clear error if the API is unavailable.

## Building

Each variant is a self-contained npm package:

```bash
cd indexeddb && npm install && npm run build
cd nedb && npm install && npm run build
cd sqlite && npm install && npm run build
cd sql-js && npm install && npm run build
cd nostr && npm install && npm run build
```

The paste demo uses Bun:

```bash
cd paste && bun run build.ts
```

## API reference

See the README in each variant's folder for variant-specific details.

### `FolderSyncDB`

| Method | Description |
|--------|-------------|
| `FolderSyncDB.open(options?)` | Create and initialize the database |
| `selectFolder()` | Prompt user to pick a sync folder |
| `hasFolderAccess()` | Check if folder permission is granted |
| `requestFolderAccess()` | Re-request permission (needs user gesture) |
| `joinRoom(roomKey)` | Join a Nostr sync room *(nostr variant only)* |
| `leaveRoom()` | Leave the current Nostr room *(nostr variant only)* |
| `isConnected()` | Check Nostr relay connection *(nostr variant only)* |
| `sync()` | Manually trigger a sync (folder + Nostr if connected) |
| `close()` | Stop auto-sync, flush writes, release resources |
| `kv` | Key-value store (see below) |
| `collection(options)` | Create/get a document collection (see below) |
| `on(event, handler)` | Listen for events, returns unsubscribe function |

### `kv` (Key-Value Store)

| Method | Description |
|--------|-------------|
| `get(key)` | Get value by key |
| `set(key, value)` | Set a key-value pair |
| `delete(key)` | Delete a key |
| `has(key)` | Check if key exists |
| `keys()` | Get all keys |
| `entries()` | Get all [key, value] pairs |

### `collection(options)` (Document Collections)

| Method | Description |
|--------|-------------|
| `get(id)` | Get document by ID |
| `put(doc)` | Insert or update a document (must have `id` field) |
| `delete(id)` | Delete a document (tombstone) |
| `all()` | Get all documents |
| `findByIndex(indexName, value)` | Exact-match query on an index |
| `queryByIndex(indexName, range)` | Range query (`{ gte, lte, gt, lt }`) |

### Events

| Event | Payload | When |
|-------|---------|------|
| `sync:start` | -- | Sync begins |
| `sync:end` | `{ imported }` or `{ error }` | Sync completes |
| `change` | `{ type, store, key, id, data }` | Any data mutation (local or synced) |
| `conflict` | `{ filename, event }` | A conflict was detected and resolved |
| `folder:lost-permission` | error? | Folder access was lost |
| `nostr:connected` | `{ roomKey }` | Joined a Nostr room *(nostr variant)* |
| `nostr:disconnected` | -- | Left a Nostr room *(nostr variant)* |