LocalHtmlDataTest/nedb

IndexSyncFile — NeDB Variant

Local-first key-value and document store using NeDB (@seald-io/nedb, the maintained fork) as the local cache, syncing with a user-selected folder via the File System Access API.

Why NeDB

• MongoDB-style queries — $gt, $lt, $in, $nin, $regex, $exists, $elemMatch, and more
• Native indexing — ensureIndex handles index creation and maintenance automatically
• Dot-notation queries — query nested fields like data.user.name naturally
• Promise-based API — @seald-io/nedb provides native *Async methods, no callback wrappers needed

Architecture

Write: app ---> NeDB in-memory (immediate) ---> /events/timestamp_hash.json
                                           ---> /data/*.db (debounced NDJSON snapshot)

Sync:  /events/*.json ---> sort by timestamp ---> skip applied ---> apply to NeDB
                                                                ---> persist to /data/*.db

Startup: load /data/*.db (instant) ---> sync only NEW events from /events/

Two persistence layers

Layer	Purpose	Format
/events/	Sync mechanism — immutable event files enable multi-browser sync without data loss	One JSON file per mutation
/data/	Fast reload — NDJSON snapshots so startup doesn't replay the entire event history	One NDJSON file per datastore

The event log is essential for sync. The data files are a startup optimization — without them, every page load would replay all events from scratch.

Data files (NDJSON)

your-folder/data/
  kv.db         # all key-value records
  docs.db       # all collection documents
  applied.db    # which event files have been processed
  meta.db       # client ID and other metadata

Each file is newline-delimited JSON (one record per line). Human-readable, diffable, and written with a 250ms debounce to avoid excessive disk writes during batch operations.

NeDB datastores (4 in-memory)

Datastore	_id	Purpose
kvDB	KV key	Key-value records
docsDB	"store\0id"	All collection documents (with _store, _docId, _deleted, data fields)
appliedDB	event filename	Tracks processed events
metaDB	meta key	Client ID, etc.

Document data is stored in a data field to avoid field name conflicts. NeDB indexes are created on data.<fieldName> using dot notation.

Install and build

npm install
npm run build

Dependency: @seald-io/nedb ^4.1.0 — actively maintained fork of NeDB with native async/await support. Zero npm vulnerabilities.

Usage

import { FolderSyncDB } from './dist/index.js';

const db = await FolderSyncDB.open({
  dbName: 'MyApp',
  autoSyncIntervalMs: 5000,
});

await db.selectFolder();

// KV
await db.kv.set('user', { name: 'Alice', role: 'admin' });

// Collections with indexes
const logs = db.collection({
  name: 'logs',
  indexes: [
    { name: 'byLevel', fields: ['level'] },
    { name: 'byTimestamp', fields: ['ts'] },
  ],
});

await logs.put({ id: 'log1', level: 'error', ts: Date.now(), msg: 'disk full' });
await logs.put({ id: 'log2', level: 'info', ts: Date.now(), msg: 'started' });

// Exact match (uses NeDB index)
const errors = await logs.findByIndex('byLevel', 'error');

// Range query (uses NeDB $gte/$lte operators)
const recent = await logs.queryByIndex('byTimestamp', {
  gte: Date.now() - 3600000,
});

// Multi-field indexes work too
const items = db.collection({
  name: 'items',
  indexes: [{ name: 'byTypeAndStatus', fields: ['type', 'status'] }],
});
const match = await items.findByIndex('byTypeAndStatus', ['widget', 'active']);

Variant-specific notes

• NeDB runs in-memory only in the browser (no Node.js fs access)
• Persistence comes from two sources: the event log (for sync) and the NDJSON snapshots (for fast reload)
• Directory handle is stored in a tiny IDB sidecar (NeDB can't store DOM objects)
• Range queries are only supported on single-field indexes (NeDB limitation)
• Consumer's bundler must handle NeDB's CommonJS-to-ESM conversion and the browser field in NeDB's package.json
• getAllData() is used for synchronous zero-copy dumps (no async overhead for persistence writes)