CanMan/README.md

# CAN Service

**Containerized Asset Network** -- A self-healing local storage daemon with HTTP REST and Protobuf APIs for ingesting, managing, and retrieving files.

CAN stores any file you throw at it, tags it with metadata, verifies integrity in the background, and syncs between machines over encrypted P2P connections. Think of it as a personal S3 that runs on your laptop and replicates to your other devices automatically.

---

## Quick Start

```bash
# Build and run (listens on port 3210)
cargo run
```

The service reads `config.yaml` from the current directory:

```yaml
storage_root: "./can_data"
admin_token: "super_secret_rebuild"
enable_thumbnail_cache: true
verify_interval_hours: 12
sync_api_key: "can-sync-default-key"   # enables P2P sync endpoints
```

Override the port with `CAN_PORT=8080 cargo run`.

### Upload a file

```bash
curl -X POST http://localhost:3210/api/v1/can/0/ingest \
  -F "file=@photo.jpg" \
  -F "tags=vacation,summer" \
  -F "application=my-app"
```

### Upload JSON data (agent-friendly)

```bash
curl -X POST http://localhost:3210/api/v1/can/0/ingest/data \
  -H "Content-Type: application/json" \
  -d '{"data": {"key": "value"}, "tags": "config,backup"}'
```

### Download a file

```bash
curl http://localhost:3210/api/v1/can/0/asset/{hash} -o file.jpg
```

---

## How It Works

```
                 +-----------+
    Upload  ---->|           |----> SQLite index (millisecond queries)
                 | CAN       |
    Download <---|  Service  |----> Flat file storage (one file per asset)
                 |           |
    Search  ---->| port 3210 |----> OS file attributes (disaster recovery)
                 |           |
    SSE     <----|           |----> Background verifier (integrity checks)
                 +-----------+
                      |
              P2P Sync (protobuf over QUIC)
                      |
                 +-----------+
                 | CAN       |
                 |  Service  |  (another machine)
                 | port 3210 |
                 +-----------+
```

Each asset is saved as `{timestamp}_{sha256hash}_{tags}.{ext}` in a flat directory. Metadata lives in SQLite for fast queries and is redundantly written to OS-level file attributes (xattr on macOS/Linux, NTFS ADS on Windows) so you can recover even if the database is lost.

A background verifier re-hashes every file periodically and flags corruption. It also watches for filesystem changes in real time.

---

## API

All endpoints live under `/api/v1/can/0/`. See [API.md](API.md) for the full specification.

| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/ingest` | Upload a file (multipart form) |
| `POST` | `/ingest/data` | Upload JSON data (no multipart needed) |
| `GET` | `/asset/{hash}` | Download an asset by its SHA-256 hash |
| `GET` | `/asset/{hash}/meta` | Get metadata as JSON |
| `PATCH` | `/asset/{hash}` | Update tags and/or description |
| `GET` | `/asset/{hash}/thumb/{w}/{h}` | Get a resized thumbnail (images only) |
| `GET` | `/list` | Paginated listing with filters |
| `GET` | `/search` | Search by hash prefix, time range, MIME, tags, etc. |
| `GET` | `/events` | SSE stream of new asset notifications |

Private sync endpoints (`/sync/*`) use protobuf and require the `X-Sync-Key` header.

---

## Examples

Four example apps show what you can build on top of CAN:

| Example | Port | Description | README |
|---------|------|-------------|--------|
| **[Paste](examples/paste/)** | 3211 | Pastebin -- type text or paste images, auto-tags with #hashtags | [README](examples/paste/README.md) |
| **[File Manager](examples/filemanager/)** | 3212 | Web file browser with grid/list views, search, and filters | [README](examples/filemanager/README.md) |
| **[CAN Sync](examples/can-sync/)** | -- | P2P replication agent -- encrypted sync via shared passphrase | [README](examples/can-sync/README.md) |
| **[CanFS](examples/canfs/)** | -- | Mount assets as a read-only Windows drive (WinFSP) | [README](examples/canfs/README.md) |

### Run everything at once

```powershell
# Windows
.\go_example_1.ps1

# macOS / Linux
./go_example_1.sh
```

Starts CAN Service + Sync Agent + Paste, builds everything, cleans up on Ctrl+C.

---

## Configuration

| Field | Default | Description |
|-------|---------|-------------|
| `storage_root` | (required) | Directory where assets and the database are stored |
| `admin_token` | `"changeme"` | Bearer token for admin endpoints |
| `enable_thumbnail_cache` | `true` | Cache resized thumbnails in `.thumbs/` |
| `rebuild_error_threshold` | `50` | Max errors before triggering a full rebuild |
| `verify_interval_hours` | `12` | Hours between full integrity scans |
| `sync_api_key` | (none) | API key for sync endpoints; omit to disable sync |

---

## Project Structure

```
src/
  main.rs          Entry point: config, DB, verifier, HTTP server
  config.rs        YAML config loading
  db.rs            SQLite CRUD (assets, tags, search)
  hash.rs          SHA-256 content hashing
  storage.rs       File I/O (write, read, trash, filename parsing)
  verifier.rs      Background integrity checker + file watcher
  xattr.rs         OS-level file attributes (xattr / NTFS ADS)
  routes/          HTTP API handlers (ingest, asset, list, search, thumb, sync, events)
examples/
  paste/           Pastebin web app
  filemanager/     File browser web app
  can-sync/        P2P sync agent (iroh + gossip + pkarr)
  canfs/           Windows virtual filesystem (WinFSP)
```

## Requirements

- **Rust** 1.75+
- **SQLite** bundled (no system install needed)
- **WinFSP** only for the canfs example (Windows only)