# CAN Sync

P2P full-mirror replication for [CAN Service](../../). Two machines with the same passphrase automatically discover each other and sync all assets over encrypted connections. No port forwarding or static IPs needed.

```
┌─────────────┐  protobuf   ┌─────────────┐  iroh (QUIC)  ┌─────────────┐  protobuf   ┌─────────────┐
│ CAN Service │◄───────────►│  CAN Sync   │◄─────────────►│  CAN Sync   │◄───────────►│ CAN Service │
│  Machine A  │  sync API   │  Agent A    │   encrypted   │  Agent B    │  sync API   │  Machine B  │
│  port 3210  │             │             │               │             │             │  port 3210  │
└─────────────┘             └─────────────┘               └─────────────┘             └─────────────┘
```

## Quick Start

1. **Start CAN Service** on each machine (port 3210):
   ```bash
   cargo run
   ```

2. **Configure the sync agent** -- edit `config.yaml`:
   ```yaml
   can_service_url: "http://127.0.0.1:3210"
   sync_api_key: "can-sync-default-key"
   sync_passphrase: "my-secret-phrase"    # must be the same on all machines
   poll_interval_secs: 30
   ```

3. **Start the sync agent** on each machine:
   ```bash
   cd examples/can-sync
   cargo run -- config.yaml
   ```

That's it. Any file uploaded to either CAN Service will appear on the other within seconds.

## How It Works

### Peer Discovery

Peers find each other through two mechanisms (both run simultaneously):

- **Gossip** -- [iroh-gossip](https://docs.rs/iroh-gossip) uses a topic derived from the shared passphrase. Peers on the same local network or connected to the same relay discover each other by broadcasting their node IDs.
- **Internet rendezvous** -- Each agent publishes its node ID to [pkarr](https://pkarr.org) relay servers using deterministic DNS-like "slots" derived from the passphrase. All agents scan these slots periodically to find peers worldwide.

### Sync Protocol

Once two peers connect over iroh's encrypted QUIC transport:

1. **Hash exchange** -- Both sides send their full list of asset hashes
2. **Diff** -- Each side computes what the other is missing
3. **Transfer** -- Missing assets are sent concurrently in both directions (metadata + file content bundled together as protobuf)
4. **Live sync** -- After the initial reconciliation, each agent subscribes to SSE events from its local CAN Service. When a new asset is ingested locally, it's pushed to the connected peer instantly.

The live sync uses SSE events (not polling) for instant propagation. A fallback incremental poll runs every 30 seconds as a safety net.

### Echo Prevention

When peer A sends an asset to peer B, B's CAN Service emits an SSE event for the new ingest. Without protection, B would try to push that asset right back to A. The sync agent tracks which hashes were received from each peer and filters them out of the push loop.

## Configuration

| Field | Default | Description |
|-------|---------|-------------|
| `can_service_url` | (required) | URL of the local CAN Service |
| `sync_api_key` | (required) | Must match `sync_api_key` in CAN Service's config |
| `sync_passphrase` | (required) | Shared secret for peer discovery (all peers must match) |
| `poll_interval_secs` | `3` | Fallback poll interval for catching missed events |
| `ticket_file` | (none) | Write this node's address to a file (for direct connection in tests) |
| `connect_ticket_file` | (none) | Read a peer's address from a file (for direct connection in tests) |

CAN Service must have `sync_api_key` set in its `config.yaml` for the sync endpoints to be enabled.

## Security

- **Transport** -- All peer traffic is encrypted with QUIC + TLS 1.3 (mandatory in iroh)
- **Identity** -- Each node gets an Ed25519 keypair on first run
- **Discovery** -- Only peers with the same passphrase can find each other
- **Hash verification** -- Every received asset is re-hashed and compared before being stored

## Project Structure

```
src/
  main.rs          Entry point: config, iroh endpoint, discovery, peer connections
  config.rs        YAML config loading
  can_client.rs    HTTP client for CAN Service's sync API (protobuf + SSE)
  protocol.rs      Protobuf message types (shared with CAN Service)
  discovery.rs     Peer discovery via iroh-gossip
  rendezvous.rs    Internet peer discovery via pkarr relay
  peer.rs          Per-peer sync: reconciliation, live push/receive, echo prevention
```