CanMan/API.md

CAN Service API Reference

Base URL: http://localhost:3210 API Prefix: /api/v1/can/0 Content-Type: application/json (all responses are JSON)

The 0 in the path is the can_id. MVP supports only container 0.

---

Quick Start

# Start the server
cargo run

# Store a file
curl -X POST http://localhost:3210/api/v1/can/0/ingest \
  -F "file=@photo.jpg" \
  -F "tags=vacation,beach" \
  -F "description=Summer trip"

# Store data (no file needed)
curl -X POST http://localhost:3210/api/v1/can/0/ingest/data \
  -H "Content-Type: application/json" \
  -d '{"data": {"sensor": "temp", "value": 22.5}, "tags": "iot,sensor"}'

# List everything
curl http://localhost:3210/api/v1/can/0/list

# Search by tag
curl "http://localhost:3210/api/v1/can/0/search?tags=vacation"

---

Response Envelope

All responses use a standard wrapper:

// Success
{
  "status": "success",
  "data": { ... }
}

// Error
{
  "status": "error",
  "error": "Human-readable error message"
}

HTTP Status Codes:

Code	Meaning
200	Success
400	Bad request (missing/invalid parameters)
404	Asset not found
500	Internal error / corrupted asset

---

Endpoints

1. Ingest File (Multipart)

Upload a binary file with optional metadata.

POST /api/v1/can/0/ingest
Content-Type: multipart/form-data

Form Fields:

Field	Type	Required	Description
file	Binary	Yes	The file to store
mime_type	String	No	Override MIME type (auto-detected from filename if omitted)
human_file_name	String	No	Logical filename (e.g. report.pdf)
human_readable_path	String	No	Logical folder path (e.g. /docs/reports/)
application	String	No	Application that produced this file
user	String	No	User or agent identity
tags	String	No	Comma-separated tags (e.g. finance,Q4,report)
description	String	No	Human-readable description

Response:

{
  "status": "success",
  "data": {
    "timestamp": 1773014400123,
    "hash": "a3b2c4d5e6f7...",
    "filename": "1773014400123_a3b2c4d5e6f7_finance_Q4.pdf"
  }
}

Example:

curl -X POST http://localhost:3210/api/v1/can/0/ingest \
  -F "file=@quarterly_report.pdf" \
  -F "application=WebUI" \
  -F "user=jason" \
  -F "tags=finance,Q4,report" \
  -F "description=Q4 2025 financial report" \
  -F "human_file_name=quarterly_report.pdf" \
  -F "human_readable_path=/finance/reports/"

---

2. Ingest Data (JSON)

Store any JSON value directly -- no multipart needed. Designed for agents and programmatic use.

POST /api/v1/can/0/ingest/data
Content-Type: application/json

JSON Body:

Field	Type	Required	Default	Description
data	Any JSON	Yes	--	The payload to store. Object, array, string, number, boolean, or null.
mime_type	String	No	application/json	Override MIME type (e.g. text/plain to store as .txt)
human_file_name	String	No		Logical filename
human_readable_path	String	No		Logical folder path
application	String	No		Application/agent that produced this data
user	String	No		User or agent identity
tags	String	No		Comma-separated tags
description	String	No		Human-readable description

The data field is serialized to pretty-printed JSON and stored as a .json file (or .txt etc. if you override mime_type).

Response: Same as file ingest.

Examples:

# Minimal -- just dump an object
curl -X POST http://localhost:3210/api/v1/can/0/ingest/data \
  -H "Content-Type: application/json" \
  -d '{"data": {"key": "value"}}'

# Agent saving structured output
curl -X POST http://localhost:3210/api/v1/can/0/ingest/data \
  -H "Content-Type: application/json" \
  -d '{
    "data": {
      "agent_id": "planner-v2",
      "session": "abc-123",
      "steps": ["research", "outline", "draft"]
    },
    "application": "AgentOrchestrator",
    "user": "planner_agent",
    "tags": "agent,plan,session",
    "description": "Planning output for session abc-123",
    "human_file_name": "plan_output.json",
    "human_readable_path": "/agents/planner/"
  }'

# Store a plain string
curl -X POST http://localhost:3210/api/v1/can/0/ingest/data \
  -H "Content-Type: application/json" \
  -d '{"data": "Log: task completed at 14:30", "tags": "log"}'

# Store an array
curl -X POST http://localhost:3210/api/v1/can/0/ingest/data \
  -H "Content-Type: application/json" \
  -d '{"data": [1, 2, 3, "four"], "tags": "test"}'

# Store as plain text instead of JSON
curl -X POST http://localhost:3210/api/v1/can/0/ingest/data \
  -H "Content-Type: application/json" \
  -d '{"data": "plain text content", "mime_type": "text/plain"}'

---

3. Retrieve Asset

Download the physical file by its hash.

GET /api/v1/can/0/asset/{hash}

Path Parameters:

Param	Type	Description
hash	String	The SHA-256 hash returned from ingest

Response: Raw file bytes with headers:

• Content-Type set to the asset's MIME type
• Content-Disposition: attachment; filename="<human_filename>"

Returns 500 if the asset is flagged as corrupted.

Example:

curl -o output.pdf http://localhost:3210/api/v1/can/0/asset/a3b2c4d5e6f7...

---

4. Get Asset Metadata

Retrieve all metadata for an asset without downloading the file.

GET /api/v1/can/0/asset/{hash}/meta

Response:

{
  "status": "success",
  "data": {
    "hash": "a3b2c4d5e6f7...",
    "mime_type": "application/pdf",
    "application": "WebUI",
    "user": "jason",
    "tags": ["finance", "Q4", "report"],
    "description": "Q4 2025 financial report",
    "human_filename": "quarterly_report.pdf",
    "human_path": "/finance/reports/",
    "timestamp": 1773014400123,
    "is_trashed": false,
    "is_corrupted": false
  }
}

---

5. Update Metadata

Modify tags and/or description for an existing asset. The physical file is unchanged.

PATCH /api/v1/can/0/asset/{hash}
Content-Type: application/json

JSON Body:

Field	Type	Required	Description
tags	String[]	No	New tag list (replaces all existing tags)
description	String	No	New description

Both fields are optional. Only provided fields are updated.

Example:

curl -X PATCH http://localhost:3210/api/v1/can/0/asset/a3b2c4d5e6f7... \
  -H "Content-Type: application/json" \
  -d '{
    "tags": ["finance", "Q4", "report", "reviewed"],
    "description": "Q4 2025 report - reviewed by CFO"
  }'

Response:

{
  "status": "success",
  "data": "updated"
}

---

6. List Assets

Paginated listing of all assets with optional filtering.

GET /api/v1/can/0/list

Query Parameters:

Param	Type	Default	Description
limit	Integer	50	Page size
offset	Integer	0	Starting position
offset_time	Integer	--	Epoch ms cursor. Lists items strictly before/after this timestamp (based on order). Faster than offset for large datasets.
order	String	desc	Sort direction: asc or desc (by timestamp)
application	String	--	Filter to assets from this application
include_trashed	Boolean	false	Include soft-deleted assets
include_corrupted	Boolean	false	Include corrupted assets

Response:

{
  "status": "success",
  "data": {
    "items": [
      {
        "hash": "a3b2...",
        "mime_type": "application/pdf",
        "application": "WebUI",
        "user": "jason",
        "tags": ["finance"],
        "description": "...",
        "human_filename": "report.pdf",
        "human_path": "/docs/",
        "timestamp": 1773014400123,
        "is_trashed": false,
        "is_corrupted": false
      }
    ],
    "pagination": {
      "limit": 50,
      "offset": 0,
      "total": 142
    }
  }
}

Examples:

# First page, 10 items
curl "http://localhost:3210/api/v1/can/0/list?limit=10"

# Second page
curl "http://localhost:3210/api/v1/can/0/list?limit=10&offset=10"

# Only assets from a specific app, oldest first
curl "http://localhost:3210/api/v1/can/0/list?application=IoTAgent&order=asc"

# Include trashed assets
curl "http://localhost:3210/api/v1/can/0/list?include_trashed=true"

---

7. Search Assets

Search with multiple filters. All filters are AND-combined.

GET /api/v1/can/0/search

Query Parameters:

Param	Type	Default	Description
hash	String	--	Exact hash or prefix match (e.g. a3b2 matches a3b2c4d5...)
start_time	Integer	--	Epoch ms lower bound (inclusive)
end_time	Integer	--	Epoch ms upper bound (inclusive)
tags	String	--	Comma-separated. AND logic: asset must have all specified tags.
mime_type	String	--	Exact MIME type match (e.g. image/jpeg)
user	String	--	Exact user/agent identity match
application	String	--	Exact application match
limit	Integer	50	Page size
offset	Integer	0	Starting position
order	String	desc	Sort direction: asc or desc
include_trashed	Boolean	false	Include soft-deleted assets
include_corrupted	Boolean	false	Include corrupted assets

Response: Same structure as List (items + pagination).

Examples:

# Find by hash prefix
curl "http://localhost:3210/api/v1/can/0/search?hash=a3b2"

# All JPEG images from last 24 hours
curl "http://localhost:3210/api/v1/can/0/search?mime_type=image/jpeg&start_time=1773014400000"

# Assets tagged with BOTH "sensor" AND "temperature"
curl "http://localhost:3210/api/v1/can/0/search?tags=sensor,temperature"

# Everything from a specific agent
curl "http://localhost:3210/api/v1/can/0/search?application=PlannerAgent&user=agent_v2"

# Combine filters
curl "http://localhost:3210/api/v1/can/0/search?tags=report&application=WebUI&order=asc&limit=5"

---

8. Get Thumbnail

Generate a resized thumbnail for image assets. Non-image assets return a placeholder SVG.

GET /api/v1/can/0/asset/{hash}/thumb/{max_width}/{max_height}

Path Parameters:

Param	Type	Description
hash	String	Asset hash
max_width	Integer	Maximum width in pixels
max_height	Integer	Maximum height in pixels

Aspect ratio is always preserved. The image fits within the max_width x max_height box.

Response:

• Image assets: JPEG bytes (Content-Type: image/jpeg). Cached in .thumbs/ if enabled.
• Non-image assets: SVG placeholder icon (Content-Type: image/svg+xml).

Example:

# 200x200 thumbnail
curl -o thumb.jpg http://localhost:3210/api/v1/can/0/asset/a3b2.../thumb/200/200

---

Configuration

config.yaml at the project root (or pass a custom path as the first CLI argument):

storage_root: "/var/lib/can_data"       # Where files are stored
admin_token: "super_secret_rebuild"     # Bearer token for admin operations
enable_thumbnail_cache: true            # Cache thumbnails in .thumbs/
rebuild_error_threshold: 50             # Error tolerance before hard rebuild
verify_interval_hours: 12               # Hours between integrity scrubs

---

Concepts

Hash

Every asset gets a unique SHA-256 hash computed as SHA256(timestamp_be_bytes + file_content). This hash is the primary identifier used in all API calls. Because the timestamp is mixed in, even identical file content produces different hashes at different times.

Physical Filename

Files are stored as: {timestamp}_{hash}_{tags}.{extension}

For example: 1773014400123_a3b2c4d5e6f7_finance_Q4.pdf

This naming allows offline integrity verification -- you can recompute the hash from the timestamp and file contents and compare it to the filename.

Tags

Tags are comma-separated strings. On ingest and in search, pass them as a single string: "finance,Q4,report". In metadata responses, they come back as an array: ["finance", "Q4", "report"].

When searching, tag matching uses AND logic: ?tags=finance,Q4 finds only assets that have both tags.

When patching, tags are replaced entirely (not merged).

Integrity Verification

A background verifier runs automatically:

1. On startup: Scrubs all assets against their hashes.
2. On file change: Watches the storage directory for modifications.
3. Periodically: Every verify_interval_hours.

Corrupted assets are flagged (is_corrupted: true) and excluded from standard list/search results unless include_corrupted=true is passed. Retrieving a corrupted asset via GET returns a 500 error.

OS File Attributes

Critical metadata is written to the host OS as file attributes for disaster recovery:

• Linux/macOS: Extended attributes (xattr) under the user.can.* namespace
• Windows: NTFS Alternate Data Streams (:can.*)

This means the SQLite database can be rebuilt from scratch by scanning the storage directory.