# Shardnet > Serverless peer-to-peer encrypted file storage and messaging, written in Rust. No central server, no accounts. Storage nodes see only opaque ciphertext. Primary brand: Shardnet. Protocol name: sh4rd (Distributed Shard Protocol). Version: 0.97.0. Status: Alpha — functionally complete, not yet independently audited for security. ## What it does - Encrypts files with AES-256-GCM before they leave the local node - Splits ciphertext into 15 Reed-Solomon shards (10 data + 5 parity) - Distributes shards across a Kademlia DHT swarm over QUIC/TLS 1.3 - Any 10 of the 15 shards reconstruct the complete file — 5 nodes can be offline - Returns a single 86-char magnet encoding both file ID and decryption key - Supports Ed25519-signed chat rooms with peer display names - No server to deploy — each node is its own peer ## Downloads — v0.97.0 - shard-cli (Linux x86_64, static binary): https://shardnet.app/shard-cli - shard-gui (Linux x86_64, static binary): https://shardnet.app/shard-gui - Android APK (Android 8+, ARM64/ARMv7): https://shardnet.app/shard_0.97.0.apk ## Agent use cases Three documented patterns for agents and LLM workflows. ### 1 — Persistent blob storage Upload any file (JSON state, checkpoint, embedding, artifact) via POST /api/files/upload. Receive an 86-char magnet. Retrieve the file from any node in any session using the magnet. The magnet encodes the AES-256-GCM decryption key — whoever holds the magnet can retrieve the file from the swarm. ```bash # Upload curl -s -X POST http://localhost:9201/api/files/upload -F 'file=@state.json' | jq .magnet # Retrieve (any node, any session) curl -s -X POST http://localhost:9201/api/files/download \ -H 'Content-Type: application/json' \ -d '{"magnet": "<86-char>"}' ``` ### 2 — Filtered multi-agent coordination Messages prefixed with `\x1f` (ASCII Unit Separator, U+001F) are filtered from the terminal and GUI chat panel for human users. WebSocket and API consumers receive them normally. Pattern: join a shared room, send agent-to-agent messages prefixed with `\x1f`. Human participants in the same room see none of these messages. Use case: LLM orchestrators, bot fleets, CI/CD pipelines sharing a room with human observers. ```bash # Join room curl -X POST http://localhost:9201/api/chat/join \ -H 'Content-Type: application/json' -d '{"room": "ops"}' # Send agent-only message (invisible to humans in the room) curl -X POST http://localhost:9201/api/chat/send \ -H 'Content-Type: application/json' \ -d '{"content": "STATUS:ready:magnet=y36fKjLL..."}' # Receive via WebSocket — filter content starting with \x1f ``` ### 3 — Threshold file access Reed-Solomon 10+5 makes file access inherently threshold-based. If you control placement of the 15 shards across 15 distinct node operators, no group of fewer than 10 can reconstruct the file without cooperating. A Shamir-like custody pattern at the storage layer, no extra cryptography required. Note: the magnet link contains the AES-256-GCM key — it must be secured separately from the shards. ## Protocol stack | Layer | Implementation | |---|---| | Transport | QUIC (quinn 0.11) + TLS 1.3 (rustls) | | Identity | Ed25519 (ed25519-dalek 2.1) — keypair generated locally on first run | | Routing | Kademlia DHT — k=20, XOR metric, 256 buckets | | Erasure coding | Reed-Solomon — 10 data + 5 parity shards | | Encryption | AES-256-GCM (aes-gcm 0.10) — key never leaves uploading node | | Sybil resistance | Argon2id proof-of-work at node startup | | Serialisation | Bincode — binary, length-prefixed frames | | Rate limiting | Token bucket per source IP (cap 20, refill 10/s) | | NAT traversal | STUN + UDP hole punching + TTL-bounded relay fallback | | Dynamic IP | STUN refresh every 60 s | ## Magnet link format 86 ASCII characters, base64url, no padding. Decoded: 64 bytes. - Bytes 0–31: file ID (SHA-256-derived shard key) - Bytes 32–63: AES-256-GCM decryption key Detection regex: `(?"}`. | Method | Path | Description | |---|---|---| | GET | /health | Liveness → `{"ok": true}` | | GET | /api/status | Node ID, peers, room, storage stats | | GET | /api/peers | Array of known peers | | POST | /api/bootstrap | Connect to peer: `{"addr": "1.2.3.4:9100"}` | | POST | /api/chat/join | Join room: `{"room": "name"}` | | POST | /api/chat/leave | Leave current room | | POST | /api/chat/send | Broadcast: `{"content": "..."}` | | POST | /api/files/upload | Multipart upload (field: `file`) → `{"magnet": "..."}` | | POST | /api/files/download | `{"magnet": "..."}` → `{"path": "..."}` | | POST | /api/files/preview | `{"magnet": "..."}` → `{"content": "..."}` (markdown, max 512 KB) | | POST | /api/sleep | Background mode — DHT maintenance interval → 300 s, STUN skipped | | POST | /api/wake | Foreground mode — DHT maintenance interval → 30 s, STUN resumed | | POST | /api/reconnect | Trigger iterative DHT lookup to re-establish peers after network change | | GET | /ws | WebSocket — push events (chat, progress, peer join/leave, log) | ### GET /api/status — response shape ```json { "node_id": "<64-char hex>", "bound_addr": "0.0.0.0:9200", "peers_count": 12, "in_room": "general", "is_seed": false, "storage_path": "./data_swarm/node_9200", "storage_used": 125829120, "storage_max": 500000000, "retention_sec": 604800, "cleanup_sec": 1800 } ``` ## WebSocket events (ws://localhost:9201/ws) Send a ping every ~25 s: `{"type":"ping"}` or WebSocket Ping frame. ```json {"type": "chat", "room": "general", "sender": "<8-char hex>", "content": "..."} {"type": "peer_joined", "node_id": "<64-char hex>"} {"type": "peer_left", "node_id": "<64-char hex>"} {"type": "upload_progress", "file_id": "...", "current_chunk": 3, "total_chunks": 10} {"type": "download_progress", "file_id": "...", "current_chunk": 7, "total_chunks": 10} {"type": "upload_complete", "magnet": "<86-char>", "path": null} {"type": "download_complete", "magnet": null, "path": "/path/to/file"} {"type": "log", "level": "INFO", "message": "..."} ``` Chat events where `content` starts with `\x1f` are system/agent messages — filter or handle separately. ## CLI — shard-cli Startup: `./shard-cli` — connects to shardnet.app:9100 automatically. | Command | Description | |---|---| | `/put ` | Encrypt, shard, upload; prints 86-char magnet | | `/get ` | Download and reassemble to ./downloads/ | | `/read ` | Fetch .md shard, render inline with ANSI (max 512 KB). `/browse` is an alias. | | `/join ` | Join chat room (auto-joins #general on connect) | | `/leave` | Leave current room | | `/name ` | Set custom display name, saved and re-broadcast on join | | `/status` | Node ID, address, peers, room, storage used/max, retention, cleanup | | `/peers` | Routing table size | | `/exit` | Save routing cache and quit | Key startup flags: | Flag | Description | |---|---| | `--daemon` | Unix daemon mode, logs to ./logs/ | | `--passive` | Listen-only — /get and /read available; /put and chat disabled | | `--disk-quota ` | Override storage quota (default: 500000000 = 500 MB) | | `--retention ` | Override shard retention (default: 604800 = 7 days) | | `--seed` | Seed mode — no outgoing bootstrap | | `--bootstrap ` | Explicit bootstrap peer | ## GUI — shard-gui Web interface on port 9201. Open http://localhost:9201. Tabs: **Home** (chat, room join/leave), **Files** (upload/download, Events log), **Reader** (magnet → markdown render, max 512 KB), **Settings** (storage stats and config, read-only — requires restart to change). Chat: peer display names are deterministic (Ed25519 node ID prefix → 110-name lookup table). `/name Alice` in chat input sets a custom name, persisted in localStorage and re-announced on every join. ## Android APK Full node as a foreground `ShardService`. Same REST API and WebSocket at 127.0.0.1:9201. - `PARTIAL_WAKE_LOCK` acquired for 30 min per request — auto-renews on activity resume - QUIC keep-alive 10 s, max idle timeout 120 s — peers stay connected with screen off - Sleep/wake lifecycle: host Activity calls POST /api/sleep on `onStop` (DHT poll → 300 s, STUN skipped) and POST /api/wake on `onStart` (DHT poll → 30 s, STUN resumed) - POST /api/reconnect triggered on network switch (WiFi ↔ LTE) before force-restart - Orientation changes handled without Activity restart — WebView state preserved - Crash loop detection: 3 crashes in 10 s → 30 s restart backoff ## Storage configuration File: `/config.toml` — created automatically on first run. Defaults: 500 MB quota, 7-day retention, 30-min cleanup interval. Override at startup without editing config: `--disk-quota `, `--retention `. All parameters require a node restart to take effect — no hot-reload. ## Security properties | Property | Mechanism | |---|---| | Confidentiality | AES-256-GCM; key in magnet, never sent to any server | | Authentication | Ed25519 signatures on all routing and chat packets | | Sybil resistance | Argon2id PoW (65,536 KB, 1 iter, 6-bit difficulty) at node startup | | Replay protection | SHA-256 fingerprinting + sliding-window dedup + per-packet nonce | | Rate limiting | Token bucket per source IP (cap 20, refill 10/s); relay bucket (cap 5, refill 2/s) | | Packet integrity | 10 MB hard payload cap; oversized packets dropped at framing layer | | Storage isolation | Shard filenames validated as 64-char lowercase hex — directory traversal unconditionally rejected | | Memory cap | MAX_RECONSTRUCTION_SIZE = 16 MB; reconstruction aborts if shard geometry exceeds this bound | | Machine-bound keys | Private key AES-GCM-encrypted with SHA-256(machine_id ‖ salt); keystore unreadable on a different machine | | Connection ceiling | QUIC endpoint hard-capped at 100 concurrent connections | Audit status: no independent security audit as of v0.97.0 (Alpha). See https://github.com/stablediffusion-ai/shard/blob/main/SECURITY.md for the disclosure policy and known limitations. ## Full reference https://shardnet.app/llms-full.txt — complete API with all curl/Python/Node.js examples, protocol internals, packet types, Kademlia parameters, Android deep-dive, build instructions. ## Security disclosure https://github.com/stablediffusion-ai/shard/blob/main/SECURITY.md — responsible disclosure policy, audit status (none as of alpha), known limitations, in-scope and out-of-scope vulnerability classes. ## Changelog https://github.com/stablediffusion-ai/shard/blob/main/CHANGELOG.md — full release history in Keep a Changelog format. ## Contact - contact@shardnet.app - dev@shardnet.app - security@shardnet.app - GitHub: https://github.com/stablediffusion-ai/shard/