laantungir/ginxsom
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

v0.1.1 - Cleaning things up.

Browse Source
This commit is contained in:
Your Name
2025-10-16 15:24:41 -04:00
parent 25a871bb31
commit bb18ffcdce
18 changed files with 0 additions and 1894 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
docs/AGENTS.md Normal file
View File

@@ -0,0 +1,41 @@
# AGENTS.md
This file provides guidance to agents when working with code in this repository.
## Critical Project-Specific Rules
- **Database Path**: Always use `db/ginxsom.db` - this is hardcoded, not configurable
- **SQLite Build**: Uses local SQLite 3.37.2 in `sqlite3-build/` directory, NOT system SQLite
- **Local Development**: Everything runs locally on port 9001, never use system nginx on port 80
- **FastCGI Socket**: Uses `/tmp/ginxsom-fcgi.sock` for FastCGI communication
- **Config Priority**: File-based config (XDG locations) overrides database config
- **Admin Auth**: Uses Nostr events for admin API authentication (kind 24242 with "admin" tag)
- **Blob Storage**: Files stored as `blobs/<sha256>.<ext>` where extension comes from MIME type
- **Build Directory**: Must create `build/` directory before compilation
- **Test Files**: Tests are in the tests/ directory, and they should be run from the root, i.e. 'tests/mirror_test_bud04.sh'
## Non-Standard Commands
```bash
# make clean && make && Restart nginx
./restart-all.sh
# Start FastCGI daemon
./scripts/start-fcgi.sh
# Test admin API with authentication
source .admin_keys && ./scripts/test_admin.sh
# Setup wizard (creates signed config events)
./scripts/setup.sh
```
## Critical Architecture Notes
- FastCGI app handles HEAD/PUT/DELETE requests, nginx serves GET directly from disk
- Two-tier config: File config (signed Nostr events) + database config (key-value)
- Admin API requires Nostr event signatures with specific tag structure
- Database is single source of truth for blob existence (not filesystem)
- Extension handling: URLs work with any extension, files stored with correct extension

504
docs/AUTH_API.md Normal file
View File

@@ -0,0 +1,504 @@
# Authentication API Documentation
## Authentication Flow and Order of Operations
### Authentication Flow Diagram
```
┌─────────────────────┐
│ Request Received │
└──────────┬──────────┘
│
▼
┌─────────────┐ ╔═══════════════════╗
│ Input Valid?├─No─►║ REJECT: Invalid ║
└──────┬──────┘ ║ Input (~1μs) ║
│Yes ╚═══════════════════╝
▼
┌─────────────┐ ╔═══════════════════╗
│System Init? ├─No─►║ REJECT: Not ║
└──────┬──────┘ ║ Initialized ║
│Yes ╚═══════════════════╝
▼
┌─────────────┐
│Auth Header? │
└──────┬──────┘
│Yes
▼ ┌─────────────────────┐
┌─────────────┐ No │ │
│Parse Header ├────────────┤ Skip Nostr │
└──────┬──────┘ │ Validation │
│ │ │
▼ └──────────┬──────────┘
┌─────────────┐ ╔═══════════════════╗ │
│Valid Base64?├─No─►║ REJECT: Malformed ║ │
└──────┬──────┘ ║ Header (~10μs) ║ │
│Yes ╚═══════════════════╝ │
▼ │
┌─────────────┐ ╔═══════════════════╗ │
│Valid JSON? ├─No─►║ REJECT: Invalid ║ │
└──────┬──────┘ ║ JSON (~50μs) ║ │
│Yes ╚═══════════════════╝ │
▼ │
┌─────────────┐ ╔═══════════════════╗ │
│Valid Struct?├─No─►║ REJECT: Invalid ║ │
└──────┬──────┘ ║ Structure (~100μs)║ │
│Yes ╚═══════════════════╝ │
▼ │
┌─────────────┐ ╔═══════════════════╗ │
│Event Kind? │ ║ ║ │
└──────┬──────┘ ║ DUAL AUTH MODES ║ │
│ ╚═══════════════════╝ │
▼ │
┌─────────────────────────────────────────────────────┐
│ │
▼ ▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Kind │ │ Kind │ │ Other │ │ Invalid │
│ 22242 │ │ 24242 │ │ Kinds │ │ Kind │
│ (NIP-42) │ │(Blossom) │ │ (Skip) │ │(Reject) │
└─────┬────┘ └─────┬────┘ └─────┬────┘ └─────┬────┘
│ │ │ │
▼ │ ▼ ▼
┌──────────┐ │ ┌──────────┐ ╔═══════════════════╗
│ NIP-42 │ │ │ Skip │ ║ REJECT: Invalid ║
│Challenge │ │ │ Nostr │ ║ Event Kind ║
│Validate │ │ │Validate │ ╚═══════════════════╝
│(~500μs) │ │ └─────┬────┘ │
└─────┬────┘ │ │ │
│ │ ▼ │
│ │ ┌──────────┐ │
│ │ │ Extract │ │
│ │ │ Context │ │
│ │ └─────┬────┘ │
│ │ │ │
▼ ▼ ▼ ▼
┌────────────────────────────────────────────────────────────┐
│ ECDSA SIGNATURE VERIFICATION │
│ (~2ms) │
└───────────────────────────┬────────────────────────────────┘
│Yes
▼ │
┌─────────────────┐ ╔═══════════════════╗
│Operation Match? │No ║ REJECT: Unauth. ║
│(Kind 24242 only)├──►║ Operation (~200μs)║
└─────────┬───────┘ ╚═══════════════════╝
│Yes/Skip(Kind 22242)
▼ │
┌─────────────────┐ ╔═══════════════════╗ │
│Event Expired? │Yes║ REJECT: Expired ║ │
└─────────┬───────┘ ║ Event (~50μs) ║ │
│No ╚═══════════════════╝ │
▼ │
┌─────────────────┐ │
│Extract Pubkey │ │
│& Auth Context │ │
└─────────┬───────┘ │
│ │
▼◄───────────────────────────────────────┘
┌─────────────────┐ ╔═══════════════════╗
│Auth Rules │ No ║ ALLOW: Rules ║
│Enabled? ├────►║ Disabled ║
└─────────┬───────┘ ╚═══════════════════╝
│Yes
▼
┌─────────────────┐
│Generate Cache │
│Key (SHA-256) │
└─────────┬───────┘
│
▼
┌─────────────────┐ ╔═══════════════════╗
│Cache Hit? │ Yes ║ RETURN: Cached ║
│(~100μs lookup) ├────►║ Decision (~100μs) ║
└─────────┬───────┘ ╚═══════════════════╝
│No
▼
╔═══════════════════════════════════════╗
║ RULE EVALUATION ENGINE ║
║ (Priority Order) ║
╚═══════════════════════════════════════╝
│
▼
┌─────────────────┐ ╔═══════════════════╗
│1. Pubkey │ Yes ║ DENY: Pubkey ║
│ Blacklisted? ├────►║ Blocked ║
└─────────┬───────┘ ╚═══════════════════╝
│No
▼
┌─────────────────┐ ╔═══════════════════╗
│2. Hash │ Yes ║ DENY: Hash ║
│ Blacklisted? ├────►║ Blocked ║
└─────────┬───────┘ ╚═══════════════════╝
│No
▼
┌─────────────────┐ ╔═══════════════════╗
│3. MIME Type │ Yes ║ DENY: MIME ║
│ Blacklisted? ├────►║ Blocked ║
└─────────┬───────┘ ╚═══════════════════╝
│No
▼
┌─────────────────┐ ╔═══════════════════╗
│4. Size Limit │ Yes ║ DENY: File ║
│ Exceeded? ├────►║ Too Large ║
└─────────┬───────┘ ╚═══════════════════╝
│No
▼
┌─────────────────┐ ╔═══════════════════╗
│5. Pubkey │ Yes ║ ALLOW: Pubkey ║
│ Whitelisted? ├────►║ Whitelisted ║
└─────────┬───────┘ ╚═══════════════════╝
│No
▼
┌─────────────────┐ ╔═══════════════════╗
│6. MIME Type │ Yes ║ ALLOW: MIME ║
│ Whitelisted? ├────►║ Whitelisted ║
└─────────┬───────┘ ╚═══════════════════╝
│No
▼
┌─────────────────┐ ╔═══════════════════╗
│Whitelist Rules │ Yes ║ DENY: Not in ║
│Exist? ├────►║ Whitelist ║
└─────────┬───────┘ ╚═══════════════════╝
│No
▼
╔═══════════════════╗
║ ALLOW: Default ║
║ Allow Policy ║
╚═══════════════════╝
│
▼
┌─────────────────┐
│ Cache Decision │
│ (5min TTL) │
└─────────┬───────┘
│
▼
┌─────────────────┐
│ Return Result │
│ to Application │
└─────────────────┘
```
### Authentication Flow (Mermaid)
```mermaid
flowchart TD
A[Request Received] --> B{Input Valid?}
B -->|No| R1[REJECT: Invalid Input ~1μs]
B -->|Yes| C{System Init?}
C -->|No| R2[REJECT: Not Initialized]
C -->|Yes| D{Auth Header?}
D -->|No| SKIP[Skip Nostr Validation]
D -->|Yes| E{Valid Base64?}
E -->|No| R3[REJECT: Malformed Header ~10μs]
E -->|Yes| F{Valid JSON?}
F -->|No| R4[REJECT: Invalid JSON ~50μs]
F -->|Yes| G{Valid Struct?}
G -->|No| R5[REJECT: Invalid Structure ~100μs]
G -->|Yes| H{Event Kind?}
%% Event Kind Branching
H --> I[Kind 22242<br/>NIP-42]
H --> J[Kind 24242<br/>Blossom]
H --> K[Other Kinds<br/>Skip]
H --> L[Invalid Kind]
L --> R6[REJECT: Invalid Event Kind]
%% NIP-42 Path
I --> M[NIP-42 Challenge<br/>Validation ~500μs]
M --> N[ECDSA Signature<br/>Verification ~2ms]
%% Blossom Path
J --> N
%% Skip Path
K --> SKIP
SKIP --> O[Extract Context]
O --> N
%% Signature Verification
N -->|No| R7[REJECT: Invalid Signature ~2ms]
N -->|Yes| P{Operation Match?<br/>Kind 24242 only}
P -->|No| R8[REJECT: Unauthorized Operation ~200μs]
P -->|Yes/Skip 22242| Q{Event Expired?}
Q -->|Yes| R9[REJECT: Expired Event ~50μs]
Q -->|No| S[Extract Pubkey & Auth Context]
%% Rules Engine
S --> T{Auth Rules Enabled?}
T -->|No| ALLOW1[ALLOW: Rules Disabled]
T -->|Yes| U[Generate Cache Key SHA-256]
U --> V{Cache Hit?}
V -->|Yes| CACHED[RETURN: Cached Decision ~100μs]
V -->|No| W[RULE EVALUATION ENGINE<br/>Priority Order]
%% Rule Checks
W --> X1{1. Pubkey Blacklisted?}
X1 -->|Yes| DENY1[DENY: Pubkey Blocked]
X1 -->|No| X2{2. Hash Blacklisted?}
X2 -->|Yes| DENY2[DENY: Hash Blocked]
X2 -->|No| X3{3. MIME Blacklisted?}
X3 -->|Yes| DENY3[DENY: MIME Blocked]
X3 -->|No| X4{4. Size Limit Exceeded?}
X4 -->|Yes| DENY4[DENY: File Too Large]
X4 -->|No| X5{5. Pubkey Whitelisted?}
X5 -->|Yes| ALLOW2[ALLOW: Pubkey Whitelisted]
X5 -->|No| X6{6. MIME Whitelisted?}
X6 -->|Yes| ALLOW3[ALLOW: MIME Whitelisted]
X6 -->|No| X7{Whitelist Rules Exist?}
X7 -->|Yes| DENY5[DENY: Not in Whitelist]
X7 -->|No| ALLOW4[ALLOW: Default Policy]
%% Final Steps
ALLOW2 --> CACHE[Cache Decision 5min TTL]
ALLOW3 --> CACHE
ALLOW4 --> CACHE
DENY1 --> CACHE
DENY2 --> CACHE
DENY3 --> CACHE
DENY4 --> CACHE
DENY5 --> CACHE
CACHE --> RESULT[Return Result to Application]
%% Styling
classDef rejectBox fill:#ff4444,stroke:#ffffff,color:#ffffff
classDef allowBox fill:#44ff44,stroke:#ffffff,color:#000000
classDef processBox fill:#4444ff,stroke:#ffffff,color:#ffffff
classDef decisionBox fill:#ffff44,stroke:#000000,color:#000000
class R1,R2,R3,R4,R5,R6,R7,R8,R9,DENY1,DENY2,DENY3,DENY4,DENY5 rejectBox
class ALLOW1,ALLOW2,ALLOW3,ALLOW4,CACHED allowBox
class A,M,N,S,U,W,CACHE,RESULT,SKIP,O processBox
class B,C,D,E,F,G,H,P,Q,T,V,X1,X2,X3,X4,X5,X6,X7 decisionBox
```
### Performance Timeline (ASCII)
```
Fast Path (Cache Hit) - Total: ~101μs
┌─────┬─────────────────────────────────────────────────────────────────┬──────┐
│ 1μs │ 100μs Cache Lookup │ 1μs │
└─────┴─────────────────────────────────────────────────────────────────┴──────┘
Input │ │ Return
Valid │ SQLite SELECT │ Result
Typical Path (Valid Request) - Total: ~2.4ms
┌──┬───┬────┬─────────────────────────┬────────┬────┬──┐
│1μ│50μ│100μ│ 2000μs │ 200μs │100μ│1μ│
└──┴───┴────┴─────────────────────────┴────────┴────┴──┘
│ │ │ │ │ │ │
│ │ │ │ ECDSA Signature │ Rule │Cache│Return
│ │ │ │ Verification │ Eval │Store│Result
│ │ │ │ (Most Expensive) │ │ │
│ │ │
│ │ JSON Parse
│ Header Parse
Input Validation
Worst Case (Full Validation) - Total: ~2.7ms
┌──┬───┬────┬─────────────────────────┬─────────┬────┬──┐
│1μ│50μ│100μ│ 2000μs │ 500μs │100μ│1μ│
└──┴───┴────┴─────────────────────────┴─────────┴────┴──┘
│
All 6 Rule Checks
(Multiple DB Queries)
```
### Dual Authentication Architecture
The system supports **two authentication modes** that can work independently or together:
#### **NIP-42 Authentication (Kind 22242)**
- **Purpose**: Client identity authentication ("who you are")
- **Use Case**: Relay client authentication, WebSocket connections
- **Event Structure**: Contains `relay` URL and `challenge` tags
- **Flow**: Challenge/response pattern with relay-generated challenges
- **Validation**: Verifies client identity against relay URL and challenge
- **Database Storage**: Client sessions and challenge tracking
#### **Blossom Protocol Authentication (Kind 24242)**
- **Purpose**: Operation authorization ("what you can do")
- **Use Case**: File upload/delete/list operations
- **Event Structure**: Contains operation tag `t=upload|delete|list` and content hash `x=hash`
- **Flow**: Direct operation authorization with expiration
- **Validation**: Verifies operation permissions and content integrity
- **Database Storage**: Operation-specific authentication rules
#### **Integration Strategy**
```
┌─────────────────┐ ┌─────────────────┐
│ NIP-42 Auth │ │ Blossom Auth │
│ (Kind 22242) │ │ (Kind 24242) │
│ │ │ │
│ Client Identity │ │ Operation Perms │
│ Challenge/Resp │ │ File Operations │
└─────────┬───────┘ └─────────┬───────┘
│ │
└──────┬─────────┬─────┘
│ │
▼ ▼
┌─────────────────┐
│ Unified Rules │
│ Engine │
│ │
│ • Pubkey Rules │
│ • Hash Rules │
│ • MIME Rules │
│ • Size Limits │
└─────────────────┘
```
#### **Event Kind Processing**
- **Kind 22242** (NIP-42): Validates against stored challenge + relay URL
- **Kind 24242** (Blossom): Validates operation tags + content integrity
- **Other Kinds**: Skip Nostr validation, proceed to rule evaluation
- **Invalid Kind**: Reject immediately
#### **Dual Mode Benefits**
- **Backwards Compatibility**: Existing Blossom clients continue working
- **Enhanced Security**: NIP-42 provides cryptographic client identity
- **Flexible Deployment**: Relays can require either or both methods
- **Performance**: Separate validation paths optimize for each use case
### Request Processing Flow (DDoS-Optimized)
The authentication system is designed with **performance and DDoS protection** as primary concerns. Here's the exact order of operations:
#### Phase 1: Input Validation (Immediate Rejection)
1. **Null Pointer Checks** - Reject malformed requests instantly (lines 122-128)
2. **Initialization Check** - Verify system is properly initialized
3. **Basic Structure Validation** - Ensure required fields are present
#### Phase 2: Nostr Event Validation (CPU Intensive)
4. **Authorization Header Parsing** (lines 139-148)
- Extract base64-encoded Nostr event from `Authorization: Nostr <base64>` header
- Decode base64 to JSON (memory allocation + decoding)
- **Early exit**: Invalid base64 or malformed header rejected immediately
5. **JSON Parsing** (lines 150-156)
- Parse Nostr event JSON using cJSON
- **Early exit**: Invalid JSON rejected before signature verification
6. **Nostr Event Structure Validation** (lines 159-166)
- Validate event has required fields (kind, pubkey, sig, etc.)
- **Early exit**: Invalid structure rejected before expensive crypto operations
7. **Event Kind Routing** (NEW - Dual Authentication)
- **Kind 22242** (NIP-42): Route to NIP-42 challenge validation
- **Kind 24242** (Blossom): Route to Blossom operation validation
- **Other Kinds**: Skip Nostr validation, proceed to rules
- **Invalid Kind**: Reject immediately
8. **NIP-42 Challenge Validation** (Kind 22242 Only)
- Validate `relay` tag matches configured relay URL
- Verify `challenge` tag exists and matches stored challenge
- Check challenge expiration and single-use constraints
- **Performance**: ~500μs additional validation overhead
9. **Cryptographic Signature Verification** (Both Paths)
- **Most CPU-intensive operation** - ECDSA signature verification
- Validates event authenticity using secp256k1
- **Early exit**: Invalid signatures rejected before database queries
10. **Operation-Specific Validation** (Kind 24242 Only)
- Verify event authorizes the requested operation (upload/delete/list)
- Check required tags (t=operation, x=hash, expiration)
- Validate timestamp and expiration
- **Early exit**: Expired or mismatched events rejected
11. **Public Key Extraction** (Both Paths)
- Extract validated public key from event for rule evaluation
- Store authentication context (NIP-42 vs Blossom) for rule processing
#### Phase 3: Authentication Rules (Database Queries)
10. **Rules System Check** (line 191)
- Quick config check if authentication rules are enabled
- **Early exit**: If disabled, allow request immediately
11. **Cache Lookup** (lines 1051-1054)
- Generate SHA-256 cache key from request parameters
- Check SQLite cache for previous decision
- **Early exit**: Cache hit returns cached decision (5-minute TTL)
12. **Rule Evaluation** (Priority Order - lines 1061-1094):
- **a. Pubkey Blacklist** (highest priority) - Immediate denial if matched
- **b. Hash Blacklist** - Block specific content hashes
- **c. MIME Type Blacklist** - Block dangerous file types
- **d. File Size Limits** - Enforce upload size restrictions
- **e. Pubkey Whitelist** - Allow specific users (only if not denied above)
- **f. MIME Type Whitelist** - Allow specific file types
13. **Whitelist Default Denial** (lines 1097-1121)
- If whitelist rules exist but none matched, deny request
- Prevents whitelist bypass attacks
14. **Cache Storage** (line 1124)
- Store decision in cache for future requests (5-minute TTL)
### DDoS Protection Features
#### **Fail-Fast Design**
- **Input validation** happens before any expensive operations
- **Authorization header parsing** fails fast on malformed data
- **JSON parsing** rejects invalid data before signature verification
- **Structure validation** happens before cryptographic operations
#### **Expensive Operations Last**
- **Signature verification** only after structure validation
- **Database queries** only after successful Nostr validation
- **Cache prioritized** over database queries
#### **Caching Strategy**
- **SHA-256 cache keys** prevent cache pollution attacks
- **5-minute TTL** balances performance with rule changes
- **LRU eviction** prevents memory exhaustion
- **Per-request caching** includes all parameters (pubkey, operation, hash, MIME, size)
#### **Resource Limits**
- **JSON parsing** limited to 4KB buffer size
- **Cache entries** limited to prevent memory exhaustion
- **Database connection pooling** (single connection with proper cleanup)
- **String length limits** on all inputs
#### **Attack Mitigation**
- **Base64 bombs** - Limited decode buffer size (4KB)
- **JSON bombs** - cJSON library handles malformed JSON safely
- **Cache poisoning** - Cryptographic cache keys prevent collisions
- **Rule bypass** - Whitelist default denial prevents unauthorized access
- **Replay attacks** - Timestamp and expiration validation
- **Hash collision attacks** - Full SHA-256 verification
### Performance Characteristics
#### **Best Case** (Cached Decision):
1. Input validation: ~1μs
2. Cache lookup: ~100μs (SQLite SELECT)
3. **Total: ~101μs**
#### **Worst Case** (Full Validation + Rule Evaluation):
1. Input validation: ~1μs
2. Base64 decoding: ~50μs
3. JSON parsing: ~100μs
4. Signature verification: ~2000μs (ECDSA)
5. Database queries: ~500μs (6 rule checks)
6. Cache storage: ~100μs
7. **Total: ~2751μs (~2.7ms)**
#### **Typical Case** (Valid Request, Rules Enabled):
1. Full validation: ~2200μs
2. Cache miss, 2-3 rule checks: ~200μs
3. **Total: ~2400μs (~2.4ms)**
### Security Order Rationale
The rule evaluation order is specifically designed for security:
1. **Blacklists First** - Immediate denial of known bad actors
2. **Resource Limits** - Prevent resource exhaustion attacks
3. **Whitelists Last** - Only allow after passing all security checks
4. **Default Deny** - If whitelists exist but don't match, deny
This ensures that even if an attacker bypasses one layer, subsequent layers will catch the attack.

509
docs/BLOSSOM_FLOW.md Normal file
View File

@@ -0,0 +1,509 @@
# Blossom Protocol Flow Charts
This document provides ASCII flow charts illustrating how each Blossom Upgrade Document (BUD) works in practice.
---
## BUD-01: Basic Blob Retrieval
### GET Request Flow
```
Client Request nginx Database
| | |
| GET /<sha256> | |
|--------------------------->| |
| | |
| | Check file exists |
| | in blobs/ directory |
| | |
| File Found | |
|<---------------------------| |
| 200 OK + File Content | |
| | |
| File Not Found | |
|<---------------------------| |
| 404 Not Found | |
```
### HEAD Request Flow
```
Client Request FastCGI App Database
| | |
| HEAD /<sha256> | |
|------------------------->| |
| | |
| | Query blob metadata |
| |-------------------------->|
| | |
| | Blob exists |
| |<--------------------------|
| | size, type, uploaded_at |
| | |
| 200 OK + Headers | |
|<-------------------------| |
| Content-Type: image/png | |
| Content-Length: 12345 | |
| | |
| | Blob not found |
| |<--------------------------|
| | |
| 404 Not Found | |
|<-------------------------| |
```
---
## BUD-02: Blob Upload & Authentication
### Upload Without Authentication
```
Client FastCGI App File System Database
| | | |
| PUT /upload | | |
|--------------------------->| | |
| Content: <binary data> | | |
| | | |
| | Calculate SHA-256 | |
| | hash of data | |
| | | |
| | Write to blobs/ | |
| |-------------------------->| |
| | | File saved |
| |<--------------------------| |
| | | |
| | Store metadata | |
| |----------------------------------------------->|
| | | | INSERT |
| |<-----------------------------------------------|
| | | |
| 200 OK | | |
|<---------------------------| | |
| { | | |
| "url": "https://...", | | |
| "sha256": "abc123...", | | |
| "size": 12345, | | |
| "type": "image/png", | | |
| "uploaded": 1234567890 | | |
| } | | |
```
### Upload With Nostr Authentication
```
Client FastCGI App Nostr Validation File System Database
| | | | |
| PUT /upload | | | |
|--------------------------->| | | |
| Authorization: Nostr <evt> | | | |
| Content: <binary data> | | | |
| | | | |
| | Parse auth event | | |
| |-------------------------->| | |
| | | • Verify signature | |
| | | • Check expiration | |
| | | • Validate tags | |
| |<--------------------------| | |
| | Valid ✓ | | |
| | | | |
| | Calculate hash | | |
| | | | |
| | Compare with 'x' tag | | |
| | in auth event | | |
| | | | |
| | Hash matches ✓ | | |
| | | | |
| | Save file | | |
| |----------------------------------------------->| |
| | | | Write | |
| |<-----------------------------------------------| |
| | | | |
| | Store metadata | | |
| |-------------------------------------------------------------->|
| | (include uploader_pubkey) | | | INSERT |
| |<--------------------------------------------------------------|
| | | | |
| 200 OK + Blob Descriptor | | | |
|<---------------------------| | | |
```
### Authentication Failure Flows
```
Invalid Signature:
Client --> FastCGI --> Nostr Validation --> 401 Unauthorized
Expired Event:
Client --> FastCGI --> Nostr Validation --> 401 Unauthorized
Hash Mismatch:
Client --> FastCGI --> Hash Check --> 409 Conflict
```
---
## BUD-03: User Server Lists
### Server List Publication
```
User/Client Nostr Relay Other Clients
| | |
| Publish kind:10063 | |
| event with server tags | |
|------------------------>| |
| { | |
| "kind": 10063, | |
| "tags": [ | |
| ["server", "cdn1"], | |
| ["server", "cdn2"] | |
| ] | |
| } | |
| | |
| | Store event |
| | |
| | |
| | Query for user's |
| | server list |
| |<--------------------------|
| | |
| | Return server list |
| |-------------------------->|
```
### Client Blob Discovery
```
Client Original URL Author's Servers
| | |
| GET blob from URL | |
|------------------------->| |
| | |
| 404 Not Found | |
|<-------------------------| |
| | |
| Extract SHA-256 hash | |
| from URL | |
| | |
| Query nostr for author's | |
| server list (kind:10063) | |
| | |
| Try each server in order | |
|--------------------------------------------------->|
| | |
| GET /<sha256> | |
| | |
| 200 OK + File Content | |
|<---------------------------------------------------|
```
---
## BUD-04: Blob Mirroring
### Mirror Request Flow
```
Client Server B Server A Database
| | | |
| PUT /mirror | | |
|------------------------>| | |
| { | | |
| "url": "https:// | | |
| serverA/abc123..." | | |
| } | | |
| Authorization: <event> | | |
| | | |
| | Validate auth event | |
| | (check x tag matches) | |
| | | |
| | Download from URL | |
| |------------------------>| |
| | | |
| | Stream blob content | |
| |<------------------------| |
| | | |
| | Calculate SHA-256 | |
| | during download | |
| | | |
| | Verify hash matches | |
| | x tag in auth event | |
| | | |
| | Save blob locally | |
| | | |
| | Store metadata | |
| |------------------------------------------------>|
| | | | INSERT |
| |<------------------------------------------------|
| | | |
| 200 OK | | |
|<------------------------| | |
| Blob Descriptor | | |
```
---
## BUD-05: Media Optimization
### Media Processing Flow
```
Client Media Server Optimization Engine File System
| | | |
| PUT /media | | |
|------------------------>| | |
| Content: <raw image> | | |
| Content-Type: image/png | | |
| Authorization: <event> | | |
| | | |
| | Validate auth | |
| | (type="media") | |
| | | |
| | Process media | |
| |------------------------>| |
| | | • Resize/compress |
| | | • Format conversion |
| | | • Quality optimization|
| |<------------------------| Optimized media |
| | | |
| | Calculate new hash | |
| | | |
| | Save optimized blob | |
| |------------------------------------------------>|
| | | | Write |
| |<------------------------------------------------|
| | | |
| 200 OK | | |
|<------------------------| | |
| { | | |
| "url": "new_hash...", | | |
| "sha256": "def456...",| | |
| "size": 8765, | | |
| "type": "image/webp" | | |
| } | | |
```
---
## BUD-06: Upload Requirements
### Upload Requirement Check
```
Client FastCGI App Configuration
| | |
| HEAD /upload | |
|------------------------>| |
| Authorization: <event> | |
| (optional) | |
| | |
| | Check server config |
| |------------------------>|
| | | • Max file size
| | | • Auth required?
| | | • Allowed types
| |<------------------------| • Rate limits
| | |
| | Validate auth if |
| | provided |
| | |
| 200 OK | |
|<------------------------| |
| X-Upload-Size-Limit: | |
| 10485760 | |
| X-Upload-Auth-Required: | |
| true | |
| X-Upload-Types: | |
| image/*,video/* | |
```
### Upload Policy Enforcement
```
Client FastCGI App Policy Check
| | |
| PUT /upload | |
|------------------------>| |
| Content-Length: 50MB | |
| | |
| | Check against limits |
| |------------------------>|
| | | Size: 50MB > 10MB ✗
| |<------------------------| REJECT
| | |
| 413 Payload Too Large | |
|<------------------------| |
| { | |
| "error": "File too | |
| large. Max: 10MB" | |
| } | |
```
---
## BUD-07: Paid Upload/Download
### Payment Required Flow
```
Client Paid Server Payment Provider
| | |
| PUT /upload | |
|------------------------>| |
| | |
| | Check payment required |
| | |
| 402 Payment Required | |
|<------------------------| |
| X-Lightning: lnbc... | |
| X-Cashu: creq... | |
| | |
| User pays invoice | |
|-------------------------------------------------->|
| | |
| PUT /upload (retry) | |
|------------------------>| |
| X-Lightning: <preimage> | |
| | |
| | Verify payment proof |
| |------------------------>|
| | | Valid ✓
| |<------------------------|
| | |
| | Process upload |
| | |
| 200 OK + Blob Desc | |
|<------------------------| |
```
### Payment Methods
```
Lightning Payment:
Client --> Server --> Lightning Node --> Payment Verification --> Upload Success
Cashu Payment:
Client --> Server --> Cashu Mint --> Token Validation --> Upload Success
```
---
## BUD-08: NIP-94 File Metadata
### Enhanced Blob Descriptor
```
Client Upload FastCGI App Metadata Generation Response
| | | |
| PUT /upload | | |
|---------------------> | | |
| | | |
| | Process file | |
| | | |
| | Generate NIP-94 tags | |
| |------------------------>| |
| | | • ["url", "..."] |
| | | • ["m", "image/png"] |
| | | • ["x", "hash..."] |
| | | • ["size", "12345"] |
| | | • ["magnet", "..."] |
| |<------------------------| NIP-94 tags |
| | | |
| | Build response | |
| |------------------------------------------------>|
| Enhanced Response | | |
|<--------------------- | | |
| { | | |
| "url": "...", | | |
| "sha256": "...", | | |
| "nip94": [ | | |
| ["url", "..."], | | |
| ["m", "..."], | | |
| ["x", "..."] | | |
| ] | | |
| } | | |
```
---
## BUD-09: Blob Reporting
### Content Moderation Flow
```
Client/User FastCGI App Moderation System Action
| | | |
| PUT /report | | |
|---------------------> | | |
| NIP-56 report event | | |
| { | | |
| "kind": 1984, | | |
| "tags": [ | | |
| ["x", "hash..."] | | |
| ], | | |
| "content": "spam" | | |
| } | | |
| | | |
| | Validate report | |
| | | |
| | Store report | |
| |------------------------>| |
| | | • Log report |
| | | • Check reporter |
| | | • Queue for review |
| |<------------------------| Stored |
| | | |
| 200 OK | | |
|<--------------------- | | |
| | | |
| | | Manual Review |
| | |---------------------> |
| | | | Remove blob
| | | | Block hash
| | | | Ban user
```
### Automated Moderation
```
Trusted Reporter Report --> Immediate Action --> Blob Removed
Multiple Reports --> Temporary Hide --> Manual Review --> Final Decision
```
---
## Complete Ginxsom Architecture Flow
### Nginx + FastCGI Integration
```
Internet nginx FastCGI App Database File System
| | | | |
| GET /<sha256> | | | |
|-------------------->| | | |
| | Direct file serve | | |
| |----------------------------------------->| |
| | | | blobs/ |
| File Content |<-----------------------------------------| <hash> |
|<--------------------| | | |
| | | | |
| HEAD /<sha256> | | | |
|-------------------->| | | |
| | Forward to FastCGI | | |
| |-------------------->| | |
| | | Query metadata | |
| | |------------------->| |
| | | | SELECT |
| | |<-------------------| |
| | Headers response | | |
| |<--------------------| | |
| Metadata Headers | | | |
|<--------------------| | | |
| | | | |
| PUT /upload | | | |
|-------------------->| | | |
| | Forward to FastCGI | | |
| |-------------------->| | |
| | | Process upload | |
| | | | |
| | | Store metadata | |
| | |------------------->| |
| | | | INSERT |
| | |<-------------------| |
| | | | |
| | | Save file | |
| | |-------------------------------------->|
| | | | Write |
| | |<--------------------------------------|
| | JSON response | | |
| |<--------------------| | |
| Blob Descriptor | | | |
|<--------------------| | | |
```

368
docs/FASTCGI.md Normal file
View File

@@ -0,0 +1,368 @@
# FastCGI Protocol Flow Documentation
This document provides ASCII flow charts to understand how FastCGI works with nginx for the ginxsom blossom server.
## FastCGI Overview
FastCGI is a binary protocol that allows web servers (like nginx) to communicate with application servers efficiently. Unlike CGI which spawns a new process per request, FastCGI applications are persistent and can handle multiple concurrent requests.
## 1. FastCGI Connection Setup Flow
```
┌─────────────────┐ Socket ┌─────────────────┐
│ nginx │◄──────────────►│ FastCGI App │
│ Web Server │ Connection │ (ginxsom) │
└─────────────────┘ └─────────────────┘
│ │
│ 1. Create Unix socket │
│ /tmp/ginxsom.sock │
│ │
│ 2. FastCGI app binds & listens │
│◄───────────────────────────────────│
│ │
│ 3. nginx connects when needed │
│───────────────────────────────────►│
│ │
│ 4. Connection established │
│◄──────────────────────────────────►│
```
## 2. HTTP Request Processing Flow
```
┌─────────┐ HTTP ┌─────────┐ FastCGI ┌─────────┐
│ Client │────────────►│ nginx │─────────────►│ ginxsom │
│Browser │ │ │ │ FastCGI │
└─────────┘ └─────────┘ └─────────┘
│ │ │
│ HTTP Request │ │
│ PUT /upload │ │
│────────────────────────► │
│ │ │
│ │ FCGI_BEGIN_REQUEST │
│ │───────────────────────►│
│ │ │
│ │ FCGI_PARAMS │
│ │ (HTTP headers, etc) │
│ │───────────────────────►│
│ │ │
│ │ FCGI_STDIN │
│ │ (Request body) │
│ │───────────────────────►│
│ │ │
│ │ Process │
│ │◄───────Request─────────│
│ │ │
│ │ FCGI_STDOUT │
│ │ (Response headers) │
│ │◄───────────────────────│
│ │ │
│ │ FCGI_STDOUT │
│ │ (Response body) │
│ │◄───────────────────────│
│ │ │
│ │ FCGI_END_REQUEST │
│ │◄───────────────────────│
│ │ │
│ HTTP Response │ │
│◄──────────────────────│ │
```
## 3. FastCGI Record Structure
```
FastCGI Record Format:
┌─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Version │ Type │RequestID│RequestID│ Length │ Length │ Padding │Reserved │
│ (1) │ (1) │ Hi │ Lo │ Hi │ Lo │ Length │ (1) │
│ │ │ (1) │ (1) │ (1) │ (1) │ (1) │ │
└─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘
│◄────────────────── 8 bytes header ──────────────────────►│
│ │
│◄─────────────────── Content (Length bytes) ─────────────►│
│ │
│◄───── Padding (Padding Length bytes) ────►│
Record Types:
- FCGI_BEGIN_REQUEST (1) - Start new request
- FCGI_PARAMS (4) - Environment variables
- FCGI_STDIN (5) - Request body data
- FCGI_STDOUT (6) - Response data
- FCGI_END_REQUEST (3) - End of request
```
## 4. Ginxsom Endpoint Handling Flow
### 4a. Static File Request (Direct nginx)
```
┌─────────┐ ┌─────────┐
│ Client │ │ nginx │
└─────────┘ └─────────┘
│ │
│ GET /{sha256hash} │
│───────────────────────►│
│ │
│ │ Check file exists:
│ │ /var/lib/ginxsom/files/
│ │ {first2}/{remaining}
│ │
│ │ ┌─ File exists? ─┐
│ │ │ YES │
│ │ └────────────────┘
│ │ │
│ HTTP 200 + File │ │ Serve directly
│◄──────────────────────│◄──────┘
│ │
Alternative flow:
│ │ ┌─ File exists? ─┐
│ │ │ NO │
│ │ └────────────────┘
│ │ │
│ HTTP 404 Not Found │ │
│◄──────────────────────│◄──────┘
```
### 4b. Upload Request (FastCGI)
```
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐
│ Client │ │ nginx │ │ ginxsom │ │ File │
│ │ │ │ │ FastCGI │ │ System │
└─────────┘ └─────────┘ └─────────┘ └─────────┘
│ │ │ │
│ PUT /upload │ │ │
│ Auth: xyz │ │ │
│ Body: file │ │ │
│─────────────►│ │ │
│ │ │ │
│ │ FCGI_PARAMS │ │
│ │ METHOD=PUT │ │
│ │ URI=/upload │ │
│ │ AUTH=xyz │ │
│ │─────────────►│ │
│ │ │ │
│ │ FCGI_STDIN │ │
│ │ (file data) │ │
│ │─────────────►│ │
│ │ │ │
│ │ │ Verify Auth │
│ │ │ (nostr sig) │
│ │ │ │
│ │ │ Calculate │
│ │ │ SHA-256 │
│ │ │ │
│ │ │ Write file │
│ │ │─────────────►│
│ │ │ │
│ │ │ Store │
│ │ │ metadata │
│ │ │────────────────────►
│ │ │ │ DB
│ │ │ │
│ │ FCGI_STDOUT │ │
│ │ 200 OK │ │
│ │ {sha256} │ │
│ │◄─────────────│ │
│ │ │ │
│ HTTP 200 OK │ │ │
│ {sha256} │ │ │
│◄─────────────│ │ │
```
### 4c. HEAD Request for Metadata (FastCGI)
```
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Client │ │ nginx │ │ ginxsom │
│ │ │ │ │ FastCGI │
└─────────┘ └─────────┘ └─────────┘
│ │ │
│ HEAD /{sha256} │ │
│──────────────────►│ │
│ │ │
│ │ FCGI_PARAMS │
│ │ METHOD=HEAD │
│ │ BLOSSOM_HASH=... │
│ │──────────────────►│
│ │ │
│ │ │ Query DB for
│ │ │ file metadata
│ │ │
│ │ FCGI_STDOUT │
│ │ Content-Length: X │
│ │ Content-Type: Y │
│ │ x-sha256: hash │
│ │◄──────────────────│
│ │ │
│ HTTP Headers Only │ │
│◄──────────────────│ │
```
## 5. Development vs Production Deployment
### Development Mode
```
┌─────────────┐ ┌─────────────┐
│ Terminal │ │ nginx │
│ │ │ │
│ ./ginxsom │◄──────────────►│ Config: │
│ --fastcgi │ Unix Socket │ fastcgi_pass│
│ --socket │ /tmp/ginxsom │ unix:/tmp/ │
│ /tmp/... │ .sock │ ginxsom.sock│
└─────────────┘ └─────────────┘
│ │
│ Direct execution │ sudo systemctl
│ Easy debugging │ reload nginx
│ Ctrl+C to stop │ (config changes)
│ Real-time logs │
```
### Production Mode
```
┌─────────────┐ ┌─────────────┐
│ systemd │ │ nginx │
│ │ │ │
│ ginxsom │◄──────────────►│ Config: │
│ .service │ Unix Socket │ fastcgi_pass│
│ │ /run/ginxsom/ │ unix:/run/ │
│ Auto-start │ ginxsom.sock │ ginxsom/... │
│ Auto-restart│ │ │
│ Log to file │ │ │
└─────────────┘ └─────────────┘
```
## 6. Complete nginx + FastCGI Request Flow
```
┌───────┐ ┌─────────────────────────────────────────────────┐ ┌─────────┐
│Client │ │ nginx │ │ginxsom │
└───────┘ └─────────────────────────────────────────────────┘ │FastCGI │
│ │ └─────────┘
│ GET /abc123...def │ │
│───────────────────────►│ │
│ │ │
│ │ location ~ ^/([a-f0-9]{64})$ { │
│ │ # Check if static file exists │
│ │ try_files /$prefix/$suffix =404 │
│ │ } │
│ │ │
│ │ ┌─ File exists? ─┐ │
│ │ │ YES │ │
│ │ └────────────────┘ │
│ │ │ │
│ │ │ Serve directly (fast!) │
│ HTTP 200 + File │ │ │
│◄──────────────────────│◄──────┘ │
│ │ │
│ │ │
│ PUT /upload │ │
│ Authorization: xyz │ │
│───────────────────────►│ │
│ │ │
│ │ location /upload { │
│ │ fastcgi_pass unix:/tmp/ │
│ │ ginxsom.sock; │
│ │ } │
│ │ │
│ │ ┌─ FastCGI Protocol ─┐ │
│ │ │ FCGI_BEGIN_REQUEST │─────────────►│
│ │ │ FCGI_PARAMS │─────────────►│
│ │ │ FCGI_STDIN │─────────────►│
│ │ └─────────────────────┘ │
│ │ │
│ │ │ Verify
│ │ │ nostr
│ │ │ signature
│ │ │
│ │ │ Calculate
│ │ │ SHA-256
│ │ │
│ │ │ Store file
│ │ │ & metadata
│ │ │
│ │ ┌─ FastCGI Response ─┐ │
│ │ │ FCGI_STDOUT │◄─────────────│
│ │ │ FCGI_END_REQUEST │◄─────────────│
│ │ └─────────────────────┘ │
│ │ │
│ HTTP 200 OK │ │
│ {"sha256": "..."} │ │
│◄──────────────────────│ │
```
## 7. libfcgi Library Usage
### Basic FastCGI Application Structure
```c
#include <fcgiapp.h>
int main() {
FCGX_Request request;
// Initialize FastCGI library
FCGX_Init();
FCGX_InitRequest(&request, 0, 0);
// Main request processing loop
while (FCGX_Accept_r(&request) == 0) {
// Read environment variables (HTTP headers, etc)
char* method = FCGX_GetParam("REQUEST_METHOD", request.envp);
char* uri = FCGX_GetParam("REQUEST_URI", request.envp);
char* auth = FCGX_GetParam("HTTP_AUTHORIZATION", request.envp);
// Route requests
if (strcmp(uri, "/health") == 0) {
handle_health(&request);
} else if (strcmp(uri, "/upload") == 0) {
handle_upload(&request);
} else if (strncmp(uri, "/head/", 6) == 0) {
handle_head(&request);
}
// Finish this request
FCGX_Finish_r(&request);
}
return 0;
}
```
### Reading Request Data
```
┌─────────────────┐ ┌─────────────────┐
│ nginx sends: │ │ ginxsom reads: │
├─────────────────┤ ├─────────────────┤
│ FCGI_PARAMS │────────►│ FCGX_GetParam() │
│ REQUEST_METHOD │ │ "PUT" │
│ REQUEST_URI │ │ "/upload" │
│ CONTENT_LENGTH │ │ "1024" │
│ HTTP_* │ │ headers │
├─────────────────┤ ├─────────────────┤
│ FCGI_STDIN │────────►│ FCGX_GetStr() │
│ (request body) │ │ file data │
└─────────────────┘ └─────────────────┘
```
### Writing Response Data
```
┌─────────────────┐ ┌─────────────────┐
│ ginxsom writes: │ │ nginx sends: │
├─────────────────┤ ├─────────────────┤
│ FCGX_PutS() │────────►│ HTTP Response │
│ "Content-Type: │ │ Headers │
│ application/ │ │ │
│ json\r\n\r\n" │ │ │
├─────────────────┤ ├─────────────────┤
│ FCGX_PutS() │────────►│ HTTP Response │
│ '{"status": │ │ Body │
│ "ok"}' │ │ │
└─────────────────┘ └─────────────────┘
```
This documentation shows how FastCGI enables nginx to efficiently serve static blossom files directly while delegating authenticated operations to the ginxsom FastCGI application.

287
docs/IMPLEMENTATION.md Normal file
View File

@@ -0,0 +1,287 @@
# Ginxsom Blossom Server Implementation Status
This document tracks the implementation status of ginxsom, a high-performance FastCGI-based Blossom server designed to work with nginx.
## Architecture Overview
- **nginx**: Handles static file serving (GET /<sha256>) for maximum performance
- **FastCGI Application**: Handles authenticated operations, metadata queries, uploads
- **SQLite Database**: Stores blob metadata and server configuration
- **File Storage**: Flat directory structure in `blobs/` directory
---
## BUD-01: Blob Retrieval ✅ **COMPLETE**
### Infrastructure & Database
- [x] Create directory structure (`blobs/`, `db/`, `logs/`)
- [x] SQLite schema with `blobs` table (sha256, size, type, uploaded_at, uploader_pubkey, filename)
- [x] Database initialization scripts and proper indexes
### GET /<sha256> Endpoint
- [x] nginx static file serving with extension support (.txt, .jpg, .png, etc.)
- [x] Extension fallback via `try_files` directive
- [x] Proper MIME type detection and headers
- [x] Cache headers (Cache-Control, immutable)
- [x] 404 handling for missing blobs
### HEAD /<sha256> Endpoint
- [x] FastCGI metadata handler
- [x] Database metadata queries
- [x] Proper HTTP headers (Content-Type, Content-Length)
- [x] SHA-256 extraction from URL paths
- [x] 404 responses for missing blobs
### Testing Status
- [x] File serving works with all supported extensions
- [x] HEAD requests return correct metadata
- [x] 404 responses for missing files
- [ ] Performance testing with large files
---
## BUD-02: File Upload & Authentication ✅ **COMPLETE**
### Nostr Authentication System
- [x] nostr_core_lib integration and compilation
- [x] secp256k1 context initialization (CRITICAL BUG FIXED)
- [x] Authentication functions:
- [x] `parse_authorization_header()` - Extract JSON from "Nostr base64(event)"
- [x] `validate_blossom_event()` - Validate kind 24242 events
- [x] `authenticate_request()` - Main authentication orchestrator
- [x] Enhanced error handling with specific error types (event_expired, invalid_signature, etc.)
- [x] API refactoring - upgraded from low-level crypto to `nostr_crypto_init()` API
### PUT /upload Endpoint
- [x] Authorization header parsing and validation
- [x] File upload streaming to temporary location
- [x] SHA-256 hash calculation during upload
- [x] Hash validation against authorization
- [x] File permissions (644) for nginx serving
- [x] Database metadata storage (uploader_pubkey, filename)
- [x] Blob descriptor JSON response
### GET /list/<pubkey> Endpoint
- [x] Extract pubkey from URL path
- [x] Database queries for user's blobs
- [x] Optional authorization with kind 24242 event validation
- [x] Support for `since`/`until` query parameters
- [x] JSON array responses with blob descriptors
### DELETE /<sha256> Endpoint
- [x] SHA-256 extraction from URL
- [x] Required authorization with kind 24242 validation
- [x] Ownership verification (uploader_pubkey matching)
- [x] File and database cleanup
- [x] Proper error handling for missing files
### Testing Status
- [x] Upload with valid nostr authentication (HTTP 200)
- [x] Upload without authentication (proper error responses)
- [x] Hash mismatch validation (409 Conflict)
- [x] List endpoint returns proper JSON
- [x] Delete endpoint with ownership checks
- [x] File retrieval after upload working
- [ ] File size limit testing
---
## BUD-03: Server List (User Server Lists) ⚪ **FOR CLIENTS, NOT SERVERS**
## BUD-04: Blob Mirroring ✅ **COMPLETE**
### HTTP Client Implementation
- [x] CURL library integration and HTTP client functions
- [x] `write_callback()` - Download response data with dynamic buffering
- [x] `header_callback()` - Extract Content-Type headers
- [x] `download_blob_from_url()` - Complete HTTP download with security controls
- [x] Memory management and error handling
### PUT /mirror Endpoint
- [x] nginx endpoint configured (`PUT /mirror`)
- [x] FastCGI routing and request handling
- [x] JSON request body parsing (extract `url` field)
- [x] URL validation and security checks (HTTPS-only, SSRF protection)
- [x] Remote blob downloading with CURL
- [x] SHA-256 hash calculation and verification
- [x] Content-Type detection (headers, URL extension, file signature)
- [x] File storage with proper extensions (.png, .jpg, etc.)
- [x] Database metadata storage
- [x] Blob descriptor JSON response
### Security Features
- [x] HTTPS-only URL validation (no HTTP allowed)
- [x] SSRF protection (blocks localhost, private IPs: 127.x, 192.168.x, 10.x, 172.16-31.x)
- [x] File size limits (100MB maximum)
- [x] Request timeouts (30s total, 10s connect)
- [x] SSL certificate verification
- [x] Authorization hash verification (when provided)
### Testing Status
- [x] Mirror request with valid HTTPS URL (HTTP 200)
- [x] Hash verification against downloaded content
- [x] Content-Type detection from PNG file
- [x] File accessibility after mirroring
- [x] HEAD request metadata retrieval
- [x] Error handling for invalid URLs
- [x] Security validation (private IP blocking)
---
## BUD-05: Media Optimization ⚪ **PARTIAL**
### Current Status
- [x] nginx endpoint configured (`HEAD/PUT /media`)
- [x] FastCGI routing established
- [ ] Media processing libraries integration
- [ ] Optimization algorithms implementation
- [ ] Multi-format media handling
- [ ] Optimization pipeline testing
---
## BUD-06: Upload Requirements ✅ **COMPLETE**
### HEAD /upload Pre-flight Validation
- [x] `HEAD /upload` endpoint implementation
- [x] Client header parsing (X-SHA-256, X-Content-Length, X-Content-Type)
- [x] Pre-flight validation without file transfer:
- [x] SHA-256 format validation
- [x] File size limit checking (100MB default)
- [ ] MIME type restrictions (policy 415 not enforced yet)
- [x] Authentication validation (optional via rules system)
- [x] Duplicate detection (policy configurable)
- [x] Banned hash checking (via rules engine)
- [x] Proper HTTP status codes (200, 400, 401, 409, 411, 413; 415 reserved for future MIME policy)
- [x] X-Reason headers for error messages
### Upload Policy Configuration
- [ ] Server configuration system
- [ ] Maximum file size limits (currently hard limit in code; move to config)
- [ ] Allowed MIME type restrictions
- [ ] Rate limiting implementation
- [ ] DOS protection benefits
---
## BUD-07: Payment Integration ⚪ **NOT IMPLEMENTED**
*Optional feature - not currently planned*
- [ ] 402 Payment Required responses
- [ ] Lightning payment support
- [ ] Cashu payment integration
- [ ] Payment verification flows
---
## BUD-08: NIP-94 File Metadata Tags ✅ **COMPLETE**
### NIP-94 Integration
- [x] Configuration system with SQLite server_config table
- [x] `nip94_enabled` configuration key (default: true)
- [x] `cdn_origin` configuration key (default: "http://localhost:9001")
- [x] Centralized MIME type to extension mapping [`mime_to_extension()`](src/main.c:2024)
- [x] Canonical blob URL generation [`nip94_build_blob_url()`](src/main.c:2055)
- [x] NIP-94 metadata field emission [`nip94_emit_field()`](src/main.c:2201)
### Image Dimension Detection
- [x] PNG dimension parsing from IHDR chunk [`parse_png_dimensions()`](src/main.c:2065)
- [x] JPEG dimension parsing from SOF0/SOF2 markers [`parse_jpeg_dimensions()`](src/main.c:2089)
- [x] WebP dimension parsing for VP8/VP8L/VP8X formats [`parse_webp_dimensions()`](src/main.c:2141)
- [x] Universal dimension dispatcher [`nip94_get_dimensions()`](src/main.c:2184)
### Integration Points
- [x] PUT /upload endpoint enhanced with NIP-94 metadata
- [x] PUT /mirror endpoint enhanced with NIP-94 metadata
- [x] Configuration-driven origin override for CDN support
- [x] Conditional metadata emission based on `nip94_enabled` setting
- [x] Proper JSON comma handling for optional nip94 field
### NIP-94 Tags Implemented
- [x] `url` tag - Canonical blob URL with proper origin and extension
- [x] `m` tag - MIME type (Content-Type)
- [x] `x` tag - SHA-256 hash (lowercase hex)
- [x] `size` tag - File size in bytes
- [x] `dim` tag - Image dimensions (e.g., "1x1", "1920x1080") when available
### Configuration Keys
- `nip94_enabled` (boolean): Enable/disable NIP-94 metadata emission (default: true)
- `cdn_origin` (string): Base URL for blob URLs (default: "http://localhost:9001")
### Testing Status
- [x] NIP-94 minimal tags (url, m, x, size) present in responses
- [x] Image dimension detection working for PNG 1x1 test case
- [x] Configuration-based enable/disable functionality
- [x] CDN origin override affecting both descriptor and nip94 URLs
- [x] Mirror endpoint NIP-94 integration (network-dependent)
---
## BUD-09: Content Reporting ✅ **COMPLETE**
### NIP-56 Report Event System
- [x] Full NIP-56 kind 1984 report event implementation
- [x] Report event structure validation (kind, required fields, x tags)
- [x] SHA-256 blob hash extraction from x tags [`extract_blob_hashes_from_report()`](src/main.c:2408)
- [x] Report type validation (nudity, malware, profanity, illegal, spam, impersonation, other)
- [x] Nostr cryptographic signature verification
- [x] Event timestamp and expiration validation
### PUT /report Endpoint
- [x] nginx endpoint configuration (`PUT /report`)
- [x] FastCGI routing and request handling [`handle_report_request()`](src/main.c:2516)
- [x] JSON request body parsing and validation
- [x] Content-Type enforcement (application/json required)
- [x] Request size limits (1 byte to 10KB)
- [x] HTTP method validation (PUT only, 405 for others)
### Report Validation Features
- [x] Event structure validation [`validate_report_event_structure()`](src/main.c:2355)
- [x] NIP-56 compliance checking (kind 1984, x tags required)
- [x] SHA-256 hash format validation (64-char hex)
- [x] Report type extensibility (accepts unknown types per NIP-56)
- [x] Multiple blob reporting (multiple x tags supported)
- [x] Optional tag support (e, p, server tags)
### Report Storage System
- [x] Optional report storage in database [`store_blob_report()`](src/main.c:2485)
- [x] Reporter pubkey extraction and storage
- [x] Report type and content preservation
- [x] Timestamp tracking for moderation review
- [x] Configurable storage policy (server MAY store reports)
### Error Handling & Security
- [x] Comprehensive error responses with specific error codes:
- `invalid_json` - Malformed JSON requests
- `invalid_content_length` - Size limit violations
- `invalid_report_event` - NIP-56 structure violations
- `no_blob_hashes` - Missing or invalid SHA-256 hashes
- `unsupported_media_type` - Non-JSON Content-Type
- [x] HTTP status code compliance (200, 400, 405, 415)
- [x] Detailed error messages with troubleshooting information
### NIP-56 Compliance Features
- [x] Report types: nudity, malware, profanity, illegal, spam, impersonation, other
- [x] Extensible report type system (unknown types accepted)
- [x] Multiple blob reporting via multiple x tags
- [x] Optional metadata tags (e, p, server)
- [x] Content field support (reports may include descriptions)
- [x] Cryptographic signature verification via nostr event validation
### Testing Status
- [x] Comprehensive test suite with 22 test cases [`tests/report_test_bud09.sh`](tests/report_test_bud09.sh)
- [x] All NIP-56 report types tested (nudity, malware, profanity, illegal, spam, impersonation, other)
- [x] Event validation testing (missing x tags, wrong kind, invalid JSON)
- [x] Error handling testing (empty body, wrong Content-Type, unsupported methods)
- [x] Multiple blob reporting functionality
- [x] Optional tag support with comprehensive NIP-56 structure
- [x] Unknown report type extensibility
- [x] Edge cases (empty content, long content, invalid hashes)
### Integration Points
- [x] Full integration with existing nostr authentication system
- [x] Database integration for optional report storage
- [x] nginx FastCGI routing configuration
- [x] Error response standardization with existing endpoints

232
docs/NIP42_INTEGRATION_PLAN.md Normal file
View File

@@ -0,0 +1,232 @@
# NIP-42 Authentication Integration Plan
## Overview
This document outlines the integration of NIP-42 client-to-relay authentication into the existing ginxsom authentication system. The goal is to support both NIP-42 (kind 22242) and Blossom (kind 24242) authentication methods in a unified system.
## Current System Analysis
### Existing Authentication Flow
1. **Blossom Protocol (kind 24242)**: Operation-specific authorization
- Tags: `t` (method), `x` (hash), `expiration`
- Purpose: Authorize specific file operations (upload, delete, etc.)
- Location: `validate_nostr_event()` in `request_validator.c:379`
2. **Rules Engine**: Policy-based access control
- Pubkey whitelist/blacklist
- Hash blacklist
- MIME type restrictions
- File size limits
- Location: `evaluate_auth_rules()` in `request_validator.c:1029`
### Integration Points Identified
1. **Header Structure**: Extend `nostr_request_t` to support NIP-42 context
2. **Event Validation**: Support both kind 22242 (NIP-42) and 24242 (Blossom)
3. **Challenge Management**: Add challenge generation and storage
4. **Database Schema**: Add challenge storage tables
5. **Authentication Flow**: Support dual authentication modes
## NIP-42 Integration Architecture
### Dual Authentication Strategy
```
Client Authentication (NIP-42) → Operation Authorization (Blossom) → Rules Engine
```
- **NIP-42 (kind 22242)**: Proves client identity to relay
- **Blossom (kind 24242)**: Authorizes specific operations
- **Rules Engine**: Applies policy-based restrictions
### Authentication Flow Options
1. **NIP-42 Only**: Client authenticates identity, all operations allowed by rules
2. **Blossom Only**: Current system - operation-specific authorization
3. **Dual Mode**: NIP-42 identity + Blossom operation authorization
4. **Rules Only**: No cryptographic auth, rules-based access only
## Technical Implementation Plan
### 1. Extend Request Validator Header (`request_validator.h`)
```c
// Add NIP-42 support structures
typedef struct {
char challenge[256]; // Current challenge for client
time_t challenge_created; // Challenge timestamp
char relay_url[256]; // Relay URL for challenge
int is_authenticated; // NIP-42 auth status
char authenticated_pubkey[65]; // Pubkey from NIP-42 auth
} nostr_nip42_context_t;
// Extend nostr_request_t
typedef struct {
// ... existing fields ...
nostr_nip42_context_t* nip42_context; // NIP-42 authentication context
const char* challenge; // Challenge for this request
} nostr_request_t;
```
### 2. Database Schema Extensions
```sql
-- NIP-42 challenge storage
CREATE TABLE IF NOT EXISTS nip42_challenges (
challenge_id TEXT PRIMARY KEY,
challenge TEXT NOT NULL,
relay_url TEXT NOT NULL,
created_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL,
client_ip TEXT
);
-- NIP-42 authentication sessions
CREATE TABLE IF NOT EXISTS nip42_sessions (
session_id TEXT PRIMARY KEY,
pubkey TEXT NOT NULL,
challenge_id TEXT NOT NULL,
authenticated_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL,
client_ip TEXT,
FOREIGN KEY (challenge_id) REFERENCES nip42_challenges(challenge_id)
);
```
### 3. Core Functions to Add
```c
// NIP-42 challenge management
int nostr_nip42_generate_server_challenge(char* challenge_out, size_t challenge_size);
int nostr_nip42_store_challenge(const char* challenge, const char* relay_url, const char* client_ip);
int nostr_nip42_verify_challenge_response(const char* auth_header, const char* relay_url, nostr_request_result_t* result);
// Enhanced validation
int validate_nip42_event(struct cJSON* event, const char* expected_challenge, const char* relay_url);
int validate_blossom_event(struct cJSON* event, const char* expected_hash, const char* method);
// Dual authentication support
int nostr_validate_request_dual(const nostr_request_t* request, nostr_request_result_t* result);
```
### 4. Enhanced Event Validation
Extend `validate_nostr_event()` to handle both event kinds:
```c
static int validate_nostr_event(struct cJSON* event, const char* expected_hash, const char* method) {
struct cJSON* kind_json = cJSON_GetObjectItem(event, "kind");
int kind = cJSON_GetNumberValue(kind_json);
switch (kind) {
case 22242: // NIP-42 authentication
return validate_nip42_event(event, expected_hash, method);
case 24242: // Blossom authorization
return validate_blossom_event(event, expected_hash, method);
default:
return NOSTR_ERROR_EVENT_INVALID_KIND;
}
}
```
### 5. Ginxsom Integration Points
#### Main.c Changes
```c
// Add challenge endpoint
if (strcmp(request_method, "GET") == 0 && strstr(request_uri, "/.well-known/nostr/challenge")) {
handle_nip42_challenge_request(response);
return;
}
// Enhanced authentication in existing endpoints
int handle_upload_request(FCGX_Request* request) {
// ... existing code ...
// Check for NIP-42 challenge header
const char* challenge_header = FCGX_GetParam("HTTP_X_NOSTR_CHALLENGE", request->envp);
nostr_request_t nostr_request = {
.operation = "upload",
.auth_header = auth_header,
.resource_hash = file_hash,
.mime_type = content_type,
.file_size = content_length,
.challenge = challenge_header,
.nip42_context = &nip42_ctx
};
}
```
## Configuration Options
### New Configuration Keys
```sql
-- NIP-42 specific settings
INSERT OR REPLACE INTO auth_config VALUES ('nip42_enabled', 'true');
INSERT OR REPLACE INTO auth_config VALUES ('nip42_challenge_ttl', '600');
INSERT OR REPLACE INTO auth_config VALUES ('nip42_session_ttl', '3600');
INSERT OR REPLACE INTO auth_config VALUES ('auth_mode', 'dual'); -- dual, nip42_only, blossom_only, rules_only
INSERT OR REPLACE INTO auth_config VALUES ('relay_url', 'http://localhost:9001');
```
### Authentication Modes
1. **dual**: NIP-42 + Blossom + Rules (most secure)
2. **nip42_only**: NIP-42 + Rules (client identity required)
3. **blossom_only**: Blossom + Rules (current system)
4. **rules_only**: Rules only (no cryptographic auth)
## Testing Strategy
### Test Cases to Add
1. **NIP-42 Challenge Flow**: Generate challenge, verify response
2. **Kind 22242 Validation**: Test NIP-42 event structure and signatures
3. **Dual Authentication**: NIP-42 auth + Blossom authorization
4. **Challenge Expiration**: Test challenge TTL and cleanup
5. **Session Management**: Test session creation and expiration
6. **Mixed Mode Testing**: Different auth modes in same server
7. **Error Handling**: Invalid challenges, expired sessions, etc.
### Test File Structure
```bash
tests/
├── nip42_basic_test.sh # Basic NIP-42 functionality
├── nip42_challenge_test.sh # Challenge generation and validation
├── nip42_dual_auth_test.sh # Dual authentication mode
└── nip42_integration_test.sh # Full integration testing
```
## Build System Integration
### Files to Update
1. **build.sh**: Already includes `nip042.c` in sources
2. **nostr_core.h**: Already includes `nip042.h`
3. **Makefile**: May need to link additional NIP-42 functions
## Implementation Order
1. ✅ **Analysis Complete**: Current system understood
2. **Extend Headers**: Add NIP-42 structures to request_validator.h
3. **Database Schema**: Add challenge and session tables
4. **Core Functions**: Implement challenge management
5. **Event Validation**: Extend validation for kind 22242
6. **Request Validation**: Update nostr_validate_request()
7. **Ginxsom Integration**: Add challenge endpoints and auth checks
8. **Build Integration**: Ensure NIP-42 is compiled
9. **Testing**: Create comprehensive test suite
10. **Documentation**: Update API documentation
## Benefits of This Integration
1. **Standards Compliance**: Full NIP-42 support for Nostr ecosystem compatibility
2. **Flexible Authentication**: Support multiple auth modes based on use case
3. **Enhanced Security**: Dual authentication provides defense in depth
4. **Backward Compatibility**: Existing Blossom auth continues to work
5. **Future Proof**: Ready for Nostr relay ecosystem integration
## Migration Path
### Phase 1: Basic NIP-42 Support
- Add NIP-42 validation alongside existing system
- No breaking changes to current auth
### Phase 2: Enhanced Integration
- Add challenge management and session handling
- Optional dual authentication mode
### Phase 3: Full Integration
- Complete NIP-42 relay compatibility
- Advanced session management features
This integration maintains full backward compatibility while adding powerful new authentication capabilities that align with Nostr ecosystem standards.

219
docs/README_ADMIN_API.md Normal file
View File

@@ -0,0 +1,219 @@
# Ginxsom Admin API
A Nostr-compliant admin interface for the Ginxsom Blossom server that provides programmatic access to server statistics, configuration, and file management operations.
## Overview
The admin API allows server administrators to:
- View server statistics (file counts, storage usage, user metrics)
- Retrieve and update server configuration settings
- Browse recent uploaded files with pagination
- Monitor server health and disk usage
All operations require Nostr authentication using admin-authorized public keys.
## Quick Start
### 1. Configure Admin Access
Add your admin pubkey to the server configuration:
```bash
# Generate admin keys (keep private key secure!)
ADMIN_PRIVKEY=$(nak key generate)
ADMIN_PUBKEY=$(echo "$ADMIN_PRIVKEY" | nak key public)
# Configure server
sqlite3 db/ginxsom.db << EOF
INSERT OR REPLACE INTO server_config (key, value, description) VALUES
('admin_pubkey', '$ADMIN_PUBKEY', 'Nostr public key authorized for admin operations'),
('admin_enabled', 'true', 'Enable admin interface');
EOF
```
### 2. Build and Start Server
```bash
make clean && make
spawn-fcgi -s /tmp/ginxsom-fcgi.sock -n ./build/ginxsom-fcgi
nginx -c $(pwd)/config/local-nginx.conf
```
### 3. Test the API
```bash
# Run the complete test suite
./tests/admin_test.sh
# Or test individual endpoints
export ADMIN_PRIVKEY="your_private_key_here"
./tests/admin_test.sh
```
## API Endpoints
### GET /api/health
System health check (no authentication required).
```bash
curl http://localhost:9001/api/health
```
### GET /api/stats
Server statistics and metrics.
```json
{
"status": "success",
"data": {
"total_files": 1234,
"total_bytes": 104857600,
"total_size_mb": 100.0,
"unique_uploaders": 56,
"avg_file_size": 85049,
"file_types": {
"image/png": 45,
"image/jpeg": 32
}
}
}
```
### GET /api/config
Current server configuration.
```json
{
"status": "success",
"data": {
"max_file_size": "104857600",
"require_auth": "false",
"server_name": "ginxsom",
"nip94_enabled": "true"
}
}
```
### PUT /api/config
Update server configuration.
```json
{
"max_file_size": "209715200",
"require_auth": "true",
"nip94_enabled": "true"
}
```
### GET /api/files
Recent uploaded files with pagination.
```json
{
"status": "success",
"data": {
"files": [
{
"sha256": "abc123...",
"size": 184292,
"type": "application/pdf",
"uploaded_at": 1725105921,
"uploader_pubkey": "def456...",
"url": "http://localhost:9001/abc123.pdf"
}
],
"total": 1234,
"limit": 50,
"offset": 0
}
}
```
## Manual API Usage with nak and curl
### Generate Admin Authentication Event
```bash
# Create an authenticated event
EVENT=$(nak event -k 24242 -c "admin_request" \
--tag t="GET" \
--tag expiration="$(date -d '+1 hour' +%s)" \
--sec "$ADMIN_PRIVKEY")
# Send authenticated request
AUTH_HEADER="Nostr $(echo "$EVENT" | base64 -w 0)"
curl -H "Authorization: $AUTH_HEADER" http://localhost:9001/api/stats
```
### Update Configuration
```bash
# Create PUT event (method in tag)
EVENT=$(nak event -k 24242 -c "admin_request" \
--tag t="PUT" \
--tag expiration="$(date -d '+1 hour' +%s)" \
--sec "$ADMIN_PRIVKEY")
AUTH_HEADER="Nostr $(echo "$EVENT" | base64 -w 0)"
curl -X PUT -H "Authorization: $AUTH_HEADER" \
-H "Content-Type: application/json" \
-d '{"max_file_size": "209715200", "require_auth": "true"}' \
http://localhost:9001/api/config
```
## Security Features
- **Nostr Authentication**: All admin operations require valid Nostr kind 24242 events
- **Pubkey Verification**: Only events signed by configured admin pubkeys are accepted
- **Event Expiration**: Admin events must include expiration timestamps for security
- **Access Control**: Separate enable/disable flag for admin interface
## Development and Testing
### Prerequisites
- nak (https://github.com/fiatjaf/nak)
- curl, jq
- sqlite3
### Run Tests
```bash
# Make test script executable
chmod +x tests/admin_test.sh
# Run complete test suite
./tests/admin_test.sh
# Run with specific admin key
export ADMIN_PRIVKEY="your_private_key"
./tests/admin_test.sh
```
### Build System
```bash
# Clean build
make clean
# Build with debug info
make debug
# Run FastCGI process
make run
```
## Files Added/Modified
- `src/admin_api.h` - Admin API function declarations
- `src/admin_api.c` - Complete admin API implementation
- `src/main.c` - Updated with admin API routing
- `config/local-nginx.conf` - Updated with admin API routes
- `tests/admin_test.sh` - Complete test suite
- `Makefile` - Updated to compile admin_api.c
- `README_ADMIN_API.md` - This documentation
## Future Enhancements
- **Nostr Relay Integration**: Automatic relay subscription for remote admin control
- **Admin Pubkey Rotation**: Support for multiple admin keys and key rotation
- **Audit Logging**: Detailed logging of admin operations
- **Rate Limiting**: Prevent abuse of admin endpoints
- **Web Dashboard**: Optional HTML/CSS/JavaScript frontend
---
The admin API provides a secure, Nostr-compliant interface for server management through command-line tools while maintaining full compatibility with the existing Blossom protocol implementation.

259
docs/SUBMODULES.md Normal file
View File

@@ -0,0 +1,259 @@
# Git Submodules Guide for Ginxsom
This project uses git submodules to include external dependencies that we reference but never modify.
## Current Submodules
- **blossom**: The official Blossom protocol specification and BUD documents
- **nostr_core_lib**: Core nostr functionality for authentication and event validation
## Key Principles
1. **Read-Only**: We never modify submodule content directly
2. **Version Pinning**: Submodules are pinned to specific commits for stability
3. **Upstream Updates**: We periodically update to newer versions when needed
---
## Common Git Submodule Commands
### Initial Setup (for new clones)
```bash
# Clone the main repository
git clone ssh://git@git.laantungir.net:222/laantungir/ginxsom.git
# Initialize and update all submodules
git submodule update --init --recursive
```
### One-Line Clone (includes submodules)
```bash
git clone --recursive ssh://git@git.laantungir.net:222/laantungir/ginxsom.git
```
### Update Submodules to Latest
```bash
# Update to latest commit on default branch
git submodule update --remote
# Update specific submodule
git submodule update --remote blossom
git submodule update --remote nostr_core_lib
```
### Pin Submodule to Specific Version
```bash
# Go into submodule directory
cd blossom
# Checkout specific commit/tag
git checkout v1.2.3 # or specific commit hash
# Go back to main repo
cd ..
# Stage the submodule change
git add blossom
# Commit the pin
git commit -m "Pin blossom submodule to v1.2.3"
```
### Check Submodule Status
```bash
# Show current submodule commits
git submodule status
# Show if submodules have uncommitted changes
git status
# Show submodule branch/commit info
git submodule foreach git log --oneline -1
```
---
## Workflow Best Practices
### Daily Development
1. **Never edit submodule files directly** - they're read-only references
2. **Use `git status` regularly** - it shows submodule state changes
3. **Commit submodule updates separately** - makes history cleaner
### When Submodule Updates Available
```bash
# Check what's new upstream
cd blossom
git fetch
git log HEAD..origin/main --oneline
cd ..
# If updates look good, update the pin
git submodule update --remote blossom
git add blossom
git commit -m "Update blossom submodule to latest"
```
### Team Collaboration
```bash
# After pulling main repo changes
git pull
# Update submodules if they changed
git submodule update --init --recursive
# Or use this one-liner for pulls
git pull --recurse-submodules
```
---
## Common Issues & Solutions
### "Submodule path contains unstaged changes"
```bash
# This means files in submodule were accidentally modified
cd <submodule_path>
git checkout . # Discard changes
cd ..
```
### "Submodule not initialized"
```bash
# Initialize and update
git submodule update --init <submodule_name>
```
### "Detached HEAD state in submodule"
```bash
# This is normal! Submodules are pinned to specific commits
# Only worry if you need to track a branch instead of a commit
```
### Accidentally Committed Changes in Submodule
```bash
# Go to submodule
cd <submodule_path>
# Reset to the pinned commit
git reset --hard <pinned_commit_hash>
cd ..
# The main repo should now show no changes to submodule
```
---
## Integration with Build System
### Makefile Integration
```makefile
# Ensure submodules are updated before building
build: update-submodules
# build commands here
update-submodules:
git submodule update --init --recursive
.PHONY: build update-submodules
```
### CI/CD Integration
```bash
# In your CI/CD pipeline, always initialize submodules
git submodule update --init --recursive
```
---
## Project-Specific Usage
### Blossom Submodule
- **Purpose**: Read BUD specifications and protocol documentation
- **Usage**: Reference only, never modify
- **Update frequency**: When new BUDs are released or existing ones updated
### nostr_core_lib Submodule
- **Purpose**: Core nostr functionality for ginxsom authentication
- **Usage**: Link against in build process, never modify source
- **Update frequency**: When bug fixes or new features needed
---
## Advanced Submodule Operations
### Change Submodule URL
```bash
# Edit .gitmodules file
vim .gitmodules
# Update git config
git submodule sync
# Update the submodule
git submodule update --init
```
### Remove a Submodule
```bash
# Remove from .gitmodules
git config -f .gitmodules --remove-section submodule.<name>
# Remove from .git/config
git config -f .git/config --remove-section submodule.<name>
# Remove directory
git rm --cached <submodule_path>
rm -rf <submodule_path>
# Commit changes
git add .gitmodules
git commit -m "Remove <name> submodule"
```
### Track a Branch Instead of Commit
```bash
# Edit .gitmodules to add branch
[submodule "blossom"]
path = blossom
url = ssh://git@git.laantungir.net:222/laantungir/blossom.git
branch = main
# Update to track branch
git submodule update --remote --merge
```
---
## Security Considerations
1. **Verify submodule sources** - only use trusted repositories
2. **Pin to specific commits** - avoid automatically tracking latest
3. **Review submodule updates** - check changes before updating pins
4. **Use SSH URLs** - more secure than HTTPS for private repos
---
## Troubleshooting Checklist
- [ ] Did you run `git submodule update --init --recursive` after cloning?
- [ ] Are you trying to edit files in submodule directories? (Don't!)
- [ ] Did you commit submodule changes to the main repository?
- [ ] Are your SSH keys set up for the submodule repositories?
- [ ] Is the submodule URL accessible from your current environment?
---
## Quick Reference
| Action | Command |
|--------|---------|
| Clone with submodules | `git clone --recursive <repo>` |
| Initialize after clone | `git submodule update --init --recursive` |
| Update all submodules | `git submodule update --remote` |
| Update one submodule | `git submodule update --remote <name>` |
| Check status | `git submodule status` |
| Reset submodule | `cd <sub> && git checkout . && cd ..` |
Remember: **Submodules are for code we use but never change!**

1088
docs/WEB_ADMIN_SPECIFICATION.md Normal file
View File

File diff suppressed because it is too large Load Diff

387
docs/admin_specification.md Normal file
View File

@@ -0,0 +1,387 @@
# Ginxsom Admin System - Comprehensive Specification
## Overview
The Ginxsom admin system provides both programmatic (API-based) and interactive (web-based) administration capabilities for the Ginxsom Blossom server. The system is designed around Nostr-based authentication and supports multiple administration workflows including first-run setup, ongoing configuration management, and operational monitoring.
## Architecture Components
### 1. Configuration System
- **File-based configuration**: Signed Nostr events stored as JSON files following XDG Base Directory specification
- **Database configuration**: Key-value pairs stored in SQLite for runtime configuration
- **Interactive setup**: Command-line wizard for initial server configuration
- **Manual setup**: Scripts for generating signed configuration events
### 2. Authentication & Authorization
- **Nostr-based auth**: All admin operations require valid Nostr event signatures
- **Admin pubkey verification**: Only configured admin public keys can perform admin operations
- **Event validation**: Full cryptographic verification of Nostr events including structure, signature, and expiration
- **Method-specific authorization**: Different event types for different operations (upload, admin, delete, etc.)
### 3. API System
- **RESTful endpoints**: `/api/*` routes for programmatic administration
- **Command-line testing**: Complete test suite using `nak` and `curl`
- **JSON responses**: Structured data for all admin operations
- **CORS support**: Cross-origin requests for web admin interface
### 4. Web Interface (Future)
- **Single-page application**: Self-contained HTML file with inline CSS/JS
- **Real-time monitoring**: Statistics and system health dashboards
- **Configuration management**: GUI for server settings
- **File management**: Browse and manage uploaded blobs
## Configuration System Architecture
### File-based Configuration (Priority 1)
**Location**: Follows XDG Base Directory Specification
- `$XDG_CONFIG_HOME/ginxsom/ginxsom_config_event.json`
- Falls back to `$HOME/.config/ginxsom/ginxsom_config_event.json`
**Format**: Signed Nostr event containing server configuration
```json
{
"kind": 33333,
"created_at": 1704067200,
"tags": [
["server_privkey", "server_private_key_hex"],
["cdn_origin", "https://cdn.example.com"],
["max_file_size", "104857600"],
["nip94_enabled", "true"]
],
"content": "Ginxsom server configuration",
"pubkey": "admin_public_key_hex",
"id": "event_id_hash",
"sig": "event_signature"
}
```
**Loading Process**:
1. Check for file-based config at XDG location
2. Validate Nostr event structure and signature
3. Extract configuration from event tags
4. Apply settings to server (database storage)
5. Fall back to database-only config if file missing/invalid
### Database Configuration (Priority 2)
**Table**: `server_config`
```sql
CREATE TABLE server_config (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
description TEXT,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```
**Key Configuration Items**:
- `admin_pubkey`: Authorized admin public key
- `admin_enabled`: Enable/disable admin interface
- `cdn_origin`: Base URL for blob access
- `max_file_size`: Maximum upload size in bytes
- `nip94_enabled`: Enable NIP-94 metadata emission
- `auth_rules_enabled`: Enable authentication rules system
### Setup Workflows
#### Interactive Setup (Command Line)
```bash
# First-run detection
if [[ ! -f "$XDG_CONFIG_HOME/ginxsom/ginxsom_config_event.json" ]]; then
echo "=== Ginxsom First-Time Setup Required ==="
echo "1. Run interactive setup wizard"
echo "2. Exit and create config manually"
read -p "Choice (1/2): " choice
if [[ "$choice" == "1" ]]; then
./scripts/setup.sh
else
echo "Manual setup: Run ./scripts/generate_config.sh"
exit 1
fi
fi
```
#### Manual Setup (Script-based)
```bash
# Generate configuration event
./scripts/generate_config.sh --admin-key <admin_pubkey> \
--server-key <server_privkey> \
--cdn-origin "https://cdn.example.com" \
--output "$XDG_CONFIG_HOME/ginxsom/ginxsom_config_event.json"
```
### C Implementation Functions
#### Configuration Loading
```c
// Get XDG-compliant config file path
int get_config_file_path(char* path, size_t path_size);
// Load and validate config event from file
int load_server_config(const char* config_path);
// Extract config from validated event and apply to server
int apply_config_from_event(cJSON* event);
// Interactive setup runner for first-run
int run_interactive_setup(const char* config_path);
```
#### Security Features
- Server private key stored only in memory (never in database)
- Config file must be signed Nostr event
- Full cryptographic validation of config events
- Admin pubkey verification for all operations
## Admin API Specification
### Authentication Model
All admin API endpoints (except `/api/health`) require Nostr authentication:
**Authorization Header Format**:
```
Authorization: Nostr <base64-encoded-event>
```
**Required Event Structure**:
```json
{
"kind": 24242,
"created_at": 1704067200,
"tags": [
["t", "GET"],
["expiration", "1704070800"]
],
"content": "admin_request",
"pubkey": "admin_public_key",
"id": "event_id",
"sig": "event_signature"
}
```
### API Endpoints
#### GET /api/health
**Purpose**: System health check (no authentication required)
**Response**:
```json
{
"status": "success",
"data": {
"database": "connected",
"blob_directory": "accessible",
"server_time": 1704067200,
"uptime": 3600,
"disk_usage": {
"total_bytes": 1073741824,
"used_bytes": 536870912,
"available_bytes": 536870912,
"usage_percent": 50.0
}
}
}
```
#### GET /api/stats
**Purpose**: Server statistics and metrics
**Authentication**: Required (admin pubkey)
**Response**:
```json
{
"status": "success",
"data": {
"total_files": 1234,
"total_bytes": 104857600,
"total_size_mb": 100.0,
"unique_uploaders": 56,
"first_upload": 1693929600,
"last_upload": 1704067200,
"avg_file_size": 85049,
"file_types": {
"image/png": 45,
"image/jpeg": 32,
"application/pdf": 12,
"other": 8
}
}
}
```
#### GET /api/config
**Purpose**: Retrieve current server configuration
**Authentication**: Required (admin pubkey)
**Response**:
```json
{
"status": "success",
"data": {
"cdn_origin": "http://localhost:9001",
"max_file_size": "104857600",
"nip94_enabled": "true",
"auth_rules_enabled": "false",
"auth_cache_ttl": "300"
}
}
```
#### PUT /api/config
**Purpose**: Update server configuration
**Authentication**: Required (admin pubkey)
**Request Body**:
```json
{
"max_file_size": "209715200",
"nip94_enabled": "true",
"cdn_origin": "https://cdn.example.com"
}
```
**Response**:
```json
{
"status": "success",
"message": "Configuration updated successfully",
"updated_keys": ["max_file_size", "cdn_origin"]
}
```
#### GET /api/files
**Purpose**: List recent files with pagination
**Authentication**: Required (admin pubkey)
**Parameters**:
- `limit` (default: 50): Number of files to return
- `offset` (default: 0): Pagination offset
**Response**:
```json
{
"status": "success",
"data": {
"files": [
{
"sha256": "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553",
"size": 184292,
"type": "application/pdf",
"uploaded_at": 1725105921,
"uploader_pubkey": "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798",
"filename": "document.pdf",
"url": "http://localhost:9001/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf"
}
],
"total": 1234,
"limit": 50,
"offset": 0
}
}
```
## Implementation Status
### ✅ Completed Components
1. **Database-based configuration loading** - Implemented in main.c
2. **Admin API authentication system** - Implemented in admin_api.c
3. **Nostr event validation** - Full cryptographic verification
4. **Admin pubkey verification** - Database-backed authorization
5. **Basic API endpoints** - Health, stats, config, files
### ✅ Recently Completed Components
1. **File-based configuration system** - Fully implemented in main.c with XDG compliance
2. **Interactive setup wizard** - Complete shell script with guided setup process (`scripts/setup.sh`)
3. **Manual config generation** - Full-featured command-line config generator (`scripts/generate_config.sh`)
4. **Testing infrastructure** - Comprehensive admin API test suite (`scripts/test_admin.sh`)
5. **Documentation system** - Complete setup and usage documentation (`scripts/README.md`)
### 📋 Planned Components
1. **Web admin interface** - Single-page HTML application
2. **Enhanced monitoring** - Real-time statistics dashboard
3. **Bulk operations** - Multi-file management APIs
4. **Configuration validation** - Advanced config checking
5. **Audit logging** - Admin action tracking
## Setup Instructions
### 1. Enable Admin Interface
```bash
# Configure admin pubkey and enable interface
sqlite3 db/ginxsom.db << EOF
INSERT OR REPLACE INTO server_config (key, value, description) VALUES
('admin_pubkey', 'your_admin_public_key_here', 'Authorized admin public key'),
('admin_enabled', 'true', 'Enable admin interface');
EOF
```
### 2. Test API Access
```bash
# Generate admin authentication event
ADMIN_PRIVKEY="your_admin_private_key"
EVENT=$(nak event -k 24242 -c "admin_request" \
--tag t="GET" \
--tag expiration="$(date -d '+1 hour' +%s)" \
--sec "$ADMIN_PRIVKEY")
# Test admin API
AUTH_HEADER="Nostr $(echo "$EVENT" | base64 -w 0)"
curl -H "Authorization: $AUTH_HEADER" http://localhost:9001/api/stats
```
### 3. Configure File-based Setup (Future)
```bash
# Create XDG config directory
mkdir -p "$XDG_CONFIG_HOME/ginxsom"
# Generate signed config event
./scripts/generate_config.sh \
--admin-key "your_admin_pubkey" \
--server-key "generated_server_privkey" \
--output "$XDG_CONFIG_HOME/ginxsom/ginxsom_config_event.json"
```
## Security Considerations
### Authentication Security
- **Event expiration**: All admin events must include expiration timestamps
- **Signature validation**: Full secp256k1 cryptographic verification
- **Replay protection**: Event IDs tracked to prevent reuse
- **Admin key rotation**: Support for updating admin pubkeys
### Configuration Security
- **File permissions**: Config files should be readable only by server user
- **Private key handling**: Server private keys never stored in database
- **Config validation**: All configuration changes validated before application
- **Backup verification**: Config events cryptographically verifiable
### Operational Security
- **Access logging**: All admin operations logged with timestamps
- **Rate limiting**: API endpoints protected against abuse
- **Input validation**: All user input sanitized and validated
- **Database security**: Prepared statements prevent SQL injection
## Future Enhancements
### 1. Web Admin Interface
- Self-contained HTML file with inline CSS/JavaScript
- Real-time monitoring dashboards
- Visual configuration management
- File upload/management interface
### 2. Advanced Monitoring
- Performance metrics collection
- Alert system for critical events
- Historical data trending
- Resource usage tracking
### 3. Multi-admin Support
- Multiple authorized admin pubkeys
- Role-based permissions (read-only vs full admin)
- Admin action audit trails
- Delegation capabilities
### 4. Integration Features
- Nostr relay integration for admin events
- Webhook notifications for admin actions
- External authentication providers
- API key management for programmatic access
This specification represents the current understanding and planned development of the Ginxsom admin system, focusing on security, usability, and maintainability.