ginxsom/IMPLEMENTATION.md

# Ginxsom Blossom Server Implementation Checklist

This document outlines the implementation plan for ginxsom, a FastCGI-based Blossom server designed to work with nginx for optimal performance.

## Architecture Overview

- **nginx**: Handles static file serving (GET /<sha256>) for maximum performance
- **FastCGI Application**: Handles authenticated operations, metadata queries, uploads
- **SQLite Database**: Stores blob metadata and server configuration
- **File Storage**: Flat directory structure initially, hierarchical optimization later

---

## Phase 1: Basic File Serving & Retrieval (BUD-01)

### 1.1 Infrastructure Setup
- [x] Create basic directory structure
  - [x] Create `blobs/` directory for file storage
  - [x] Create `db/` directory for SQLite database
  - [x] Create `logs/` directory for application logs
  - [x] Set up proper permissions (nginx readable, app writable)

### 1.2 Database Schema
- [x] Design SQLite schema for blob metadata
  - [x] `blobs` table: sha256, size, type, uploaded_at, uploader_pubkey, filename
  - [x] `server_config` table: key-value pairs for server settings
  - [x] Create database initialization script
  - [x] Add proper indexes on sha256 hash

### 1.3 nginx Configuration
- [x] Configure nginx for static file serving
  - [x] Set up location block for `GET /<sha256>` pattern with extension support
  - [x] Configure try_files directive for multiple extension fallbacks
  - [x] Configure proper MIME type detection
  - [x] Add proper headers (Cache-Control, ETag, etc.)
  - [x] Handle 404s gracefully when blob doesn't exist
  - [x] Configure FastCGI pass-through for HEAD and non-GET requests

**Future Enhancement Note**: Consider implementing nginx Lua extension for true Blossom compliance with dynamic file discovery. The current approach uses explicit extension lists in `try_files`, which works well for common extensions but may not serve files with unusual extensions. Lua module would allow runtime directory scanning for hash-matching files regardless of extension.

### 1.4 Basic HEAD Endpoint
- [x] Implement FastCGI handler for `HEAD /<sha256>`
  - [x] Query database for blob metadata (single source of truth)
  - [x] Extract SHA-256 from URL (strip extensions)
  - [x] Return proper headers (Content-Type, Content-Length, etc.)
  - [x] Return 404 if blob doesn't exist in database
  - [x] Add server timing headers for debugging

### 1.5 Testing & Validation
- [x] Create test blobs with known SHA-256 hashes
- [x] Verify nginx serves files correctly with extension support
- [x] Verify HEAD requests return proper metadata
- [x] Test with missing files (404 responses)
- [x] Test HEAD requests with and without extensions
- [ ] Performance test with large files

---

## Phase 2: Upload & Authentication (BUD-02)

### 2.1 Nostr Authentication Setup
- [x] Integrate nostr_core_lib submodule
  - [x] Update Makefile to include nostr_core_lib paths and static library
  - [x] Build libnostr_core_x64.a using provided build.sh script
  - [x] Add system dependencies: -lsecp256k1 -lssl -lcrypto -lcurl -lz -ldl -lpthread -lm

- [x] Implement authentication functions in main.c (BUD-02 section):
  - [x] `parse_authorization_header()` - Extract JSON from "Nostr base64(event)" header
  - [x] `validate_blossom_event()` - Validate Blossom-specific requirements (kind 24242, content hash, method, expiration)
  - [x] `authenticate_request()` - Main orchestrator function

- [x] Leverage existing nostr_core_lib functions:
  - [x] Use `nostr_validate_event()` for structure + signature validation (from nip001.h)
  - [x] Use standardized error codes from nostr_common.h (NOSTR_SUCCESS, NOSTR_ERROR_EVENT_INVALID_SIGNATURE, etc.)
  - [x] Use `nostr_strerror()` for error message translation

### 2.2 Upload Endpoint Implementation
- [x] Implement `PUT /upload` endpoint
  - [x] Parse Authorization header (Nostr base64 event extraction)
  - [x] Stream file upload to temporary location
  - [x] Calculate SHA-256 hash during upload
  - [x] Validate hash matches authorization if provided
  - [x] Move file to permanent location
  - [x] Store metadata in database (including uploader_pubkey and filename)
  - [x] Return blob descriptor JSON response

### 2.3 Blob Descriptor Response
- [x] Implement blob descriptor structure
  - [x] Required fields: url, sha256, size, type, uploaded
  - [x] Handle MIME type detection
  - [x] Generate proper blob URLs
  - [x] Add optional server-specific fields (uploader_pubkey, filename)

### 2.4 Error Handling
- [x] Implement proper HTTP status codes
  - [x] 400 Bad Request for invalid data
  - [x] 401 Unauthorized for auth failures
  - [x] 409 Conflict for hash mismatches
  - [x] 413 Payload Too Large for size limits
  - [x] 500 Internal Server Error for system issues
- [x] Add detailed error messages
- [x] Implement request logging

### 2.5 List Blobs Endpoint
- [ ] Implement `GET /list/<pubkey>` endpoint
  - [ ] Extract pubkey from URL path
  - [ ] Query database for blobs uploaded by specified pubkey
  - [ ] Support `since` and `until` query parameters for date filtering
  - [ ] Return JSON array of blob descriptors
  - [ ] Handle empty results gracefully
  - [ ] Implement optional authorization with kind 24242 event validation
    - [ ] Validate `t` tag is set to "list"
    - [ ] Check authorization expiration
    - [ ] Verify event signature and structure

### 2.6 Delete Blob Endpoint
- [ ] Implement `DELETE /<sha256>` endpoint
  - [ ] Extract SHA-256 hash from URL path
  - [ ] Require authorization with kind 24242 event validation
    - [ ] Validate `t` tag is set to "delete"
    - [ ] Verify at least one `x` tag matches the requested hash
    - [ ] Check authorization expiration
    - [ ] Verify event signature and structure
  - [ ] Check blob exists in database
  - [ ] Verify uploader_pubkey matches authorized pubkey (ownership check)
  - [ ] Remove blob file from filesystem
  - [ ] Remove blob metadata from database
  - [ ] Handle file deletion errors gracefully
  - [ ] Return appropriate success/error responses

### 2.7 Testing & Validation
- [x] Test uploads without authentication
- [x] Test uploads with valid nostr auth
- [x] Test uploads with invalid auth
- [x] Test hash mismatch scenarios
- [ ] Test file size limits
- [x] Verify blob descriptors are correct
- [x] Verify database metadata storage (uploader_pubkey and filename)

---

## Phase 3: Upload Requirements (BUD-06)

### 3.1 Upload Policy Configuration
- [ ] Add server configuration options
  - [ ] Maximum file size limits
  - [ ] Allowed MIME types
  - [ ] Authentication requirements
  - [ ] Rate limiting settings
  - [ ] Storage quota limits

### 3.2 HEAD /upload Endpoint
- [ ] Implement `HEAD /upload` endpoint
  - [ ] Return upload requirements in headers
  - [ ] Handle optional Authorization header
  - [ ] Return proper status codes for policy checks
  - [ ] Add custom headers for requirements

### 3.3 Upload Validation
- [ ] Implement pre-upload validation
  - [ ] Check file size before processing
  - [ ] Validate MIME types if restricted
  - [ ] Check authentication requirements
  - [ ] Verify user permissions/quotas

### 3.4 Testing & Validation
- [ ] Test upload requirements endpoint
- [ ] Test policy enforcement
- [ ] Test with various client scenarios
- [ ] Verify error responses match spec

---

## Phase 4: Optional Features

### 4.1 User Server Lists (BUD-03) - Optional
- [ ] Implement server list advertisement
- [ ] Handle kind:10063 events
- [ ] Create server discovery endpoint
- [ ] Test client fallback scenarios

### 4.2 Blob Mirroring (BUD-04) - Optional
- [ ] Implement `PUT /mirror` endpoint
- [ ] Add URL downloading capability
- [ ] Implement hash verification
- [ ] Handle authorization for mirroring
- [ ] Test inter-server mirroring

### 4.3 Media Optimization (BUD-05) - Optional
- [ ] Implement `PUT /media` endpoint
- [ ] Add media processing libraries
- [ ] Implement optimization algorithms
- [ ] Handle various media formats
- [ ] Test optimization pipeline

### 4.4 Payment Integration (BUD-07) - Optional
- [ ] Implement 402 Payment Required responses
- [ ] Add Lightning payment support
- [ ] Add Cashu payment support
- [ ] Implement payment verification
- [ ] Test payment flows

### 4.5 NIP-94 Metadata (BUD-08) - Optional
- [ ] Add NIP-94 tag generation
- [ ] Extend blob descriptor responses
- [ ] Generate magnet links if supported
- [ ] Test metadata compatibility

### 4.6 Blob Reporting (BUD-09) - Optional
- [ ] Implement `PUT /report` endpoint
- [ ] Handle NIP-56 report events
- [ ] Add moderation interface
- [ ] Implement content filtering
- [ ] Test reporting workflow

---

## Development Milestones

### Milestone 1: Basic Functionality (Phase 1 Complete)
- [x] nginx serves files by hash with extension support
- [x] HEAD requests return metadata from database
- [x] Database stores blob information with proper schema

### Milestone 2: Full Upload Support (Phase 2 Pending)
- [x] Basic upload functionality working (PUT requests accepted)
- [x] SHA-256 hash calculation during upload
- [x] File storage to blobs/ directory
- [x] Blob descriptor JSON response
- [x] Authenticated uploads working (Nostr kind 24242 event validation)
- [x] Proper error handling for upload scenarios
- [x] Database metadata storage during upload (with uploader_pubkey and filename)
- [ ] List blobs endpoint implemented (GET /list/<pubkey>)
- [ ] Delete blob endpoint implemented (DELETE /<sha256>)

### Milestone 3: Policy Compliance (Phase 3 Pending)
- [ ] Upload requirements implemented
- [ ] Server policies configurable
- [ ] Spec compliance verified

### Milestone 4: Production Ready (Phase 4 Complete)
- Optional features implemented as needed
- Performance optimized
- Security hardened
- Documentation complete

---

## Testing Strategy

### Unit Tests
- [ ] Authentication validation functions
- [ ] SHA-256 hash calculation
- [ ] Database operations
- [ ] Configuration parsing

### Integration Tests
- [ ] nginx + FastCGI integration
- [ ] End-to-end upload/download flows
- [ ] Error scenario handling
- [ ] Multi-client concurrent access

### Performance Tests
- [ ] Large file uploads/downloads
- [ ] Concurrent request handling
- [ ] Database query performance
- [ ] Memory usage optimization

### Compliance Tests
- [ ] Blossom protocol compliance
- [ ] Nostr event validation
- [ ] HTTP specification compliance
- [ ] Security best practices

---

## Security Considerations

- [ ] Input validation on all endpoints
- [ ] Rate limiting to prevent abuse
- [ ] Secure file storage permissions
- [ ] Database injection prevention
- [ ] Memory safety in C implementation
- [ ] Proper error message sanitization
- [ ] Log security (no sensitive data)

---

## Performance Optimizations

- [ ] nginx direct file serving (bypasses application)
- [ ] FastCGI connection pooling
- [ ] Database connection management
- [ ] Efficient hash calculation
- [ ] Memory-mapped file operations
- [ ] Hierarchical file storage (future)
- [ ] CDN integration support

---

## Deployment Checklist

- [ ] nginx configuration template
- [ ] FastCGI service configuration
- [ ] Database initialization scripts
- [ ] Log rotation setup
- [ ] Monitoring and health checks
- [ ] Backup procedures
- [ ] Security hardening guide
- [ ] Documentation and examples