# Ginxsom Blossom Server Implementation Checklist

This document outlines the implementation plan for ginxsom, a FastCGI-based Blossom server designed to work with nginx for optimal performance.

## Architecture Overview

- **nginx**: Handles static file serving (GET /<sha256>) for maximum performance
- **FastCGI Application**: Handles authenticated operations, metadata queries, uploads
- **SQLite Database**: Stores blob metadata and server configuration
- **File Storage**: Flat directory structure initially, hierarchical optimization later

---

## Phase 1: Basic File Serving & Retrieval (BUD-01)

### 1.1 Infrastructure Setup
- [x] Create basic directory structure
  - [x] Create `blobs/` directory for file storage
  - [x] Create `db/` directory for SQLite database
  - [x] Create `logs/` directory for application logs
  - [x] Set up proper permissions (nginx readable, app writable)

### 1.2 Database Schema
- [x] Design SQLite schema for blob metadata
  - [x] `blobs` table: sha256, size, type, uploaded_at, uploader_pubkey, filename
  - [x] `server_config` table: key-value pairs for server settings
  - [x] Create database initialization script
  - [x] Add proper indexes on sha256 hash

### 1.3 nginx Configuration
- [x] Configure nginx for static file serving
  - [x] Set up location block for `GET /<sha256>` pattern with extension support
  - [x] Configure try_files directive for multiple extension fallbacks
  - [x] Configure proper MIME type detection
  - [x] Add proper headers (Cache-Control, ETag, etc.)
  - [x] Handle 404s gracefully when blob doesn't exist
  - [x] Configure FastCGI pass-through for HEAD and non-GET requests

**Future Enhancement Note**: Consider implementing nginx Lua extension for true Blossom compliance with dynamic file discovery. The current approach uses explicit extension lists in `try_files`, which works well for common extensions but may not serve files with unusual extensions. Lua module would allow runtime directory scanning for hash-matching files regardless of extension.

### 1.4 Basic HEAD Endpoint
- [x] Implement FastCGI handler for `HEAD /<sha256>`
  - [x] Query database for blob metadata (single source of truth)
  - [x] Extract SHA-256 from URL (strip extensions)
  - [x] Return proper headers (Content-Type, Content-Length, etc.)
  - [x] Return 404 if blob doesn't exist in database
  - [x] Add server timing headers for debugging

### 1.5 Testing & Validation
- [x] Create test blobs with known SHA-256 hashes
- [x] Verify nginx serves files correctly with extension support
- [x] Verify HEAD requests return proper metadata
- [x] Test with missing files (404 responses)
- [x] Test HEAD requests with and without extensions
- [ ] Performance test with large files

---

## Phase 2: Upload & Authentication (BUD-02)

### 2.1 Nostr Authentication Setup
- [x] Integrate nostr_core_lib submodule
  - [x] Update Makefile to include nostr_core_lib paths and static library
  - [x] Build libnostr_core_x64.a using provided build.sh script
  - [x] Add system dependencies: -lsecp256k1 -lssl -lcrypto -lcurl -lz -ldl -lpthread -lm

- [x] Implement authentication functions in main.c (BUD-02 section):
  - [x] `parse_authorization_header()` - Extract JSON from "Nostr base64(event)" header
  - [x] `validate_blossom_event()` - Validate Blossom-specific requirements (kind 24242, content hash, method, expiration)
  - [x] `authenticate_request()` - Main orchestrator function

- [x] Leverage existing nostr_core_lib functions:
  - [x] Use `nostr_validate_event()` for structure + signature validation (from nip001.h)
  - [x] Use standardized error codes from nostr_common.h (NOSTR_SUCCESS, NOSTR_ERROR_EVENT_INVALID_SIGNATURE, etc.)
  - [x] Use `nostr_strerror()` for error message translation

### 2.2 Upload Endpoint Implementation
- [x] Implement `PUT /upload` endpoint
  - [x] Parse Authorization header (Nostr base64 event extraction)
  - [x] Stream file upload to temporary location
  - [x] Calculate SHA-256 hash during upload
  - [x] Validate hash matches authorization if provided
  - [x] Move file to permanent location
  - [x] Store metadata in database (including uploader_pubkey and filename)
  - [x] Return blob descriptor JSON response

### 2.3 Blob Descriptor Response
- [x] Implement blob descriptor structure
  - [x] Required fields: url, sha256, size, type, uploaded
  - [x] Handle MIME type detection
  - [x] Generate proper blob URLs
  - [x] Add optional server-specific fields (uploader_pubkey, filename)

### 2.4 Error Handling
- [x] Implement proper HTTP status codes
  - [x] 400 Bad Request for invalid data
  - [x] 401 Unauthorized for auth failures
  - [x] 409 Conflict for hash mismatches
  - [x] 413 Payload Too Large for size limits
  - [x] 500 Internal Server Error for system issues
- [x] Add detailed error messages
- [x] Implement request logging

### 2.5 List Blobs Endpoint
- [x] Implement `GET /list/<pubkey>` endpoint
  - [x] Extract pubkey from URL path
  - [x] Query database for blobs uploaded by specified pubkey
  - [x] Support `since` and `until` query parameters for date filtering
  - [x] Return JSON array of blob descriptors
  - [x] Handle empty results gracefully
  - [x] Implement optional authorization with kind 24242 event validation
    - [x] Validate `t` tag is set to "list"
    - [x] Check authorization expiration
    - [x] Verify event signature and structure

### 2.6 Delete Blob Endpoint
- [x] Implement `DELETE /<sha256>` endpoint
  - [x] Extract SHA-256 hash from URL path
  - [x] Require authorization with kind 24242 event validation
    - [x] Validate `t` tag is set to "delete"
    - [x] Verify at least one `x` tag matches the requested hash
    - [x] Check authorization expiration
    - [x] Verify event signature and structure
  - [x] Check blob exists in database
  - [x] Verify uploader_pubkey matches authorized pubkey (ownership check)
  - [x] Remove blob file from filesystem
  - [x] Remove blob metadata from database
  - [x] Handle file deletion errors gracefully
  - [x] Return appropriate success/error responses

### 2.7 Testing & Validation
- [x] Test uploads without authentication
- [x] Test uploads with valid nostr auth
- [x] Test uploads with invalid auth
- [x] Test hash mismatch scenarios
- [ ] Test file size limits
- [x] Verify blob descriptors are correct
- [x] Verify database metadata storage (uploader_pubkey and filename)

---

## Phase 3: Upload Requirements (BUD-06)

### 3.1 Upload Policy Configuration
- [ ] Add server configuration options
  - [ ] Maximum file size limits
  - [ ] Allowed MIME types
  - [ ] Authentication requirements
  - [ ] Rate limiting settings
  - [ ] Storage quota limits
  - [ ] Hash-based banning/filtering

### 3.2 HEAD /upload Endpoint Implementation
- [ ] Implement `HEAD /upload` endpoint for pre-flight upload validation
  - [ ] Parse client headers:
    - [ ] `X-SHA-256`: blob's SHA-256 hash
    - [ ] `X-Content-Length`: blob size in bytes
    - [ ] `X-Content-Type`: blob's MIME type
  - [ ] Handle optional Authorization header (same as PUT /upload)
  - [ ] Perform validation checks without file transfer:
    - [ ] Validate SHA-256 format
    - [ ] Check file size against limits
    - [ ] Validate MIME type restrictions
    - [ ] Check authentication if required
    - [ ] Check if hash already exists (duplicate detection)
    - [ ] Check if hash is banned
  - [ ] Return appropriate HTTP status codes:
    - [ ] `200 OK` - upload can proceed
    - [ ] `400 Bad Request` - invalid headers
    - [ ] `401 Unauthorized` - auth required
    - [ ] `403 Forbidden` - not permitted (banned hash, etc.)
    - [ ] `411 Length Required` - missing content length
    - [ ] `413 Content Too Large` - file too large
    - [ ] `415 Unsupported Media Type` - invalid MIME type
  - [ ] Add `X-Reason` header with human-readable error messages

### 3.3 Upload Pre-validation Logic
- [ ] Create validation functions that can be shared between HEAD and PUT endpoints
  - [ ] `validate_upload_headers()` - check required headers present and valid
  - [ ] `check_file_size_limits()` - enforce maximum size restrictions
  - [ ] `check_mime_type_allowed()` - validate against allowed types list
  - [ ] `check_hash_restrictions()` - check banned hashes, duplicates
  - [ ] `check_upload_permissions()` - user-specific upload rights

### 3.4 DOS Protection Benefits
- [ ] Implement early rejection before file transfer:
  - [ ] Authentication happens before any file data sent
  - [ ] Size validation prevents large file uploads that would be rejected
  - [ ] MIME type checking prevents unwanted file types
  - [ ] Hash checking prevents duplicate uploads
- [ ] Update PUT /upload to use same validation functions for consistency

### 3.5 Client Integration Support
- [ ] Update nginx configuration to properly handle HEAD requests to /upload
- [ ] Ensure FastCGI handles HEAD method for /upload endpoint
- [ ] Add CORS headers for preflight requests

### 3.6 Testing & Validation
- [ ] Test HEAD /upload with valid headers
- [ ] Test various error scenarios (missing headers, invalid formats)
- [ ] Test authorization requirements
- [ ] Test policy enforcement (size limits, MIME types, banned hashes)
- [ ] Verify error responses match BUD-06 specification
- [ ] Test client workflow: HEAD check → PUT upload
- [ ] Verify DOS protection effectiveness

---

## Phase 4: Advanced Authentication & Administration System

### 4.1 Flexible Authentication Rules System

#### 4.1.1 Database Schema Extension
- [ ] Create authentication rules tables
  - [ ] `auth_rules` table: rule_type, rule_target, operation, rule_value, enabled, expires_at
  - [ ] `auth_cache` table: performance caching for rule evaluation results
  - [ ] Add indexes on rule_type, rule_target, operation for performance

#### 4.1.2 Authentication Rule Types Implementation
- [ ] Basic rule types:
  - [ ] `pubkey_whitelist`: Only specific pubkeys allowed
  - [ ] `pubkey_blacklist`: Specific pubkeys banned
  - [ ] `hash_blacklist`: Specific file hashes cannot be uploaded
  - [ ] `mime_type_whitelist`: Only specific content types allowed
  - [ ] `mime_type_blacklist`: Specific content types banned
- [ ] Advanced rule types:
  - [ ] `rate_limit`: Limit operations per pubkey/IP per time period
  - [ ] `size_limit`: Per-pubkey or global size limits
  - [ ] `conditional`: Complex JSON-based rules (time-based, size-based, etc.)

#### 4.1.3 Rule Evaluation Engine
- [ ] Core authentication functions:
  - [ ] `evaluate_auth_rules()`: Main rule evaluation with caching
  - [ ] `check_rule_cache()`: Performance optimization layer
  - [ ] `process_rule_priority()`: Handle rule precedence and conflicts
  - [ ] `update_auth_cache()`: Store evaluation results for reuse
- [ ] Integration points:
  - [ ] Extend `handle_upload_request()` with rule evaluation
  - [ ] Extend `handle_delete_request()` with rule evaluation
  - [ ] Extend `handle_list_request()` with rule evaluation (optional)

#### 4.1.4 Rule Management Interface
- [ ] SQL-based rule management:
  - [ ] `add_auth_rule()`: Add new authentication rules
  - [ ] `remove_auth_rule()`: Remove rules by ID
  - [ ] `list_auth_rules()`: Query existing rules with filters
  - [ ] `update_auth_rule()`: Modify existing rule parameters

### 4.2 Nostr-Native Administrative Interface

#### 4.2.1 Server Identity Management
- [ ] Server keypair generation and storage:
  - [ ] Generate server public/private keypair on first run
  - [ ] Store server pubkey in `server_config` table
  - [ ] Secure private key storage (encrypted file or environment)
  - [ ] Key rotation capabilities for security

#### 4.2.2 Administrator Management System
- [ ] Administrator database schema:
  - [ ] `administrators` table: pubkey, permissions, added_by, expires_at
  - [ ] Permission levels: rules, config, users, stats, * (full access)
  - [ ] Initial admin setup during server deployment
- [ ] Administrative functions:
  - [ ] `check_admin_permissions()`: Verify admin authorization
  - [ ] `add_administrator()`: Grant admin privileges
  - [ ] `remove_administrator()`: Revoke admin privileges
  - [ ] `list_administrators()`: Query admin list with permissions

#### 4.2.3 Administrative Event Types
- [ ] Event kind definitions:
  - [ ] Kind 30242: Administrative commands (rule_add, rule_remove, config_set, etc.)
  - [ ] Kind 30243: Administrative queries (stats_get, rule_list, audit_log, etc.)
  - [ ] Kind 30244: Administrative responses (command results, query data)
- [ ] Command implementations:
  - [ ] Rule management: `rule_add`, `rule_remove`, `rule_update`, `rule_list`
  - [ ] System management: `config_set`, `config_get`, `admin_add`, `admin_remove`
  - [ ] Query operations: `stats_get`, `blob_list`, `audit_log`, `storage_cleanup`

#### 4.2.4 Administrative Event Processing
- [ ] HTTP administrative endpoint:
  - [ ] `POST /admin` with nostr event authorization
  - [ ] JSON command interface with parameter validation
  - [ ] Synchronous response with operation results
- [ ] Direct nostr relay integration (future enhancement):
  - [ ] Subscribe to administrative events on configured relays
  - [ ] Real-time event processing and response
  - [ ] Publish response events back to relays

#### 4.2.5 Administrative Audit Trail
- [ ] Administrative logging system:
  - [ ] `admin_log` table: track all administrative actions
  - [ ] Event ID references for nostr event traceability
  - [ ] Success/failure tracking with detailed error messages
  - [ ] Audit query capabilities for compliance

#### 4.2.6 Security & Permission Framework
- [ ] Multi-level permission system:
  - [ ] Granular permissions: rules, config, users, stats
  - [ ] Permission inheritance and delegation
  - [ ] Time-limited administrative access (expires_at)
- [ ] Authentication security:
  - [ ] Strong nostr signature validation
  - [ ] Administrator authorization chain verification
  - [ ] Command-specific permission checks
  - [ ] Rate limiting for administrative operations

### 4.3 Integration & Testing
- [ ] Authentication system integration:
  - [ ] Integrate rule evaluation into existing authentication flow
  - [ ] Maintain backward compatibility with nostr-only authentication
  - [ ] Performance testing with rule caching
- [ ] Administrative system testing:
  - [ ] Test all administrative commands and queries
  - [ ] Verify permission enforcement and security
  - [ ] Test audit logging and compliance features
  - [ ] Load testing for administrative operations

---

## Phase 5: Optional Features
---

## Phase 5: Optional Features

### 4.1 User Server Lists (BUD-03) - Optional
- [ ] Implement server list advertisement
- [ ] Handle kind:10063 events
- [ ] Create server discovery endpoint
- [ ] Test client fallback scenarios

### 4.2 Blob Mirroring (BUD-04) - Optional
- [ ] Implement `PUT /mirror` endpoint
- [ ] Add URL downloading capability
- [ ] Implement hash verification
- [ ] Handle authorization for mirroring
- [ ] Test inter-server mirroring

### 4.3 Media Optimization (BUD-05) - Optional
- [ ] Implement `PUT /media` endpoint
- [ ] Add media processing libraries
- [ ] Implement optimization algorithms
- [ ] Handle various media formats
- [ ] Test optimization pipeline

### 4.4 Payment Integration (BUD-07) - Optional
- [ ] Implement 402 Payment Required responses
- [ ] Add Lightning payment support
- [ ] Add Cashu payment support
- [ ] Implement payment verification
- [ ] Test payment flows

### 4.5 NIP-94 Metadata (BUD-08) - Optional
- [ ] Add NIP-94 tag generation
- [ ] Extend blob descriptor responses
- [ ] Generate magnet links if supported
- [ ] Test metadata compatibility

### 4.6 Blob Reporting (BUD-09) - Optional
- [ ] Implement `PUT /report` endpoint
- [ ] Handle NIP-56 report events
- [ ] Add moderation interface
- [ ] Implement content filtering
- [ ] Test reporting workflow

---

## Development Milestones

### Milestone 1: Basic Functionality (Phase 1 Complete)
- [x] nginx serves files by hash with extension support
- [x] HEAD requests return metadata from database
- [x] Database stores blob information with proper schema

### Milestone 2: Full Upload Support (Phase 2 Pending)
- [x] Basic upload functionality working (PUT requests accepted)
- [x] SHA-256 hash calculation during upload
- [x] File storage to blobs/ directory
- [x] Blob descriptor JSON response
- [x] Authenticated uploads working (Nostr kind 24242 event validation)
- [x] Proper error handling for upload scenarios
- [x] Database metadata storage during upload (with uploader_pubkey and filename)
- [x] List blobs endpoint implemented (GET /list/<pubkey>)
- [x] Delete blob endpoint implemented (DELETE /<sha256>)

### Milestone 3: Policy Compliance (Phase 3 Pending)
- [ ] Upload requirements implemented
- [ ] Server policies configurable
- [ ] Spec compliance verified

### Milestone 4: Advanced Authentication (Phase 4 Complete)
- [ ] Flexible authentication rules system operational
- [ ] Nostr-native administrative interface implemented
- [ ] Rule evaluation engine with caching performance
- [ ] Administrative audit trail and compliance features

### Milestone 5: Production Ready (Phase 5 Complete)
- Optional features implemented as needed
- Performance optimized
- Security hardened
- Documentation complete

---

## Testing Strategy

### Unit Tests
- [ ] Authentication validation functions
- [ ] SHA-256 hash calculation
- [ ] Database operations
- [ ] Configuration parsing

### Integration Tests
- [ ] nginx + FastCGI integration
- [ ] End-to-end upload/download flows
- [ ] Error scenario handling
- [ ] Multi-client concurrent access

### Performance Tests
- [ ] Large file uploads/downloads
- [ ] Concurrent request handling
- [ ] Database query performance
- [ ] Memory usage optimization

### Compliance Tests
- [ ] Blossom protocol compliance
- [ ] Nostr event validation
- [ ] HTTP specification compliance
- [ ] Security best practices

---

## Future Improvements

### Upload Security & Performance Enhancements

**Current Issue**: The existing upload flow has a DOS vulnerability where large files are loaded entirely into memory before authentication occurs. This allows unauthenticated attackers to exhaust server memory.

**Current Flow**:
```
Client → nginx → FastCGI ginxsom
                 ├─ reads entire file into memory (malloc + fread)
                 ├─ validates auth (after file in memory)
                 └─ saves to blobs/ or errors
```

**Proposed Solution - nginx Upload Module**:
```
Client → nginx upload module → temp file → FastCGI ginxsom
         ├─ saves to /tmp/uploads/           ├─ validates auth quickly
         └─ passes metadata only            ├─ moves file to blobs/ 
                                            └─ or deletes temp file
```

**Benefits**:
- Eliminates DOS vulnerability - nginx handles large files efficiently
- Fast auth validation - no waiting for full upload
- Leverages nginx strengths - what it's designed for
- Better scalability - memory usage independent of file size

**Implementation Requirements**:
- nginx upload module configuration
- Temp file cleanup handling
- Modified FastCGI code to process file paths instead of stdin
- Proper error handling for temp file operations

**Alternative Enhancement - HTTP 100 Continue**:
Could propose new Blossom BUD for two-phase upload:
1. Client sends headers with `Expect: 100-continue` + auth event
2. Server validates early (signature, expiration, pubkey) 
3. Server responds `100 Continue` or `401 Unauthorized`
4. Client only sends file data if authorized

**Priority**: Implement after core BUD compliance is complete.

---

## Security Considerations

- [ ] Input validation on all endpoints
- [ ] Rate limiting to prevent abuse
- [ ] Secure file storage permissions
- [ ] Database injection prevention
- [ ] Memory safety in C implementation
- [ ] Proper error message sanitization
- [ ] Log security (no sensitive data)
- [ ] **Upload DOS vulnerability** - Current implementation vulnerable to memory exhaustion attacks

---

## Performance Optimizations

- [ ] nginx direct file serving (bypasses application)
- [ ] FastCGI connection pooling
- [ ] Database connection management
- [ ] Efficient hash calculation
- [ ] Memory-mapped file operations
- [ ] Hierarchical file storage (future)
- [ ] CDN integration support

---

## Deployment Checklist

- [ ] nginx configuration template
- [ ] FastCGI service configuration
- [ ] Database initialization scripts
- [ ] Log rotation setup
- [ ] Monitoring and health checks
- [ ] Backup procedures
- [ ] Security hardening guide
- [ ] Documentation and examples