290 lines
10 KiB
Markdown
290 lines
10 KiB
Markdown
# Ginxsom Blossom Server Implementation Checklist
|
|
|
|
This document outlines the implementation plan for ginxsom, a FastCGI-based Blossom server designed to work with nginx for optimal performance.
|
|
|
|
## Architecture Overview
|
|
|
|
- **nginx**: Handles static file serving (GET /<sha256>) for maximum performance
|
|
- **FastCGI Application**: Handles authenticated operations, metadata queries, uploads
|
|
- **SQLite Database**: Stores blob metadata and server configuration
|
|
- **File Storage**: Flat directory structure initially, hierarchical optimization later
|
|
|
|
---
|
|
|
|
## Phase 1: Basic File Serving & Retrieval (BUD-01)
|
|
|
|
### 1.1 Infrastructure Setup
|
|
- [x] Create basic directory structure
|
|
- [x] Create `blobs/` directory for file storage
|
|
- [x] Create `db/` directory for SQLite database
|
|
- [x] Create `logs/` directory for application logs
|
|
- [x] Set up proper permissions (nginx readable, app writable)
|
|
|
|
### 1.2 Database Schema
|
|
- [x] Design SQLite schema for blob metadata
|
|
- [x] `blobs` table: sha256, size, type, uploaded_at, uploader_pubkey, filename
|
|
- [x] `server_config` table: key-value pairs for server settings
|
|
- [x] Create database initialization script
|
|
- [x] Add proper indexes on sha256 hash
|
|
|
|
### 1.3 nginx Configuration
|
|
- [x] Configure nginx for static file serving
|
|
- [x] Set up location block for `GET /<sha256>` pattern with extension support
|
|
- [x] Configure try_files directive for multiple extension fallbacks
|
|
- [x] Configure proper MIME type detection
|
|
- [x] Add proper headers (Cache-Control, ETag, etc.)
|
|
- [x] Handle 404s gracefully when blob doesn't exist
|
|
- [x] Configure FastCGI pass-through for HEAD and non-GET requests
|
|
|
|
**Future Enhancement Note**: Consider implementing nginx Lua extension for true Blossom compliance with dynamic file discovery. The current approach uses explicit extension lists in `try_files`, which works well for common extensions but may not serve files with unusual extensions. Lua module would allow runtime directory scanning for hash-matching files regardless of extension.
|
|
|
|
### 1.4 Basic HEAD Endpoint
|
|
- [x] Implement FastCGI handler for `HEAD /<sha256>`
|
|
- [x] Query database for blob metadata (single source of truth)
|
|
- [x] Extract SHA-256 from URL (strip extensions)
|
|
- [x] Return proper headers (Content-Type, Content-Length, etc.)
|
|
- [x] Return 404 if blob doesn't exist in database
|
|
- [x] Add server timing headers for debugging
|
|
|
|
### 1.5 Testing & Validation
|
|
- [x] Create test blobs with known SHA-256 hashes
|
|
- [x] Verify nginx serves files correctly with extension support
|
|
- [x] Verify HEAD requests return proper metadata
|
|
- [x] Test with missing files (404 responses)
|
|
- [x] Test HEAD requests with and without extensions
|
|
- [ ] Performance test with large files
|
|
|
|
---
|
|
|
|
## Phase 2: Upload & Authentication (BUD-02)
|
|
|
|
### 2.1 Nostr Authentication Setup
|
|
- [ ] Integrate nostr_core_lib submodule
|
|
- [ ] Update Makefile to include nostr_core_lib paths and static library
|
|
- [ ] Build libnostr_core_x64.a using provided build.sh script
|
|
- [ ] Add system dependencies: -lsecp256k1 -lssl -lcrypto -lcurl -lz -ldl -lpthread -lm
|
|
|
|
- [ ] Implement authentication functions in main.c (BUD-02 section):
|
|
- [ ] `parse_authorization_header()` - Extract JSON from "Nostr base64(event)" header
|
|
- [ ] `validate_blossom_event()` - Validate Blossom-specific requirements (kind 24242, content hash, method, expiration)
|
|
- [ ] `authenticate_request()` - Main orchestrator function
|
|
|
|
- [ ] Leverage existing nostr_core_lib functions:
|
|
- [ ] Use `nostr_validate_event()` for structure + signature validation (from nip001.h)
|
|
- [ ] Use standardized error codes from nostr_common.h (NOSTR_SUCCESS, NOSTR_ERROR_EVENT_INVALID_SIGNATURE, etc.)
|
|
- [ ] Use `nostr_strerror()` for error message translation
|
|
|
|
**Function Specifications:**
|
|
```c
|
|
// Custom functions to implement:
|
|
int parse_authorization_header(const char* auth_header, char* event_json, size_t json_size);
|
|
int validate_blossom_event(struct cJSON* event, const char* expected_hash, const char* method);
|
|
int authenticate_request(const char* auth_header, const char* method, const char* file_hash);
|
|
|
|
// Existing nostr_core_lib functions to use directly:
|
|
// - nostr_validate_event(cJSON* event) - handles structure + signature validation
|
|
// - nostr_validate_event_structure(cJSON* event) - if separate validation needed
|
|
// - nostr_verify_event_signature(cJSON* event) - if separate signature check needed
|
|
```
|
|
|
|
### 2.2 Upload Endpoint Implementation
|
|
- [ ] Implement `PUT /upload` endpoint
|
|
- [ ] Parse Authorization header (optional but recommended)
|
|
- [ ] Stream file upload to temporary location
|
|
- [ ] Calculate SHA-256 hash during upload
|
|
- [ ] Validate hash matches authorization if provided
|
|
- [ ] Move file to permanent location
|
|
- [ ] Store metadata in database
|
|
- [ ] Return blob descriptor JSON response
|
|
|
|
### 2.3 Blob Descriptor Response
|
|
- [ ] Implement blob descriptor structure
|
|
- [ ] Required fields: url, sha256, size, type, uploaded
|
|
- [ ] Handle MIME type detection
|
|
- [ ] Generate proper blob URLs
|
|
- [ ] Add optional server-specific fields
|
|
|
|
### 2.4 Error Handling
|
|
- [ ] Implement proper HTTP status codes
|
|
- [ ] 400 Bad Request for invalid data
|
|
- [ ] 401 Unauthorized for auth failures
|
|
- [ ] 409 Conflict for hash mismatches
|
|
- [ ] 413 Payload Too Large for size limits
|
|
- [ ] 500 Internal Server Error for system issues
|
|
- [ ] Add detailed error messages
|
|
- [ ] Implement request logging
|
|
|
|
### 2.5 Testing & Validation
|
|
- [ ] Test uploads without authentication
|
|
- [ ] Test uploads with valid nostr auth
|
|
- [ ] Test uploads with invalid auth
|
|
- [ ] Test hash mismatch scenarios
|
|
- [ ] Test file size limits
|
|
- [ ] Verify blob descriptors are correct
|
|
|
|
---
|
|
|
|
## Phase 3: Upload Requirements (BUD-06)
|
|
|
|
### 3.1 Upload Policy Configuration
|
|
- [ ] Add server configuration options
|
|
- [ ] Maximum file size limits
|
|
- [ ] Allowed MIME types
|
|
- [ ] Authentication requirements
|
|
- [ ] Rate limiting settings
|
|
- [ ] Storage quota limits
|
|
|
|
### 3.2 HEAD /upload Endpoint
|
|
- [ ] Implement `HEAD /upload` endpoint
|
|
- [ ] Return upload requirements in headers
|
|
- [ ] Handle optional Authorization header
|
|
- [ ] Return proper status codes for policy checks
|
|
- [ ] Add custom headers for requirements
|
|
|
|
### 3.3 Upload Validation
|
|
- [ ] Implement pre-upload validation
|
|
- [ ] Check file size before processing
|
|
- [ ] Validate MIME types if restricted
|
|
- [ ] Check authentication requirements
|
|
- [ ] Verify user permissions/quotas
|
|
|
|
### 3.4 Testing & Validation
|
|
- [ ] Test upload requirements endpoint
|
|
- [ ] Test policy enforcement
|
|
- [ ] Test with various client scenarios
|
|
- [ ] Verify error responses match spec
|
|
|
|
---
|
|
|
|
## Phase 4: Optional Features
|
|
|
|
### 4.1 User Server Lists (BUD-03) - Optional
|
|
- [ ] Implement server list advertisement
|
|
- [ ] Handle kind:10063 events
|
|
- [ ] Create server discovery endpoint
|
|
- [ ] Test client fallback scenarios
|
|
|
|
### 4.2 Blob Mirroring (BUD-04) - Optional
|
|
- [ ] Implement `PUT /mirror` endpoint
|
|
- [ ] Add URL downloading capability
|
|
- [ ] Implement hash verification
|
|
- [ ] Handle authorization for mirroring
|
|
- [ ] Test inter-server mirroring
|
|
|
|
### 4.3 Media Optimization (BUD-05) - Optional
|
|
- [ ] Implement `PUT /media` endpoint
|
|
- [ ] Add media processing libraries
|
|
- [ ] Implement optimization algorithms
|
|
- [ ] Handle various media formats
|
|
- [ ] Test optimization pipeline
|
|
|
|
### 4.4 Payment Integration (BUD-07) - Optional
|
|
- [ ] Implement 402 Payment Required responses
|
|
- [ ] Add Lightning payment support
|
|
- [ ] Add Cashu payment support
|
|
- [ ] Implement payment verification
|
|
- [ ] Test payment flows
|
|
|
|
### 4.5 NIP-94 Metadata (BUD-08) - Optional
|
|
- [ ] Add NIP-94 tag generation
|
|
- [ ] Extend blob descriptor responses
|
|
- [ ] Generate magnet links if supported
|
|
- [ ] Test metadata compatibility
|
|
|
|
### 4.6 Blob Reporting (BUD-09) - Optional
|
|
- [ ] Implement `PUT /report` endpoint
|
|
- [ ] Handle NIP-56 report events
|
|
- [ ] Add moderation interface
|
|
- [ ] Implement content filtering
|
|
- [ ] Test reporting workflow
|
|
|
|
---
|
|
|
|
## Development Milestones
|
|
|
|
### Milestone 1: Basic Functionality (Phase 1 Complete)
|
|
- [x] nginx serves files by hash with extension support
|
|
- [x] HEAD requests return metadata from database
|
|
- [x] Database stores blob information with proper schema
|
|
|
|
### Milestone 2: Full Upload Support (Phase 2 Complete)
|
|
- Authenticated uploads working
|
|
- Proper error handling
|
|
- Blob descriptors returned correctly
|
|
|
|
### Milestone 3: Policy Compliance (Phase 3 Complete)
|
|
- Upload requirements implemented
|
|
- Server policies configurable
|
|
- Spec compliance verified
|
|
|
|
### Milestone 4: Production Ready (Phase 4 Complete)
|
|
- Optional features implemented as needed
|
|
- Performance optimized
|
|
- Security hardened
|
|
- Documentation complete
|
|
|
|
---
|
|
|
|
## Testing Strategy
|
|
|
|
### Unit Tests
|
|
- [ ] Authentication validation functions
|
|
- [ ] SHA-256 hash calculation
|
|
- [ ] Database operations
|
|
- [ ] Configuration parsing
|
|
|
|
### Integration Tests
|
|
- [ ] nginx + FastCGI integration
|
|
- [ ] End-to-end upload/download flows
|
|
- [ ] Error scenario handling
|
|
- [ ] Multi-client concurrent access
|
|
|
|
### Performance Tests
|
|
- [ ] Large file uploads/downloads
|
|
- [ ] Concurrent request handling
|
|
- [ ] Database query performance
|
|
- [ ] Memory usage optimization
|
|
|
|
### Compliance Tests
|
|
- [ ] Blossom protocol compliance
|
|
- [ ] Nostr event validation
|
|
- [ ] HTTP specification compliance
|
|
- [ ] Security best practices
|
|
|
|
---
|
|
|
|
## Security Considerations
|
|
|
|
- [ ] Input validation on all endpoints
|
|
- [ ] Rate limiting to prevent abuse
|
|
- [ ] Secure file storage permissions
|
|
- [ ] Database injection prevention
|
|
- [ ] Memory safety in C implementation
|
|
- [ ] Proper error message sanitization
|
|
- [ ] Log security (no sensitive data)
|
|
|
|
---
|
|
|
|
## Performance Optimizations
|
|
|
|
- [ ] nginx direct file serving (bypasses application)
|
|
- [ ] FastCGI connection pooling
|
|
- [ ] Database connection management
|
|
- [ ] Efficient hash calculation
|
|
- [ ] Memory-mapped file operations
|
|
- [ ] Hierarchical file storage (future)
|
|
- [ ] CDN integration support
|
|
|
|
---
|
|
|
|
## Deployment Checklist
|
|
|
|
- [ ] nginx configuration template
|
|
- [ ] FastCGI service configuration
|
|
- [ ] Database initialization scripts
|
|
- [ ] Log rotation setup
|
|
- [ ] Monitoring and health checks
|
|
- [ ] Backup procedures
|
|
- [ ] Security hardening guide
|
|
- [ ] Documentation and examples
|