readme.md and implementation.md
This commit is contained in:
Your Name
265
IMPLEMENTATION.md
Normal file
265
IMPLEMENTATION.md
Normal file
@@ -0,0 +1,265 @@
|
||||
# Ginxsom Blossom Server Implementation Checklist
|
||||
|
||||
This document outlines the implementation plan for ginxsom, a FastCGI-based Blossom server designed to work with nginx for optimal performance.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
- **nginx**: Handles static file serving (GET /<sha256>) for maximum performance
|
||||
- **FastCGI Application**: Handles authenticated operations, metadata queries, uploads
|
||||
- **SQLite Database**: Stores blob metadata and server configuration
|
||||
- **File Storage**: Flat directory structure initially, hierarchical optimization later
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Basic File Serving & Retrieval (BUD-01)
|
||||
|
||||
### 1.1 Infrastructure Setup
|
||||
- [ ] Create basic directory structure
|
||||
- [ ] Create `blobs/` directory for file storage
|
||||
- [ ] Create `db/` directory for SQLite database
|
||||
- [ ] Create `logs/` directory for application logs
|
||||
- [ ] Set up proper permissions (nginx readable, app writable)
|
||||
|
||||
### 1.2 Database Schema
|
||||
- [ ] Design SQLite schema for blob metadata
|
||||
- [ ] `blobs` table: sha256, size, type, uploaded_at, uploader_pubkey
|
||||
- [ ] `server_config` table: key-value pairs for server settings
|
||||
- [ ] Create database initialization script
|
||||
- [ ] Add proper indexes on sha256 hash
|
||||
|
||||
### 1.3 nginx Configuration
|
||||
- [ ] Configure nginx for static file serving
|
||||
- [ ] Set up location block for `GET /<sha256>` pattern
|
||||
- [ ] Configure proper MIME type detection
|
||||
- [ ] Add proper headers (Cache-Control, ETag, etc.)
|
||||
- [ ] Handle 404s gracefully when blob doesn't exist
|
||||
- [ ] Configure FastCGI pass-through for non-GET requests
|
||||
|
||||
### 1.4 Basic HEAD Endpoint
|
||||
- [ ] Implement FastCGI handler for `HEAD /<sha256>`
|
||||
- [ ] Query database for blob metadata
|
||||
- [ ] Return proper headers (Content-Type, Content-Length, etc.)
|
||||
- [ ] Return 404 if blob doesn't exist
|
||||
- [ ] Add server timing headers for debugging
|
||||
|
||||
### 1.5 Testing & Validation
|
||||
- [ ] Create test blobs with known SHA-256 hashes
|
||||
- [ ] Verify nginx serves files correctly
|
||||
- [ ] Verify HEAD requests return proper metadata
|
||||
- [ ] Test with missing files (404 responses)
|
||||
- [ ] Performance test with large files
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Upload & Authentication (BUD-02)
|
||||
|
||||
### 2.1 Nostr Authentication Setup
|
||||
- [ ] Integrate nostr_core_lib submodule
|
||||
- [ ] Implement nostr event validation
|
||||
- [ ] Verify event signature (schnorr)
|
||||
- [ ] Validate event structure (kind 24242)
|
||||
- [ ] Check required fields (t, expiration, x tags)
|
||||
- [ ] Implement expiration checking
|
||||
- [ ] Create authentication middleware
|
||||
|
||||
### 2.2 Upload Endpoint Implementation
|
||||
- [ ] Implement `PUT /upload` endpoint
|
||||
- [ ] Parse Authorization header (optional but recommended)
|
||||
- [ ] Stream file upload to temporary location
|
||||
- [ ] Calculate SHA-256 hash during upload
|
||||
- [ ] Validate hash matches authorization if provided
|
||||
- [ ] Move file to permanent location
|
||||
- [ ] Store metadata in database
|
||||
- [ ] Return blob descriptor JSON response
|
||||
|
||||
### 2.3 Blob Descriptor Response
|
||||
- [ ] Implement blob descriptor structure
|
||||
- [ ] Required fields: url, sha256, size, type, uploaded
|
||||
- [ ] Handle MIME type detection
|
||||
- [ ] Generate proper blob URLs
|
||||
- [ ] Add optional server-specific fields
|
||||
|
||||
### 2.4 Error Handling
|
||||
- [ ] Implement proper HTTP status codes
|
||||
- [ ] 400 Bad Request for invalid data
|
||||
- [ ] 401 Unauthorized for auth failures
|
||||
- [ ] 409 Conflict for hash mismatches
|
||||
- [ ] 413 Payload Too Large for size limits
|
||||
- [ ] 500 Internal Server Error for system issues
|
||||
- [ ] Add detailed error messages
|
||||
- [ ] Implement request logging
|
||||
|
||||
### 2.5 Testing & Validation
|
||||
- [ ] Test uploads without authentication
|
||||
- [ ] Test uploads with valid nostr auth
|
||||
- [ ] Test uploads with invalid auth
|
||||
- [ ] Test hash mismatch scenarios
|
||||
- [ ] Test file size limits
|
||||
- [ ] Verify blob descriptors are correct
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Upload Requirements (BUD-06)
|
||||
|
||||
### 3.1 Upload Policy Configuration
|
||||
- [ ] Add server configuration options
|
||||
- [ ] Maximum file size limits
|
||||
- [ ] Allowed MIME types
|
||||
- [ ] Authentication requirements
|
||||
- [ ] Rate limiting settings
|
||||
- [ ] Storage quota limits
|
||||
|
||||
### 3.2 HEAD /upload Endpoint
|
||||
- [ ] Implement `HEAD /upload` endpoint
|
||||
- [ ] Return upload requirements in headers
|
||||
- [ ] Handle optional Authorization header
|
||||
- [ ] Return proper status codes for policy checks
|
||||
- [ ] Add custom headers for requirements
|
||||
|
||||
### 3.3 Upload Validation
|
||||
- [ ] Implement pre-upload validation
|
||||
- [ ] Check file size before processing
|
||||
- [ ] Validate MIME types if restricted
|
||||
- [ ] Check authentication requirements
|
||||
- [ ] Verify user permissions/quotas
|
||||
|
||||
### 3.4 Testing & Validation
|
||||
- [ ] Test upload requirements endpoint
|
||||
- [ ] Test policy enforcement
|
||||
- [ ] Test with various client scenarios
|
||||
- [ ] Verify error responses match spec
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Optional Features
|
||||
|
||||
### 4.1 User Server Lists (BUD-03) - Optional
|
||||
- [ ] Implement server list advertisement
|
||||
- [ ] Handle kind:10063 events
|
||||
- [ ] Create server discovery endpoint
|
||||
- [ ] Test client fallback scenarios
|
||||
|
||||
### 4.2 Blob Mirroring (BUD-04) - Optional
|
||||
- [ ] Implement `PUT /mirror` endpoint
|
||||
- [ ] Add URL downloading capability
|
||||
- [ ] Implement hash verification
|
||||
- [ ] Handle authorization for mirroring
|
||||
- [ ] Test inter-server mirroring
|
||||
|
||||
### 4.3 Media Optimization (BUD-05) - Optional
|
||||
- [ ] Implement `PUT /media` endpoint
|
||||
- [ ] Add media processing libraries
|
||||
- [ ] Implement optimization algorithms
|
||||
- [ ] Handle various media formats
|
||||
- [ ] Test optimization pipeline
|
||||
|
||||
### 4.4 Payment Integration (BUD-07) - Optional
|
||||
- [ ] Implement 402 Payment Required responses
|
||||
- [ ] Add Lightning payment support
|
||||
- [ ] Add Cashu payment support
|
||||
- [ ] Implement payment verification
|
||||
- [ ] Test payment flows
|
||||
|
||||
### 4.5 NIP-94 Metadata (BUD-08) - Optional
|
||||
- [ ] Add NIP-94 tag generation
|
||||
- [ ] Extend blob descriptor responses
|
||||
- [ ] Generate magnet links if supported
|
||||
- [ ] Test metadata compatibility
|
||||
|
||||
### 4.6 Blob Reporting (BUD-09) - Optional
|
||||
- [ ] Implement `PUT /report` endpoint
|
||||
- [ ] Handle NIP-56 report events
|
||||
- [ ] Add moderation interface
|
||||
- [ ] Implement content filtering
|
||||
- [ ] Test reporting workflow
|
||||
|
||||
---
|
||||
|
||||
## Development Milestones
|
||||
|
||||
### Milestone 1: Basic Functionality ✓ (Phase 1 Complete)
|
||||
- nginx serves files by hash
|
||||
- HEAD requests return metadata
|
||||
- Database stores blob information
|
||||
|
||||
### Milestone 2: Full Upload Support (Phase 2 Complete)
|
||||
- Authenticated uploads working
|
||||
- Proper error handling
|
||||
- Blob descriptors returned correctly
|
||||
|
||||
### Milestone 3: Policy Compliance (Phase 3 Complete)
|
||||
- Upload requirements implemented
|
||||
- Server policies configurable
|
||||
- Spec compliance verified
|
||||
|
||||
### Milestone 4: Production Ready (Phase 4 Complete)
|
||||
- Optional features implemented as needed
|
||||
- Performance optimized
|
||||
- Security hardened
|
||||
- Documentation complete
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
- [ ] Authentication validation functions
|
||||
- [ ] SHA-256 hash calculation
|
||||
- [ ] Database operations
|
||||
- [ ] Configuration parsing
|
||||
|
||||
### Integration Tests
|
||||
- [ ] nginx + FastCGI integration
|
||||
- [ ] End-to-end upload/download flows
|
||||
- [ ] Error scenario handling
|
||||
- [ ] Multi-client concurrent access
|
||||
|
||||
### Performance Tests
|
||||
- [ ] Large file uploads/downloads
|
||||
- [ ] Concurrent request handling
|
||||
- [ ] Database query performance
|
||||
- [ ] Memory usage optimization
|
||||
|
||||
### Compliance Tests
|
||||
- [ ] Blossom protocol compliance
|
||||
- [ ] Nostr event validation
|
||||
- [ ] HTTP specification compliance
|
||||
- [ ] Security best practices
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- [ ] Input validation on all endpoints
|
||||
- [ ] Rate limiting to prevent abuse
|
||||
- [ ] Secure file storage permissions
|
||||
- [ ] Database injection prevention
|
||||
- [ ] Memory safety in C implementation
|
||||
- [ ] Proper error message sanitization
|
||||
- [ ] Log security (no sensitive data)
|
||||
|
||||
---
|
||||
|
||||
## Performance Optimizations
|
||||
|
||||
- [ ] nginx direct file serving (bypasses application)
|
||||
- [ ] FastCGI connection pooling
|
||||
- [ ] Database connection management
|
||||
- [ ] Efficient hash calculation
|
||||
- [ ] Memory-mapped file operations
|
||||
- [ ] Hierarchical file storage (future)
|
||||
- [ ] CDN integration support
|
||||
|
||||
---
|
||||
|
||||
## Deployment Checklist
|
||||
|
||||
- [ ] nginx configuration template
|
||||
- [ ] FastCGI service configuration
|
||||
- [ ] Database initialization scripts
|
||||
- [ ] Log rotation setup
|
||||
- [ ] Monitoring and health checks
|
||||
- [ ] Backup procedures
|
||||
- [ ] Security hardening guide
|
||||
- [ ] Documentation and examples
|
||||
|
||||
Reference in New Issue
Block a user