laantungir/ginxsom
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
ec976ab090df7547a25a3e2b1a2d5888136d2e48
ginxsom/IMPLEMENTATION.md
Your Name ec976ab090 Bud 2 mostly done
2025-08-19 10:23:40 -04:00

9.8 KiB
Raw Blame History

Ginxsom Blossom Server Implementation Checklist

This document outlines the implementation plan for ginxsom, a FastCGI-based Blossom server designed to work with nginx for optimal performance.

Architecture Overview

  • nginx: Handles static file serving (GET /) for maximum performance
  • FastCGI Application: Handles authenticated operations, metadata queries, uploads
  • SQLite Database: Stores blob metadata and server configuration
  • File Storage: Flat directory structure initially, hierarchical optimization later

Phase 1: Basic File Serving & Retrieval (BUD-01)

1.1 Infrastructure Setup

  • Create basic directory structure
    • Create blobs/ directory for file storage
    • Create db/ directory for SQLite database
    • Create logs/ directory for application logs
    • Set up proper permissions (nginx readable, app writable)

1.2 Database Schema

  • Design SQLite schema for blob metadata
    • blobs table: sha256, size, type, uploaded_at, uploader_pubkey, filename
    • server_config table: key-value pairs for server settings
    • Create database initialization script
    • Add proper indexes on sha256 hash

1.3 nginx Configuration

  • Configure nginx for static file serving
    • Set up location block for GET /<sha256> pattern with extension support
    • Configure try_files directive for multiple extension fallbacks
    • Configure proper MIME type detection
    • Add proper headers (Cache-Control, ETag, etc.)
    • Handle 404s gracefully when blob doesn't exist
    • Configure FastCGI pass-through for HEAD and non-GET requests

Future Enhancement Note: Consider implementing nginx Lua extension for true Blossom compliance with dynamic file discovery. The current approach uses explicit extension lists in try_files, which works well for common extensions but may not serve files with unusual extensions. Lua module would allow runtime directory scanning for hash-matching files regardless of extension.

1.4 Basic HEAD Endpoint

  • Implement FastCGI handler for HEAD /<sha256>
    • Query database for blob metadata (single source of truth)
    • Extract SHA-256 from URL (strip extensions)
    • Return proper headers (Content-Type, Content-Length, etc.)
    • Return 404 if blob doesn't exist in database
    • Add server timing headers for debugging

1.5 Testing & Validation

  • Create test blobs with known SHA-256 hashes
  • Verify nginx serves files correctly with extension support
  • Verify HEAD requests return proper metadata
  • Test with missing files (404 responses)
  • Test HEAD requests with and without extensions
  • Performance test with large files

Phase 2: Upload & Authentication (BUD-02)

2.1 Nostr Authentication Setup

  • Integrate nostr_core_lib submodule

    • Update Makefile to include nostr_core_lib paths and static library
    • Build libnostr_core_x64.a using provided build.sh script
    • Add system dependencies: -lsecp256k1 -lssl -lcrypto -lcurl -lz -ldl -lpthread -lm

  • Implement authentication functions in main.c (BUD-02 section):

    • parse_authorization_header() - Extract JSON from "Nostr base64(event)" header
    • validate_blossom_event() - Validate Blossom-specific requirements (kind 24242, content hash, method, expiration)
    • authenticate_request() - Main orchestrator function

  • Leverage existing nostr_core_lib functions:

    • Use nostr_validate_event() for structure + signature validation (from nip001.h)
    • Use standardized error codes from nostr_common.h (NOSTR_SUCCESS, NOSTR_ERROR_EVENT_INVALID_SIGNATURE, etc.)
    • Use nostr_strerror() for error message translation

2.2 Upload Endpoint Implementation

  • Implement PUT /upload endpoint
    • Parse Authorization header (Nostr base64 event extraction)
    • Stream file upload to temporary location
    • Calculate SHA-256 hash during upload
    • Validate hash matches authorization if provided
    • Move file to permanent location
    • Store metadata in database (including uploader_pubkey and filename)
    • Return blob descriptor JSON response

2.3 Blob Descriptor Response

  • Implement blob descriptor structure
    • Required fields: url, sha256, size, type, uploaded
    • Handle MIME type detection
    • Generate proper blob URLs
    • Add optional server-specific fields (uploader_pubkey, filename)

2.4 Error Handling

  • Implement proper HTTP status codes
    • 400 Bad Request for invalid data
    • 401 Unauthorized for auth failures
    • 409 Conflict for hash mismatches
    • 413 Payload Too Large for size limits
    • 500 Internal Server Error for system issues
  • Add detailed error messages
  • Implement request logging

2.5 Testing & Validation

  • Test uploads without authentication
  • Test uploads with valid nostr auth
  • Test uploads with invalid auth
  • Test hash mismatch scenarios
  • Test file size limits
  • Verify blob descriptors are correct
  • Verify database metadata storage (uploader_pubkey and filename)

Phase 3: Upload Requirements (BUD-06)

3.1 Upload Policy Configuration

  • Add server configuration options
    • Maximum file size limits
    • Allowed MIME types
    • Authentication requirements
    • Rate limiting settings
    • Storage quota limits

3.2 HEAD /upload Endpoint

  • Implement HEAD /upload endpoint
    • Return upload requirements in headers
    • Handle optional Authorization header
    • Return proper status codes for policy checks
    • Add custom headers for requirements

3.3 Upload Validation

  • Implement pre-upload validation
    • Check file size before processing
    • Validate MIME types if restricted
    • Check authentication requirements
    • Verify user permissions/quotas

3.4 Testing & Validation

  • Test upload requirements endpoint
  • Test policy enforcement
  • Test with various client scenarios
  • Verify error responses match spec

Phase 4: Optional Features

4.1 User Server Lists (BUD-03) - Optional

  • Implement server list advertisement
  • Handle kind:10063 events
  • Create server discovery endpoint
  • Test client fallback scenarios

4.2 Blob Mirroring (BUD-04) - Optional

  • Implement PUT /mirror endpoint
  • Add URL downloading capability
  • Implement hash verification
  • Handle authorization for mirroring
  • Test inter-server mirroring

4.3 Media Optimization (BUD-05) - Optional

  • Implement PUT /media endpoint
  • Add media processing libraries
  • Implement optimization algorithms
  • Handle various media formats
  • Test optimization pipeline

4.4 Payment Integration (BUD-07) - Optional

  • Implement 402 Payment Required responses
  • Add Lightning payment support
  • Add Cashu payment support
  • Implement payment verification
  • Test payment flows

4.5 NIP-94 Metadata (BUD-08) - Optional

  • Add NIP-94 tag generation
  • Extend blob descriptor responses
  • Generate magnet links if supported
  • Test metadata compatibility

4.6 Blob Reporting (BUD-09) - Optional

  • Implement PUT /report endpoint
  • Handle NIP-56 report events
  • Add moderation interface
  • Implement content filtering
  • Test reporting workflow

Development Milestones

Milestone 1: Basic Functionality (Phase 1 Complete)

  • nginx serves files by hash with extension support
  • HEAD requests return metadata from database
  • Database stores blob information with proper schema

Milestone 2: Full Upload Support (Phase 2 Complete)

  • Basic upload functionality working (PUT requests accepted)
  • SHA-256 hash calculation during upload
  • File storage to blobs/ directory
  • Blob descriptor JSON response
  • Authenticated uploads working (Nostr kind 24242 event validation)
  • Proper error handling for upload scenarios
  • Database metadata storage during upload (with uploader_pubkey and filename)

Milestone 3: Policy Compliance (Phase 3 Pending)

  • Upload requirements implemented
  • Server policies configurable
  • Spec compliance verified

Milestone 4: Production Ready (Phase 4 Complete)

  • Optional features implemented as needed
  • Performance optimized
  • Security hardened
  • Documentation complete

Testing Strategy

Unit Tests

  • Authentication validation functions
  • SHA-256 hash calculation
  • Database operations
  • Configuration parsing

Integration Tests

  • nginx + FastCGI integration
  • End-to-end upload/download flows
  • Error scenario handling
  • Multi-client concurrent access

Performance Tests

  • Large file uploads/downloads
  • Concurrent request handling
  • Database query performance
  • Memory usage optimization

Compliance Tests

  • Blossom protocol compliance
  • Nostr event validation
  • HTTP specification compliance
  • Security best practices

Security Considerations

  • Input validation on all endpoints
  • Rate limiting to prevent abuse
  • Secure file storage permissions
  • Database injection prevention
  • Memory safety in C implementation
  • Proper error message sanitization
  • Log security (no sensitive data)

Performance Optimizations

  • nginx direct file serving (bypasses application)
  • FastCGI connection pooling
  • Database connection management
  • Efficient hash calculation
  • Memory-mapped file operations
  • Hierarchical file storage (future)
  • CDN integration support

Deployment Checklist

  • nginx configuration template
  • FastCGI service configuration
  • Database initialization scripts
  • Log rotation setup
  • Monitoring and health checks
  • Backup procedures
  • Security hardening guide
  • Documentation and examples