readme.md and implementation.md

2025-08-18 17:42:51 -04:00
parent 5f7d2197e8
commit e641c813eb
8 changed files with 1505 additions and 1 deletions
--- a/README.md
+++ b/README.md
@@ -1 +1,315 @@
-Ginxsom
+# ginxsom 🌸
+
+A high-performance [Blossom](https://github.com/hzrd149/blossom) server implementation in C, designed for optimal integration with nginx for blazing-fast file serving.
+
+## Overview
+
+ginxsom is a Blossom protocol server implemented as a FastCGI application that integrates seamlessly with nginx. nginx handles static file serving directly while ginxsom processes authenticated operations (uploads, deletes, management) via FastCGI. This architecture provides optimal performance with nginx's excellent static file serving and C's efficiency for cryptographic operations.
+
+### Why ginxsom?
+
+- **Performance**: C application with nginx static serving = maximum throughput
+- **Simplicity**: Clean separation between static serving (nginx) and dynamic operations (C app)
+- **Scalability**: nginx handles the heavy file serving load, C app handles auth and metadata
+- **Standards Compliant**: Full Blossom protocol implementation with nostr authentication
+
+## Architecture
+
+```
+┌─────────────┐    ┌──────────────┐    ┌─────────────────┐
+│   Client    │    │    nginx     │    │ ginxsom FastCGI │
+│             │───▶│              │───▶│                 │
+│             │    │              │    │                 │
+└─────────────┘    └──────────────┘    └─────────────────┘
+                          │                      │
+                          ▼                      ▼
+                   ┌─────────────┐    ┌─────────────────┐
+                   │    Blobs    │    │   SQLite DB     │
+                   │ (flat files)│    │  (metadata)     │
+                   └─────────────┘    └─────────────────┘
+```
+
+### Request Flow
+
+1. **File Retrieval (`GET /<sha256>`)**: nginx serves directly from disk
+2. **File Upload (`PUT /upload`)**: nginx calls ginxsom via FastCGI for authentication + storage
+3. **File Management (`DELETE`, `LIST`)**: nginx calls ginxsom via FastCGI
+4. **Metadata Operations**: ginxsom manages SQLite database
+
+## Blossom Protocol Support
+
+ginxsom implements the following Blossom Upgrade Documents (BUDs):
+
+- **BUD-01**: Server requirements and blob retrieval ✅
+- **BUD-02**: Blob upload and management ✅  
+- **BUD-06**: Upload requirements ✅
+
+### Supported Endpoints
+
+| Endpoint | Method | Description | Handler |
+|----------|---------|-------------|---------|
+| `/<sha256>` | GET | Retrieve blob | nginx → disk |
+| `/<sha256>` | HEAD | Check blob exists | nginx → disk |
+| `/upload` | PUT | Upload new blob | nginx → FastCGI ginxsom |
+| `/upload` | HEAD | Check upload requirements | nginx → FastCGI ginxsom |
+| `/list/<pubkey>` | GET | List user's blobs | nginx → FastCGI ginxsom |
+| `/<sha256>` | DELETE | Delete blob | nginx → FastCGI ginxsom |
+
+## Installation
+
+### Prerequisites
+
+```bash
+# Ubuntu/Debian
+sudo apt-get update
+sudo apt-get install build-essential libssl-dev libsqlite3-dev libfcgi-dev pkg-config
+
+# RHEL/CentOS/Fedora  
+sudo dnf install gcc make openssl-devel sqlite-devel fcgi-devel pkgconfig
+
+# macOS
+brew install openssl sqlite fcgi pkg-config
+```
+
+### Building ginxsom
+
+```bash
+git clone https://github.com/yourusername/ginxsom.git
+cd ginxsom
+make
+```
+
+### Installation
+
+```bash
+sudo make install
+# Installs to /usr/local/bin/ginxsom
+```
+
+## Configuration
+
+### ginxsom Configuration
+
+Create `/etc/ginxsom/config.conf`:
+
+```ini
+[server]
+socket_path = /run/ginxsom.sock
+max_file_size = 104857600  # 100MB
+storage_path = /var/lib/ginxsom/blobs
+database_path = /var/lib/ginxsom/blobs.db
+
+[auth]
+require_auth_upload = true
+require_auth_get = false
+require_auth_list = false
+
+[limits]
+max_blobs_per_user = 1000
+rate_limit_uploads = 10  # per minute
+```
+
+### nginx Configuration
+
+Add to your nginx configuration:
+
+```nginx
+server {
+    listen 80;
+    server_name your-blossom-server.com;
+    
+    # CORS headers for all responses
+    add_header Access-Control-Allow-Origin *;
+    add_header Access-Control-Allow-Methods "GET, HEAD, PUT, DELETE, OPTIONS";
+    add_header Access-Control-Allow-Headers "Authorization, Content-Type, Content-Length, X-SHA-256, X-Content-Type, X-Content-Length";
+    add_header Access-Control-Max-Age 86400;
+    
+    # Handle preflight OPTIONS requests
+    if ($request_method = 'OPTIONS') {
+        return 204;
+    }
+    
+    # Static blob serving - nginx handles directly
+    location ~ ^/([a-f0-9]{64})(\.[a-zA-Z0-9]+)?$ {
+        root /var/lib/ginxsom/blobs;
+        try_files /$1$2 =404;
+        add_header Accept-Ranges bytes;
+        
+        # Set content type based on file extension if not detected
+        location ~* \.(pdf)$ { add_header Content-Type application/pdf; }
+        location ~* \.(jpg|jpeg)$ { add_header Content-Type image/jpeg; }
+        location ~* \.(png)$ { add_header Content-Type image/png; }
+        location ~* \.(mp4)$ { add_header Content-Type video/mp4; }
+    }
+    
+    # All other requests go to ginxsom FastCGI
+    location / {
+        include fastcgi_params;
+        fastcgi_pass unix:/run/ginxsom.sock;
+        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
+        
+        # Handle large uploads
+        client_max_body_size 100M;
+        fastcgi_request_buffering off;
+    }
+}
+```
+
+## Usage
+
+### Starting the Server
+
+```bash
+# Start ginxsom FastCGI daemon
+sudo spawn-fcgi -s /run/ginxsom.sock -f /usr/local/bin/ginxsom -u www-data -g www-data
+
+# Or with systemd
+sudo systemctl enable ginxsom
+sudo systemctl start ginxsom
+```
+
+### Testing
+
+```bash
+# Check server is running
+curl -I http://localhost/upload
+
+# Upload a file (requires nostr authorization)
+curl -X PUT http://localhost/upload \
+  -H "Authorization: Nostr eyJ..." \
+  -H "Content-Type: application/pdf" \
+  --data-binary @document.pdf
+
+# Retrieve a file
+curl http://localhost/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf
+```
+
+## API Documentation
+
+### Authentication
+
+All write operations require nostr event authentication via the `Authorization` header:
+
+```
+Authorization: Nostr <base64-encoded-nostr-event>
+```
+
+The nostr event must be kind `24242` with appropriate tags:
+- `t` tag: `upload`, `delete`, `list`, or `get`  
+- `x` tag: SHA-256 hash for blob-specific operations
+- `expiration` tag: Unix timestamp when event expires
+
+### Blob Descriptors
+
+Successful uploads return blob descriptors:
+
+```json
+{
+  "url": "https://cdn.example.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf",
+  "sha256": "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553",
+  "size": 184292,
+  "type": "application/pdf",
+  "uploaded": 1725105921
+}
+```
+
+## File Storage
+
+### Current (Flat) Structure
+```
+/var/lib/ginxsom/blobs/
+├── b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf
+├── a8472f6d93e42c1e5b4e9f3a7b2c8d4e6f9a1b3c5d7e8f0a1b2c3d4e5f6a7b8.png
+└── ...
+```
+
+### Future (Optimized) Structure
+```
+/var/lib/ginxsom/blobs/
+├── b1/
+│   └── 67/
+│       └── b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf
+├── a8/
+│   └── 47/
+│       └── a8472f6d93e42c1e5b4e9f3a7b2c8d4e6f9a1b3c5d7e8f0a1b2c3d4e5f6a7b8.png
+```
+
+## Development
+
+### Project Structure
+
+```
+ginxsom/
+├── src/
+│   ├── main.c              # Main server loop
+│   ├── config.c            # Configuration parsing
+│   ├── server.c            # HTTP server logic
+│   ├── auth.c              # Nostr authentication
+│   ├── storage.c           # File storage operations
+│   ├── database.c          # SQLite operations
+│   └── utils.c             # Utility functions
+├── include/
+│   └── ginxsom.h           # Main header file
+├── tests/
+│   └── test_*.c            # Unit tests
+├── docs/
+│   └── api.md              # API documentation
+├── Makefile
+└── README.md
+```
+
+### Building for Development
+
+```bash
+make debug    # Build with debug symbols
+make test     # Run test suite
+make clean    # Clean build artifacts
+```
+
+### Dependencies
+
+- **libssl**: For cryptographic operations (nostr event verification)
+- **libsqlite3**: For metadata storage
+- **libfcgi**: For FastCGI functionality
+
+## Performance Characteristics
+
+- **Static File Serving**: nginx performance (typically >10k req/s)
+- **Upload Processing**: Limited by disk I/O and crypto verification
+- **Memory Usage**: Minimal - FastCGI processes are lightweight
+- **Concurrent Operations**: nginx manages FastCGI process pool efficiently
+- **Process Management**: nginx spawns/manages ginxsom FastCGI processes
+
+## Security Considerations
+
+- All uploads require valid nostr event signatures
+- SHA-256 verification prevents data corruption
+- Rate limiting prevents abuse
+- File size limits prevent disk exhaustion
+- No script execution - blobs stored as static files
+
+## Monitoring
+
+ginxsom provides metrics via `/metrics` endpoint:
+
+```
+# HELP ginxsom_uploads_total Total number of uploads processed
+# TYPE ginxsom_uploads_total counter
+ginxsom_uploads_total 1234
+
+# HELP ginxsom_storage_bytes Total bytes stored
+# TYPE ginxsom_storage_bytes gauge  
+ginxsom_storage_bytes 1073741824
+```
+
+## License
+
+[License information here]
+
+## Contributing
+
+[Contributing guidelines here]
+
+---
+
+**Note**: This project is under active development. Please report bugs and feature requests via GitHub issues.