laantungir/ginxsom
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Stuck on a bug with auth, but got to push anyway.

Browse Source
This commit is contained in:
Your Name
2025-08-20 06:20:32 -04:00
parent b2b1240136
commit 8c3d2b1aac
18 changed files with 10443 additions and 151 deletions
Download Patch File Download Diff File Expand all files Collapse all files

289
IMPLEMENTATION.md
View File

@@ -102,31 +102,31 @@ This document outlines the implementation plan for ginxsom, a FastCGI-based Blos
- [x] Implement request logging
### 2.5 List Blobs Endpoint
- [ ] Implement `GET /list/<pubkey>` endpoint
- [ ] Extract pubkey from URL path
- [ ] Query database for blobs uploaded by specified pubkey
- [ ] Support `since` and `until` query parameters for date filtering
- [ ] Return JSON array of blob descriptors
- [ ] Handle empty results gracefully
- [ ] Implement optional authorization with kind 24242 event validation
- [ ] Validate `t` tag is set to "list"
- [ ] Check authorization expiration
- [ ] Verify event signature and structure
- [x] Implement `GET /list/<pubkey>` endpoint
- [x] Extract pubkey from URL path
- [x] Query database for blobs uploaded by specified pubkey
- [x] Support `since` and `until` query parameters for date filtering
- [x] Return JSON array of blob descriptors
- [x] Handle empty results gracefully
- [x] Implement optional authorization with kind 24242 event validation
- [x] Validate `t` tag is set to "list"
- [x] Check authorization expiration
- [x] Verify event signature and structure
### 2.6 Delete Blob Endpoint
- [ ] Implement `DELETE /<sha256>` endpoint
- [ ] Extract SHA-256 hash from URL path
- [ ] Require authorization with kind 24242 event validation
- [ ] Validate `t` tag is set to "delete"
- [ ] Verify at least one `x` tag matches the requested hash
- [ ] Check authorization expiration
- [ ] Verify event signature and structure
- [ ] Check blob exists in database
- [ ] Verify uploader_pubkey matches authorized pubkey (ownership check)
- [ ] Remove blob file from filesystem
- [ ] Remove blob metadata from database
- [ ] Handle file deletion errors gracefully
- [ ] Return appropriate success/error responses
- [x] Implement `DELETE /<sha256>` endpoint
- [x] Extract SHA-256 hash from URL path
- [x] Require authorization with kind 24242 event validation
- [x] Validate `t` tag is set to "delete"
- [x] Verify at least one `x` tag matches the requested hash
- [x] Check authorization expiration
- [x] Verify event signature and structure
- [x] Check blob exists in database
- [x] Verify uploader_pubkey matches authorized pubkey (ownership check)
- [x] Remove blob file from filesystem
- [x] Remove blob metadata from database
- [x] Handle file deletion errors gracefully
- [x] Return appropriate success/error responses
### 2.7 Testing & Validation
- [x] Test uploads without authentication
@@ -148,30 +148,179 @@ This document outlines the implementation plan for ginxsom, a FastCGI-based Blos
- [ ] Authentication requirements
- [ ] Rate limiting settings
- [ ] Storage quota limits
- [ ] Hash-based banning/filtering
### 3.2 HEAD /upload Endpoint
- [ ] Implement `HEAD /upload` endpoint
- [ ] Return upload requirements in headers
- [ ] Handle optional Authorization header
- [ ] Return proper status codes for policy checks
- [ ] Add custom headers for requirements
### 3.2 HEAD /upload Endpoint Implementation
- [ ] Implement `HEAD /upload` endpoint for pre-flight upload validation
- [ ] Parse client headers:
- [ ] `X-SHA-256`: blob's SHA-256 hash
- [ ] `X-Content-Length`: blob size in bytes
- [ ] `X-Content-Type`: blob's MIME type
- [ ] Handle optional Authorization header (same as PUT /upload)
- [ ] Perform validation checks without file transfer:
- [ ] Validate SHA-256 format
- [ ] Check file size against limits
- [ ] Validate MIME type restrictions
- [ ] Check authentication if required
- [ ] Check if hash already exists (duplicate detection)
- [ ] Check if hash is banned
- [ ] Return appropriate HTTP status codes:
- [ ] `200 OK` - upload can proceed
- [ ] `400 Bad Request` - invalid headers
- [ ] `401 Unauthorized` - auth required
- [ ] `403 Forbidden` - not permitted (banned hash, etc.)
- [ ] `411 Length Required` - missing content length
- [ ] `413 Content Too Large` - file too large
- [ ] `415 Unsupported Media Type` - invalid MIME type
- [ ] Add `X-Reason` header with human-readable error messages
### 3.3 Upload Validation
- [ ] Implement pre-upload validation
- [ ] Check file size before processing
- [ ] Validate MIME types if restricted
- [ ] Check authentication requirements
- [ ] Verify user permissions/quotas
### 3.3 Upload Pre-validation Logic
- [ ] Create validation functions that can be shared between HEAD and PUT endpoints
- [ ] `validate_upload_headers()` - check required headers present and valid
- [ ] `check_file_size_limits()` - enforce maximum size restrictions
- [ ] `check_mime_type_allowed()` - validate against allowed types list
- [ ] `check_hash_restrictions()` - check banned hashes, duplicates
- [ ] `check_upload_permissions()` - user-specific upload rights
### 3.4 Testing & Validation
- [ ] Test upload requirements endpoint
- [ ] Test policy enforcement
- [ ] Test with various client scenarios
- [ ] Verify error responses match spec
### 3.4 DOS Protection Benefits
- [ ] Implement early rejection before file transfer:
- [ ] Authentication happens before any file data sent
- [ ] Size validation prevents large file uploads that would be rejected
- [ ] MIME type checking prevents unwanted file types
- [ ] Hash checking prevents duplicate uploads
- [ ] Update PUT /upload to use same validation functions for consistency
### 3.5 Client Integration Support
- [ ] Update nginx configuration to properly handle HEAD requests to /upload
- [ ] Ensure FastCGI handles HEAD method for /upload endpoint
- [ ] Add CORS headers for preflight requests
### 3.6 Testing & Validation
- [ ] Test HEAD /upload with valid headers
- [ ] Test various error scenarios (missing headers, invalid formats)
- [ ] Test authorization requirements
- [ ] Test policy enforcement (size limits, MIME types, banned hashes)
- [ ] Verify error responses match BUD-06 specification
- [ ] Test client workflow: HEAD check → PUT upload
- [ ] Verify DOS protection effectiveness
---
## Phase 4: Optional Features
## Phase 4: Advanced Authentication & Administration System
### 4.1 Flexible Authentication Rules System
#### 4.1.1 Database Schema Extension
- [ ] Create authentication rules tables
- [ ] `auth_rules` table: rule_type, rule_target, operation, rule_value, enabled, expires_at
- [ ] `auth_cache` table: performance caching for rule evaluation results
- [ ] Add indexes on rule_type, rule_target, operation for performance
#### 4.1.2 Authentication Rule Types Implementation
- [ ] Basic rule types:
- [ ] `pubkey_whitelist`: Only specific pubkeys allowed
- [ ] `pubkey_blacklist`: Specific pubkeys banned
- [ ] `hash_blacklist`: Specific file hashes cannot be uploaded
- [ ] `mime_type_whitelist`: Only specific content types allowed
- [ ] `mime_type_blacklist`: Specific content types banned
- [ ] Advanced rule types:
- [ ] `rate_limit`: Limit operations per pubkey/IP per time period
- [ ] `size_limit`: Per-pubkey or global size limits
- [ ] `conditional`: Complex JSON-based rules (time-based, size-based, etc.)
#### 4.1.3 Rule Evaluation Engine
- [ ] Core authentication functions:
- [ ] `evaluate_auth_rules()`: Main rule evaluation with caching
- [ ] `check_rule_cache()`: Performance optimization layer
- [ ] `process_rule_priority()`: Handle rule precedence and conflicts
- [ ] `update_auth_cache()`: Store evaluation results for reuse
- [ ] Integration points:
- [ ] Extend `handle_upload_request()` with rule evaluation
- [ ] Extend `handle_delete_request()` with rule evaluation
- [ ] Extend `handle_list_request()` with rule evaluation (optional)
#### 4.1.4 Rule Management Interface
- [ ] SQL-based rule management:
- [ ] `add_auth_rule()`: Add new authentication rules
- [ ] `remove_auth_rule()`: Remove rules by ID
- [ ] `list_auth_rules()`: Query existing rules with filters
- [ ] `update_auth_rule()`: Modify existing rule parameters
### 4.2 Nostr-Native Administrative Interface
#### 4.2.1 Server Identity Management
- [ ] Server keypair generation and storage:
- [ ] Generate server public/private keypair on first run
- [ ] Store server pubkey in `server_config` table
- [ ] Secure private key storage (encrypted file or environment)
- [ ] Key rotation capabilities for security
#### 4.2.2 Administrator Management System
- [ ] Administrator database schema:
- [ ] `administrators` table: pubkey, permissions, added_by, expires_at
- [ ] Permission levels: rules, config, users, stats, * (full access)
- [ ] Initial admin setup during server deployment
- [ ] Administrative functions:
- [ ] `check_admin_permissions()`: Verify admin authorization
- [ ] `add_administrator()`: Grant admin privileges
- [ ] `remove_administrator()`: Revoke admin privileges
- [ ] `list_administrators()`: Query admin list with permissions
#### 4.2.3 Administrative Event Types
- [ ] Event kind definitions:
- [ ] Kind 30242: Administrative commands (rule_add, rule_remove, config_set, etc.)
- [ ] Kind 30243: Administrative queries (stats_get, rule_list, audit_log, etc.)
- [ ] Kind 30244: Administrative responses (command results, query data)
- [ ] Command implementations:
- [ ] Rule management: `rule_add`, `rule_remove`, `rule_update`, `rule_list`
- [ ] System management: `config_set`, `config_get`, `admin_add`, `admin_remove`
- [ ] Query operations: `stats_get`, `blob_list`, `audit_log`, `storage_cleanup`
#### 4.2.4 Administrative Event Processing
- [ ] HTTP administrative endpoint:
- [ ] `POST /admin` with nostr event authorization
- [ ] JSON command interface with parameter validation
- [ ] Synchronous response with operation results
- [ ] Direct nostr relay integration (future enhancement):
- [ ] Subscribe to administrative events on configured relays
- [ ] Real-time event processing and response
- [ ] Publish response events back to relays
#### 4.2.5 Administrative Audit Trail
- [ ] Administrative logging system:
- [ ] `admin_log` table: track all administrative actions
- [ ] Event ID references for nostr event traceability
- [ ] Success/failure tracking with detailed error messages
- [ ] Audit query capabilities for compliance
#### 4.2.6 Security & Permission Framework
- [ ] Multi-level permission system:
- [ ] Granular permissions: rules, config, users, stats
- [ ] Permission inheritance and delegation
- [ ] Time-limited administrative access (expires_at)
- [ ] Authentication security:
- [ ] Strong nostr signature validation
- [ ] Administrator authorization chain verification
- [ ] Command-specific permission checks
- [ ] Rate limiting for administrative operations
### 4.3 Integration & Testing
- [ ] Authentication system integration:
- [ ] Integrate rule evaluation into existing authentication flow
- [ ] Maintain backward compatibility with nostr-only authentication
- [ ] Performance testing with rule caching
- [ ] Administrative system testing:
- [ ] Test all administrative commands and queries
- [ ] Verify permission enforcement and security
- [ ] Test audit logging and compliance features
- [ ] Load testing for administrative operations
---
## Phase 5: Optional Features
---
## Phase 5: Optional Features
### 4.1 User Server Lists (BUD-03) - Optional
- [ ] Implement server list advertisement
@@ -230,15 +379,21 @@ This document outlines the implementation plan for ginxsom, a FastCGI-based Blos
- [x] Authenticated uploads working (Nostr kind 24242 event validation)
- [x] Proper error handling for upload scenarios
- [x] Database metadata storage during upload (with uploader_pubkey and filename)
- [ ] List blobs endpoint implemented (GET /list/<pubkey>)
- [ ] Delete blob endpoint implemented (DELETE /<sha256>)
- [x] List blobs endpoint implemented (GET /list/<pubkey>)
- [x] Delete blob endpoint implemented (DELETE /<sha256>)
### Milestone 3: Policy Compliance (Phase 3 Pending)
- [ ] Upload requirements implemented
- [ ] Server policies configurable
- [ ] Spec compliance verified
### Milestone 4: Production Ready (Phase 4 Complete)
### Milestone 4: Advanced Authentication (Phase 4 Complete)
- [ ] Flexible authentication rules system operational
- [ ] Nostr-native administrative interface implemented
- [ ] Rule evaluation engine with caching performance
- [ ] Administrative audit trail and compliance features
### Milestone 5: Production Ready (Phase 5 Complete)
- Optional features implemented as needed
- Performance optimized
- Security hardened
@@ -274,6 +429,51 @@ This document outlines the implementation plan for ginxsom, a FastCGI-based Blos
---
## Future Improvements
### Upload Security & Performance Enhancements
**Current Issue**: The existing upload flow has a DOS vulnerability where large files are loaded entirely into memory before authentication occurs. This allows unauthenticated attackers to exhaust server memory.
**Current Flow**:
```
Client → nginx → FastCGI ginxsom
├─ reads entire file into memory (malloc + fread)
├─ validates auth (after file in memory)
└─ saves to blobs/ or errors
```
**Proposed Solution - nginx Upload Module**:
```
Client → nginx upload module → temp file → FastCGI ginxsom
├─ saves to /tmp/uploads/ ├─ validates auth quickly
└─ passes metadata only ├─ moves file to blobs/
└─ or deletes temp file
```
**Benefits**:
- Eliminates DOS vulnerability - nginx handles large files efficiently
- Fast auth validation - no waiting for full upload
- Leverages nginx strengths - what it's designed for
- Better scalability - memory usage independent of file size
**Implementation Requirements**:
- nginx upload module configuration
- Temp file cleanup handling
- Modified FastCGI code to process file paths instead of stdin
- Proper error handling for temp file operations
**Alternative Enhancement - HTTP 100 Continue**:
Could propose new Blossom BUD for two-phase upload:
1. Client sends headers with `Expect: 100-continue` + auth event
2. Server validates early (signature, expiration, pubkey)
3. Server responds `100 Continue` or `401 Unauthorized`
4. Client only sends file data if authorized
**Priority**: Implement after core BUD compliance is complete.
---
## Security Considerations
- [ ] Input validation on all endpoints
@@ -283,6 +483,7 @@ This document outlines the implementation plan for ginxsom, a FastCGI-based Blos
- [ ] Memory safety in C implementation
- [ ] Proper error message sanitization
- [ ] Log security (no sensitive data)
- [ ] **Upload DOS vulnerability** - Current implementation vulnerable to memory exhaustion attacks
---