v0.1.4 - Make response at root JSON

2025-11-11 07:08:27 -04:00
parent 30e4408b28
commit fe2495f897
20 changed files with 2095 additions and 328 deletions
--- a/docs/PRODUCTION_MIGRATION_PLAN.md
+++ b/docs/PRODUCTION_MIGRATION_PLAN.md
@@ -0,0 +1,356 @@
+# Production Directory Structure Migration Plan
+
+## Overview
+
+This document outlines the plan to migrate the ginxsom production deployment from the current configuration to a new, more organized directory structure.
+
+## Current Configuration (As-Is)
+
+```
+Binary Location:     /var/www/html/blossom/ginxsom.fcgi
+Database Location:   /var/www/html/blossom/ginxsom.db
+Data Directory:      /var/www/html/blossom/
+Working Directory:   /var/www/html/blossom/ (set via spawn-fcgi -d)
+Socket:              /tmp/ginxsom-fcgi.sock
+```
+
+**Issues with Current Setup:**
+1. Binary and database mixed with data files in web-accessible directory
+2. Database path hardcoded as relative path `db/ginxsom.db` but database is at root of working directory
+3. No separation between application files and user data
+4. Security concern: application files in web root
+
+## Target Configuration (To-Be)
+
+```
+Binary Location:     /home/ubuntu/ginxsom/ginxsom.fcgi
+Database Location:   /home/ubuntu/ginxsom/db/ginxsom.db
+Data Directory:      /var/www/html/blossom/
+Working Directory:   /home/ubuntu/ginxsom/ (set via spawn-fcgi -d)
+Socket:              /tmp/ginxsom-fcgi.sock
+```
+
+**Benefits of New Setup:**
+1. Application files separated from user data
+2. Database in proper subdirectory structure
+3. Application files outside web root (better security)
+4. Clear separation of concerns
+5. Easier backup and maintenance
+
+## Directory Structure
+
+### Application Directory: `/home/ubuntu/ginxsom/`
+```
+/home/ubuntu/ginxsom/
+├── ginxsom.fcgi          # FastCGI binary
+├── db/
+│   └── ginxsom.db        # SQLite database
+├── build/                # Build artifacts (from rsync)
+├── src/                  # Source code (from rsync)
+├── include/              # Headers (from rsync)
+├── config/               # Config files (from rsync)
+└── scripts/              # Utility scripts (from rsync)
+```
+
+### Data Directory: `/var/www/html/blossom/`
+```
+/var/www/html/blossom/
+├── <sha256>.jpg          # User uploaded files
+├── <sha256>.png
+├── <sha256>.mp4
+└── ...
+```
+
+## Command-Line Arguments
+
+The ginxsom binary supports these arguments (from [`src/main.c`](src/main.c:1488-1509)):
+
+```bash
+--db-path PATH       # Database file path (default: db/ginxsom.db)
+--storage-dir DIR    # Storage directory for files (default: .)
+--help, -h          # Show help message
+```
+
+## Migration Steps
+
+### 1. Update deploy_lt.sh Configuration
+
+Update the configuration variables in [`deploy_lt.sh`](deploy_lt.sh:16-23):
+
+```bash
+# Configuration
+REMOTE_HOST="laantungir.net"
+REMOTE_USER="ubuntu"
+REMOTE_DIR="/home/ubuntu/ginxsom"
+REMOTE_DB_PATH="/home/ubuntu/ginxsom/db/ginxsom.db"
+REMOTE_NGINX_CONFIG="/etc/nginx/conf.d/default.conf"
+REMOTE_BINARY_PATH="/home/ubuntu/ginxsom/ginxsom.fcgi"
+REMOTE_SOCKET="/tmp/ginxsom-fcgi.sock"
+REMOTE_DATA_DIR="/var/www/html/blossom"
+```
+
+### 2. Update Binary Deployment
+
+Modify the binary copy section (lines 82-97) to use new path:
+
+```bash
+# Copy binary to application directory (not web directory)
+print_status "Copying ginxsom binary to application directory..."
+ssh $REMOTE_USER@$REMOTE_HOST << EOF
+    # Stop any running process first
+    sudo pkill -f ginxsom-fcgi || true
+    sleep 1
+
+    # Remove old binary if it exists
+    rm -f $REMOTE_BINARY_PATH
+
+    # Copy new binary
+    cp $REMOTE_DIR/build/ginxsom-fcgi $REMOTE_BINARY_PATH
+    chmod +x $REMOTE_BINARY_PATH
+    chown ubuntu:ubuntu $REMOTE_BINARY_PATH
+
+    echo "Binary copied successfully"
+EOF
+```
+
+### 3. Create Database Directory Structure
+
+Add database setup before starting FastCGI:
+
+```bash
+# Setup database directory
+print_status "Setting up database directory..."
+ssh $REMOTE_USER@$REMOTE_HOST << EOF
+    # Create db directory if it doesn't exist
+    mkdir -p $REMOTE_DIR/db
+    
+    # Copy database if it exists in old location
+    if [ -f /var/www/html/blossom/ginxsom.db ]; then
+        echo "Migrating database from old location..."
+        cp /var/www/html/blossom/ginxsom.db $REMOTE_DB_PATH
+    elif [ ! -f $REMOTE_DB_PATH ]; then
+        echo "Initializing new database..."
+        # Database will be created by application on first run
+    fi
+    
+    # Set proper permissions
+    chown -R ubuntu:ubuntu $REMOTE_DIR/db
+    chmod 755 $REMOTE_DIR/db
+    chmod 644 $REMOTE_DB_PATH 2>/dev/null || true
+    
+    echo "Database directory setup complete"
+EOF
+```
+
+### 4. Update spawn-fcgi Command
+
+Modify the FastCGI startup (line 164) to include command-line arguments:
+
+```bash
+# Start FastCGI process with explicit paths
+echo "Starting ginxsom FastCGI..."
+sudo spawn-fcgi \
+    -M 666 \
+    -u www-data \
+    -g www-data \
+    -s $REMOTE_SOCKET \
+    -U www-data \
+    -G www-data \
+    -d $REMOTE_DIR \
+    -- $REMOTE_BINARY_PATH \
+        --db-path "$REMOTE_DB_PATH" \
+        --storage-dir "$REMOTE_DATA_DIR"
+```
+
+**Key Changes:**
+- `-d $REMOTE_DIR`: Sets working directory to `/home/ubuntu/ginxsom/`
+- `--db-path "$REMOTE_DB_PATH"`: Explicit database path
+- `--storage-dir "$REMOTE_DATA_DIR"`: Explicit data directory
+
+### 5. Verify Permissions
+
+Ensure proper permissions for all directories:
+
+```bash
+# Application directory - owned by ubuntu
+sudo chown -R ubuntu:ubuntu /home/ubuntu/ginxsom
+sudo chmod 755 /home/ubuntu/ginxsom
+sudo chmod +x /home/ubuntu/ginxsom/ginxsom.fcgi
+
+# Database directory - readable by www-data
+sudo chmod 755 /home/ubuntu/ginxsom/db
+sudo chmod 644 /home/ubuntu/ginxsom/db/ginxsom.db
+
+# Data directory - writable by www-data
+sudo chown -R www-data:www-data /var/www/html/blossom
+sudo chmod 755 /var/www/html/blossom
+```
+
+## Path Resolution Logic
+
+### How Paths Work with spawn-fcgi -d Option
+
+When spawn-fcgi starts the FastCGI process:
+
+1. **Working Directory**: Set to `/home/ubuntu/ginxsom/` via `-d` option
+2. **Relative Paths**: Resolved from working directory
+3. **Absolute Paths**: Used as-is
+
+### Default Behavior (Without Arguments)
+
+From [`src/main.c`](src/main.c:30-31):
+```c
+char g_db_path[MAX_PATH_LEN] = "db/ginxsom.db";  // Relative to working dir
+char g_storage_dir[MAX_PATH_LEN] = ".";          // Current working dir
+```
+
+With working directory `/home/ubuntu/ginxsom/`:
+- Database: `/home/ubuntu/ginxsom/db/ginxsom.db` ✓
+- Storage: `/home/ubuntu/ginxsom/` ✗ (wrong - we want `/var/www/html/blossom/`)
+
+### With Command-Line Arguments
+
+```bash
+--db-path "/home/ubuntu/ginxsom/db/ginxsom.db"
+--storage-dir "/var/www/html/blossom"
+```
+
+Result:
+- Database: `/home/ubuntu/ginxsom/db/ginxsom.db` ✓
+- Storage: `/var/www/html/blossom/` ✓
+
+## Testing Plan
+
+### 1. Pre-Migration Verification
+```bash
+# Check current setup
+ssh ubuntu@laantungir.net "
+    echo 'Current binary location:'
+    ls -la /var/www/html/blossom/ginxsom.fcgi
+    
+    echo 'Current database location:'
+    ls -la /var/www/html/blossom/ginxsom.db
+    
+    echo 'Current process:'
+    ps aux | grep ginxsom-fcgi | grep -v grep
+"
+```
+
+### 2. Post-Migration Verification
+```bash
+# Check new setup
+ssh ubuntu@laantungir.net "
+    echo 'New binary location:'
+    ls -la /home/ubuntu/ginxsom/ginxsom.fcgi
+    
+    echo 'New database location:'
+    ls -la /home/ubuntu/ginxsom/db/ginxsom.db
+    
+    echo 'Data directory:'
+    ls -la /var/www/html/blossom/ | head -10
+    
+    echo 'Process working directory:'
+    sudo ls -la /proc/\$(pgrep -f ginxsom.fcgi)/cwd
+    
+    echo 'Process command line:'
+    ps aux | grep ginxsom-fcgi | grep -v grep
+"
+```
+
+### 3. Functional Testing
+```bash
+# Test health endpoint
+curl -k https://blossom.laantungir.net/health
+
+# Test file upload
+./tests/file_put_production.sh
+
+# Test file retrieval
+curl -k -I https://blossom.laantungir.net/<sha256>
+
+# Test list endpoint
+curl -k https://blossom.laantungir.net/list/<pubkey>
+```
+
+## Rollback Plan
+
+If migration fails:
+
+1. **Stop new process:**
+   ```bash
+   sudo pkill -f ginxsom-fcgi
+   ```
+
+2. **Restore old binary location:**
+   ```bash
+   sudo cp /home/ubuntu/ginxsom/build/ginxsom-fcgi /var/www/html/blossom/ginxsom.fcgi
+   sudo chown www-data:www-data /var/www/html/blossom/ginxsom.fcgi
+   ```
+
+3. **Restart with old configuration:**
+   ```bash
+   sudo spawn-fcgi -M 666 -u www-data -g www-data \
+       -s /tmp/ginxsom-fcgi.sock \
+       -U www-data -G www-data \
+       -d /var/www/html/blossom \
+       /var/www/html/blossom/ginxsom.fcgi
+   ```
+
+## Additional Considerations
+
+### 1. Database Backup
+Before migration, backup the current database:
+```bash
+ssh ubuntu@laantungir.net "
+    cp /var/www/html/blossom/ginxsom.db /var/www/html/blossom/ginxsom.db.backup
+"
+```
+
+### 2. NIP-94 Origin Configuration
+After migration, update [`src/bud08.c`](src/bud08.c) to return production domain:
+```c
+void nip94_get_origin(char *origin, size_t origin_size) {
+    snprintf(origin, origin_size, "https://blossom.laantungir.net");
+}
+```
+
+### 3. Monitoring
+Monitor logs after migration:
+```bash
+# Application logs
+ssh ubuntu@laantungir.net "sudo journalctl -u nginx -f"
+
+# FastCGI process
+ssh ubuntu@laantungir.net "ps aux | grep ginxsom-fcgi"
+```
+
+## Success Criteria
+
+Migration is successful when:
+
+1. ✓ Binary running from `/home/ubuntu/ginxsom/ginxsom.fcgi`
+2. ✓ Database accessible at `/home/ubuntu/ginxsom/db/ginxsom.db`
+3. ✓ Files stored in `/var/www/html/blossom/`
+4. ✓ Health endpoint returns 200 OK
+5. ✓ File upload works correctly
+6. ✓ File retrieval works correctly
+7. ✓ Database queries succeed
+8. ✓ No permission errors in logs
+
+## Timeline
+
+1. **Preparation**: Update deploy_lt.sh script (15 minutes)
+2. **Backup**: Backup current database (5 minutes)
+3. **Migration**: Run updated deployment script (10 minutes)
+4. **Testing**: Verify all endpoints (15 minutes)
+5. **Monitoring**: Watch for issues (30 minutes)
+
+**Total Estimated Time**: ~75 minutes
+
+## References
+
+- Current deployment script: [`deploy_lt.sh`](deploy_lt.sh)
+- Main application: [`src/main.c`](src/main.c)
+- Command-line parsing: [`src/main.c:1488-1509`](src/main.c:1488-1509)
+- Global configuration: [`src/main.c:30-31`](src/main.c:30-31)
+- Database operations: [`src/main.c:333-385`](src/main.c:333-385)