c-relay/docs/sql_query_admin_api.md

# SQL Query Admin API Design

## Overview

This document describes the design for a general-purpose SQL query interface for the C-Relay admin API. This allows administrators to execute read-only SQL queries against the relay database through cryptographically signed kind 23456 events with NIP-44 encrypted command arrays.

## Security Model

### Authentication
- All queries must be sent as kind 23456 events with NIP-44 encrypted content
- Events must be signed by the admin's private key
- Admin pubkey verified against `config.admin_pubkey`
- Follows the same authentication pattern as existing admin commands

### Query Restrictions
While authentication is cryptographically secure, we implement defensive safeguards:

1. **Read-Only Enforcement**
   - Only SELECT statements allowed
   - Block: INSERT, UPDATE, DELETE, DROP, CREATE, ALTER, PRAGMA (write operations)
   - Allow: SELECT, WITH (for CTEs)

2. **Resource Limits**
   - Query timeout: 5 seconds (configurable)
   - Result row limit: 1000 rows (configurable)
   - Result size limit: 1MB (configurable)

3. **Query Logging**
   - All queries logged with timestamp, admin pubkey, execution time
   - Failed queries logged with error message

## Command Format

### Admin Event Structure (Kind 23456)
```json
{
  "id": "event_id",
  "pubkey": "admin_public_key",
  "created_at": 1234567890,
  "kind": 23456,
  "content": "AqHBUgcM7dXFYLQuDVzGwMST1G8jtWYyVvYxXhVGEu4nAb4LVw...",
  "tags": [
    ["p", "relay_public_key"]
  ],
  "sig": "event_signature"
}
```

The `content` field contains a NIP-44 encrypted JSON array:
```json
["sql_query", "SELECT * FROM events LIMIT 10"]
```

### Response Format (Kind 23457)
```json
["EVENT", "temp_sub_id", {
  "id": "response_event_id",
  "pubkey": "relay_public_key",
  "created_at": 1234567890,
  "kind": 23457,
  "content": "nip44_encrypted_content",
  "tags": [
    ["p", "admin_public_key"],
    ["e", "request_event_id"]
  ],
  "sig": "response_event_signature"
}]
```

The `content` field contains NIP-44 encrypted JSON:
```json
{
  "query_type": "sql_query",
  "request_id": "request_event_id",
  "timestamp": 1234567890,
  "query": "SELECT * FROM events LIMIT 10",
  "execution_time_ms": 45,
  "row_count": 10,
  "columns": ["id", "pubkey", "created_at", "kind", "content"],
  "rows": [
    ["abc123...", "def456...", 1234567890, 1, "Hello world"],
    ...
  ]
}
```

**Note:** The response includes the request event ID in two places:
1. **In tags**: `["e", "request_event_id"]` - Standard Nostr convention for event references
2. **In content**: `"request_id": "request_event_id"` - For easy access after decryption

### Error Response Format (Kind 23457)
```json
["EVENT", "temp_sub_id", {
  "id": "response_event_id",
  "pubkey": "relay_public_key",
  "created_at": 1234567890,
  "kind": 23457,
  "content": "nip44_encrypted_content",
  "tags": [
    ["p", "admin_public_key"],
    ["e", "request_event_id"]
  ],
  "sig": "response_event_signature"
}]
```

The `content` field contains NIP-44 encrypted JSON:
```json
{
  "query_type": "sql_query",
  "request_id": "request_event_id",
  "timestamp": 1234567890,
  "query": "DELETE FROM events",
  "status": "error",
  "error": "Query blocked: DELETE statements not allowed",
  "error_type": "blocked_statement"
}
```

## Available Database Tables and Views

### Core Tables
- **events** - All Nostr events (id, pubkey, created_at, kind, content, tags, sig)
- **config** - Configuration key-value pairs
- **auth_rules** - Authentication and authorization rules
- **subscription_events** - Subscription lifecycle events
- **event_broadcasts** - Event broadcast log

### Useful Views
- **recent_events** - Last 1000 events
- **event_stats** - Event statistics by type
- **configuration_events** - Kind 33334 configuration events
- **subscription_analytics** - Subscription metrics by date
- **active_subscriptions_log** - Currently active subscriptions
- **event_kinds_view** - Event distribution by kind
- **top_pubkeys_view** - Top 10 pubkeys by event count
- **time_stats_view** - Time-based statistics (24h, 7d, 30d)

## Implementation Plan

### Backend (dm_admin.c)

#### 1. Query Validation Function
```c
int validate_sql_query(const char* query, char* error_msg, size_t error_size);
```
- Check for blocked keywords (case-insensitive)
- Validate query syntax (basic checks)
- Return 0 on success, -1 on failure

#### 2. Query Execution Function
```c
char* execute_sql_query(const char* query, char* error_msg, size_t error_size);
```
- Set query timeout using sqlite3_busy_timeout()
- Execute query with row/size limits
- Build JSON response with results
- Log query execution
- Return JSON string or NULL on error

#### 3. Command Handler Integration
Add to `process_dm_admin_command()` in [`dm_admin.c`](src/dm_admin.c:131):
```c
else if (strcmp(command_type, "sql_query") == 0) {
    const char* query = get_tag_value(event, "sql_query", 1);
    if (!query) {
        DEBUG_ERROR("DM Admin: Missing sql_query parameter");
        snprintf(error_message, error_size, "invalid: missing SQL query");
    } else {
        result = handle_sql_query_unified(event, query, error_message, error_size, wsi);
    }
}
```

Add unified handler function:
```c
int handle_sql_query_unified(cJSON* event, const char* query,
                             char* error_message, size_t error_size,
                             struct lws* wsi) {
    // Get request event ID for response correlation
    cJSON* request_id_obj = cJSON_GetObjectItem(event, "id");
    if (!request_id_obj || !cJSON_IsString(request_id_obj)) {
        snprintf(error_message, error_size, "Missing request event ID");
        return -1;
    }
    const char* request_id = cJSON_GetStringValue(request_id_obj);
    
    // Validate query
    if (!validate_sql_query(query, error_message, error_size)) {
        return -1;
    }
    
    // Execute query and include request_id in result
    char* result_json = execute_sql_query(query, request_id, error_message, error_size);
    if (!result_json) {
        return -1;
    }
    
    // Send response as kind 23457 event with request ID in tags
    cJSON* sender_pubkey_obj = cJSON_GetObjectItem(event, "pubkey");
    if (!sender_pubkey_obj || !cJSON_IsString(sender_pubkey_obj)) {
        free(result_json);
        snprintf(error_message, error_size, "Missing sender pubkey");
        return -1;
    }
    
    const char* sender_pubkey = cJSON_GetStringValue(sender_pubkey_obj);
    int send_result = send_admin_response(sender_pubkey, result_json, request_id,
                                         error_message, error_size, wsi);
    free(result_json);
    
    return send_result;
}
```

### Frontend (api/index.html)

#### SQL Query Section UI
Add to [`api/index.html`](api/index.html:1):
```html
<section id="sql-query-section" class="admin-section">
  <h2>SQL Query Console</h2>
  
  <div class="query-selector">
    <label for="query-dropdown">Quick Queries & History:</label>
    <select id="query-dropdown" onchange="loadSelectedQuery()">
      <option value="">-- Select a query --</option>
      <optgroup label="Common Queries">
        <option value="recent_events">Recent Events</option>
        <option value="event_stats">Event Statistics</option>
        <option value="subscriptions">Active Subscriptions</option>
        <option value="top_pubkeys">Top Pubkeys</option>
        <option value="event_kinds">Event Kinds Distribution</option>
        <option value="time_stats">Time-based Statistics</option>
      </optgroup>
      <optgroup label="Query History" id="history-group">
        <!-- Dynamically populated from localStorage -->
      </optgroup>
    </select>
  </div>
  
  <div class="query-editor">
    <label for="sql-input">SQL Query:</label>
    <textarea id="sql-input" rows="5" placeholder="SELECT * FROM events LIMIT 10"></textarea>
    <div class="query-actions">
      <button onclick="executeSqlQuery()" class="primary-button">Execute Query</button>
      <button onclick="clearSqlQuery()">Clear</button>
      <button onclick="clearQueryHistory()" class="danger-button">Clear History</button>
    </div>
  </div>
  
  <div class="query-results">
    <h3>Results</h3>
    <div id="query-info" class="info-box"></div>
    <div id="query-table" class="table-container"></div>
  </div>
</section>
```

#### JavaScript Functions (api/index.js)
Add to [`api/index.js`](api/index.js:1):
```javascript
// Predefined query templates
const SQL_QUERY_TEMPLATES = {
  recent_events: "SELECT id, pubkey, created_at, kind, substr(content, 1, 50) as content FROM events ORDER BY created_at DESC LIMIT 20",
  event_stats: "SELECT * FROM event_stats",
  subscriptions: "SELECT * FROM active_subscriptions_log ORDER BY created_at DESC",
  top_pubkeys: "SELECT * FROM top_pubkeys_view",
  event_kinds: "SELECT * FROM event_kinds_view ORDER BY count DESC",
  time_stats: "SELECT * FROM time_stats_view"
};

// Query history management (localStorage)
const QUERY_HISTORY_KEY = 'c_relay_sql_history';
const MAX_HISTORY_ITEMS = 20;

// Load query history from localStorage
function loadQueryHistory() {
  try {
    const history = localStorage.getItem(QUERY_HISTORY_KEY);
    return history ? JSON.parse(history) : [];
  } catch (e) {
    console.error('Failed to load query history:', e);
    return [];
  }
}

// Save query to history
function saveQueryToHistory(query) {
  if (!query || query.trim().length === 0) return;
  
  try {
    let history = loadQueryHistory();
    
    // Remove duplicate if exists
    history = history.filter(q => q !== query);
    
    // Add to beginning
    history.unshift(query);
    
    // Limit size
    if (history.length > MAX_HISTORY_ITEMS) {
      history = history.slice(0, MAX_HISTORY_ITEMS);
    }
    
    localStorage.setItem(QUERY_HISTORY_KEY, JSON.stringify(history));
    updateQueryDropdown();
  } catch (e) {
    console.error('Failed to save query history:', e);
  }
}

// Clear query history
function clearQueryHistory() {
  if (confirm('Clear all query history?')) {
    localStorage.removeItem(QUERY_HISTORY_KEY);
    updateQueryDropdown();
  }
}

// Update dropdown with history
function updateQueryDropdown() {
  const historyGroup = document.getElementById('history-group');
  if (!historyGroup) return;
  
  // Clear existing history options
  historyGroup.innerHTML = '';
  
  const history = loadQueryHistory();
  if (history.length === 0) {
    const option = document.createElement('option');
    option.value = '';
    option.textContent = '(no history)';
    option.disabled = true;
    historyGroup.appendChild(option);
    return;
  }
  
  history.forEach((query, index) => {
    const option = document.createElement('option');
    option.value = `history_${index}`;
    // Truncate long queries for display
    const displayQuery = query.length > 60 ? query.substring(0, 60) + '...' : query;
    option.textContent = displayQuery;
    option.dataset.query = query;
    historyGroup.appendChild(option);
  });
}

// Load selected query from dropdown
function loadSelectedQuery() {
  const dropdown = document.getElementById('query-dropdown');
  const selectedValue = dropdown.value;
  
  if (!selectedValue) return;
  
  let query = '';
  
  // Check if it's a template
  if (SQL_QUERY_TEMPLATES[selectedValue]) {
    query = SQL_QUERY_TEMPLATES[selectedValue];
  }
  // Check if it's from history
  else if (selectedValue.startsWith('history_')) {
    const selectedOption = dropdown.options[dropdown.selectedIndex];
    query = selectedOption.dataset.query;
  }
  
  if (query) {
    document.getElementById('sql-input').value = query;
  }
  
  // Reset dropdown to placeholder
  dropdown.value = '';
}

// Initialize query history on page load
document.addEventListener('DOMContentLoaded', function() {
  updateQueryDropdown();
});

// Clear the SQL query input
function clearSqlQuery() {
  document.getElementById('sql-input').value = '';
  document.getElementById('query-info').innerHTML = '';
  document.getElementById('query-table').innerHTML = '';
}

// Track pending SQL queries by request ID
const pendingSqlQueries = new Map();

// Execute SQL query via admin API
async function executeSqlQuery() {
  const query = document.getElementById('sql-input').value;
  if (!query.trim()) {
    showError('Please enter a SQL query');
    return;
  }
  
  try {
    // Show loading state
    document.getElementById('query-info').innerHTML = '<div class="loading">Executing query...</div>';
    document.getElementById('query-table').innerHTML = '';
    
    // Save to history (before execution, so it's saved even if query fails)
    saveQueryToHistory(query.trim());
    
    // Send query as kind 23456 admin command
    const command = ["sql_query", query];
    const requestEvent = await sendAdminCommand(command);
    
    // Store query info for when response arrives
    if (requestEvent && requestEvent.id) {
      pendingSqlQueries.set(requestEvent.id, {
        query: query,
        timestamp: Date.now()
      });
    }
    
    // Note: Response will be handled by the event listener
    // which will call displaySqlQueryResults() when response arrives
  } catch (error) {
    showError('Failed to execute query: ' + error.message);
  }
}

// Handle SQL query response (called by event listener)
function handleSqlQueryResponse(response) {
  // Check if this is a response to one of our queries
  if (response.request_id && pendingSqlQueries.has(response.request_id)) {
    const queryInfo = pendingSqlQueries.get(response.request_id);
    pendingSqlQueries.delete(response.request_id);
    
    // Display results
    displaySqlQueryResults(response);
  }
}

// Display SQL query results
function displaySqlQueryResults(response) {
  const infoDiv = document.getElementById('query-info');
  const tableDiv = document.getElementById('query-table');
  
  if (response.status === 'error' || response.error) {
    infoDiv.innerHTML = `<div class="error-message">❌ ${response.error || 'Query failed'}</div>`;
    tableDiv.innerHTML = '';
    return;
  }
  
  // Show query info with request ID for debugging
  const rowCount = response.row_count || 0;
  const execTime = response.execution_time_ms || 0;
  const requestId = response.request_id ? response.request_id.substring(0, 8) + '...' : 'unknown';
  infoDiv.innerHTML = `
    <div class="query-info-success">
      <span>✅ Query executed successfully</span>
      <span>Rows: ${rowCount}</span>
      <span>Execution Time: ${execTime}ms</span>
      <span class="request-id" title="${response.request_id || ''}">Request: ${requestId}</span>
    </div>
  `;
  
  // Build results table
  if (response.rows && response.rows.length > 0) {
    let html = '<table class="sql-results-table"><thead><tr>';
    response.columns.forEach(col => {
      html += `<th>${escapeHtml(col)}</th>`;
    });
    html += '</tr></thead><tbody>';
    
    response.rows.forEach(row => {
      html += '<tr>';
      row.forEach(cell => {
        const cellValue = cell === null ? '<em>NULL</em>' : escapeHtml(String(cell));
        html += `<td>${cellValue}</td>`;
      });
      html += '</tr>';
    });
    
    html += '</tbody></table>';
    tableDiv.innerHTML = html;
  } else {
    tableDiv.innerHTML = '<p class="no-results">No results returned</p>';
  }
}

// Helper function to escape HTML
function escapeHtml(text) {
  const div = document.createElement('div');
  div.textContent = text;
  return div.innerHTML;
}
```

## Example Queries

### Subscription Statistics
```sql
SELECT 
  date,
  subscriptions_created,
  subscriptions_ended,
  avg_duration_seconds,
  unique_clients
FROM subscription_analytics
ORDER BY date DESC
LIMIT 7;
```

### Event Distribution by Kind
```sql
SELECT kind, count, percentage
FROM event_kinds_view
ORDER BY count DESC;
```

### Recent Events by Specific Pubkey
```sql
SELECT id, created_at, kind, content
FROM events
WHERE pubkey = 'abc123...'
ORDER BY created_at DESC
LIMIT 20;
```

### Active Subscriptions with Details
```sql
SELECT 
  subscription_id,
  client_ip,
  events_sent,
  duration_seconds,
  filter_json
FROM active_subscriptions_log
ORDER BY created_at DESC;
```

### Database Size and Event Count
```sql
SELECT 
  (SELECT COUNT(*) FROM events) as total_events,
  (SELECT COUNT(*) FROM subscription_events) as total_subscriptions,
  (SELECT COUNT(*) FROM auth_rules WHERE active = 1) as active_rules;
```

## Configuration Options

Add to config table:
```sql
INSERT INTO config (key, value, data_type, description, category) VALUES
  ('sql_query_enabled', 'true', 'boolean', 'Enable SQL query admin API', 'admin'),
  ('sql_query_timeout', '5', 'integer', 'Query timeout in seconds', 'admin'),
  ('sql_query_row_limit', '1000', 'integer', 'Maximum rows per query', 'admin'),
  ('sql_query_size_limit', '1048576', 'integer', 'Maximum result size in bytes', 'admin'),
  ('sql_query_log_enabled', 'true', 'boolean', 'Log all SQL queries', 'admin');
```

## Security Considerations

### What This Protects Against
1. **Unauthorized Access** - Only admin can execute queries (cryptographic verification)
2. **Data Modification** - Read-only enforcement prevents accidental/malicious changes
3. **Resource Exhaustion** - Timeouts and limits prevent DoS
4. **Audit Trail** - All queries logged for security review

### What This Does NOT Protect Against
1. **Admin Compromise** - If admin private key is stolen, attacker has full read access
2. **Information Disclosure** - Admin can read all data (by design)
3. **Complex Attacks** - Sophisticated SQL injection might bypass simple keyword blocking

### Recommendations
1. **Secure Admin Key** - Store admin private key securely, never commit to git
2. **Monitor Query Logs** - Review query logs regularly for suspicious activity
3. **Backup Database** - Regular backups in case of issues
4. **Test Queries** - Test complex queries on development relay first

## Testing Plan

### Unit Tests
1. Query validation (blocked keywords, syntax)
2. Result formatting (JSON structure)
3. Error handling (timeouts, limits)

### Integration Tests
1. Execute queries through NIP-17 DM
2. Verify authentication (admin vs non-admin)
3. Test resource limits (timeout, row limit)
4. Test error responses

### Security Tests
1. Attempt blocked statements (INSERT, DELETE, etc.)
2. Attempt SQL injection patterns
3. Test query timeout with slow queries
4. Test row limit with large result sets

## Future Enhancements

1. **Query History** - Store recent queries for quick re-execution
2. **Query Favorites** - Save frequently used queries
3. **Export Results** - Download results as CSV/JSON
4. **Query Builder** - Visual query builder for common operations
5. **Real-time Updates** - WebSocket updates for live data
6. **Query Sharing** - Share queries with other admins (if multi-admin support added)

## Migration Path

### Phase 1: Backend Implementation
1. Add query validation function
2. Add query execution function
3. Integrate with NIP-17 command handler
4. Add configuration options
5. Add query logging

### Phase 2: Frontend Implementation
1. Add SQL query section to index.html
2. Add query execution JavaScript
3. Add predefined query templates
4. Add results display formatting

### Phase 3: Testing and Documentation
1. Write unit tests
2. Write integration tests
3. Update user documentation
4. Create query examples guide

### Phase 4: Enhancement
1. Add query history
2. Add export functionality
3. Optimize performance
4. Add more predefined templates