c-relay/docs/monitoring_simplified_plan.md

Simplified Monitoring Implementation Plan

Kind 34567 Event Kind Distribution Reporting

Date: 2025-10-16
Status: Implementation Ready

---

Overview

Simplified real-time monitoring system that:

• Reports event kind distribution (which includes total event count)
• Uses kind 34567 addressable events with d=event_kinds
• Controlled by two config variables
• Enabled on-demand when admin logs in
• Uses simple throttling to prevent performance impact

---

Configuration Variables

Database Config Table

Add two new configuration keys:

INSERT INTO config (key, value, data_type, description, category) VALUES
  ('kind_34567_reporting_enabled', 'false', 'boolean', 
   'Enable/disable kind 34567 event kind distribution reporting', 'monitoring'),
  ('kind_34567_reporting_throttling_sec', '5', 'integer', 
   'Minimum seconds between kind 34567 reports (throttling)', 'monitoring');

Configuration Access

// In src/monitoring.c or src/api.c
int is_monitoring_enabled(void) {
    return get_config_bool("kind_34567_reporting_enabled", 0);
}

int get_monitoring_throttle_seconds(void) {
    return get_config_int("kind_34567_reporting_throttling_sec", 5);
}

---

Event Structure

Kind 34567 Event Format

{
  "id": "<event_id>",
  "pubkey": "<relay_pubkey>",
  "created_at": 1697123456,
  "kind": 34567,
  "content": "{\"data_type\":\"event_kinds\",\"timestamp\":1697123456,\"data\":{\"total_events\":125000,\"distribution\":[{\"kind\":1,\"count\":45000,\"percentage\":36.0},{\"kind\":3,\"count\":12500,\"percentage\":10.0}]}}",
  "tags": [
    ["d", "event_kinds"],
    ["relay", "<relay_pubkey>"]
  ],
  "sig": "<signature>"
}

Content JSON Structure

{
  "data_type": "event_kinds",
  "timestamp": 1697123456,
  "data": {
    "total_events": 125000,
    "distribution": [
      {
        "kind": 1,
        "count": 45000,
        "percentage": 36.0
      },
      {
        "kind": 3,
        "count": 12500,
        "percentage": 10.0
      }
    ]
  },
  "metadata": {
    "query_time_ms": 18
  }
}

---

Implementation

File Structure

src/
  monitoring.h          # New file - monitoring system header
  monitoring.c          # New file - monitoring implementation
  main.c               # Modified - add trigger hook
  config.c             # Modified - add config keys (or use migration)

1. Header File: src/monitoring.h

#ifndef MONITORING_H
#define MONITORING_H

#include <time.h>
#include <cjson/cJSON.h>

// Initialize monitoring system
int init_monitoring_system(void);

// Cleanup monitoring system
void cleanup_monitoring_system(void);

// Called when an event is stored (from main.c)
void monitoring_on_event_stored(void);

// Enable/disable monitoring (called from admin API)
int set_monitoring_enabled(int enabled);

// Get monitoring status
int is_monitoring_enabled(void);

// Get throttle interval
int get_monitoring_throttle_seconds(void);

#endif /* MONITORING_H */

2. Implementation: src/monitoring.c

#include "monitoring.h"
#include "config.h"
#include "debug.h"
#include "../nostr_core_lib/nostr_core/nostr_core.h"
#include <sqlite3.h>
#include <string.h>
#include <time.h>

// External references
extern sqlite3* g_db;
extern int broadcast_event_to_subscriptions(cJSON* event);
extern int store_event(cJSON* event);
extern const char* get_config_value(const char* key);
extern int get_config_bool(const char* key, int default_value);
extern int get_config_int(const char* key, int default_value);
extern char* get_relay_private_key(void);

// Throttling state
static time_t last_report_time = 0;

// Initialize monitoring system
int init_monitoring_system(void) {
    DEBUG_LOG("Monitoring system initialized");
    last_report_time = 0;
    return 0;
}

// Cleanup monitoring system
void cleanup_monitoring_system(void) {
    DEBUG_LOG("Monitoring system cleaned up");
}

// Check if monitoring is enabled
int is_monitoring_enabled(void) {
    return get_config_bool("kind_34567_reporting_enabled", 0);
}

// Get throttle interval
int get_monitoring_throttle_seconds(void) {
    return get_config_int("kind_34567_reporting_throttling_sec", 5);
}

// Enable/disable monitoring
int set_monitoring_enabled(int enabled) {
    // Update config table
    const char* value = enabled ? "true" : "false";
    
    // This would call update_config_in_table() or similar
    // For now, assume we have a function to update config
    extern int update_config_in_table(const char* key, const char* value);
    return update_config_in_table("kind_34567_reporting_enabled", value);
}

// Query event kind distribution from database
static char* query_event_kind_distribution(void) {
    if (!g_db) {
        DEBUG_ERROR("Database not available for monitoring query");
        return NULL;
    }
    
    struct timespec start_time;
    clock_gettime(CLOCK_MONOTONIC, &start_time);
    
    // Query total events
    sqlite3_stmt* stmt;
    int total_events = 0;
    
    if (sqlite3_prepare_v2(g_db, "SELECT COUNT(*) FROM events", -1, &stmt, NULL) == SQLITE_OK) {
        if (sqlite3_step(stmt) == SQLITE_ROW) {
            total_events = sqlite3_column_int(stmt, 0);
        }
        sqlite3_finalize(stmt);
    }
    
    // Query kind distribution
    cJSON* response = cJSON_CreateObject();
    cJSON_AddStringToObject(response, "data_type", "event_kinds");
    cJSON_AddNumberToObject(response, "timestamp", (double)time(NULL));
    
    cJSON* data = cJSON_CreateObject();
    cJSON_AddNumberToObject(data, "total_events", total_events);
    
    cJSON* distribution = cJSON_CreateArray();
    
    const char* sql = 
        "SELECT kind, COUNT(*) as count, "
        "ROUND(COUNT(*) * 100.0 / (SELECT COUNT(*) FROM events), 2) as percentage "
        "FROM events GROUP BY kind ORDER BY count DESC";
    
    if (sqlite3_prepare_v2(g_db, sql, -1, &stmt, NULL) == SQLITE_OK) {
        while (sqlite3_step(stmt) == SQLITE_ROW) {
            cJSON* kind_obj = cJSON_CreateObject();
            cJSON_AddNumberToObject(kind_obj, "kind", sqlite3_column_int(stmt, 0));
            cJSON_AddNumberToObject(kind_obj, "count", sqlite3_column_int64(stmt, 1));
            cJSON_AddNumberToObject(kind_obj, "percentage", sqlite3_column_double(stmt, 2));
            cJSON_AddItemToArray(distribution, kind_obj);
        }
        sqlite3_finalize(stmt);
    }
    
    cJSON_AddItemToObject(data, "distribution", distribution);
    cJSON_AddItemToObject(response, "data", data);
    
    // Calculate query time
    struct timespec end_time;
    clock_gettime(CLOCK_MONOTONIC, &end_time);
    double query_time_ms = (end_time.tv_sec - start_time.tv_sec) * 1000.0 +
                          (end_time.tv_nsec - start_time.tv_nsec) / 1000000.0;
    
    cJSON* metadata = cJSON_CreateObject();
    cJSON_AddNumberToObject(metadata, "query_time_ms", query_time_ms);
    cJSON_AddItemToObject(response, "metadata", metadata);
    
    char* json_string = cJSON_Print(response);
    cJSON_Delete(response);
    
    return json_string;
}

// Generate and broadcast kind 34567 event
static int generate_monitoring_event(const char* json_content) {
    if (!json_content) return -1;
    
    // Get relay keys
    const char* relay_pubkey = get_config_value("relay_pubkey");
    char* relay_privkey_hex = get_relay_private_key();
    if (!relay_pubkey || !relay_privkey_hex) {
        if (relay_privkey_hex) free(relay_privkey_hex);
        DEBUG_ERROR("Could not get relay keys for monitoring event");
        return -1;
    }
    
    // Convert relay private key to bytes
    unsigned char relay_privkey[32];
    if (nostr_hex_to_bytes(relay_privkey_hex, relay_privkey, sizeof(relay_privkey)) != 0) {
        free(relay_privkey_hex);
        DEBUG_ERROR("Failed to convert relay private key");
        return -1;
    }
    free(relay_privkey_hex);
    
    // Create tags array
    cJSON* tags = cJSON_CreateArray();
    
    // d tag for addressable event
    cJSON* d_tag = cJSON_CreateArray();
    cJSON_AddItemToArray(d_tag, cJSON_CreateString("d"));
    cJSON_AddItemToArray(d_tag, cJSON_CreateString("event_kinds"));
    cJSON_AddItemToArray(tags, d_tag);
    
    // relay tag
    cJSON* relay_tag = cJSON_CreateArray();
    cJSON_AddItemToArray(relay_tag, cJSON_CreateString("relay"));
    cJSON_AddItemToArray(relay_tag, cJSON_CreateString(relay_pubkey));
    cJSON_AddItemToArray(tags, relay_tag);
    
    // Create and sign event
    cJSON* event = nostr_create_and_sign_event(
        34567,              // kind
        json_content,       // content
        tags,               // tags
        relay_privkey,      // private key
        time(NULL)          // timestamp
    );
    
    if (!event) {
        DEBUG_ERROR("Failed to create and sign monitoring event");
        return -1;
    }
    
    // Broadcast to subscriptions
    broadcast_event_to_subscriptions(event);
    
    // Store in database
    int result = store_event(event);
    
    cJSON_Delete(event);
    
    return result;
}

// Called when an event is stored
void monitoring_on_event_stored(void) {
    // Check if monitoring is enabled
    if (!is_monitoring_enabled()) {
        return;
    }
    
    // Check throttling
    time_t now = time(NULL);
    int throttle_seconds = get_monitoring_throttle_seconds();
    
    if (now - last_report_time < throttle_seconds) {
        return;  // Too soon, skip this update
    }
    
    // Query event kind distribution
    char* json_content = query_event_kind_distribution();
    if (!json_content) {
        DEBUG_ERROR("Failed to query event kind distribution");
        return;
    }
    
    // Generate and broadcast monitoring event
    int result = generate_monitoring_event(json_content);
    free(json_content);
    
    if (result == 0) {
        last_report_time = now;
        DEBUG_LOG("Generated kind 34567 monitoring event");
    } else {
        DEBUG_ERROR("Failed to generate monitoring event");
    }
}

3. Integration: Modify src/main.c

Add monitoring hook to event storage:

// At top of file
#include "monitoring.h"

// In main() function, after init_database()
if (init_monitoring_system() != 0) {
    DEBUG_WARN("Failed to initialize monitoring system");
    // Continue anyway - monitoring is optional
}

// In store_event() function, after successful storage
int store_event(cJSON* event) {
    // ... existing code ...
    
    if (rc != SQLITE_DONE) {
        // ... error handling ...
    }
    
    free(tags_json);
    
    // Trigger monitoring update
    monitoring_on_event_stored();
    
    return 0;
}

// In cleanup section of main()
cleanup_monitoring_system();

4. Admin API: Enable/Disable Monitoring

Add admin command to enable monitoring (in src/dm_admin.c or src/api.c):

// Handle admin command to enable monitoring
if (strcmp(command, "enable_monitoring") == 0) {
    set_monitoring_enabled(1);
    send_nip17_response(sender_pubkey, 
        "✅ Kind 34567 monitoring enabled", 
        error_msg, sizeof(error_msg));
    return 0;
}

// Handle admin command to disable monitoring
if (strcmp(command, "disable_monitoring") == 0) {
    set_monitoring_enabled(0);
    send_nip17_response(sender_pubkey, 
        "🔴 Kind 34567 monitoring disabled", 
        error_msg, sizeof(error_msg));
    return 0;
}

// Handle admin command to set throttle interval
if (strncmp(command, "set_monitoring_throttle ", 24) == 0) {
    int seconds = atoi(command + 24);
    if (seconds >= 1 && seconds <= 3600) {
        char value[16];
        snprintf(value, sizeof(value), "%d", seconds);
        update_config_in_table("kind_34567_reporting_throttling_sec", value);
        
        char response[128];
        snprintf(response, sizeof(response), 
            "✅ Monitoring throttle set to %d seconds", seconds);
        send_nip17_response(sender_pubkey, response, error_msg, sizeof(error_msg));
    }
    return 0;
}

---

Frontend Integration

Admin Dashboard Subscription

// When admin logs in to dashboard
async function enableMonitoring() {
  // Send admin command to enable monitoring
  await sendAdminCommand(['enable_monitoring']);
  
  // Subscribe to kind 34567 events
  const subscription = {
    kinds: [34567],
    authors: [relayPubkey],
    "#d": ["event_kinds"]
  };
  
  relay.subscribe([subscription], {
    onevent: (event) => {
      handleMonitoringEvent(event);
    }
  });
}

// Handle incoming monitoring events
function handleMonitoringEvent(event) {
  const content = JSON.parse(event.content);
  
  if (content.data_type === 'event_kinds') {
    updateEventKindsChart(content.data);
    updateTotalEventsDisplay(content.data.total_events);
  }
}

// When admin logs out or closes dashboard
async function disableMonitoring() {
  await sendAdminCommand(['disable_monitoring']);
}

Display Event Kind Distribution

function updateEventKindsChart(data) {
  const { total_events, distribution } = data;
  
  // Update total events display
  document.getElementById('total-events').textContent = 
    total_events.toLocaleString();
  
  // Update chart/table with distribution
  const tableBody = document.getElementById('kind-distribution-table');
  tableBody.innerHTML = '';
  
  distribution.forEach(item => {
    const row = document.createElement('tr');
    row.innerHTML = `
      <td>Kind ${item.kind}</td>
      <td>${item.count.toLocaleString()}</td>
      <td>${item.percentage}%</td>
    `;
    tableBody.appendChild(row);
  });
}

---

Configuration Migration

Add to Schema or Migration Script

-- Add monitoring configuration
INSERT INTO config (key, value, data_type, description, category) VALUES
  ('kind_34567_reporting_enabled', 'false', 'boolean', 
   'Enable/disable kind 34567 event kind distribution reporting', 'monitoring'),
  ('kind_34567_reporting_throttling_sec', '5', 'integer', 
   'Minimum seconds between kind 34567 reports (throttling)', 'monitoring');

Or add to existing config initialization in src/config.c.

---

Testing

1. Enable Monitoring

# Via admin command (NIP-17 DM)
echo '["enable_monitoring"]' | nak event --kind 14 --content - ws://localhost:8888

2. Subscribe to Monitoring Events

# Subscribe to kind 34567 events
nak req --kinds 34567 --authors <relay_pubkey> ws://localhost:8888

3. Generate Events

# Send some test events to trigger monitoring
for i in {1..10}; do
  nak event -c "Test event $i" ws://localhost:8888
  sleep 1
done

4. Verify Monitoring Events

You should see kind 34567 events every 5 seconds (or configured throttle interval) with event kind distribution.

---

Performance Impact

With 3 events/second (relay.damus.io scale)

Query execution:

• Frequency: Every 5 seconds (throttled)
• Query time: ~700ms (for 1M events)
• Overhead: 700ms / 5000ms = 14% (acceptable)

Per-event overhead:

• Check if enabled: < 0.01ms
• Check throttle: < 0.01ms
• Total: < 0.02ms per event (negligible)

Overall impact: < 1% on event processing, 14% on query thread (separate from event processing)

---

Future Enhancements

Once this is working, easy to add:

1. More data types: Add d=connections, d=subscriptions, etc.
2. Materialized counters: Optimize queries for very large databases
3. Historical data: Store monitoring events for trending
4. Alerts: Trigger on thresholds (e.g., > 90% capacity)

---

Summary

This simplified plan provides:

✅ Single data type: Event kind distribution (includes total events)
✅ Two config variables: Enable/disable and throttle control
✅ On-demand activation: Enabled when admin logs in
✅ Simple throttling: Prevents performance impact
✅ Clean implementation: ~200 lines of code
✅ Easy to extend: Add more data types later

Estimated implementation time: 4-6 hours

Files to create/modify:

• Create: src/monitoring.h (~30 lines)
• Create: src/monitoring.c (~200 lines)
• Modify: src/main.c (~10 lines)
• Modify: src/config.c or migration (~5 lines)
• Modify: src/dm_admin.c or src/api.c (~30 lines)
• Create: api/monitoring.js (frontend, ~100 lines)

Total new code: ~375 lines