c-relay/docs/unified_startup_design.md

Unified Startup Sequence Design

Overview

This document describes the new unified startup sequence where all config values are created first, then CLI overrides are applied as a separate atomic operation. This eliminates the current 3-step incremental building process.

Current Problems

1. Incremental Config Building: Config is built in 3 steps:

   • Step 1: populate_default_config_values() - adds defaults
   • Step 2: CLI overrides applied via update_config_in_table()
   • Step 3: add_pubkeys_to_config_table() - adds generated keys

2. Race Conditions: Cache can be refreshed between steps, causing incomplete config reads

3. Complexity: Multiple code paths for first-time vs restart scenarios

New Design Principles

1. Atomic Config Creation: All config values created in single transaction
2. Separate Override Phase: CLI overrides applied after complete config exists
3. Unified Code Path: Same logic for first-time and restart scenarios
4. Cache Safety: Cache only loaded after config is complete

---

Scenario 1: First-Time Startup (No Database)

Sequence

1. Key Generation Phase
   ├─ generate_random_private_key_bytes() → admin_privkey_bytes
   ├─ nostr_bytes_to_hex() → admin_privkey (hex)
   ├─ nostr_ec_public_key_from_private_key() → admin_pubkey_bytes
   ├─ nostr_bytes_to_hex() → admin_pubkey (hex)
   ├─ generate_random_private_key_bytes() → relay_privkey_bytes
   ├─ nostr_bytes_to_hex() → relay_privkey (hex)
   ├─ nostr_ec_public_key_from_private_key() → relay_pubkey_bytes
   └─ nostr_bytes_to_hex() → relay_pubkey (hex)

2. Database Creation Phase
   ├─ create_database_with_relay_pubkey(relay_pubkey)
   │  └─ Sets g_database_path = "<relay_pubkey>.db"
   └─ init_database(g_database_path)
      └─ Creates database with embedded schema (includes config table)

3. Complete Config Population Phase (ATOMIC)
   ├─ BEGIN TRANSACTION
   ├─ populate_all_config_values_atomic()
   │  ├─ Insert ALL default config values from DEFAULT_CONFIG_VALUES[]
   │  ├─ Insert admin_pubkey
   │  └─ Insert relay_pubkey
   └─ COMMIT TRANSACTION

4. CLI Override Phase (ATOMIC)
   ├─ BEGIN TRANSACTION
   ├─ apply_cli_overrides()
   │  ├─ IF cli_options.port_override > 0:
   │  │  └─ UPDATE config SET value = ? WHERE key = 'relay_port'
   │  ├─ IF cli_options.admin_pubkey_override[0]:
   │  │  └─ UPDATE config SET value = ? WHERE key = 'admin_pubkey'
   │  └─ IF cli_options.relay_privkey_override[0]:
   │     └─ UPDATE config SET value = ? WHERE key = 'relay_privkey'
   └─ COMMIT TRANSACTION

5. Secure Key Storage Phase
   └─ store_relay_private_key(relay_privkey)
      └─ INSERT INTO relay_seckey (private_key_hex) VALUES (?)

6. Cache Initialization Phase
   └─ refresh_unified_cache_from_table()
      └─ Loads complete config into g_unified_cache

Function Call Sequence

// In main.c - first_time_startup branch
if (is_first_time_startup()) {
    // 1. Key Generation
    first_time_startup_sequence(&cli_options);
    // → Generates keys, stores in g_unified_cache
    // → Sets g_database_path
    // → Does NOT populate config yet
    
    // 2. Database Creation
    init_database(g_database_path);
    // → Creates database with schema
    
    // 3. Complete Config Population (NEW FUNCTION)
    populate_all_config_values_atomic(&cli_options);
    // → Inserts ALL defaults + pubkeys in single transaction
    // → Does NOT apply CLI overrides yet
    
    // 4. CLI Override Phase (NEW FUNCTION)
    apply_cli_overrides_atomic(&cli_options);
    // → Updates config table with CLI overrides
    // → Separate transaction after complete config exists
    
    // 5. Secure Key Storage
    store_relay_private_key(relay_privkey);
    
    // 6. Cache Initialization
    refresh_unified_cache_from_table();
}

New Functions Needed

// In config.c
int populate_all_config_values_atomic(const cli_options_t* cli_options) {
    // BEGIN TRANSACTION
    // Insert ALL defaults from DEFAULT_CONFIG_VALUES[]
    // Insert admin_pubkey from g_unified_cache
    // Insert relay_pubkey from g_unified_cache
    // COMMIT TRANSACTION
    return 0;
}

int apply_cli_overrides_atomic(const cli_options_t* cli_options) {
    // BEGIN TRANSACTION
    // IF port_override: UPDATE config SET value = ? WHERE key = 'relay_port'
    // IF admin_pubkey_override: UPDATE config SET value = ? WHERE key = 'admin_pubkey'
    // IF relay_privkey_override: UPDATE config SET value = ? WHERE key = 'relay_privkey'
    // COMMIT TRANSACTION
    // invalidate_config_cache()
    return 0;
}

---

Scenario 2: Restart with Existing Database + CLI Options

Sequence

1. Database Discovery Phase
   ├─ find_existing_db_files() → ["<relay_pubkey>.db"]
   ├─ extract_pubkey_from_filename() → relay_pubkey
   └─ Sets g_database_path = "<relay_pubkey>.db"

2. Database Initialization Phase
   └─ init_database(g_database_path)
      └─ Opens existing database

3. Config Validation Phase
   └─ validate_config_table_completeness()
      ├─ Check if all required keys exist
      └─ IF missing keys: populate_missing_config_values()

4. CLI Override Phase (ATOMIC)
   ├─ BEGIN TRANSACTION
   ├─ apply_cli_overrides()
   │  └─ UPDATE config SET value = ? WHERE key = ?
   └─ COMMIT TRANSACTION

5. Cache Initialization Phase
   └─ refresh_unified_cache_from_table()
      └─ Loads complete config into g_unified_cache

Function Call Sequence

// In main.c - existing relay branch
else {
    // 1. Database Discovery
    char** existing_files = find_existing_db_files();
    char* relay_pubkey = extract_pubkey_from_filename(existing_files[0]);
    startup_existing_relay(relay_pubkey);
    // → Sets g_database_path
    
    // 2. Database Initialization
    init_database(g_database_path);
    
    // 3. Config Validation (NEW FUNCTION)
    validate_config_table_completeness();
    // → Checks for missing keys
    // → Populates any missing defaults
    
    // 4. CLI Override Phase (REUSE FUNCTION)
    if (has_cli_overrides(&cli_options)) {
        apply_cli_overrides_atomic(&cli_options);
    }
    
    // 5. Cache Initialization
    refresh_unified_cache_from_table();
}

New Functions Needed

// In config.c
int validate_config_table_completeness(void) {
    // Check if all DEFAULT_CONFIG_VALUES keys exist
    // IF missing: populate_missing_config_values()
    return 0;
}

int populate_missing_config_values(void) {
    // BEGIN TRANSACTION
    // For each key in DEFAULT_CONFIG_VALUES:
    //   IF NOT EXISTS: INSERT INTO config
    // COMMIT TRANSACTION
    return 0;
}

int has_cli_overrides(const cli_options_t* cli_options) {
    return (cli_options->port_override > 0 ||
            cli_options->admin_pubkey_override[0] != '\0' ||
            cli_options->relay_privkey_override[0] != '\0');
}

---

Scenario 3: Restart with Existing Database + No CLI Options

Sequence

1. Database Discovery Phase
   ├─ find_existing_db_files() → ["<relay_pubkey>.db"]
   ├─ extract_pubkey_from_filename() → relay_pubkey
   └─ Sets g_database_path = "<relay_pubkey>.db"

2. Database Initialization Phase
   └─ init_database(g_database_path)
      └─ Opens existing database

3. Config Validation Phase
   └─ validate_config_table_completeness()
      ├─ Check if all required keys exist
      └─ IF missing keys: populate_missing_config_values()

4. Cache Initialization Phase (IMMEDIATE)
   └─ refresh_unified_cache_from_table()
      └─ Loads complete config into g_unified_cache

Function Call Sequence

// In main.c - existing relay branch (no CLI overrides)
else {
    // 1. Database Discovery
    char** existing_files = find_existing_db_files();
    char* relay_pubkey = extract_pubkey_from_filename(existing_files[0]);
    startup_existing_relay(relay_pubkey);
    
    // 2. Database Initialization
    init_database(g_database_path);
    
    // 3. Config Validation
    validate_config_table_completeness();
    
    // 4. Cache Initialization (IMMEDIATE - no overrides to apply)
    refresh_unified_cache_from_table();
}

---

Key Improvements

1. Atomic Config Creation

Before:

populate_default_config_values();  // Step 1
update_config_in_table("relay_port", port_str);  // Step 2
add_pubkeys_to_config_table();  // Step 3

After:

populate_all_config_values_atomic(&cli_options);  // Single transaction
apply_cli_overrides_atomic(&cli_options);  // Separate transaction

2. Elimination of Race Conditions

Before:

• Cache could refresh between steps 1-3
• Incomplete config could be read

After:

• Config created atomically
• Cache only refreshed after complete config exists

3. Unified Code Path

Before:

• Different logic for first-time vs restart
• populate_default_config_values() vs add_pubkeys_to_config_table()

After:

• Same validation logic for both scenarios
• validate_config_table_completeness() handles both cases

4. Clear Separation of Concerns

Before:

• CLI overrides mixed with default population
• Unclear when overrides are applied

After:

• Phase 1: Complete config creation
• Phase 2: CLI overrides (if any)
• Phase 3: Cache initialization

---

Implementation Changes Required

1. New Functions in config.c

// Atomic config population for first-time startup
int populate_all_config_values_atomic(const cli_options_t* cli_options);

// Atomic CLI override application
int apply_cli_overrides_atomic(const cli_options_t* cli_options);

// Config validation for existing databases
int validate_config_table_completeness(void);
int populate_missing_config_values(void);

// Helper function
int has_cli_overrides(const cli_options_t* cli_options);

2. Modified Functions in config.c

// Simplify to only generate keys and set database path
int first_time_startup_sequence(const cli_options_t* cli_options);

// Remove config population logic
int add_pubkeys_to_config_table(void);  // DEPRECATED - logic moved to populate_all_config_values_atomic()

3. Modified Startup Flow in main.c

// First-time startup
if (is_first_time_startup()) {
    first_time_startup_sequence(&cli_options);
    init_database(g_database_path);
    populate_all_config_values_atomic(&cli_options);  // NEW
    apply_cli_overrides_atomic(&cli_options);  // NEW
    store_relay_private_key(relay_privkey);
    refresh_unified_cache_from_table();
}

// Existing relay
else {
    startup_existing_relay(relay_pubkey);
    init_database(g_database_path);
    validate_config_table_completeness();  // NEW
    if (has_cli_overrides(&cli_options)) {
        apply_cli_overrides_atomic(&cli_options);  // NEW
    }
    refresh_unified_cache_from_table();
}

---

Benefits

1. Atomicity: Config creation is atomic - no partial states
2. Simplicity: Clear phases with single responsibility
3. Safety: Cache only loaded after complete config exists
4. Consistency: Same validation logic for all scenarios
5. Maintainability: Easier to understand and modify
6. Testability: Each phase can be tested independently

---

Migration Path

1. Implement new functions in config.c
2. Update main.c startup flow
3. Test first-time startup scenario
4. Test restart with CLI overrides
5. Test restart without CLI overrides
6. Remove deprecated functions
7. Update documentation

---

Testing Strategy

Test Cases

1. First-time startup with defaults

   • Verify all config values created atomically
   • Verify cache loads complete config

2. First-time startup with port override

   • Verify defaults created first
   • Verify port override applied second
   • Verify cache reflects override

3. Restart with complete config

   • Verify no config changes
   • Verify cache loads immediately

4. Restart with missing config keys

   • Verify missing keys populated
   • Verify existing keys unchanged

5. Restart with CLI overrides

   • Verify overrides applied atomically
   • Verify cache invalidated and refreshed

Validation Points

• Config table row count after each phase
• Cache validity state after each phase
• Transaction boundaries (BEGIN/COMMIT)
• Error handling for failed transactions