c-relay/docs/unified_startup_design.md

# Unified Startup Sequence Design

## Overview

This document describes the new unified startup sequence where all config values are created first, then CLI overrides are applied as a separate atomic operation. This eliminates the current 3-step incremental building process.

## Current Problems

1. **Incremental Config Building**: Config is built in 3 steps:
   - Step 1: `populate_default_config_values()` - adds defaults
   - Step 2: CLI overrides applied via `update_config_in_table()`
   - Step 3: `add_pubkeys_to_config_table()` - adds generated keys

2. **Race Conditions**: Cache can be refreshed between steps, causing incomplete config reads

3. **Complexity**: Multiple code paths for first-time vs restart scenarios

## New Design Principles

1. **Atomic Config Creation**: All config values created in single transaction
2. **Separate Override Phase**: CLI overrides applied after complete config exists
3. **Unified Code Path**: Same logic for first-time and restart scenarios
4. **Cache Safety**: Cache only loaded after config is complete

---

## Scenario 1: First-Time Startup (No Database)

### Sequence

```
1. Key Generation Phase
   ├─ generate_random_private_key_bytes() → admin_privkey_bytes
   ├─ nostr_bytes_to_hex() → admin_privkey (hex)
   ├─ nostr_ec_public_key_from_private_key() → admin_pubkey_bytes
   ├─ nostr_bytes_to_hex() → admin_pubkey (hex)
   ├─ generate_random_private_key_bytes() → relay_privkey_bytes
   ├─ nostr_bytes_to_hex() → relay_privkey (hex)
   ├─ nostr_ec_public_key_from_private_key() → relay_pubkey_bytes
   └─ nostr_bytes_to_hex() → relay_pubkey (hex)

2. Database Creation Phase
   ├─ create_database_with_relay_pubkey(relay_pubkey)
   │  └─ Sets g_database_path = "<relay_pubkey>.db"
   └─ init_database(g_database_path)
      └─ Creates database with embedded schema (includes config table)

3. Complete Config Population Phase (ATOMIC)
   ├─ BEGIN TRANSACTION
   ├─ populate_all_config_values_atomic()
   │  ├─ Insert ALL default config values from DEFAULT_CONFIG_VALUES[]
   │  ├─ Insert admin_pubkey
   │  └─ Insert relay_pubkey
   └─ COMMIT TRANSACTION

4. CLI Override Phase (ATOMIC)
   ├─ BEGIN TRANSACTION
   ├─ apply_cli_overrides()
   │  ├─ IF cli_options.port_override > 0:
   │  │  └─ UPDATE config SET value = ? WHERE key = 'relay_port'
   │  ├─ IF cli_options.admin_pubkey_override[0]:
   │  │  └─ UPDATE config SET value = ? WHERE key = 'admin_pubkey'
   │  └─ IF cli_options.relay_privkey_override[0]:
   │     └─ UPDATE config SET value = ? WHERE key = 'relay_privkey'
   └─ COMMIT TRANSACTION

5. Secure Key Storage Phase
   └─ store_relay_private_key(relay_privkey)
      └─ INSERT INTO relay_seckey (private_key_hex) VALUES (?)

6. Cache Initialization Phase
   └─ refresh_unified_cache_from_table()
      └─ Loads complete config into g_unified_cache
```

### Function Call Sequence

```c
// In main.c - first_time_startup branch
if (is_first_time_startup()) {
    // 1. Key Generation
    first_time_startup_sequence(&cli_options);
    // → Generates keys, stores in g_unified_cache
    // → Sets g_database_path
    // → Does NOT populate config yet
    
    // 2. Database Creation
    init_database(g_database_path);
    // → Creates database with schema
    
    // 3. Complete Config Population (NEW FUNCTION)
    populate_all_config_values_atomic(&cli_options);
    // → Inserts ALL defaults + pubkeys in single transaction
    // → Does NOT apply CLI overrides yet
    
    // 4. CLI Override Phase (NEW FUNCTION)
    apply_cli_overrides_atomic(&cli_options);
    // → Updates config table with CLI overrides
    // → Separate transaction after complete config exists
    
    // 5. Secure Key Storage
    store_relay_private_key(relay_privkey);
    
    // 6. Cache Initialization
    refresh_unified_cache_from_table();
}
```

### New Functions Needed

```c
// In config.c
int populate_all_config_values_atomic(const cli_options_t* cli_options) {
    // BEGIN TRANSACTION
    // Insert ALL defaults from DEFAULT_CONFIG_VALUES[]
    // Insert admin_pubkey from g_unified_cache
    // Insert relay_pubkey from g_unified_cache
    // COMMIT TRANSACTION
    return 0;
}

int apply_cli_overrides_atomic(const cli_options_t* cli_options) {
    // BEGIN TRANSACTION
    // IF port_override: UPDATE config SET value = ? WHERE key = 'relay_port'
    // IF admin_pubkey_override: UPDATE config SET value = ? WHERE key = 'admin_pubkey'
    // IF relay_privkey_override: UPDATE config SET value = ? WHERE key = 'relay_privkey'
    // COMMIT TRANSACTION
    // invalidate_config_cache()
    return 0;
}
```

---

## Scenario 2: Restart with Existing Database + CLI Options

### Sequence

```
1. Database Discovery Phase
   ├─ find_existing_db_files() → ["<relay_pubkey>.db"]
   ├─ extract_pubkey_from_filename() → relay_pubkey
   └─ Sets g_database_path = "<relay_pubkey>.db"

2. Database Initialization Phase
   └─ init_database(g_database_path)
      └─ Opens existing database

3. Config Validation Phase
   └─ validate_config_table_completeness()
      ├─ Check if all required keys exist
      └─ IF missing keys: populate_missing_config_values()

4. CLI Override Phase (ATOMIC)
   ├─ BEGIN TRANSACTION
   ├─ apply_cli_overrides()
   │  └─ UPDATE config SET value = ? WHERE key = ?
   └─ COMMIT TRANSACTION

5. Cache Initialization Phase
   └─ refresh_unified_cache_from_table()
      └─ Loads complete config into g_unified_cache
```

### Function Call Sequence

```c
// In main.c - existing relay branch
else {
    // 1. Database Discovery
    char** existing_files = find_existing_db_files();
    char* relay_pubkey = extract_pubkey_from_filename(existing_files[0]);
    startup_existing_relay(relay_pubkey);
    // → Sets g_database_path
    
    // 2. Database Initialization
    init_database(g_database_path);
    
    // 3. Config Validation (NEW FUNCTION)
    validate_config_table_completeness();
    // → Checks for missing keys
    // → Populates any missing defaults
    
    // 4. CLI Override Phase (REUSE FUNCTION)
    if (has_cli_overrides(&cli_options)) {
        apply_cli_overrides_atomic(&cli_options);
    }
    
    // 5. Cache Initialization
    refresh_unified_cache_from_table();
}
```

### New Functions Needed

```c
// In config.c
int validate_config_table_completeness(void) {
    // Check if all DEFAULT_CONFIG_VALUES keys exist
    // IF missing: populate_missing_config_values()
    return 0;
}

int populate_missing_config_values(void) {
    // BEGIN TRANSACTION
    // For each key in DEFAULT_CONFIG_VALUES:
    //   IF NOT EXISTS: INSERT INTO config
    // COMMIT TRANSACTION
    return 0;
}

int has_cli_overrides(const cli_options_t* cli_options) {
    return (cli_options->port_override > 0 ||
            cli_options->admin_pubkey_override[0] != '\0' ||
            cli_options->relay_privkey_override[0] != '\0');
}
```

---

## Scenario 3: Restart with Existing Database + No CLI Options

### Sequence

```
1. Database Discovery Phase
   ├─ find_existing_db_files() → ["<relay_pubkey>.db"]
   ├─ extract_pubkey_from_filename() → relay_pubkey
   └─ Sets g_database_path = "<relay_pubkey>.db"

2. Database Initialization Phase
   └─ init_database(g_database_path)
      └─ Opens existing database

3. Config Validation Phase
   └─ validate_config_table_completeness()
      ├─ Check if all required keys exist
      └─ IF missing keys: populate_missing_config_values()

4. Cache Initialization Phase (IMMEDIATE)
   └─ refresh_unified_cache_from_table()
      └─ Loads complete config into g_unified_cache
```

### Function Call Sequence

```c
// In main.c - existing relay branch (no CLI overrides)
else {
    // 1. Database Discovery
    char** existing_files = find_existing_db_files();
    char* relay_pubkey = extract_pubkey_from_filename(existing_files[0]);
    startup_existing_relay(relay_pubkey);
    
    // 2. Database Initialization
    init_database(g_database_path);
    
    // 3. Config Validation
    validate_config_table_completeness();
    
    // 4. Cache Initialization (IMMEDIATE - no overrides to apply)
    refresh_unified_cache_from_table();
}
```

---

## Key Improvements

### 1. Atomic Config Creation

**Before:**
```c
populate_default_config_values();  // Step 1
update_config_in_table("relay_port", port_str);  // Step 2
add_pubkeys_to_config_table();  // Step 3
```

**After:**
```c
populate_all_config_values_atomic(&cli_options);  // Single transaction
apply_cli_overrides_atomic(&cli_options);  // Separate transaction
```

### 2. Elimination of Race Conditions

**Before:**
- Cache could refresh between steps 1-3
- Incomplete config could be read

**After:**
- Config created atomically
- Cache only refreshed after complete config exists

### 3. Unified Code Path

**Before:**
- Different logic for first-time vs restart
- `populate_default_config_values()` vs `add_pubkeys_to_config_table()`

**After:**
- Same validation logic for both scenarios
- `validate_config_table_completeness()` handles both cases

### 4. Clear Separation of Concerns

**Before:**
- CLI overrides mixed with default population
- Unclear when overrides are applied

**After:**
- Phase 1: Complete config creation
- Phase 2: CLI overrides (if any)
- Phase 3: Cache initialization

---

## Implementation Changes Required

### 1. New Functions in config.c

```c
// Atomic config population for first-time startup
int populate_all_config_values_atomic(const cli_options_t* cli_options);

// Atomic CLI override application
int apply_cli_overrides_atomic(const cli_options_t* cli_options);

// Config validation for existing databases
int validate_config_table_completeness(void);
int populate_missing_config_values(void);

// Helper function
int has_cli_overrides(const cli_options_t* cli_options);
```

### 2. Modified Functions in config.c

```c
// Simplify to only generate keys and set database path
int first_time_startup_sequence(const cli_options_t* cli_options);

// Remove config population logic
int add_pubkeys_to_config_table(void);  // DEPRECATED - logic moved to populate_all_config_values_atomic()
```

### 3. Modified Startup Flow in main.c

```c
// First-time startup
if (is_first_time_startup()) {
    first_time_startup_sequence(&cli_options);
    init_database(g_database_path);
    populate_all_config_values_atomic(&cli_options);  // NEW
    apply_cli_overrides_atomic(&cli_options);  // NEW
    store_relay_private_key(relay_privkey);
    refresh_unified_cache_from_table();
}

// Existing relay
else {
    startup_existing_relay(relay_pubkey);
    init_database(g_database_path);
    validate_config_table_completeness();  // NEW
    if (has_cli_overrides(&cli_options)) {
        apply_cli_overrides_atomic(&cli_options);  // NEW
    }
    refresh_unified_cache_from_table();
}
```

---

## Benefits

1. **Atomicity**: Config creation is atomic - no partial states
2. **Simplicity**: Clear phases with single responsibility
3. **Safety**: Cache only loaded after complete config exists
4. **Consistency**: Same validation logic for all scenarios
5. **Maintainability**: Easier to understand and modify
6. **Testability**: Each phase can be tested independently

---

## Migration Path

1. Implement new functions in config.c
2. Update main.c startup flow
3. Test first-time startup scenario
4. Test restart with CLI overrides
5. Test restart without CLI overrides
6. Remove deprecated functions
7. Update documentation

---

## Testing Strategy

### Test Cases

1. **First-time startup with defaults**
   - Verify all config values created atomically
   - Verify cache loads complete config

2. **First-time startup with port override**
   - Verify defaults created first
   - Verify port override applied second
   - Verify cache reflects override

3. **Restart with complete config**
   - Verify no config changes
   - Verify cache loads immediately

4. **Restart with missing config keys**
   - Verify missing keys populated
   - Verify existing keys unchanged

5. **Restart with CLI overrides**
   - Verify overrides applied atomically
   - Verify cache invalidated and refreshed

### Validation Points

- Config table row count after each phase
- Cache validity state after each phase
- Transaction boundaries (BEGIN/COMMIT)
- Error handling for failed transactions