c-relay/tests/README.md

# C-Relay Comprehensive Testing Framework

This directory contains a comprehensive testing framework for the C-Relay Nostr relay implementation. The framework provides automated testing for security vulnerabilities, performance validation, and stability assurance.

## Overview

The testing framework is designed to validate all critical security fixes and ensure stable operation of the Nostr relay. It includes multiple test suites covering different aspects of relay functionality and security.

## Test Suites

### 1. Master Test Runner (`run_all_tests.sh`)
The master test runner orchestrates all test suites and provides comprehensive reporting.

**Usage:**
```bash
./tests/run_all_tests.sh
```

**Features:**
- Automated execution of all test suites
- Comprehensive HTML and log reporting
- Success/failure tracking across all tests
- Relay status validation before testing

### 2. SQL Injection Tests (`sql_injection_tests.sh`)
Comprehensive testing of SQL injection vulnerabilities across all filter types.

**Tests:**
- Classic SQL injection payloads (`'; DROP TABLE; --`)
- Union-based injection attacks
- Error-based injection attempts
- Time-based blind injection
- Stacked query attacks
- Filter-specific injection (authors, IDs, kinds, search, tags)

**Usage:**
```bash
./tests/sql_injection_tests.sh
```

### 3. Memory Corruption Tests (`memory_corruption_tests.sh`)
Tests for buffer overflows, use-after-free, and memory safety issues.

**Tests:**
- Malformed subscription IDs (empty, very long, null bytes)
- Oversized filter arrays
- Concurrent access patterns
- Malformed JSON structures
- Large message payloads

**Usage:**
```bash
./tests/memory_corruption_tests.sh
```

### 4. Input Validation Tests (`input_validation_tests.sh`)
Comprehensive boundary condition testing for all input parameters.

**Tests:**
- Message type validation
- Message structure validation
- Subscription ID boundary tests
- Filter object validation
- Authors, IDs, kinds, timestamps, limits validation

**Usage:**
```bash
./tests/input_validation_tests.sh
```

### 5. Load Testing (`load_tests.sh`)
Performance testing under high concurrent connection scenarios.

**Test Scenarios:**
- Light load (10 concurrent clients)
- Medium load (25 concurrent clients)
- Heavy load (50 concurrent clients)
- Stress test (100 concurrent clients)

**Features:**
- Resource monitoring (CPU, memory, connections)
- Connection success rate tracking
- Message throughput measurement
- Relay responsiveness validation

**Usage:**
```bash
./tests/load_tests.sh
```

### 6. Authentication Tests (`auth_tests.sh`)
Tests NIP-42 authentication mechanisms and access control.

**Tests:**
- Authentication challenge responses
- Whitelist/blacklist functionality
- Event publishing with auth requirements
- Admin API authentication events

**Usage:**
```bash
./tests/auth_tests.sh
```

### 7. Rate Limiting Tests (`rate_limiting_tests.sh`)
Tests rate limiting and abuse prevention mechanisms.

**Tests:**
- Message rate limiting
- Connection rate limiting
- Subscription creation limits
- Abuse pattern detection

**Usage:**
```bash
./tests/rate_limiting_tests.sh
```

### 8. Performance Benchmarks (`performance_benchmarks.sh`)
Performance metrics and benchmarking tools.

**Tests:**
- Message throughput measurement
- Response time analysis
- Memory usage profiling
- CPU utilization tracking

**Usage:**
```bash
./tests/performance_benchmarks.sh
```

### 9. Resource Monitoring (`resource_monitoring.sh`)
System resource usage monitoring during testing.

**Features:**
- Real-time CPU and memory monitoring
- Connection count tracking
- Database size monitoring
- System load analysis

**Usage:**
```bash
./tests/resource_monitoring.sh
```

### 10. Configuration Tests (`config_tests.sh`)
Tests configuration management and persistence.

**Tests:**
- Configuration event processing
- Setting validation and persistence
- Admin API configuration commands
- Configuration reload behavior

**Usage:**
```bash
./tests/config_tests.sh
```

### 11. Existing Test Suites

#### Filter Validation Tests (`filter_validation_test.sh`)
Tests comprehensive input validation for REQ and COUNT messages.

#### Subscription Limits Tests (`subscription_limits.sh`)
Tests subscription limit enforcement and rate limiting.

#### Subscription Validation Tests (`subscription_validation.sh`)
Tests subscription ID handling and memory corruption fixes.

## Prerequisites

### System Requirements
- Linux/macOS environment
- `websocat` for WebSocket communication
- `bash` shell
- Standard Unix tools (`grep`, `awk`, `timeout`, etc.)

### Installing Dependencies

#### Ubuntu/Debian:
```bash
sudo apt-get update
sudo apt-get install websocat curl jq
```

#### macOS:
```bash
brew install websocat curl jq
```

#### Other systems:
Download `websocat` from: https://github.com/vi/websocat/releases

### Relay Setup
Before running tests, ensure the C-Relay is running:

```bash
# Build and start the relay
./make_and_restart_relay.sh

# Verify it's running
ps aux | grep c_relay
curl -H "Accept: application/nostr+json" http://localhost:8888
```

## Running Tests

### Quick Start
1. Start the relay:
   ```bash
   ./make_and_restart_relay.sh
   ```

2. Run all tests:
   ```bash
   ./tests/run_all_tests.sh
   ```

### Individual Test Suites
Run specific test suites for targeted testing:

```bash
# Security tests
./tests/sql_injection_tests.sh
./tests/memory_corruption_tests.sh
./tests/input_validation_tests.sh

# Performance tests
./tests/load_tests.sh

# Existing tests
./tests/filter_validation_test.sh
./tests/subscription_limits.sh
./tests/subscription_validation.sh
```

### NIP Protocol Tests
Run the existing NIP compliance tests:

```bash
# Run all NIP tests
./tests/run_nip_tests.sh

# Or run individual NIP tests
./tests/1_nip_test.sh
./tests/11_nip_information.sh
./tests/42_nip_test.sh
# ... etc
```

## Test Results and Reporting

### Master Test Runner Output
The master test runner (`run_all_tests.sh`) generates:

1. **Console Output**: Real-time test progress and results
2. **Log File**: Detailed execution log (`test_results_YYYYMMDD_HHMMSS.log`)
3. **HTML Report**: Comprehensive web report (`test_report_YYYYMMDD_HHMMSS.html`)

### Individual Test Suite Output
Each test suite provides:
- Test-by-test results with PASS/FAIL status
- Summary statistics (passed/failed/total tests)
- Detailed error information for failures

### Interpreting Results

#### Security Tests
- **PASS**: No vulnerabilities detected
- **FAIL**: Potential security issues found
- **UNCERTAIN**: Test inconclusive (may need manual verification)

#### Performance Tests
- **Connection Success Rate**: >95% = Excellent, >80% = Good, <80% = Poor
- **Resource Usage**: Monitor CPU/memory during load tests
- **Relay Responsiveness**: Must remain responsive after all tests

## Test Configuration

### Environment Variables
Customize test behavior with environment variables:

```bash
# Relay connection settings
export RELAY_HOST="127.0.0.1"
export RELAY_PORT="8888"

# Test parameters
export TEST_TIMEOUT=10
export CONCURRENT_CONNECTIONS=50
export MESSAGES_PER_SECOND=100
```

### Test Customization
Modify test parameters within individual test scripts:

- `RELAY_HOST` / `RELAY_PORT`: Relay connection details
- `TEST_TIMEOUT`: Individual test timeout (seconds)
- `TOTAL_TESTS`: Number of test iterations
- Load test parameters in `load_tests.sh`

## Troubleshooting

### Common Issues

#### "Could not connect to relay"
- Ensure relay is running: `./make_and_restart_relay.sh`
- Check port availability: `netstat -tln | grep 8888`
- Verify relay process: `ps aux | grep c_relay`

#### "websocat: command not found"
- Install websocat: `sudo apt-get install websocat`
- Or download from: https://github.com/vi/websocat/releases

#### Tests timing out
- Increase `TEST_TIMEOUT` value
- Check system resources (CPU/memory)
- Reduce concurrent connections in load tests

#### High failure rates in load tests
- Reduce `CONCURRENT_CONNECTIONS`
- Check system ulimits: `ulimit -n`
- Monitor system resources during testing

### Debug Mode
Enable verbose output for debugging:

```bash
# Set debug environment variable
export DEBUG=1

# Run tests with verbose output
./tests/run_all_tests.sh
```

## Security Testing Methodology

### SQL Injection Testing
- Tests all filter types (authors, IDs, kinds, search, tags)
- Uses comprehensive payload library
- Validates parameterized query protection
- Tests edge cases and boundary conditions

### Memory Safety Testing
- Buffer overflow detection
- Use-after-free prevention
- Concurrent access validation
- Malformed input handling

### Input Validation Testing
- Boundary condition testing
- Type validation
- Length limit enforcement
- Malformed data rejection

## Performance Benchmarking

### Load Testing Scenarios
1. **Light Load**: Basic functionality validation
2. **Medium Load**: Moderate stress testing
3. **Heavy Load**: High concurrency validation
4. **Stress Test**: Breaking point identification

### Metrics Collected
- Connection success rate
- Message throughput
- Response times
- Resource utilization (CPU, memory)
- Relay stability under load

## Integration with CI/CD

### Automated Testing
Integrate with CI/CD pipelines:

```yaml
# Example GitHub Actions workflow
- name: Run C-Relay Tests
  run: |
    ./make_and_restart_relay.sh
    ./tests/run_all_tests.sh
```

### Test Result Processing
Parse test results for automated reporting:

```bash
# Extract test summary
grep "Total tests:" test_results_*.log
grep "Passed:" test_results_*.log
grep "Failed:" test_results_*.log
```

## Contributing

### Adding New Tests
1. Create new test script in `tests/` directory
2. Follow existing naming conventions
3. Add to master test runner in `run_all_tests.sh`
4. Update this documentation

### Test Script Template
```bash
#!/bin/bash
# Test suite description

set -e

# Configuration
RELAY_HOST="${RELAY_HOST:-127.0.0.1}"
RELAY_PORT="${RELAY_PORT:-8888}"

# Test implementation here

echo "Test suite completed successfully"
```

## Security Considerations

### Test Environment
- Run tests in isolated environment
- Use test relay instance (not production)
- Monitor system resources during testing
- Clean up test data after completion

### Sensitive Data
- Tests use synthetic data only
- No real user data in test payloads
- Safe for production system testing

## Support and Issues

### Reporting Test Failures
When reporting test failures, include:
1. Test suite and specific test that failed
2. Full error output
3. System information (OS, relay version)
4. Relay configuration
5. Test environment details

### Getting Help
- Check existing issues in the project repository
- Review test logs for detailed error information
- Validate relay setup and configuration
- Test with minimal configuration to isolate issues

---

## Test Coverage Summary

| Test Suite | Security | Performance | Stability | Coverage |
|------------|----------|-------------|-----------|----------|
| SQL Injection | ✓ | | | All filter types |
| Memory Corruption | ✓ | | ✓ | Buffer overflows, race conditions |
| Input Validation | ✓ | | | Boundary conditions, type validation |
| Load Testing | | ✓ | ✓ | Concurrent connections, resource usage |
| Authentication | ✓ | | | NIP-42 auth, whitelist/blacklist |
| Rate Limiting | ✓ | ✓ | ✓ | Message rates, abuse prevention |
| Performance Benchmarks | | ✓ | | Throughput, response times |
| Resource Monitoring | | ✓ | ✓ | CPU/memory usage tracking |
| Configuration | ✓ | | ✓ | Admin API, settings persistence |
| Filter Validation | ✓ | | | REQ/COUNT message validation |
| Subscription Limits | | ✓ | ✓ | Rate limiting, connection limits |
| Subscription Validation | ✓ | | ✓ | ID validation, memory safety |

**Legend:**
- ✓ Covered
- Performance: Load and throughput testing
- Security: Vulnerability and attack vector testing
- Stability: Crash prevention and error handling