c-relay/tests

C-Relay Comprehensive Testing Framework

This directory contains a comprehensive testing framework for the C-Relay Nostr relay implementation. The framework provides automated testing for security vulnerabilities, performance validation, and stability assurance.

Overview

The testing framework is designed to validate all critical security fixes and ensure stable operation of the Nostr relay. It includes multiple test suites covering different aspects of relay functionality and security.

Test Suites

1. Master Test Runner (run_all_tests.sh)

The master test runner orchestrates all test suites and provides comprehensive reporting.

Usage:

./tests/run_all_tests.sh

Features:

• Automated execution of all test suites
• Comprehensive HTML and log reporting
• Success/failure tracking across all tests
• Relay status validation before testing

2. SQL Injection Tests (sql_injection_tests.sh)

Comprehensive testing of SQL injection vulnerabilities across all filter types.

Tests:

• Classic SQL injection payloads ('; DROP TABLE; --)
• Union-based injection attacks
• Error-based injection attempts
• Time-based blind injection
• Stacked query attacks
• Filter-specific injection (authors, IDs, kinds, search, tags)

Usage:

./tests/sql_injection_tests.sh

3. Memory Corruption Tests (memory_corruption_tests.sh)

Tests for buffer overflows, use-after-free, and memory safety issues.

Tests:

• Malformed subscription IDs (empty, very long, null bytes)
• Oversized filter arrays
• Concurrent access patterns
• Malformed JSON structures
• Large message payloads

Usage:

./tests/memory_corruption_tests.sh

4. Input Validation Tests (input_validation_tests.sh)

Comprehensive boundary condition testing for all input parameters.

Tests:

• Message type validation
• Message structure validation
• Subscription ID boundary tests
• Filter object validation
• Authors, IDs, kinds, timestamps, limits validation

Usage:

./tests/input_validation_tests.sh

5. Load Testing (load_tests.sh)

Performance testing under high concurrent connection scenarios.

Test Scenarios:

• Light load (10 concurrent clients)
• Medium load (25 concurrent clients)
• Heavy load (50 concurrent clients)
• Stress test (100 concurrent clients)

Features:

• Resource monitoring (CPU, memory, connections)
• Connection success rate tracking
• Message throughput measurement
• Relay responsiveness validation

Usage:

./tests/load_tests.sh

6. Authentication Tests (auth_tests.sh)

Tests NIP-42 authentication mechanisms and access control.

Tests:

• Authentication challenge responses
• Whitelist/blacklist functionality
• Event publishing with auth requirements
• Admin API authentication events

Usage:

./tests/auth_tests.sh

7. Rate Limiting Tests (rate_limiting_tests.sh)

Tests rate limiting and abuse prevention mechanisms.

Tests:

• Message rate limiting
• Connection rate limiting
• Subscription creation limits
• Abuse pattern detection

Usage:

./tests/rate_limiting_tests.sh

8. Performance Benchmarks (performance_benchmarks.sh)

Performance metrics and benchmarking tools.

Tests:

• Message throughput measurement
• Response time analysis
• Memory usage profiling
• CPU utilization tracking

Usage:

./tests/performance_benchmarks.sh

9. Resource Monitoring (resource_monitoring.sh)

System resource usage monitoring during testing.

Features:

• Real-time CPU and memory monitoring
• Connection count tracking
• Database size monitoring
• System load analysis

Usage:

./tests/resource_monitoring.sh

10. Configuration Tests (config_tests.sh)

Tests configuration management and persistence.

Tests:

• Configuration event processing
• Setting validation and persistence
• Admin API configuration commands
• Configuration reload behavior

Usage:

./tests/config_tests.sh

11. Existing Test Suites

Filter Validation Tests (filter_validation_test.sh)

Tests comprehensive input validation for REQ and COUNT messages.

Subscription Limits Tests (subscription_limits.sh)

Tests subscription limit enforcement and rate limiting.

Subscription Validation Tests (subscription_validation.sh)

Tests subscription ID handling and memory corruption fixes.

Prerequisites

System Requirements

• Linux/macOS environment
• websocat for WebSocket communication
• bash shell
• Standard Unix tools (grep, awk, timeout, etc.)

Installing Dependencies

Ubuntu/Debian:

sudo apt-get update
sudo apt-get install websocat curl jq

macOS:

brew install websocat curl jq

Other systems:

Download websocat from: https://github.com/vi/websocat/releases

Relay Setup

Before running tests, ensure the C-Relay is running:

# Build and start the relay
./make_and_restart_relay.sh

# Verify it's running
ps aux | grep c_relay
curl -H "Accept: application/nostr+json" http://localhost:8888

Running Tests

Quick Start

1. Start the relay:

   ./make_and_restart_relay.sh
   

2. Run all tests:

   ./tests/run_all_tests.sh
   

Individual Test Suites

Run specific test suites for targeted testing:

# Security tests
./tests/sql_injection_tests.sh
./tests/memory_corruption_tests.sh
./tests/input_validation_tests.sh

# Performance tests
./tests/load_tests.sh

# Existing tests
./tests/filter_validation_test.sh
./tests/subscription_limits.sh
./tests/subscription_validation.sh

NIP Protocol Tests

Run the existing NIP compliance tests:

# Run all NIP tests
./tests/run_nip_tests.sh

# Or run individual NIP tests
./tests/1_nip_test.sh
./tests/11_nip_information.sh
./tests/42_nip_test.sh
# ... etc

Test Results and Reporting

Master Test Runner Output

The master test runner (run_all_tests.sh) generates:

1. Console Output: Real-time test progress and results
2. Log File: Detailed execution log (test_results_YYYYMMDD_HHMMSS.log)
3. HTML Report: Comprehensive web report (test_report_YYYYMMDD_HHMMSS.html)

Individual Test Suite Output

Each test suite provides:

• Test-by-test results with PASS/FAIL status
• Summary statistics (passed/failed/total tests)
• Detailed error information for failures

Interpreting Results

Security Tests

• PASS: No vulnerabilities detected
• FAIL: Potential security issues found
• UNCERTAIN: Test inconclusive (may need manual verification)

Performance Tests

• Connection Success Rate: >95% = Excellent, >80% = Good, <80% = Poor
• Resource Usage: Monitor CPU/memory during load tests
• Relay Responsiveness: Must remain responsive after all tests

Test Configuration

Environment Variables

Customize test behavior with environment variables:

# Relay connection settings
export RELAY_HOST="127.0.0.1"
export RELAY_PORT="8888"

# Test parameters
export TEST_TIMEOUT=10
export CONCURRENT_CONNECTIONS=50
export MESSAGES_PER_SECOND=100

Test Customization

Modify test parameters within individual test scripts:

• RELAY_HOST / RELAY_PORT: Relay connection details
• TEST_TIMEOUT: Individual test timeout (seconds)
• TOTAL_TESTS: Number of test iterations
• Load test parameters in load_tests.sh

Troubleshooting

Common Issues

"Could not connect to relay"

• Ensure relay is running: ./make_and_restart_relay.sh
• Check port availability: netstat -tln | grep 8888
• Verify relay process: ps aux | grep c_relay

"websocat: command not found"

• Install websocat: sudo apt-get install websocat
• Or download from: https://github.com/vi/websocat/releases

Tests timing out

• Increase TEST_TIMEOUT value
• Check system resources (CPU/memory)
• Reduce concurrent connections in load tests

High failure rates in load tests

• Reduce CONCURRENT_CONNECTIONS
• Check system ulimits: ulimit -n
• Monitor system resources during testing

Debug Mode

Enable verbose output for debugging:

# Set debug environment variable
export DEBUG=1

# Run tests with verbose output
./tests/run_all_tests.sh

Security Testing Methodology

SQL Injection Testing

• Tests all filter types (authors, IDs, kinds, search, tags)
• Uses comprehensive payload library
• Validates parameterized query protection
• Tests edge cases and boundary conditions

Memory Safety Testing

• Buffer overflow detection
• Use-after-free prevention
• Concurrent access validation
• Malformed input handling

Input Validation Testing

• Boundary condition testing
• Type validation
• Length limit enforcement
• Malformed data rejection

Performance Benchmarking

Load Testing Scenarios

1. Light Load: Basic functionality validation
2. Medium Load: Moderate stress testing
3. Heavy Load: High concurrency validation
4. Stress Test: Breaking point identification

Metrics Collected

• Connection success rate
• Message throughput
• Response times
• Resource utilization (CPU, memory)
• Relay stability under load

Integration with CI/CD

Automated Testing

Integrate with CI/CD pipelines:

# Example GitHub Actions workflow
- name: Run C-Relay Tests
  run: |
    ./make_and_restart_relay.sh
    ./tests/run_all_tests.sh

Test Result Processing

Parse test results for automated reporting:

# Extract test summary
grep "Total tests:" test_results_*.log
grep "Passed:" test_results_*.log
grep "Failed:" test_results_*.log

Contributing

Adding New Tests

1. Create new test script in tests/ directory
2. Follow existing naming conventions
3. Add to master test runner in run_all_tests.sh
4. Update this documentation

Test Script Template

#!/bin/bash
# Test suite description

set -e

# Configuration
RELAY_HOST="${RELAY_HOST:-127.0.0.1}"
RELAY_PORT="${RELAY_PORT:-8888}"

# Test implementation here

echo "Test suite completed successfully"

Security Considerations

Test Environment

• Run tests in isolated environment
• Use test relay instance (not production)
• Monitor system resources during testing
• Clean up test data after completion

Sensitive Data

• Tests use synthetic data only
• No real user data in test payloads
• Safe for production system testing

Support and Issues

Reporting Test Failures

When reporting test failures, include:

1. Test suite and specific test that failed
2. Full error output
3. System information (OS, relay version)
4. Relay configuration
5. Test environment details

Getting Help

• Check existing issues in the project repository
• Review test logs for detailed error information
• Validate relay setup and configuration
• Test with minimal configuration to isolate issues

---

Test Coverage Summary

Test Suite	Security	Performance	Stability	Coverage
SQL Injection	✓			All filter types
Memory Corruption	✓		✓	Buffer overflows, race conditions
Input Validation	✓			Boundary conditions, type validation
Load Testing		✓	✓	Concurrent connections, resource usage
Authentication	✓			NIP-42 auth, whitelist/blacklist
Rate Limiting	✓	✓	✓	Message rates, abuse prevention
Performance Benchmarks		✓		Throughput, response times
Resource Monitoring		✓	✓	CPU/memory usage tracking
Configuration	✓		✓	Admin API, settings persistence
Filter Validation	✓			REQ/COUNT message validation
Subscription Limits		✓	✓	Rate limiting, connection limits
Subscription Validation	✓		✓	ID validation, memory safety

Legend:

• ✓ Covered
• Performance: Load and throughput testing
• Security: Vulnerability and attack vector testing
• Stability: Crash prevention and error handling