325 lines
7.4 KiB
Markdown
325 lines
7.4 KiB
Markdown
# Relay Traffic Measurement Guide
|
|
|
|
## Measuring Real-World Relay Traffic
|
|
|
|
To validate our performance assumptions, here are commands to measure actual event rates from live relays.
|
|
|
|
---
|
|
|
|
## Command: Count Events Over 1 Minute
|
|
|
|
### Basic Command
|
|
|
|
```bash
|
|
# Count events from relay.damus.io over 60 seconds
|
|
timeout 60 nak req -s $(date +%s) --stream wss://relay.damus.io | wc -l
|
|
```
|
|
|
|
This will:
|
|
1. Subscribe to all new events (`-s $(date +%s)` = since now)
|
|
2. Stream for 60 seconds (`timeout 60`)
|
|
3. Count the lines (each line = 1 event)
|
|
|
|
### With Event Rate Display
|
|
|
|
```bash
|
|
# Show events per second in real-time
|
|
timeout 60 nak req -s $(date +%s) --stream wss://relay.damus.io | \
|
|
pv -l -i 1 -r > /dev/null
|
|
```
|
|
|
|
This displays:
|
|
- Total events received
|
|
- Current rate (events/second)
|
|
- Average rate
|
|
|
|
### With Detailed Statistics
|
|
|
|
```bash
|
|
# Count events and calculate statistics
|
|
echo "Measuring relay traffic for 60 seconds..."
|
|
START=$(date +%s)
|
|
COUNT=$(timeout 60 nak req -s $START --stream wss://relay.damus.io | wc -l)
|
|
END=$(date +%s)
|
|
DURATION=$((END - START))
|
|
|
|
echo "Results:"
|
|
echo " Total events: $COUNT"
|
|
echo " Duration: ${DURATION}s"
|
|
echo " Events/second: $(echo "scale=2; $COUNT / $DURATION" | bc)"
|
|
echo " Events/minute: $COUNT"
|
|
```
|
|
|
|
### With Event Kind Distribution
|
|
|
|
```bash
|
|
# Count events by kind over 60 seconds
|
|
timeout 60 nak req -s $(date +%s) --stream wss://relay.damus.io | \
|
|
jq -r '.kind' | \
|
|
sort | uniq -c | sort -rn
|
|
```
|
|
|
|
Output example:
|
|
```
|
|
45 1 # 45 text notes
|
|
12 3 # 12 contact lists
|
|
8 7 # 8 reactions
|
|
3 6 # 3 reposts
|
|
```
|
|
|
|
### With Timestamp Analysis
|
|
|
|
```bash
|
|
# Show event timestamps and calculate intervals
|
|
timeout 60 nak req -s $(date +%s) --stream wss://relay.damus.io | \
|
|
jq -r '.created_at' | \
|
|
awk 'NR>1 {print $1-prev} {prev=$1}' | \
|
|
awk '{sum+=$1; count++} END {
|
|
print "Average interval:", sum/count, "seconds"
|
|
print "Events per second:", count/sum
|
|
}'
|
|
```
|
|
|
|
---
|
|
|
|
## Testing Multiple Relays
|
|
|
|
### Compare Traffic Across Relays
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
# test_relay_traffic.sh
|
|
|
|
RELAYS=(
|
|
"wss://relay.damus.io"
|
|
"wss://nos.lol"
|
|
"wss://relay.nostr.band"
|
|
"wss://nostr.wine"
|
|
)
|
|
|
|
DURATION=60
|
|
|
|
echo "Measuring relay traffic for ${DURATION} seconds..."
|
|
echo ""
|
|
|
|
for relay in "${RELAYS[@]}"; do
|
|
echo "Testing: $relay"
|
|
count=$(timeout $DURATION nak req -s $(date +%s) --stream "$relay" 2>/dev/null | wc -l)
|
|
rate=$(echo "scale=2; $count / $DURATION" | bc)
|
|
echo " Events: $count"
|
|
echo " Rate: ${rate}/sec"
|
|
echo ""
|
|
done
|
|
```
|
|
|
|
---
|
|
|
|
## Expected Results (Based on Real Measurements)
|
|
|
|
### relay.damus.io (Large Public Relay)
|
|
- **Expected rate**: 0.5-2 events/second
|
|
- **60-second count**: 30-120 events
|
|
- **Peak times**: Higher during US daytime hours
|
|
|
|
### nos.lol (Medium Public Relay)
|
|
- **Expected rate**: 0.2-0.8 events/second
|
|
- **60-second count**: 12-48 events
|
|
|
|
### Personal/Small Relays
|
|
- **Expected rate**: 0.01-0.1 events/second
|
|
- **60-second count**: 1-6 events
|
|
|
|
---
|
|
|
|
## Using Results to Validate Performance Assumptions
|
|
|
|
After measuring your relay's traffic:
|
|
|
|
1. **Calculate average events/second**:
|
|
```
|
|
events_per_second = total_events / 60
|
|
```
|
|
|
|
2. **Estimate query overhead**:
|
|
```
|
|
# For 100k event database:
|
|
query_time = 70ms
|
|
overhead_percentage = (query_time * events_per_second) / 1000 * 100
|
|
|
|
# Example: 0.5 events/sec
|
|
overhead = (70 * 0.5) / 1000 * 100 = 3.5%
|
|
```
|
|
|
|
3. **Determine if optimization needed**:
|
|
- < 5% overhead: No optimization needed
|
|
- 5-20% overhead: Consider 1-second throttling
|
|
- > 20% overhead: Use materialized counters
|
|
|
|
---
|
|
|
|
## Real-Time Monitoring During Development
|
|
|
|
### Monitor Your Own Relay
|
|
|
|
```bash
|
|
# Watch events in real-time with count
|
|
nak req -s $(date +%s) --stream ws://localhost:8888 | \
|
|
awk '{count++; print count, $0}'
|
|
```
|
|
|
|
### Monitor with Event Details
|
|
|
|
```bash
|
|
# Show event kind and pubkey for each event
|
|
nak req -s $(date +%s) --stream ws://localhost:8888 | \
|
|
jq -r '"[\(.kind)] \(.pubkey[0:8])... \(.content[0:50])"'
|
|
```
|
|
|
|
### Continuous Traffic Monitoring
|
|
|
|
```bash
|
|
# Monitor traffic in 10-second windows
|
|
while true; do
|
|
echo "=== $(date) ==="
|
|
count=$(timeout 10 nak req -s $(date +%s) --stream ws://localhost:8888 | wc -l)
|
|
rate=$(echo "scale=2; $count / 10" | bc)
|
|
echo "Events: $count (${rate}/sec)"
|
|
sleep 1
|
|
done
|
|
```
|
|
|
|
---
|
|
|
|
## Performance Testing Commands
|
|
|
|
### Simulate Load
|
|
|
|
```bash
|
|
# Send test events to measure query performance
|
|
for i in {1..100}; do
|
|
nak event -c "Test event $i" ws://localhost:8888
|
|
sleep 0.1 # 10 events/second
|
|
done
|
|
```
|
|
|
|
### Measure Query Response Time
|
|
|
|
```bash
|
|
# Time how long queries take with current database
|
|
time sqlite3 your_relay.db "SELECT COUNT(*) FROM events"
|
|
time sqlite3 your_relay.db "SELECT kind, COUNT(*) FROM events GROUP BY kind"
|
|
```
|
|
|
|
---
|
|
|
|
## Automated Traffic Analysis Script
|
|
|
|
Save this as `analyze_relay_traffic.sh`:
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
# Comprehensive relay traffic analysis
|
|
|
|
RELAY="${1:-ws://localhost:8888}"
|
|
DURATION="${2:-60}"
|
|
|
|
echo "Analyzing relay: $RELAY"
|
|
echo "Duration: ${DURATION} seconds"
|
|
echo ""
|
|
|
|
# Collect events
|
|
TMPFILE=$(mktemp)
|
|
timeout $DURATION nak req -s $(date +%s) --stream "$RELAY" > "$TMPFILE" 2>/dev/null
|
|
|
|
# Calculate statistics
|
|
TOTAL=$(wc -l < "$TMPFILE")
|
|
RATE=$(echo "scale=2; $TOTAL / $DURATION" | bc)
|
|
|
|
echo "=== Traffic Statistics ==="
|
|
echo "Total events: $TOTAL"
|
|
echo "Events/second: $RATE"
|
|
echo "Events/minute: $(echo "$TOTAL * 60 / $DURATION" | bc)"
|
|
echo ""
|
|
|
|
echo "=== Event Kind Distribution ==="
|
|
jq -r '.kind' "$TMPFILE" | sort | uniq -c | sort -rn | head -10
|
|
echo ""
|
|
|
|
echo "=== Top Publishers ==="
|
|
jq -r '.pubkey[0:16]' "$TMPFILE" | sort | uniq -c | sort -rn | head -5
|
|
echo ""
|
|
|
|
echo "=== Performance Estimate ==="
|
|
echo "For 100k event database:"
|
|
echo " Query time: ~70ms"
|
|
echo " Overhead: $(echo "scale=2; 70 * $RATE / 10" | bc)%"
|
|
echo ""
|
|
|
|
# Cleanup
|
|
rm "$TMPFILE"
|
|
```
|
|
|
|
Usage:
|
|
```bash
|
|
chmod +x analyze_relay_traffic.sh
|
|
./analyze_relay_traffic.sh wss://relay.damus.io 60
|
|
```
|
|
|
|
---
|
|
|
|
## Interpreting Results
|
|
|
|
### Low Traffic (< 0.1 events/sec)
|
|
- **Typical for**: Personal relays, small communities
|
|
- **Recommendation**: Trigger on every event, no optimization
|
|
- **Expected overhead**: < 1%
|
|
|
|
### Medium Traffic (0.1-0.5 events/sec)
|
|
- **Typical for**: Medium public relays
|
|
- **Recommendation**: Trigger on every event, consider throttling if database > 100k
|
|
- **Expected overhead**: 1-5%
|
|
|
|
### High Traffic (0.5-2 events/sec)
|
|
- **Typical for**: Large public relays
|
|
- **Recommendation**: Use 1-second throttling
|
|
- **Expected overhead**: 5-20% without throttling, < 1% with throttling
|
|
|
|
### Very High Traffic (> 2 events/sec)
|
|
- **Typical for**: Major public relays (rare)
|
|
- **Recommendation**: Use materialized counters
|
|
- **Expected overhead**: > 20% without optimization
|
|
|
|
---
|
|
|
|
## Continuous Monitoring in Production
|
|
|
|
### Add to Relay Startup
|
|
|
|
```bash
|
|
# In your relay startup script
|
|
echo "Starting traffic monitoring..."
|
|
nohup bash -c 'while true; do
|
|
count=$(timeout 60 nak req -s $(date +%s) --stream ws://localhost:8888 2>/dev/null | wc -l)
|
|
echo "$(date +%Y-%m-%d\ %H:%M:%S) - Events/min: $count" >> traffic.log
|
|
done' &
|
|
```
|
|
|
|
### Analyze Historical Traffic
|
|
|
|
```bash
|
|
# View traffic trends
|
|
cat traffic.log | awk '{print $4}' | \
|
|
awk '{sum+=$1; count++} END {print "Average:", sum/count, "events/min"}'
|
|
```
|
|
|
|
---
|
|
|
|
## Conclusion
|
|
|
|
Use these commands to:
|
|
1. ✅ Measure real-world traffic on your relay
|
|
2. ✅ Validate performance assumptions
|
|
3. ✅ Determine if optimization is needed
|
|
4. ✅ Monitor traffic trends over time
|
|
|
|
**Remember**: Most relays will measure < 1 event/second, making the simple "trigger on every event" approach perfectly viable. |