buds 4 and 6 implemented
This commit is contained in:
@@ -88,15 +88,43 @@ This document tracks the implementation status of ginxsom, a high-performance Fa
|
|||||||
|
|
||||||
## BUD-03: Server List (User Server Lists) ⚪ **FOR CLIENTS, NOT SERVERS**
|
## BUD-03: Server List (User Server Lists) ⚪ **FOR CLIENTS, NOT SERVERS**
|
||||||
|
|
||||||
## BUD-04: Blob Mirroring ⚪ **PARTIAL**
|
## BUD-04: Blob Mirroring ✅ **COMPLETE**
|
||||||
|
|
||||||
### Current Status
|
### HTTP Client Implementation
|
||||||
|
- [x] CURL library integration and HTTP client functions
|
||||||
|
- [x] `write_callback()` - Download response data with dynamic buffering
|
||||||
|
- [x] `header_callback()` - Extract Content-Type headers
|
||||||
|
- [x] `download_blob_from_url()` - Complete HTTP download with security controls
|
||||||
|
- [x] Memory management and error handling
|
||||||
|
|
||||||
|
### PUT /mirror Endpoint
|
||||||
- [x] nginx endpoint configured (`PUT /mirror`)
|
- [x] nginx endpoint configured (`PUT /mirror`)
|
||||||
- [x] FastCGI routing established
|
- [x] FastCGI routing and request handling
|
||||||
- [ ] URL downloading implementation
|
- [x] JSON request body parsing (extract `url` field)
|
||||||
- [ ] Hash verification after download
|
- [x] URL validation and security checks (HTTPS-only, SSRF protection)
|
||||||
- [ ] Authorization handling for mirroring
|
- [x] Remote blob downloading with CURL
|
||||||
- [ ] Inter-server mirroring testing
|
- [x] SHA-256 hash calculation and verification
|
||||||
|
- [x] Content-Type detection (headers, URL extension, file signature)
|
||||||
|
- [x] File storage with proper extensions (.png, .jpg, etc.)
|
||||||
|
- [x] Database metadata storage
|
||||||
|
- [x] Blob descriptor JSON response
|
||||||
|
|
||||||
|
### Security Features
|
||||||
|
- [x] HTTPS-only URL validation (no HTTP allowed)
|
||||||
|
- [x] SSRF protection (blocks localhost, private IPs: 127.x, 192.168.x, 10.x, 172.16-31.x)
|
||||||
|
- [x] File size limits (100MB maximum)
|
||||||
|
- [x] Request timeouts (30s total, 10s connect)
|
||||||
|
- [x] SSL certificate verification
|
||||||
|
- [x] Authorization hash verification (when provided)
|
||||||
|
|
||||||
|
### Testing Status
|
||||||
|
- [x] Mirror request with valid HTTPS URL (HTTP 200)
|
||||||
|
- [x] Hash verification against downloaded content
|
||||||
|
- [x] Content-Type detection from PNG file
|
||||||
|
- [x] File accessibility after mirroring
|
||||||
|
- [x] HEAD request metadata retrieval
|
||||||
|
- [x] Error handling for invalid URLs
|
||||||
|
- [x] Security validation (private IP blocking)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -112,24 +140,24 @@ This document tracks the implementation status of ginxsom, a high-performance Fa
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## BUD-06: Upload Requirements ⚪ **NOT IMPLEMENTED**
|
## BUD-06: Upload Requirements ✅ **COMPLETE**
|
||||||
|
|
||||||
### HEAD /upload Pre-flight Validation
|
### HEAD /upload Pre-flight Validation
|
||||||
- [ ] `HEAD /upload` endpoint implementation
|
- [x] `HEAD /upload` endpoint implementation
|
||||||
- [ ] Client header parsing (X-SHA-256, X-Content-Length, X-Content-Type)
|
- [x] Client header parsing (X-SHA-256, X-Content-Length, X-Content-Type)
|
||||||
- [ ] Pre-flight validation without file transfer:
|
- [x] Pre-flight validation without file transfer:
|
||||||
- [ ] SHA-256 format validation
|
- [x] SHA-256 format validation
|
||||||
- [ ] File size limit checking
|
- [x] File size limit checking (100MB default)
|
||||||
- [ ] MIME type restrictions
|
- [ ] MIME type restrictions (policy 415 not enforced yet)
|
||||||
- [ ] Authentication validation
|
- [x] Authentication validation (optional via rules system)
|
||||||
- [ ] Duplicate detection
|
- [x] Duplicate detection (policy configurable)
|
||||||
- [ ] Banned hash checking
|
- [x] Banned hash checking (via rules engine)
|
||||||
- [ ] Proper HTTP status codes (200, 400, 401, 403, 411, 413, 415)
|
- [x] Proper HTTP status codes (200, 400, 401, 409, 411, 413; 415 reserved for future MIME policy)
|
||||||
- [ ] X-Reason headers for error messages
|
- [x] X-Reason headers for error messages
|
||||||
|
|
||||||
### Upload Policy Configuration
|
### Upload Policy Configuration
|
||||||
- [ ] Server configuration system
|
- [ ] Server configuration system
|
||||||
- [ ] Maximum file size limits
|
- [ ] Maximum file size limits (currently hard limit in code; move to config)
|
||||||
- [ ] Allowed MIME type restrictions
|
- [ ] Allowed MIME type restrictions
|
||||||
- [ ] Rate limiting implementation
|
- [ ] Rate limiting implementation
|
||||||
- [ ] DOS protection benefits
|
- [ ] DOS protection benefits
|
||||||
|
|||||||
Binary file not shown.
BIN
build/main.o
BIN
build/main.o
Binary file not shown.
BIN
db/ginxsom.db
BIN
db/ginxsom.db
Binary file not shown.
@@ -101,3 +101,52 @@
|
|||||||
127.0.0.1 - - [02/Sep/2025:17:21:41 -0400] "DELETE /28408ccd849c970912bdd4fd10aba23697cfd91a43c1b801af254167a25ceb36 HTTP/1.1" 200 136 "-" "curl/8.15.0"
|
127.0.0.1 - - [02/Sep/2025:17:21:41 -0400] "DELETE /28408ccd849c970912bdd4fd10aba23697cfd91a43c1b801af254167a25ceb36 HTTP/1.1" 200 136 "-" "curl/8.15.0"
|
||||||
127.0.0.1 - - [02/Sep/2025:17:21:41 -0400] "GET /28408ccd849c970912bdd4fd10aba23697cfd91a43c1b801af254167a25ceb36.txt HTTP/1.1" 404 162 "-" "curl/8.15.0"
|
127.0.0.1 - - [02/Sep/2025:17:21:41 -0400] "GET /28408ccd849c970912bdd4fd10aba23697cfd91a43c1b801af254167a25ceb36.txt HTTP/1.1" 404 162 "-" "curl/8.15.0"
|
||||||
127.0.0.1 - - [02/Sep/2025:17:21:41 -0400] "HEAD /28408ccd849c970912bdd4fd10aba23697cfd91a43c1b801af254167a25ceb36 HTTP/1.1" 404 0 "-" "curl/8.15.0"
|
127.0.0.1 - - [02/Sep/2025:17:21:41 -0400] "HEAD /28408ccd849c970912bdd4fd10aba23697cfd91a43c1b801af254167a25ceb36 HTTP/1.1" 404 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [02/Sep/2025:17:32:32 -0400] "PUT /upload HTTP/1.1" 200 262 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [02/Sep/2025:17:32:32 -0400] "GET /a83b1ac3d85de650a25beabc0bef67efe6856e33d3d519f27c22a014572031bd.txt HTTP/1.1" 200 155 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [02/Sep/2025:17:32:32 -0400] "DELETE /a83b1ac3d85de650a25beabc0bef67efe6856e33d3d519f27c22a014572031bd HTTP/1.1" 200 136 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [02/Sep/2025:17:32:32 -0400] "GET /a83b1ac3d85de650a25beabc0bef67efe6856e33d3d519f27c22a014572031bd.txt HTTP/1.1" 404 162 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [02/Sep/2025:17:32:32 -0400] "HEAD /a83b1ac3d85de650a25beabc0bef67efe6856e33d3d519f27c22a014572031bd HTTP/1.1" 404 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [02/Sep/2025:17:32:48 -0400] "PUT /upload HTTP/1.1" 200 262 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:00:31 -0400] "PUT /mirror HTTP/1.1" 501 38 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:01:24 -0400] "PUT /mirror HTTP/1.1" 200 256 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:01:24 -0400] "HEAD /24308d48eb498b593e55a87b6300ccffdea8432babc0bb898b1eff21ebbb72de.png HTTP/1.1" 200 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:01:24 -0400] "HEAD /24308d48eb498b593e55a87b6300ccffdea8432babc0bb898b1eff21ebbb72de HTTP/1.1" 200 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:34:45 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:34:45 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:34:45 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:34:45 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:34:45 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:34:45 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:34:45 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:34:45 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:34:45 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:36:34 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:36:34 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:36:34 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:36:34 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:36:34 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:36:34 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:36:34 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:36:34 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:36:34 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:37:59 -0400] "PUT /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:38:57 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:11:39:20 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:12:53:03 -0400] "HEAD /upload HTTP/1.1" 409 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:12:53:03 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:12:53:03 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:12:53:03 -0400] "HEAD /upload HTTP/1.1" 411 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:12:53:03 -0400] "HEAD /upload HTTP/1.1" 413 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:12:53:03 -0400] "HEAD /upload HTTP/1.1" 409 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:12:53:03 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:12:53:03 -0400] "HEAD /upload HTTP/1.1" 409 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:12:53:03 -0400] "HEAD /upload HTTP/1.1" 409 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:13:08:35 -0400] "HEAD /upload HTTP/1.1" 200 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:13:08:35 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:13:08:35 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:13:08:36 -0400] "HEAD /upload HTTP/1.1" 411 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:13:08:36 -0400] "HEAD /upload HTTP/1.1" 413 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:13:08:36 -0400] "HEAD /upload HTTP/1.1" 200 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:13:08:36 -0400] "HEAD /upload HTTP/1.1" 400 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:13:08:36 -0400] "HEAD /upload HTTP/1.1" 200 0 "-" "curl/8.15.0"
|
||||||
|
127.0.0.1 - - [03/Sep/2025:13:08:36 -0400] "HEAD /upload HTTP/1.1" 200 0 "-" "curl/8.15.0"
|
||||||
|
|||||||
12802
logs/error.log
12802
logs/error.log
File diff suppressed because it is too large
Load Diff
@@ -1 +1 @@
|
|||||||
FastCGI starting at Tue Sep 2 05:18:26 PM EDT 2025
|
FastCGI starting at Wed Sep 3 12:52:46 PM EDT 2025
|
||||||
|
|||||||
@@ -1 +1 @@
|
|||||||
198244
|
221465
|
||||||
|
|||||||
1449
src/main.c
1449
src/main.c
File diff suppressed because it is too large
Load Diff
87
tests/mirror_test_bud04.sh
Executable file
87
tests/mirror_test_bud04.sh
Executable file
@@ -0,0 +1,87 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Mirror Test Script for BUD-04
|
||||||
|
# Tests the PUT /mirror endpoint with a sample PNG file
|
||||||
|
|
||||||
|
# Test URL - PNG file with known SHA-256 hash
|
||||||
|
TEST_URL="https://laantungir.github.io/img_repo/24308d48eb498b593e55a87b6300ccffdea8432babc0bb898b1eff21ebbb72de.png"
|
||||||
|
EXPECTED_HASH="24308d48eb498b593e55a87b6300ccffdea8432babc0bb898b1eff21ebbb72de"
|
||||||
|
|
||||||
|
echo "=== BUD-04 Mirror Endpoint Test ==="
|
||||||
|
echo "Target URL: $TEST_URL"
|
||||||
|
echo "Expected Hash: $EXPECTED_HASH"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Create JSON request body
|
||||||
|
JSON_BODY=$(cat <<EOF
|
||||||
|
{
|
||||||
|
"url": "$TEST_URL"
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
)
|
||||||
|
|
||||||
|
echo "Request Body:"
|
||||||
|
echo "$JSON_BODY"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Make the mirror request
|
||||||
|
echo "=== Making Mirror Request ==="
|
||||||
|
RESPONSE=$(curl -s -w "\nHTTP_CODE:%{http_code}\n" \
|
||||||
|
-X PUT \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d "$JSON_BODY" \
|
||||||
|
http://localhost:9001/mirror)
|
||||||
|
|
||||||
|
echo "Response:"
|
||||||
|
echo "$RESPONSE"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Extract HTTP status code
|
||||||
|
HTTP_CODE=$(echo "$RESPONSE" | grep "HTTP_CODE:" | cut -d: -f2)
|
||||||
|
echo "HTTP Status Code: $HTTP_CODE"
|
||||||
|
|
||||||
|
# Check if successful
|
||||||
|
if [ "$HTTP_CODE" = "200" ]; then
|
||||||
|
echo "✅ Mirror request successful!"
|
||||||
|
|
||||||
|
# Try to access the mirrored blob
|
||||||
|
echo ""
|
||||||
|
echo "=== Verifying Mirrored Blob ==="
|
||||||
|
echo "Attempting to fetch: http://localhost:9001/$EXPECTED_HASH.png"
|
||||||
|
|
||||||
|
BLOB_RESPONSE=$(curl -s -w "HTTP_CODE:%{http_code}" -I "http://localhost:9001/$EXPECTED_HASH.png")
|
||||||
|
BLOB_HTTP_CODE=$(echo "$BLOB_RESPONSE" | grep "HTTP_CODE:" | cut -d: -f2)
|
||||||
|
|
||||||
|
if [ "$BLOB_HTTP_CODE" = "200" ]; then
|
||||||
|
echo "✅ Mirrored blob accessible!"
|
||||||
|
echo ""
|
||||||
|
echo "=== Blob Headers ==="
|
||||||
|
echo "$BLOB_RESPONSE" | grep -v "HTTP_CODE:"
|
||||||
|
else
|
||||||
|
echo "❌ Mirrored blob not accessible (HTTP $BLOB_HTTP_CODE)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Test HEAD request for metadata
|
||||||
|
echo ""
|
||||||
|
echo "=== Testing HEAD Request ==="
|
||||||
|
HEAD_RESPONSE=$(curl -s -w "HTTP_CODE:%{http_code}" -I -X HEAD "http://localhost:9001/$EXPECTED_HASH")
|
||||||
|
HEAD_HTTP_CODE=$(echo "$HEAD_RESPONSE" | grep "HTTP_CODE:" | cut -d: -f2)
|
||||||
|
|
||||||
|
if [ "$HEAD_HTTP_CODE" = "200" ]; then
|
||||||
|
echo "✅ HEAD request successful!"
|
||||||
|
echo "Metadata headers:"
|
||||||
|
echo "$HEAD_RESPONSE" | grep -E "(Content-Type|Content-Length|ETag)" | grep -v "HTTP_CODE:"
|
||||||
|
else
|
||||||
|
echo "❌ HEAD request failed (HTTP $HEAD_HTTP_CODE)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
else
|
||||||
|
echo "❌ Mirror request failed (HTTP $HTTP_CODE)"
|
||||||
|
if [ "$HTTP_CODE" != "000" ]; then
|
||||||
|
echo "Response body:"
|
||||||
|
echo "$RESPONSE" | grep -v "HTTP_CODE:"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=== Test Complete ==="
|
||||||
310
tests/requirements_test_bud06.sh
Executable file
310
tests/requirements_test_bud06.sh
Executable file
@@ -0,0 +1,310 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# BUD-06 Upload Requirements Test Suite
|
||||||
|
# Tests HEAD /upload endpoint for pre-flight validation
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
echo "=== BUD-06 Upload Requirements Test Suite ==="
|
||||||
|
|
||||||
|
# Test configuration
|
||||||
|
SERVER_URL="http://localhost:9001"
|
||||||
|
UPLOAD_ENDPOINT="${SERVER_URL}/upload"
|
||||||
|
|
||||||
|
# Test file properties
|
||||||
|
TEST_SHA256="24308d48eb498b593e55a87b6300ccffdea8432babc0bb898b1eff21ebbb72de"
|
||||||
|
TEST_CONTENT_TYPE="image/png"
|
||||||
|
TEST_CONTENT_LENGTH="71418"
|
||||||
|
|
||||||
|
# Helper function to make HEAD request with custom headers
|
||||||
|
make_head_request() {
|
||||||
|
local sha256="$1"
|
||||||
|
local content_type="$2"
|
||||||
|
local content_length="$3"
|
||||||
|
local auth_header="$4"
|
||||||
|
|
||||||
|
local curl_cmd="curl -s -I -X HEAD \"${UPLOAD_ENDPOINT}\""
|
||||||
|
|
||||||
|
if [ -n "$sha256" ]; then
|
||||||
|
curl_cmd="${curl_cmd} -H \"X-SHA-256: ${sha256}\""
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -n "$content_type" ]; then
|
||||||
|
curl_cmd="${curl_cmd} -H \"X-Content-Type: ${content_type}\""
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -n "$content_length" ]; then
|
||||||
|
curl_cmd="${curl_cmd} -H \"X-Content-Length: ${content_length}\""
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -n "$auth_header" ]; then
|
||||||
|
curl_cmd="${curl_cmd} -H \"Authorization: ${auth_header}\""
|
||||||
|
fi
|
||||||
|
|
||||||
|
eval "$curl_cmd"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Helper function to extract HTTP status code
|
||||||
|
get_status_code() {
|
||||||
|
echo "$1" | head -n1 | grep -o '[0-9]\{3\}'
|
||||||
|
}
|
||||||
|
|
||||||
|
# Helper function to extract X-Reason header
|
||||||
|
get_x_reason() {
|
||||||
|
echo "$1" | grep -i "x-reason:" | cut -d: -f2- | sed 's/^ *//'
|
||||||
|
}
|
||||||
|
|
||||||
|
# Test 1: Valid request should return 200 OK
|
||||||
|
echo ""
|
||||||
|
echo "=== Test 1: Valid Upload Requirements ==="
|
||||||
|
echo "Testing HEAD /upload with valid headers..."
|
||||||
|
|
||||||
|
RESPONSE=$(make_head_request "$TEST_SHA256" "$TEST_CONTENT_TYPE" "$TEST_CONTENT_LENGTH")
|
||||||
|
STATUS=$(get_status_code "$RESPONSE")
|
||||||
|
|
||||||
|
echo "Request Headers:"
|
||||||
|
echo " X-SHA-256: $TEST_SHA256"
|
||||||
|
echo " X-Content-Type: $TEST_CONTENT_TYPE"
|
||||||
|
echo " X-Content-Length: $TEST_CONTENT_LENGTH"
|
||||||
|
echo ""
|
||||||
|
echo "Response:"
|
||||||
|
echo "$RESPONSE"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ "$STATUS" = "200" ]; then
|
||||||
|
echo "✅ Test 1 PASSED: Valid request accepted (HTTP 200)"
|
||||||
|
else
|
||||||
|
echo "❌ Test 1 FAILED: Expected HTTP 200, got HTTP $STATUS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Test 2: Missing X-SHA-256 header
|
||||||
|
echo ""
|
||||||
|
echo "=== Test 2: Missing X-SHA-256 Header ==="
|
||||||
|
echo "Testing HEAD /upload without X-SHA-256..."
|
||||||
|
|
||||||
|
RESPONSE=$(make_head_request "" "$TEST_CONTENT_TYPE" "$TEST_CONTENT_LENGTH")
|
||||||
|
STATUS=$(get_status_code "$RESPONSE")
|
||||||
|
REASON=$(get_x_reason "$RESPONSE")
|
||||||
|
|
||||||
|
echo "Request Headers:"
|
||||||
|
echo " X-SHA-256: (missing)"
|
||||||
|
echo " X-Content-Type: $TEST_CONTENT_TYPE"
|
||||||
|
echo " X-Content-Length: $TEST_CONTENT_LENGTH"
|
||||||
|
echo ""
|
||||||
|
echo "Response:"
|
||||||
|
echo "$RESPONSE"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ "$STATUS" = "400" ]; then
|
||||||
|
echo "✅ Test 2 PASSED: Missing X-SHA-256 rejected (HTTP 400)"
|
||||||
|
echo " X-Reason: $REASON"
|
||||||
|
else
|
||||||
|
echo "❌ Test 2 FAILED: Expected HTTP 400, got HTTP $STATUS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Test 3: Invalid X-SHA-256 format
|
||||||
|
echo ""
|
||||||
|
echo "=== Test 3: Invalid X-SHA-256 Format ==="
|
||||||
|
echo "Testing HEAD /upload with invalid hash format..."
|
||||||
|
|
||||||
|
INVALID_SHA256="invalid_hash_format"
|
||||||
|
RESPONSE=$(make_head_request "$INVALID_SHA256" "$TEST_CONTENT_TYPE" "$TEST_CONTENT_LENGTH")
|
||||||
|
STATUS=$(get_status_code "$RESPONSE")
|
||||||
|
REASON=$(get_x_reason "$RESPONSE")
|
||||||
|
|
||||||
|
echo "Request Headers:"
|
||||||
|
echo " X-SHA-256: $INVALID_SHA256"
|
||||||
|
echo " X-Content-Type: $TEST_CONTENT_TYPE"
|
||||||
|
echo " X-Content-Length: $TEST_CONTENT_LENGTH"
|
||||||
|
echo ""
|
||||||
|
echo "Response:"
|
||||||
|
echo "$RESPONSE"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ "$STATUS" = "400" ]; then
|
||||||
|
echo "✅ Test 3 PASSED: Invalid X-SHA-256 format rejected (HTTP 400)"
|
||||||
|
echo " X-Reason: $REASON"
|
||||||
|
else
|
||||||
|
echo "❌ Test 3 FAILED: Expected HTTP 400, got HTTP $STATUS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Test 4: Missing X-Content-Length header
|
||||||
|
echo ""
|
||||||
|
echo "=== Test 4: Missing X-Content-Length Header ==="
|
||||||
|
echo "Testing HEAD /upload without X-Content-Length..."
|
||||||
|
|
||||||
|
RESPONSE=$(make_head_request "$TEST_SHA256" "$TEST_CONTENT_TYPE" "")
|
||||||
|
STATUS=$(get_status_code "$RESPONSE")
|
||||||
|
REASON=$(get_x_reason "$RESPONSE")
|
||||||
|
|
||||||
|
echo "Request Headers:"
|
||||||
|
echo " X-SHA-256: $TEST_SHA256"
|
||||||
|
echo " X-Content-Type: $TEST_CONTENT_TYPE"
|
||||||
|
echo " X-Content-Length: (missing)"
|
||||||
|
echo ""
|
||||||
|
echo "Response:"
|
||||||
|
echo "$RESPONSE"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ "$STATUS" = "411" ]; then
|
||||||
|
echo "✅ Test 4 PASSED: Missing X-Content-Length rejected (HTTP 411 Length Required)"
|
||||||
|
echo " X-Reason: $REASON"
|
||||||
|
else
|
||||||
|
echo "❌ Test 4 FAILED: Expected HTTP 411, got HTTP $STATUS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Test 5: File too large
|
||||||
|
echo ""
|
||||||
|
echo "=== Test 5: File Size Too Large ==="
|
||||||
|
echo "Testing HEAD /upload with oversized file..."
|
||||||
|
|
||||||
|
LARGE_SIZE="209715200" # 200MB (over 100MB limit)
|
||||||
|
RESPONSE=$(make_head_request "$TEST_SHA256" "$TEST_CONTENT_TYPE" "$LARGE_SIZE")
|
||||||
|
STATUS=$(get_status_code "$RESPONSE")
|
||||||
|
REASON=$(get_x_reason "$RESPONSE")
|
||||||
|
|
||||||
|
echo "Request Headers:"
|
||||||
|
echo " X-SHA-256: $TEST_SHA256"
|
||||||
|
echo " X-Content-Type: $TEST_CONTENT_TYPE"
|
||||||
|
echo " X-Content-Length: $LARGE_SIZE (200MB)"
|
||||||
|
echo ""
|
||||||
|
echo "Response:"
|
||||||
|
echo "$RESPONSE"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ "$STATUS" = "413" ]; then
|
||||||
|
echo "✅ Test 5 PASSED: Oversized file rejected (HTTP 413 Content Too Large)"
|
||||||
|
echo " X-Reason: $REASON"
|
||||||
|
else
|
||||||
|
echo "❌ Test 5 FAILED: Expected HTTP 413, got HTTP $STATUS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Test 6: Invalid Content-Type
|
||||||
|
echo ""
|
||||||
|
echo "=== Test 6: Unsupported Media Type ==="
|
||||||
|
echo "Testing HEAD /upload with potentially unsupported MIME type..."
|
||||||
|
|
||||||
|
UNSUPPORTED_TYPE="application/x-malware"
|
||||||
|
RESPONSE=$(make_head_request "$TEST_SHA256" "$UNSUPPORTED_TYPE" "$TEST_CONTENT_LENGTH")
|
||||||
|
STATUS=$(get_status_code "$RESPONSE")
|
||||||
|
REASON=$(get_x_reason "$RESPONSE")
|
||||||
|
|
||||||
|
echo "Request Headers:"
|
||||||
|
echo " X-SHA-256: $TEST_SHA256"
|
||||||
|
echo " X-Content-Type: $UNSUPPORTED_TYPE"
|
||||||
|
echo " X-Content-Length: $TEST_CONTENT_LENGTH"
|
||||||
|
echo ""
|
||||||
|
echo "Response:"
|
||||||
|
echo "$RESPONSE"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ "$STATUS" = "415" ]; then
|
||||||
|
echo "✅ Test 6 PASSED: Unsupported media type rejected (HTTP 415)"
|
||||||
|
echo " X-Reason: $REASON"
|
||||||
|
elif [ "$STATUS" = "200" ]; then
|
||||||
|
echo "⚠️ Test 6 INFO: Unsupported media type accepted (no MIME restrictions configured)"
|
||||||
|
else
|
||||||
|
echo "❌ Test 6 FAILED: Unexpected status code: HTTP $STATUS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Test 7: Zero file size
|
||||||
|
echo ""
|
||||||
|
echo "=== Test 7: Zero File Size ==="
|
||||||
|
echo "Testing HEAD /upload with zero byte file..."
|
||||||
|
|
||||||
|
RESPONSE=$(make_head_request "$TEST_SHA256" "$TEST_CONTENT_TYPE" "0")
|
||||||
|
STATUS=$(get_status_code "$RESPONSE")
|
||||||
|
REASON=$(get_x_reason "$RESPONSE")
|
||||||
|
|
||||||
|
echo "Request Headers:"
|
||||||
|
echo " X-SHA-256: $TEST_SHA256"
|
||||||
|
echo " X-Content-Type: $TEST_CONTENT_TYPE"
|
||||||
|
echo " X-Content-Length: 0"
|
||||||
|
echo ""
|
||||||
|
echo "Response:"
|
||||||
|
echo "$RESPONSE"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ "$STATUS" = "400" ]; then
|
||||||
|
echo "✅ Test 7 PASSED: Zero byte file rejected (HTTP 400)"
|
||||||
|
echo " X-Reason: $REASON"
|
||||||
|
else
|
||||||
|
echo "❌ Test 7 FAILED: Expected HTTP 400, got HTTP $STATUS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Test 8: Existing blob (duplicate)
|
||||||
|
echo ""
|
||||||
|
echo "=== Test 8: Duplicate Blob Detection ==="
|
||||||
|
echo "Testing HEAD /upload with hash of existing blob..."
|
||||||
|
|
||||||
|
# Use the hash from our previous mirror test that should exist
|
||||||
|
EXISTING_SHA256="24308d48eb498b593e55a87b6300ccffdea8432babc0bb898b1eff21ebbb72de"
|
||||||
|
RESPONSE=$(make_head_request "$EXISTING_SHA256" "$TEST_CONTENT_TYPE" "$TEST_CONTENT_LENGTH")
|
||||||
|
STATUS=$(get_status_code "$RESPONSE")
|
||||||
|
REASON=$(get_x_reason "$RESPONSE")
|
||||||
|
|
||||||
|
echo "Request Headers:"
|
||||||
|
echo " X-SHA-256: $EXISTING_SHA256 (should already exist from mirror test)"
|
||||||
|
echo " X-Content-Type: $TEST_CONTENT_TYPE"
|
||||||
|
echo " X-Content-Length: $TEST_CONTENT_LENGTH"
|
||||||
|
echo ""
|
||||||
|
echo "Response:"
|
||||||
|
echo "$RESPONSE"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ "$STATUS" = "409" ]; then
|
||||||
|
echo "✅ Test 8 PASSED: Duplicate blob detected (HTTP 409 Conflict)"
|
||||||
|
echo " X-Reason: $REASON"
|
||||||
|
elif [ "$STATUS" = "200" ]; then
|
||||||
|
echo "⚠️ Test 8 INFO: Duplicate upload allowed (server allows overwrites)"
|
||||||
|
else
|
||||||
|
echo "❌ Test 8 FAILED: Unexpected status code: HTTP $STATUS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Test 9: Authorization test (if server requires auth)
|
||||||
|
echo ""
|
||||||
|
echo "=== Test 9: Authorization Handling ==="
|
||||||
|
echo "Testing HEAD /upload authorization requirements..."
|
||||||
|
|
||||||
|
# Test without authorization first
|
||||||
|
RESPONSE=$(make_head_request "$TEST_SHA256" "$TEST_CONTENT_TYPE" "$TEST_CONTENT_LENGTH")
|
||||||
|
STATUS=$(get_status_code "$RESPONSE")
|
||||||
|
REASON=$(get_x_reason "$RESPONSE")
|
||||||
|
|
||||||
|
echo "Request Headers (no authorization):"
|
||||||
|
echo " X-SHA-256: $TEST_SHA256"
|
||||||
|
echo " X-Content-Type: $TEST_CONTENT_TYPE"
|
||||||
|
echo " X-Content-Length: $TEST_CONTENT_LENGTH"
|
||||||
|
echo " Authorization: (missing)"
|
||||||
|
echo ""
|
||||||
|
echo "Response:"
|
||||||
|
echo "$RESPONSE"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ "$STATUS" = "401" ]; then
|
||||||
|
echo "✅ Test 9a PASSED: Authorization required (HTTP 401)"
|
||||||
|
echo " X-Reason: $REASON"
|
||||||
|
elif [ "$STATUS" = "200" ]; then
|
||||||
|
echo "ℹ️ Test 9a INFO: No authorization required (anonymous uploads allowed)"
|
||||||
|
else
|
||||||
|
echo "❌ Test 9a FAILED: Unexpected status code: HTTP $STATUS"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=== Test Summary ==="
|
||||||
|
echo "BUD-06 Upload Requirements pre-flight validation test complete."
|
||||||
|
echo ""
|
||||||
|
echo "Key Test Scenarios:"
|
||||||
|
echo " ✓ Valid request validation"
|
||||||
|
echo " ✓ Missing/invalid X-SHA-256 header"
|
||||||
|
echo " ✓ Missing X-Content-Length header"
|
||||||
|
echo " ✓ File size limit enforcement"
|
||||||
|
echo " ✓ Media type validation (if configured)"
|
||||||
|
echo " ✓ Zero byte file rejection"
|
||||||
|
echo " ✓ Duplicate blob detection"
|
||||||
|
echo " ✓ Authorization handling"
|
||||||
|
echo ""
|
||||||
|
echo "Note: Some tests may show INFO/WARNING if server policies differ"
|
||||||
|
echo "from BUD-06 specification (e.g., no MIME restrictions, allows duplicates)."
|
||||||
|
echo ""
|
||||||
|
echo "=== End of BUD-06 Test Suite ==="
|
||||||
Reference in New Issue
Block a user