BUD-10

Multi-part uploads

draft optional

This bud defines a new PATCH method for the /upload endpoint to allow clients to upload blobs in multiple parts.

Signaling support for multi-part uploads

The server SHOULD respond to an OPTIONS /upload request with a 204 "No Content" response according to RFC-9110 including the Allow header with PATCH

Upload requirements

The server SHOULD implement BUD-06 "Upload requirements" to allow clients to check if a blob can be uploaded before uploading any chunks.

Chunking strategy

The client MAY split the blob into as many chunks as needed and MAY include overlap in the chunks if needed.

The server MUST concatenate the chunks based on the Upload-Offset and Content-Length headers to reconstruct the final blob to ensure any overlap is accounted for.

Uploading chunks

Clients MUST send the following headers in each PATCH /upload request:

X-SHA-256: The sha256 hash of the final blob. this should be considered the "ID" of the multi-part upload.
Upload-Type: The mine type of the final blob. should be set like Content-Type defined in RFC-9110
Upload-Length: The total length of the blob. should be set like Content-Length defined in RFC-9110
Content-Length: The length of the chunk in bytes.
Upload-Offset: The offset of the chunk in the blob.
Content-Type: The type of the chunk. MUST be set to application/octet-stream

The server MUST respond with a 204 "No Content" response if the chunk was accepted or a 4xx status code if it was not.

Uploading the final chunk

Once the server has received enough chunks to cover the Upload-Length of the blob the server MUST respond with a 2xx status code following BUD-02 or a 4xx status code if the blob hash does not match the X-SHA-256 header

Authorization

The server MAY require authorization for uploads by checking the Authorization header similar to the PUT /upload endpoint

The server SHOULD validate that the authorization event contains the following tags:

A t tag set to upload
One x tag for each chunk with the sha256 hash of the chunk
A final x tag with the sha256 hash of the final blob

Resuming uploads

The client SHOULD keep track of a chunks it has uploaded in order to resume uploads after a failure.

Example upload flow

# Client splits the blob into 4 chunks
split -b 46073 bitcoin.pdf chunk_

# Client uploads the first chunk
curl -X PATCH http://cdn.example.com/upload \
  -H "X-SHA-256: b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553" \
  -H "Upload-Type: application/pdf" \
  -H "Upload-Length: 184292" \
  -H "Upload-Offset: 0" \
  -H "Content-Length: 46073" \
  -H "Content-Type: application/octet-stream" \
  --data-binary "@chunk_aa"

# Server accepts the chunk and responds with a 204
HTTP/1.1 204 No Content

# CLient uploads remaining chunks (2-4)
curl -X PATCH http://cdn.example.com/upload
	# ..
	--data-binary "@chunk_ab"
curl -X PATCH http://cdn.example.com/upload
	# ...
	--data-binary "@chunk_ac"
curl -X PATCH http://cdn.example.com/upload
	# ...
	--data-binary "@chunk_ad"

# Server responds with 200 OK
HTTP/1.1 200 OK
Content-Type: application/pdf
Content-Length: 184292

{
  "url": "https://cdn.example.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf",
	"sha256": "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553",
  "uploaded": 1725105921,
	"type": "application/pdf",
	"length": 184292
}

3.7 KiB Raw Permalink Blame History