Merge branch 'master' into list-pagination

README.md

@@ -4,13 +4,30 @@ Blossom uses [nostr](https://github.com/nostr-protocol/nostr) public / private k
 
 ## What is it?
 
-Blossom is a spec for a set of HTTP endpoints that allow users to store blobs of data on publicly accessible servers
+Blossom is a specification for a set of HTTP endpoints that allow users to store blobs of data on publicly accessible servers
 
 ## What are blobs
 
 Blobs are packs of binary data addressed by their sha256 hash
 
-## How does it work?
+## Protocol specification (BUDs)
+
+BUDs or **Blossom Upgrade Documents** are short documents that outline an additional feature that a blossom server may implement.
+
+## BUDs
+
+- [BUD-00: Blossom Upgrade Documents](./buds/00.md)
+- [BUD-01: Server requirements and blob retrieval](./buds/01.md)
+- [BUD-02: Blob upload and management](./buds/02.md)
+- [BUD-03: User Server List](./buds/03.md)
+- [BUD-04: Mirroring blobs](./buds/04.md)
+- [BUD-05: Media optimization](./buds/05.md)
+- [BUD-06: Upload requirements](./buds/06.md)
+- [BUD-07: Payment required](./buds/07.md)
+- [BUD-08: Nostr File Metadata Tags](./buds/08.md)
+- [BUD-09: Blob Report](./buds/09.md)
+
+## Endpoints
 
 Blossom Servers expose a few endpoints for managing blobs
 
@@ -32,23 +49,6 @@ Blossom Servers expose a few endpoints for managing blobs
   - `Authentication`: Signed [nostr event](./buds/05.md#upload-authorization)
 - `PUT /report` [BUD-09](./buds/09.md)
 
-## Protocol specification (BUDs)
-
-BUDs stand for **Blossom Upgrade Documents**.
-
-See the [BUDs](./buds) folder and specifically [BUD-01](./buds/01.md) and [BUD-02](./buds/02.md) for a detailed explanation of the endpoints
-
-## BUDs
-
-- [BUD-01: Server requirements and blob retrieval](./buds/01.md)
-- [BUD-02: Blob upload and management](./buds/02.md)
-- [BUD-03: User Server List](./buds/03.md)
-- [BUD-04: Mirroring blobs](./buds/04.md)
-- [BUD-05: Media optimization](./buds/05.md)
-- [BUD-06: Upload requirements](./buds/06.md)
-- [BUD-08: Nostr File Metadata Tags](./buds/08.md)
-- [BUD-09: Blob Report](./buds/09.md)
-
 ## Event kinds
 
 | kind    | description         | BUD                |

buds/00.md

@@ -0,0 +1,19 @@
+# BUD-00
+
+## Blossom Upgrade Documents
+
+`draft` `mandatory`
+
+This document details the common language for all following BUDs
+
+## Language
+
+All occurences of "MUST", "MUST NOT", "SHOULD", "SHOULD NOT" MUST be interpreted as per [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119)
+
+## BUDs
+
+BUDs or "Blossom Upgrade Documents" are short documents that outline an additional requirement or feature that a blossom server MUST or MAY implement.
+
+## Blobs
+
+Blobs are raw binary data addressed by the sha256 hash of the data.

buds/01.md

@@ -11,7 +11,7 @@ _All pubkeys MUST be in hex format_
 Servers MUST set the `Access-Control-Allow-Origin: *` header on all responses to ensure compatibility with applications hosted on other domains.
 
 For [preflight](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#preflighted_requests) (`OPTIONS`) requests,
-servers MUST also set, at minimum, the `Access-Control-Allow-Headers: Authorization, *` and `Access-Control-Allow-Methods: GET, PUT,
+servers MUST also set, at minimum, the `Access-Control-Allow-Headers: Authorization, *` and `Access-Control-Allow-Methods: GET, HEAD, PUT,
 DELETE` headers.
 
 The header `Access-Control-Max-Age: 86400` MAY be set to cache the results of a preflight request for 24 hours.
@@ -28,7 +28,7 @@ Authorization events must be generic and must NOT be scoped to specific servers.
 
 Events MUST be kind `24242` and have a `t` tag with a verb of `get`, `upload`, `list`, or `delete`
 
-Events MUST have the `content` set to a human readable string explaining to the user what the events inteded use is. For example `Upload Blob`, `Delete dog-picture.png`, `List Images`, etc
+Events MUST have the `content` set to a human readable string explaining to the user what the events intended use is. For example `Upload Blob`, `Delete dog-picture.png`, `List Images`, etc
 
 All events MUST have a [NIP-40](https://github.com/nostr-protocol/nips/blob/master/40.md) `expiration` tag set to a unix timestamp at which the event should be considered expired.
 
@@ -75,12 +75,29 @@ All endpoints MUST be served from the root of the domain (eg. the `/upload` endp
 
 ## GET /sha256 - Get Blob
 
-The `GET /<sha256>` endpoint MUST return the contents of the blob with the `Content-Type` header set to the appropriate MIME-type
+The `GET /<sha256>` endpoint MUST return the contents of the blob in the response body. the `Content-Type` header SHOULD beset to the appropriate MIME-type
 
 The endpoint MUST accept an optional file extension in the URL. ie. `.pdf`, `.png`, etc
 
-If the endpoints returns a 301 or 302 redirect it MUST redirect to a URL containing the same sha256 hash as the requested blob.
+Regardless of the file extension, the server MUST return the MIME type of the blob in the `Content-Type` header. If the
-This ensures that if a user was to copy or reuse the redirect URL it would still contain the original sha256 hash
+server does not know the MIME type of the blob, it MUST default to `application/octet-stream`
+
+### Proxying and Redirection (Optional)
+
+If the endpoint returns a redirection 3xx status code such as 307 or 308 ([RFC 9110 section
+15.4](https://datatracker.ietf.org/doc/html/rfc9110#name-redirection-3xx)), it MUST redirect to a URL containing the
+same sha256 hash as the requested blob. This ensures that if a user copies or reuses the redirect URL, it will
+contain the original sha256 hash.
+
+While the final blob may not be served from a Blossom server (e.g. CDN, IPFS, object storage, etc.), the destination
+server MUST set the `Access-Control-Allow-Origin: *` header on the response to allow cross-origin requests, as well as
+the `Content-Type` and `Content-Length` headers to ensure the blob can be correctly displayed by clients. Two ways to
+guarantee this are:
+
+1. Proxying the blob through the Blossom server, allowing it to override headers such as `Content-Type`.
+2. Manipulating the redirect URL to include a file extension that matches the blob type, such as `.pdf`, `.png`, etc. If
+the server is unable to determine the MIME type of the blob, it MUST default to `application/octet-stream` and MAY
+include a file extension in the URL that reflects the blob type (e.g. `.bin`, `.dat`, etc.).
 
 ### Get Authorization (optional)
 
@@ -131,7 +148,10 @@ Example event for retrieving multiple blobs from single server:
 
 ## HEAD /sha256 - Has Blob
 
-The `HEAD /<sha256>` endpoint MUST respond with either a `200` or `404` status code
+The `HEAD /<sha256>` endpoint SHOULD be identical to the `GET /<sha256>` endpoint except that it MUST NOT return the
+blob in the reponse body per [RFC 7231](https://www.rfc-editor.org/rfc/rfc7231#section-4.3.2)
+
+The endpoint MUST respond with the same `Content-Type` and `Content-Length` headers as the `GET /<sha256>` endpoint.
 
 The endpoint MUST accept an optional file extension in the URL similar to the `GET /<sha256>` endpoint. ie. `.pdf`, `.png`, etc
 

buds/02.md

@@ -15,7 +15,7 @@ A blob descriptor is a JSON object containing `url`, `sha256`, `size`, `type`, a
 - `url` A publicly accessible URL to the [BUD-01](./01.md#get-sha256---get-blob) `GET /<sha256>` endpoint with a file extension
 - `sha256` The sha256 hash of the blob
 - `size` The size of the blob in bytes
-- `type` (optional) The MIME type of the blob
+- `type` The MIME type of the blob (falling back to `application/octet-stream` if unknown)
 - `uploaded` The unix timestamp of when the blob was uploaded to the server
 
 Servers MUST include a file extension in the URL in the `url` field to allow clients to easily embed the URL in social posts or other content
@@ -44,6 +44,13 @@ The endpoint MUST return a [Blob Descriptor](#blob-descriptor) if the upload was
 
 Servers MAY reject an upload for any reason and SHOULD respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection
 
+### File extension normalization (Optional)
+
+When storing blobs, servers MAY normalise the file extension to a standard format (e.g. `.pdf`, `.png`, etc.) based on
+the MIME type of the blob. This can be especially useful when the `GET /<sha256>` endpoint is redirected to an external
+URL (see the [proxying and redirection section from BUD-01](./01.md#proxying-and-redirection-optional)), as external
+servers may rely on the file extension to serve the blob correctly.
+
 ### Upload Authorization (Optional)
 
 Servers MAY accept an authorization event when uploading blobs and SHOULD perform additional checks

buds/03.md

@@ -68,7 +68,7 @@ Take the following event as an example
 
 Once the client discovers that the URL `https://cdn.broken-domain.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf` is no longer available. It can perform the following steps to find the blob:
 
-1. Get the SHA256 has from the URL
+1. Get the SHA256 hash from the URL
 2. Look for the authors server list `kind:10063`
 3. If found, Attempt to retrieve the blob from each `server` listed started with the first
 4. If not found, the client MAY fallback to using a well-known popular blossom server to retrieve the blob

buds/04.md

@@ -8,7 +8,7 @@ Defines the `/mirror` endpoint
 
 ## PUT /mirror - Mirror Blob
 
-A server may expose a `PUT /mirror` endpoint to allow users to copy a blob from a URL instead of uploading it
+A server MAY expose a `PUT /mirror` endpoint to allow users to copy a blob from a URL instead of uploading it
 
 Clients MUST pass the URL of the remote blob as a stringified JSON object in the request body
 
@@ -19,23 +19,28 @@ Clients MUST pass the URL of the remote blob as a stringified JSON object in the
 }
 ```
 
-Clients MAY set the `Authorization` header to an upload authorization event defined in [BUD-02](./02.md#upload-authorization-required). When using authorization, the event MUST be of type "upload".
+Clients MAY set the `Authorization` header to an upload authorization event defined in [BUD-02](./02.md#upload-authorization-optional). When using authorization, the event MUST be of type "upload".
 
 The `/mirror` endpoint MUST download the blob from the specified URL and verify that there is at least one `x` tag in the authorization event matching the sha256 hash of the download blob
 
-**Multiple `x` tags in the authorization event MUST NOT be interpreted as the user requesting a bulk mirror.**
+**Multiple `x` tags in the authorization event MUST NOT be interpreted as the user requesting to mirror multiple blobs.**
 
-The endpoint MUST return a [Blob Descriptor](#blob-descriptor) if the mirroring was successful or an error object if it was not
+The endpoint MUST return a [Blob Descriptor](#blob-descriptor) and a `2xx` status code if the mirroring was successful
+or a `4xx` status code and error message if it was not.
 
-Servers should re-use the `Content-Type` header returned from the URL to discover the mime type of the blob. if none is returned it may use the file extension in the URL
+The destination server SHOULD use the `Content-Type` header returned from the origin server to infer the mime type of
+the blob. If the `Content-Type` header is not present the destination server SHOULD attempt to detect the `Content-Type`
+from the blob contents and file extension, falling back to `application/octet-stream` if it cannot determine the type.
 
-Servers MAY reject a mirror request for any reason and should respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection
+Servers MAY use the `Content-Length` header to determine the size of the blob.
+
+Servers MAY reject a mirror request for any reason and MUST respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection.
 
 ## Example Flow
 
-1. Client signs authorization event and uploads blob to Server A
+1. Client signs an `upload` authorization event and uploads blob to Server A
-1. Server A returns blob descriptor with `url`
+1. Server A returns a [Blob Descriptor](./02.md#blob-descriptor) with the `url`
-1. Client sends the `url` to Server B `/mirror` using the original authorization event
+1. Client sends the `url` to Server B `/mirror` using the original `upload` authorization event
-1. Server B downloads blob from Server A using the url
+1. Server B downloads the blob from Server A using the `url`
-1. Server B verifies downloaded blob hash matches `x` tag in authorization event
+1. Server B verifies the downloaded blob hash matches the `x` tag in the authorization event
-1. Server B returns [Blob Descriptor](./02.md#blob-descriptor)
+1. Server B returns a [Blob Descriptor](./02.md#blob-descriptor)

buds/05.md

@@ -20,7 +20,7 @@ Servers MAY reject media uploads for any reason and should respond with the appr
 
 Servers MAY require a `media` [authorization event](./02.md#upload-authorization-required) to identify the uploader
 
-If a server requires a `media` authorization event it MUST preform the following checks
+If a server requires a `media` authorization event it MUST perform the following checks
 
 1. The `t` tag MUST be set to `media`
 2. MUST contain at least one `x` tag matching the sha256 hash of the body of the request
@@ -33,7 +33,7 @@ Servers MUST respond to `HEAD` requests on the `/media` endpoint in a similar wa
 
 This endpoint is intentionally limited to optimizing a single blob with the goal of making it easier to distribute
 
-How the blob is optimized is the sole respirability of the server and the client should have no say in what optimization process is used
+How the blob is optimized is the sole responsibility of the server and the client should have no say in what optimization process is used
 
 The goal of this endpoint is to provide a simple "trusted" optimization endpoint clients can use to optimize media for distribution
 

buds/06.md

@@ -8,7 +8,7 @@ Defines how clients can verify if the upload can be completed before sending the
 
 ## HEAD /upload - Upload requirements
 
-The `HEAD /upload` endpoint `MUST` use the `X-SHA-256`, `X-Content-Type` and `X-Content-Length` headers sent by client to get the SHA-256 hash, MIME type and size of the blob that will be uploaded, returning a HTTP status code and a custom header `X-Reason` to indicate some human readable message about the upload requirements.
+The `HEAD /upload` endpoint MUST use the `X-SHA-256`, `X-Content-Type` and `X-Content-Length` headers sent by client to get the SHA-256 hash, MIME type and size of the blob that will be uploaded, returning a HTTP status code and a custom header `X-Reason` to indicate some human readable message about the upload requirements.
 
 ### Headers
 
@@ -38,7 +38,7 @@ Example response from the server if the upload can be done:
 HTTP/1.1 200 OK
 ```
 
-If the upload cannot proceed, the server `MUST` return an appropriate `4xx` HTTP status code and a custom header `X-Reason` with a human readable error message.
+If the upload cannot proceed, the server MUST return an appropriate `4xx` HTTP status code and a custom header `X-Reason` with a human readable error message.
 
 Some examples of error messages:
 

buds/07.md

@@ -0,0 +1,105 @@
+BUD-07
+======
+
+Paid upload and download
+---------------
+
+`draft` `optional`
+
+Payment requirements for blob storage.
+
+## Payment Required
+
+Some servers MAY require payment for uploads, downloads, or any other endpoint. In such cases, these endpoints MUST return a **402 Payment Required** status code.
+
+Some endpoints a server may require payment for:
+
+- [`HEAD /upload`](./06.md) to signal that payment is required for the `PUT` request ( if [BUD-06](./06.md) is supported )
+- [`PUT /upload`](./02.md#put-upload---upload-blob) to require payment for uploads
+- [`HEAD /<sha256>`](./01.md#head-sha256---has-blob) to signal that payment is required for the `GET` request
+- [`GET /<sha256>`](./01.md#get-sha256---get-blob) to require payment for downloads ( maybe charge by MB downloaded? )
+- [`HEAD /media`](./05.md) and [`PUT /upload`](./05.md) to require payment for media optimizations ( if [BUD-06](./06.md) is supported )
+
+When payment is required, the server MUST include one or more `X-{payment_method}` header(s), each corresponding to a supported payment method.
+
+## Server headers
+
+The 402 status code and `X-{payment_method}` header is used by the server to inform the client that a payment is required for the requested operation. The server MUST provide specific headers for each supported payment method.
+
+Supported payment methods:
+
+- `X-Cashu`: Payment details for the cashu payment method, adhering to the [NUT-24](https://github.com/cashubtc/nuts/blob/main/24.md) standard.
+- `X-Lightning`: Payment details for the lightning payment method, adhering to the [BOLT-11](https://github.com/lightning/bolts/blob/master/11-payment-encoding.md) standard.
+
+If a server supports multiple payment methods, it MAY send multiple `X-{payment_method}` headers in the same response.
+
+Schema:
+
+```http
+HTTP/1.1 402 Payment Required
+X-{payment_method}: "<encoded_payload_according_to_{payment_method}_spec>"
+```
+
+### `X-Cashu` Header
+
+When using the X-Cashu header, the server MUST adhere to the [NUT-24](https://github.com/cashubtc/nuts/blob/main/24.md) standard.
+
+Example for cashu:
+
+```http	
+HTTP/1.1 402 Payment Required
+X-Cashu: creqApWF0gaNhdGVub3N0cmFheKlucHJvZmlsZTFxeTI4d3VtbjhnaGo3dW45ZDNzaGp0bnl2OWtoMnVld2Q5aHN6OW1od2RlbjV0ZTB3ZmprY2N0ZTljdXJ4dmVuOWVlaHFjdHJ2NWhzenJ0aHdkZW41dGUwZGVoaHh0bnZkYWtxcWd5ZGFxeTdjdXJrNDM5eWtwdGt5c3Y3dWRoZGh1NjhzdWNtMjk1YWtxZWZkZWhrZjBkNDk1Y3d1bmw1YWeBgmFuYjE3YWloYjdhOTAxNzZhYQphdWNzYXRhbYF4Imh0dHBzOi8vbm9mZWVzLnRlc3RudXQuY2FzaHUuc3BhY2U
+```
+
+### `X-Lightning` Header
+
+When using the X-Lightning header, the server MUST adhere to the [BOLT-11](https://github.com/lightning/bolts/blob/master/11-payment-encoding.md) standard.
+Example for lightning:
+
+```http
+HTTP/1.1 402 Payment Required
+X-Lightning: lnbc30n1pnnmw3lpp57727jjq8zxctahfavqacymellq56l70f7lwfkmhxfjva6dgul2zqhp5w48l28v60yvythn6qvnpq0lez54422a042yaw4kq8arvd68a6n7qcqzzsxqyz5vqsp5sqezejdfaxx5hge83tf59a50h6gagwah59fjn9mw2d5mn278jkys9qxpqysgqt2q2lhjl9kgfaqz864mhlsspftzdyr642lf3zdt6ljqj6wmathdhtgcn0e6f4ym34jl0qkt6gwnllygvzkhdlpq64c6yv3rta2hyzlqp8k28pz
+```
+
+### Client implementation
+
+Clients MUST parse and validate the `X-{payment_method}` header received from the server. The client SHOULD provide a way for the user to complete the payment and retry the request using the same `X-{payment_method}` header.
+
+The client MUST provide the payment proof when re-trying the request using the same `X-{payment_method}` header that was chosen. The payment proof MUST align with the payment method specification:
+
+- For cashu the payment proof should be a serialized `cashuB` token in the `X-Cashu` header according to [NUT-24](https://github.com/cashubtc/nuts/blob/main/24.md#client-payment).
+- For lightning the payment proof should be the preimage of the payment request according to [BOLT-11](https://github.com/lightning/bolts/blob/master/11-payment-encoding.md).
+
+Schema:
+  
+```http
+X-{payment_method}: "<encoded_payment_proof_according_to_{payment_method}_spec>"
+```
+
+Example for Cashu:
+
+```http
+X-Cashu: cashuBo2F0gqJhaUgA_9SLj17PgGFwgaNhYQFhc3hAYWNjMTI0MzVlN2I4NDg0YzNjZjE4NTAxNDkyMThhZjkwZjcxNmE1MmJmNGE1ZWQzNDdlNDhlY2MxM2Y3NzM4OGFjWCECRFODGd5IXVW
+```
+
+Example for Lightning:
+
+```http
+X-Lightning: 966fcb8f153339372f9a187f725384ff4ceae0047c25b9ce607488d7c7e93bba
+```
+
+**Special Note on HEAD Requests**
+
+The HEAD endpoints are only used to retrieve blob or server information. They MUST NOT be retried with payment proof. Instead, clients should complete the payment and proceed with the `PUT` or `GET` request.
+
+### Error handling
+
+If the client fails to provide the payment proof (expired invoice, invalid token, etc.) the server MUST respond with **400 Bad request** status code and include a `X-Reason` header with a human-readable message. The client SHOULD inform the user about the error and provide a way to retry the request.
+
+### Extending with Future Payment Methods
+
+To support future payment methods (e.g., other Layer 2 solutions), the specification allows the addition of new X-{payment_method} headers. Each new method MUST adhere to the following:
+
+New methods MUST use a unique `X-{payment_method}` header containing the specific payment details.
+
+New methods MUST adhere their own specification, which MUST be publicly available and linked in the header.

buds/09.md

@@ -6,7 +6,6 @@
 
 This bud defines a new endpoint for clients and users to report blobs to servers.
 
-
 ### PUT /report - reporting a blob
 
 The request body MUST be a signed [NIP-56](https://github.com/nostr-protocol/nips/blob/master/56.md) report event with one or more `x` tags containing the hashes of the blobs being reported.
@@ -35,7 +34,7 @@ The clients can show a blob report button on posts or in blob details. Or its RE
 
 ### Server behavior
 
-The servers MAY keep the reports somewhere for operators to check and take action on them. they MAY use a list of trusted people or moderators to directly take action on blob without operator request. 
+The servers MAY keep the reports somewhere for operators to check and take action on them. they MAY use a list of trusted people or moderators to directly take action on blob without operator request.
 
 Servers MAY consider removed blobs sha256 as blocked to prevent rewrite.
-Servers SHOULD advertise a route or landing page to provide their rules and term of service which affects the report process.
+Servers SHOULD advertise a route or landing page to provide their rules and terms of service which affects the report process.