diff --git a/README.md b/README.md index 49c8586..7108f16 100644 --- a/README.md +++ b/README.md @@ -4,13 +4,30 @@ Blossom uses [nostr](https://github.com/nostr-protocol/nostr) public / private k ## What is it? -Blossom is a spec for a set of HTTP endpoints that allow users to store blobs of data on publicly accessible servers +Blossom is a specification for a set of HTTP endpoints that allow users to store blobs of data on publicly accessible servers ## What are blobs Blobs are packs of binary data addressed by their sha256 hash -## How does it work? +## Protocol specification (BUDs) + +BUDs or **Blossom Upgrade Documents** are short documents that outline an additional feature that a blossom server may implement. + +## BUDs + +- [BUD-00: Blossom Upgrade Documents](./buds/00.md) +- [BUD-01: Server requirements and blob retrieval](./buds/01.md) +- [BUD-02: Blob upload and management](./buds/02.md) +- [BUD-03: User Server List](./buds/03.md) +- [BUD-04: Mirroring blobs](./buds/04.md) +- [BUD-05: Media optimization](./buds/05.md) +- [BUD-06: Upload requirements](./buds/06.md) +- [BUD-07: Payment required](./buds/07.md) +- [BUD-08: Nostr File Metadata Tags](./buds/08.md) +- [BUD-09: Blob Report](./buds/09.md) + +## Endpoints Blossom Servers expose a few endpoints for managing blobs @@ -30,23 +47,7 @@ Blossom Servers expose a few endpoints for managing blobs - `HEAD /media` [BUD-05](./buds/05.md#head-media) - `PUT /media` [BUD-05](./buds/05.md#put-media) - `Authentication`: Signed [nostr event](./buds/05.md#upload-authorization) - -## Protocol specification (BUDs) - -BUDs stand for **Blossom Upgrade Documents**. - -See the [BUDs](./buds) folder and specifically [BUD-01](./buds/01.md) and [BUD-02](./buds/02.md) for a detailed explanation of the endpoints - -## BUDs - -- [BUD-01: Server requirements and blob retrieval](./buds/01.md) -- [BUD-02: Blob upload and management](./buds/02.md) -- [BUD-03: User Server List](./buds/03.md) -- [BUD-04: Mirroring blobs](./buds/04.md) -- [BUD-05: Media optimization](./buds/05.md) -- [BUD-06: Upload requirements](./buds/06.md) -- [BUD-07: Paid storage](./buds/07.md) -- [BUD-08: Nostr File Metadata Tags](./buds/08.md) +- `PUT /report` [BUD-09](./buds/09.md) ## Event kinds diff --git a/buds/00.md b/buds/00.md new file mode 100644 index 0000000..fc50c19 --- /dev/null +++ b/buds/00.md @@ -0,0 +1,19 @@ +# BUD-00 + +## Blossom Upgrade Documents + +`draft` `mandatory` + +This document details the common language for all following BUDs + +## Language + +All occurences of "MUST", "MUST NOT", "SHOULD", "SHOULD NOT" MUST be interpreted as per [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119) + +## BUDs + +BUDs or "Blossom Upgrade Documents" are short documents that outline an additional requirement or feature that a blossom server MUST or MAY implement. + +## Blobs + +Blobs are raw binary data addressed by the sha256 hash of the data. diff --git a/buds/01.md b/buds/01.md index a087c94..94243bf 100644 --- a/buds/01.md +++ b/buds/01.md @@ -11,7 +11,7 @@ _All pubkeys MUST be in hex format_ Servers MUST set the `Access-Control-Allow-Origin: *` header on all responses to ensure compatibility with applications hosted on other domains. For [preflight](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#preflighted_requests) (`OPTIONS`) requests, -servers MUST also set, at minimum, the `Access-Control-Allow-Headers: Authorization, *` and `Access-Control-Allow-Methods: GET, PUT, +servers MUST also set, at minimum, the `Access-Control-Allow-Headers: Authorization, *` and `Access-Control-Allow-Methods: GET, HEAD, PUT, DELETE` headers. The header `Access-Control-Max-Age: 86400` MAY be set to cache the results of a preflight request for 24 hours. @@ -28,7 +28,7 @@ Authorization events must be generic and must NOT be scoped to specific servers. Events MUST be kind `24242` and have a `t` tag with a verb of `get`, `upload`, `list`, or `delete` -Events MUST have the `content` set to a human readable string explaining to the user what the events inteded use is. For example `Upload Blob`, `Delete dog-picture.png`, `List Images`, etc +Events MUST have the `content` set to a human readable string explaining to the user what the events intended use is. For example `Upload Blob`, `Delete dog-picture.png`, `List Images`, etc All events MUST have a [NIP-40](https://github.com/nostr-protocol/nips/blob/master/40.md) `expiration` tag set to a unix timestamp at which the event should be considered expired. @@ -36,7 +36,7 @@ Authorization events MAY have multiple `x` tags for endpoints that require a sha Example event: -```json +```jsonc { "id": "bb653c815da18c089f3124b41c4b5ec072a40b87ca0f50bbbc6ecde9aca442eb", "pubkey": "b53185b9f27962ebdf76b8a9b0a84cd8b27f9f3d4abd59f715788a3bf9e7f75e", @@ -45,7 +45,7 @@ Example event: "created_at": 1708773959, "tags": [ ["t", "upload"], - // Authorization events MAY have multiple "x" tags + // Authorization events MAY have multiple "x" tags. ["x", "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553"], ["expiration", "1708858680"] ], @@ -71,22 +71,39 @@ Authorization: Nostr eyJpZCI6IjhlY2JkY2RkNTMyOTIwMDEwNTUyNGExNDI4NzkxMzg4MWIzOWQ ## Endpoints -All endpoints MUST be served from the root path (eg. `https://cdn.example.com/upload`, etc). This allows clients to talk to servers interchangeably when uploading or retrieving blobs +All endpoints MUST be served from the root of the domain (eg. the `/upload` endpoint MUST be accessible from `https://cdn.example.com/upload`, etc). This allows clients to talk to servers interchangeably when uploading or retrieving blobs ## GET /sha256 - Get Blob -The `GET /` endpoint MUST return the contents of the blob with the `Content-Type` header set to the appropriate MIME type +The `GET /` endpoint MUST return the contents of the blob in the response body. the `Content-Type` header SHOULD beset to the appropriate MIME-type The endpoint MUST accept an optional file extension in the URL. ie. `.pdf`, `.png`, etc -If the endpoints returns a 301 or 302 redirect it MUST redirect to a URL containing the same sha256 hash as requested blob. -This ensures that if a user was to copy or reuse the redirect URL it would still contain the original sha256 hash +Regardless of the file extension, the server MUST return the MIME type of the blob in the `Content-Type` header. If the +server does not know the MIME type of the blob, it MUST default to `application/octet-stream` + +### Proxying and Redirection (Optional) + +If the endpoint returns a redirection 3xx status code such as 307 or 308 ([RFC 9110 section +15.4](https://datatracker.ietf.org/doc/html/rfc9110#name-redirection-3xx)), it MUST redirect to a URL containing the +same sha256 hash as the requested blob. This ensures that if a user copies or reuses the redirect URL, it will +contain the original sha256 hash. + +While the final blob may not be served from a Blossom server (e.g. CDN, IPFS, object storage, etc.), the destination +server MUST set the `Access-Control-Allow-Origin: *` header on the response to allow cross-origin requests, as well as +the `Content-Type` and `Content-Length` headers to ensure the blob can be correctly displayed by clients. Two ways to +guarantee this are: + +1. Proxying the blob through the Blossom server, allowing it to override headers such as `Content-Type`. +2. Manipulating the redirect URL to include a file extension that matches the blob type, such as `.pdf`, `.png`, etc. If +the server is unable to determine the MIME type of the blob, it MUST default to `application/octet-stream` and MAY +include a file extension in the URL that reflects the blob type (e.g. `.bin`, `.dat`, etc.). ### Get Authorization (optional) The server may optionally require authorization when retrieving blobs from the `GET /` endpoint -In this case the server MUST perform additional checks on the authorization event +In this case, the server MUST perform additional checks on the authorization event 1. A `t` tag MUST be present and set to `get` 2. The event MUST contain either a `server` tag containing the full URL to the server or MUST contain at least one `x` tag matching the sha256 hash of the blob being retrieved @@ -131,7 +148,10 @@ Example event for retrieving multiple blobs from single server: ## HEAD /sha256 - Has Blob -The `HEAD /` endpoint MUST respond with either a `200` or `404` status code +The `HEAD /` endpoint SHOULD be identical to the `GET /` endpoint except that it MUST NOT return the +blob in the reponse body per [RFC 7231](https://www.rfc-editor.org/rfc/rfc7231#section-4.3.2) + +The endpoint MUST respond with the same `Content-Type` and `Content-Length` headers as the `GET /` endpoint. The endpoint MUST accept an optional file extension in the URL similar to the `GET /` endpoint. ie. `.pdf`, `.png`, etc diff --git a/buds/02.md b/buds/02.md index 230ecad..fa073b0 100644 --- a/buds/02.md +++ b/buds/02.md @@ -12,12 +12,14 @@ Defines the `/upload`, `/list` and `DELETE /` endpoints A blob descriptor is a JSON object containing `url`, `sha256`, `size`, `type`, and `uploaded` fields -- `url` A publicly accessible URL to the [BUD-01](./01.md#get-sha256---get-blob) `GET /` endpoint (optionally with a file extension) +- `url` A publicly accessible URL to the [BUD-01](./01.md#get-sha256---get-blob) `GET /` endpoint with a file extension - `sha256` The sha256 hash of the blob - `size` The size of the blob in bytes -- `type` (optional) The MIME type of the blob +- `type` The MIME type of the blob (falling back to `application/octet-stream` if unknown) - `uploaded` The unix timestamp of when the blob was uploaded to the server +Servers MUST include a file extension in the URL in the `url` field to allow clients to easily embed the URL in social posts or other content + Servers MAY include additional fields in the descriptor like `magnet`, `infohash`, or `ipfs` depending on other protocols they support Example: @@ -40,7 +42,15 @@ The endpoint MUST NOT modify the blob in any way and should return the exact sam The endpoint MUST return a [Blob Descriptor](#blob-descriptor) if the upload was successful or an error object if it was not -Servers MAY reject an upload for any reason and should respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection +Servers MAY reject an upload for any reason and should respond with the appropriate HTTP `4xx` status code and an error +message explaining the reason for the rejection + +### File extension normalization (Optional) + +When storing blobs, servers MAY normalise the file extension to a standard format (e.g. `.pdf`, `.png`, etc.) based on +the MIME type of the blob. This can be especially useful when the `GET /` endpoint is redirected to an external +URL (see the [proxying and redirection section from BUD-01](./01.md#proxying-and-redirection-optional)), as external +servers may rely on the file extension to serve the blob correctly. ### Upload Authorization (Optional) @@ -69,7 +79,7 @@ Example Authorization event: ## GET /list/pubkey - List Blobs (Optional) -The `/list/` endpoint MUST return a JSON array of [Blob Descriptor](#blob-descriptor) that where uploaded by the specified pubkey +The `/list/` endpoint MUST return a JSON array of [Blob Descriptor](#blob-descriptor) that were uploaded by the specified pubkey The endpoint MUST support a `since` and `until` query parameter to limit the returned blobs by their `uploaded` date diff --git a/buds/03.md b/buds/03.md index c6a1730..cebc0fb 100644 --- a/buds/03.md +++ b/buds/03.md @@ -68,7 +68,7 @@ Take the following event as an example Once the client discovers that the URL `https://cdn.broken-domain.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf` is no longer available. It can perform the following steps to find the blob: -1. Get the SHA256 has from the URL +1. Get the SHA256 hash from the URL 2. Look for the authors server list `kind:10063` 3. If found, Attempt to retrieve the blob from each `server` listed started with the first 4. If not found, the client MAY fallback to using a well-known popular blossom server to retrieve the blob diff --git a/buds/04.md b/buds/04.md index 92559c0..67b013b 100644 --- a/buds/04.md +++ b/buds/04.md @@ -8,34 +8,39 @@ Defines the `/mirror` endpoint ## PUT /mirror - Mirror Blob -A server may expose a `PUT /mirror` endpoint to allow users to copy a blob from a URL instead of uploading it +A server MAY expose a `PUT /mirror` endpoint to allow users to copy a blob from a URL instead of uploading it Clients MUST pass the URL of the remote blob as a stringified JSON object in the request body -```json -// request body +```jsonc +// request body... { "url": "https://cdn.satellite.earth/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf" } ``` -Clients MUST set the `Authorization` header to an upload authorization event defined in [BUD-02](./02.md#upload-authorization-required) +Clients MAY set the `Authorization` header to an upload authorization event defined in [BUD-02](./02.md#upload-authorization-optional). When using authorization, the event MUST be of type "upload". The `/mirror` endpoint MUST download the blob from the specified URL and verify that there is at least one `x` tag in the authorization event matching the sha256 hash of the download blob -**Multiple `x` tags MUST NOT be interpreted as the user requesting a bulk mirror.** +**Multiple `x` tags in the authorization event MUST NOT be interpreted as the user requesting to mirror multiple blobs.** -The endpoint MUST return a [Blob Descriptor](#blob-descriptor) if the mirroring was successful or an error object if it was not +The endpoint MUST return a [Blob Descriptor](#blob-descriptor) and a `2xx` status code if the mirroring was successful +or a `4xx` status code and error message if it was not. -Servers should re-use the `Content-Type` header returned from the URL to discover the mime type of the blob. if none is returned it may use the file extension in the URL +The destination server SHOULD use the `Content-Type` header returned from the origin server to infer the mime type of +the blob. If the `Content-Type` header is not present the destination server SHOULD attempt to detect the `Content-Type` +from the blob contents and file extension, falling back to `application/octet-stream` if it cannot determine the type. -Servers MAY reject a mirror request for any reason and should respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection +Servers MAY use the `Content-Length` header to determine the size of the blob. + +Servers MAY reject a mirror request for any reason and MUST respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection. ## Example Flow -1. Client signs authorization event and uploads blob to Server A -1. Server A returns blob descriptor with `url` -1. Client sends the `url` to Server B `/mirror` using the original authorization event -1. Server B downloads blob from Server A using the url -1. Server B verifies downloaded blob hash matches `x` tag in authorization event -1. Server B returns [Blob Descriptor](./02.md#blob-descriptor) +1. Client signs an `upload` authorization event and uploads blob to Server A +1. Server A returns a [Blob Descriptor](./02.md#blob-descriptor) with the `url` +1. Client sends the `url` to Server B `/mirror` using the original `upload` authorization event +1. Server B downloads the blob from Server A using the `url` +1. Server B verifies the downloaded blob hash matches the `x` tag in the authorization event +1. Server B returns a [Blob Descriptor](./02.md#blob-descriptor) diff --git a/buds/05.md b/buds/05.md index 2ad9851..a7165eb 100644 --- a/buds/05.md +++ b/buds/05.md @@ -18,9 +18,12 @@ Servers MAY reject media uploads for any reason and should respond with the appr ### Upload Authorization -Servers MAY require an `upload` [authorization event](./02.md#upload-authorization-required) to identify the uploader +Servers MAY require a `media` [authorization event](./02.md#upload-authorization-required) to identify the uploader -If a server requires an `upload` authorization event it MUST preform all the checks outlined in the [`/upload`](./02.md#upload-authorization-required) endpoint +If a server requires a `media` authorization event it MUST perform the following checks + +1. The `t` tag MUST be set to `media` +2. MUST contain at least one `x` tag matching the sha256 hash of the body of the request ## HEAD /media @@ -30,7 +33,7 @@ Servers MUST respond to `HEAD` requests on the `/media` endpoint in a similar wa This endpoint is intentionally limited to optimizing a single blob with the goal of making it easier to distribute -How the blob is optimized is the sole respirability of the server and the client should have no say in what optimization process is used +How the blob is optimized is the sole responsibility of the server and the client should have no say in what optimization process is used The goal of this endpoint is to provide a simple "trusted" optimization endpoint clients can use to optimize media for distribution @@ -40,6 +43,6 @@ If a longer optimization or transformation process is needed, or if the client n Clients MAY let a user selected a "trusted processing" server for uploading images or short videos -Once a server has been selected, the client can upload the original media to the `/media` endpoint of the trusted server and get the optimized blob back +Once a server has been selected, the client uploads the original media to the `/media` endpoint of the trusted server and get the optimized blob back -Then optionally the client can ask the user to sign another `upload` authorization event for the new optimized blob and call the `/mirror` endpoint on other servers to distribute the blob +Then the client can ask the user to sign another `upload` authorization event for the new optimized blob and call the `/mirror` endpoint on other servers to distribute the blob diff --git a/buds/06.md b/buds/06.md index 739404b..f3b9cbe 100644 --- a/buds/06.md +++ b/buds/06.md @@ -8,7 +8,7 @@ Defines how clients can verify if the upload can be completed before sending the ## HEAD /upload - Upload requirements -The `HEAD /upload` endpoint `MUST` use the `X-SHA-256`, `X-Content-Type` and `X-Content-Length` headers sent by client to get the SHA-256 hash, MIME type and size of the blob that will be uploaded, returning a HTTP status code and a custom header `X-Reason` to indicate some human readable message about the upload requirements. +The `HEAD /upload` endpoint MUST use the `X-SHA-256`, `X-Content-Type` and `X-Content-Length` headers sent by client to get the SHA-256 hash, MIME type and size of the blob that will be uploaded, returning a HTTP status code and a custom header `X-Reason` to indicate some human readable message about the upload requirements. ### Headers @@ -38,7 +38,7 @@ Example response from the server if the upload can be done: HTTP/1.1 200 OK ``` -If the upload cannot proceed, the server `MUST` return an appropriate `4xx` HTTP status code and a custom header `X-Reason` with a human readable error message. +If the upload cannot proceed, the server MUST return an appropriate `4xx` HTTP status code and a custom header `X-Reason` with a human readable error message. Some examples of error messages: diff --git a/buds/08.md b/buds/08.md index b5e2ec0..89c1668 100644 --- a/buds/08.md +++ b/buds/08.md @@ -12,7 +12,7 @@ As described in [BUD-02](./02.md#blob-descriptor) servers MAY add any additional Servers MAY return an additional `nip94` field in the [blob descriptor](./02.md#blob-descriptor) from the `/upload` or `/mirror` endpoints -The `nip94` field should contain a JSON object with the keys being the tag names defined in [NIP-94](https://github.com/nostr-protocol/nips/blob/master/94.md) +The `nip94` field should contain a JSON array with KV pairs as defined in [NIP-94](https://github.com/nostr-protocol/nips/blob/master/94.md) An example response would look like: @@ -23,13 +23,13 @@ An example response would look like: "size": 184292, "type": "application/pdf", "uploaded": 1725909682, - "nip94": { - "url": "https://cdn.example.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf", - "m": "application/pdf", - "x": "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553", - "size": 184292, - "magnet": "magnet:?xt=urn:btih:9804c5286a3fb07b2244c968b39bc3cc814313bc&dn=bitcoin.pdf", - "i": "9804c5286a3fb07b2244c968b39bc3cc814313bc" - } + "nip94": [ + ["url", "https://cdn.example.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf"], + ["m", "application/pdf"], + ["x", "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553"], + ["size", "184292"], + ["magnet", "magnet:?xt=urn:btih:9804c5286a3fb07b2244c968b39bc3cc814313bc&dn=bitcoin.pdf"], + ["i", "9804c5286a3fb07b2244c968b39bc3cc814313bc"] + ] } ``` diff --git a/buds/09.md b/buds/09.md new file mode 100644 index 0000000..94c9478 --- /dev/null +++ b/buds/09.md @@ -0,0 +1,40 @@ +# BUD-09 + +## Blob Report + +`draft` `optional` + +This bud defines a new endpoint for clients and users to report blobs to servers. + +### PUT /report - reporting a blob + +The request body MUST be a signed [NIP-56](https://github.com/nostr-protocol/nips/blob/master/56.md) report event with one or more `x` tags containing the hashes of the blobs being reported. + +Example: + +```jsonc +{ + "kind": 1984, + "tags": [ + ["x", "", ""], + ["x", "", ""] + ], + "content": "", + // other fields... +} +``` + +The clients can include `e` or `p` tags to point to the event or the profile that contains this media if they want to make this report event useful for relays as well. + +Server MUST respond to a report request with a success code or a code in the 4xx/5xx range if there was any error. + +### Client behavior + +The clients can show a blob report button on posts or in blob details. Or its RECOMMENDED to merge this with normal nostr report and send it to both relays and blossom server. other clients can receive it from relays and hide or blur reported blob from trusted friends. + +### Server behavior + +The servers MAY keep the reports somewhere for operators to check and take action on them. they MAY use a list of trusted people or moderators to directly take action on blob without operator request. + +Servers MAY consider removed blobs sha256 as blocked to prevent rewrite. +Servers SHOULD advertise a route or landing page to provide their rules and terms of service which affects the report process.