laantungir/blossom
1
0
Fork 0
mirror of https://github.com/hzrd149/blossom.git synced 2025-12-08 14:58:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

reorder branches. sync with source repo

Browse Source
This commit is contained in:
Quentin
2024-08-22 09:51:23 +02:00
parent ba4eb9b8f9
commit 3b6de30719
4 changed files with 86 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
buds/01.md
View File

@@ -60,7 +60,7 @@ Authorization: Nostr eyJpZCI6IjhlY2JkY2RkNTMyOTIwMDEwNTUyNGExNDI4NzkxMzg4MWIzOWQ
## Endpoints ## Endpoints
All endpoints MUST be served from the root path (eg. `https://cdn.example.com/upload`, etc). This allows clients to talk to servers interchangeably when uploading or reteriving blobs All endpoints MUST be served from the root path (eg. `https://cdn.example.com/upload`, etc). This allows clients to talk to servers interchangeably when uploading or retrieving blobs
## Error Responses ## Error Responses
@@ -92,7 +92,7 @@ This ensures that if a user was to copy or reuse the redirect URL it would still
### Get Authorization (optional) ### Get Authorization (optional)
The server may optionally require authorization when reteriving blobs from the `GET /<sha256>` endpoint The server may optionally require authorization when retrieving blobs from the `GET /<sha256>` endpoint
In this case the server MUST perform additional checks on the authorization event In this case the server MUST perform additional checks on the authorization event
@@ -101,7 +101,7 @@ In this case the server MUST perform additional checks on the authorization even
If the client did not send an `Authorization` header the server must respond with the appropriate HTTP status code `401` (Unauthorized) If the client did not send an `Authorization` header the server must respond with the appropriate HTTP status code `401` (Unauthorized)
Example event for retreiving a single blob: Example event for retrieving a single blob:
```json ```json
{ {
@@ -119,7 +119,7 @@ Example event for retreiving a single blob:
} }
``` ```
Example event for retreiving multiple blobs from single server: Example event for retrieving multiple blobs from single server:
```json ```json
{ {

30
buds/02.md
View File

@@ -8,7 +8,7 @@ Blob upload and management
_All pubkeys MUST be in hex format_ _All pubkeys MUST be in hex format_
Defines the `/upload`, `/list` and `DELETE /<sha256>` enpoints Defines the `/upload`, `/list` and `DELETE /<sha256>` endpoints
## Blob Descriptor ## Blob Descriptor
@@ -19,28 +19,6 @@ A blob descriptor is a JSON object containing `url`, `sha256`, `size`, `type`, a
- `size` The size of the blob in bytes - `size` The size of the blob in bytes
- `type` (optional) The MIME type of the blob - `type` (optional) The MIME type of the blob
- `uploaded` The unix timestamp of when the blob was uploaded to the server - `uploaded` The unix timestamp of when the blob was uploaded to the server
- `compressed` (optional) A JSON object containing the `sha256`, `size`, `library`, `version`, and `parameters` JSON object containing the compression parameters `quality` and `mode`. This field is optional and should only be included if the blob was compressed
Blob Descriptor example:
```json
{
"sha256": "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553",
"uploaded": 1708773959,
"size": 123456,
"type": "application/pdf",
"compressed": {
"sha256": "e2e2e1d2b9c04e2a9914c245e3f789d230e3c2d5e3a0f5c6e4b4e3f9c7d4f3e2",
"size": 56789,
"library": "brotli",
"version": "1.0.7",
"parameters": {
"quality": 11,
"mode": "text"
}
}
}
```
Servers may include additional fields in the descriptor like `magnet`, `infohash`, or `ipfs` depending on other protocols they support Servers may include additional fields in the descriptor like `magnet`, `infohash`, or `ipfs` depending on other protocols they support
@@ -83,7 +61,7 @@ Example Authorization event:
The `/list/<pubkey>` endpoint MUST return a JSON array of [Blob Descriptor](#blob-descriptor) that where uploaded by the specified pubkey The `/list/<pubkey>` endpoint MUST return a JSON array of [Blob Descriptor](#blob-descriptor) that where uploaded by the specified pubkey
The endpoint MUST support a `since` and `until` query parameter to limit the returned blobs by thier `uploaded` date The endpoint MUST support a `since` and `until` query parameter to limit the returned blobs by their `uploaded` date
Servers may reject a list for any reason and MUST respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection Servers may reject a list for any reason and MUST respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection
@@ -116,7 +94,7 @@ Example Authorization event:
Servers MUST accept `DELETE` requests to the `/<sha256>` endpoint Servers MUST accept `DELETE` requests to the `/<sha256>` endpoint
Servers may reject a delete request for any reason and should respond with the aproperate HTTP `4xx` status code and an error message explaining the reason for the rejection Servers may reject a delete request for any reason and should respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection
### Delete Authorization (required) ### Delete Authorization (required)
@@ -138,7 +116,7 @@ Example Authorization event:
"created_at": 1708774469, "created_at": 1708774469,
"tags": [ "tags": [
["t", "delete"], ["t", "delete"],
["x", "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553 "], ["x", "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553"],
["expiration", "1708858680"] ["expiration", "1708858680"]
], ],
"sig": "2ba9af680505583e3eb289a1624a08661a2f6fa2e5566a5ee0036333d517f965e0ffba7f5f7a57c2de37e00a2e85fd7999076468e52bdbcfad8abb76b37a94b0" "sig": "2ba9af680505583e3eb289a1624a08661a2f6fa2e5566a5ee0036333d517f965e0ffba7f5f7a57c2de37e00a2e85fd7999076468e52bdbcfad8abb76b37a94b0"

78
buds/03.md Normal file
View File

@@ -0,0 +1,78 @@
BUD-03
======
User Server List
-------------------------
`draft` `optional`
Defines a replaceable event using `kind:10063` to advertise the blossom servers a user uses to host their blobs.
The event MUST include at least one `server` tag containing the full server URL including the `http://` or `https://`.
The order of these tags is important and should be arranged with the users most "reliable" or "trusted" servers being first.
The `.content` field is not used.
```json
{
"id": "e4bee088334cb5d38cff1616e964369c37b6081be997962ab289d6c671975d71",
"pubkey": "781208004e09102d7da3b7345e64fd193cd1bc3fce8fdae6008d77f9cabcd036",
"content": "",
"kind": 10063,
"created_at": 1708774162,
"tags": [
["server", "https://cdn.self.hosted"],
["server", "https://cdn.satellite.earth"]
],
"sig": "cc5efa74f59e80622c77cacf4dd62076bcb7581b45e9acff471e7963a1f4d8b3406adab5ee1ac9673487480e57d20e523428e60ffcc7e7a904ac882cfccfc653"
}
```
## Client Upload Implementation
When uploading blobs clients MUST attempt to upload the blob to at least the first `server` listed in the users server list.
Optionally clients MAY upload the blob to all the servers or mirror the blob to the other servers if they support [BUD-04](./04.md)
This ensures that the blob is available in multiple locations in the case one of the servers goes offline.
## Client Retrieval Implementation
When extracting the SHA256 hash from the URL clients MUST use the last occurrence of a 64 char hex string. This allows clients to extract hashes from blossom URLs and SOME non-blossom URLs.
In all the following examples, the hash `b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553` should be selected
- Blossom URLs
- `https://blossom.example.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf`
- `https://cdn.example.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553`
- Non Blossom URLs
- `https://cdn.example.com/user/ec4425ff5e9446080d2f70440188e3ca5d6da8713db7bdeef73d0ed54d9093f0/media/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf`
- `https://cdn.example.com/media/user-name/documents/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf`
- `http://download.example.com/downloads/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553`
- `http://media.example.com/documents/b1/67/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf`
In the context of nostr events, clients SHOULD use the author's server list when looking for blobs that are no longer available at the original URL.
Take the following event as an example
```json
{
"id": "834185269f4ab72539193105060dbb1c8b2efd702d14481cea345c47beefe6eb",
"pubkey": "ec4425ff5e9446080d2f70440188e3ca5d6da8713db7bdeef73d0ed54d9093f0",
"content": "I've developed a new open source P2P e-cash system called Bitcoin. check it out\nhttps://cdn.broken-domain.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf",
"kind": 1,
"created_at": 1297484820,
"tags": [],
"sig": "bd4bb200bdd5f7ffe5dbc3e539052e27b05d6f9f528e255b1bc4261cc16b8f2ad85c89eef990c5f2eee756ef71b4c571ecf6a88ad12f7338e321dd60c6a903b5"
}
```
Once the client discovers that the URL `https://cdn.broken-domain.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf` is no longer available. It can perform the following steps to find the blob:
1. Get the SHA256 has from the URL
2. Look for the authors server list `kind:10063`
3. If found, Attempt to retrieve the blob from each `server` listed started with the first
3. If not found, the client MAY fallback to using a well-known popular blossom server to retrieve the blob
This ensures clients can quickly find missing blobs using the users list of trusted servers.

97
buds/05.md
View File

@@ -1,97 +0,0 @@
BUD-05
======
Deterministic File Compression
---------------
`draft` `optional`
## Overview
This document outlines the implementation of a deterministic file compression method for Blossom, specifying the metadata that must be stored for each compressed blob and the endpoint for retrieving this metadata.
## Server Adaptation
File storage servers `MUST` store the following metadata for each compressed blob:
- File hash (sha256)
- Uploaded timestamp (uploaded)
- Original file size (size)
- Original file type (type)
- Compressed file hash (compressed.sha256)
- Compressed file size (compressed.size)
- Compression library (compression.library)
- Compression library version (compression.version)
- Compression library parameters (compression.parameters)
Servers `MUST` normalize the blob metadata to ensure that the same compressed blob with the same parameters will always produce the same result, thereby ensuring deterministic compression and anonymizing any potential user data.
## Client adaptation
Clients `MUST` verify the metadata of the compressed blob to ensure that the compressed blob matches the original blob. Clients `MUST` sign and publish a new authorization event with the new blob hash. Clients `MUST` upload the compressed blob to the server using the new authorization event and the 'compress' field set to false or not set.
## GET /metadata - Retrieve blob metadata
The GET /metadata/<sha256> endpoint `MUST` return a [Blob Descriptor](https://github.com/hzrd149/blossom/blob/master/buds/02.md#blob-descriptor) containing the metadata fields for the requested blob or an error object if the blob does not exist.
The endpoint `MUST` accept an optional file extension in the URL. ie. `.pdf`, `.png`, etc
Example response:
```json
{
"url": "cdn.nostrcheck.me/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553",
"sha256": "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553",
"uploaded": 1708773959,
"size": 123456,
"type": "application/pdf",
"compressed": {
"sha256": "e2e2e1d2b9c04e2a9914c245e3f789d230e3c2d5e3a0f5c6e4b4e3f9c7d4f3e2",
"size": 56789,
"library": "brotli",
"version": "1.0.7",
"parameters": {
"quality": 11,
"mode": "text"
}
}
}
```
## Example Flow
1. Client signs authorization event and uploads blob to Server A with 'compress' field set to true
1. Server A compresses the blob using it's preferred library and parameters
1. Server A stores the blob metadata
1. Server A returns [Blob Descriptor](./02.md#blob-descriptor)
1. Client verify the metadata matches the original blob compressing and comparing the blob locally with the same returned parameters
1. Client signs and publishes a new authorization event with the new blob hash and the 'compress' field set to false
1. Client uploads the compressed blob to Server B using the new authorization event and the 'compress' field set to false or not set
1. Server B verifies blob hash metadata matches `x` tag in the new authorization event
1. Server B stores the blob metadata
1. Server B returns [Blob Descriptor](./02.md#blob-descriptor)
(optional using the /mirror endpoint)
1. Client sends the `url` to Server C `/mirror` using the compressed authorization event
1. Server C downloads blob from Server A or B using the url field
1. Server C verifies downloaded blob hash matches `x` tag in authorization event (using sha256 or compressed.sha256)
1. Server C returns [Blob Descriptor](./02.md#blob-descriptor)
## Accepted Libraries, Versions, and Parameters
The following libraries, versions, and parameters are accepted for deterministic compression:
(This is just a DEMO table, the actual table will be updated with the final list of libraries, versions, and parameters)
| Library | Version | Quality Range | Modes |
|---------|---------|---------------|------------------------|
| Brotli | 1.0.7 | 0-11 | text, font, generic |
| Gzip | 1.10 | 0-9 | text, font, generic |
| Zstd | 1.4.5 | 1-22 | text, font, generic |
| Lz4 | 1.9.2 | 1-12 | text, font, generic |