mirror of
https://github.com/hzrd149/blossom.git
synced 2025-12-08 14:58:49 +00:00
reorder branches. sync with source repo
This commit is contained in:
Quentin
@@ -60,7 +60,7 @@ Authorization: Nostr eyJpZCI6IjhlY2JkY2RkNTMyOTIwMDEwNTUyNGExNDI4NzkxMzg4MWIzOWQ
|
||||
|
||||
## Endpoints
|
||||
|
||||
All endpoints MUST be served from the root path (eg. `https://cdn.example.com/upload`, etc). This allows clients to talk to servers interchangeably when uploading or reteriving blobs
|
||||
All endpoints MUST be served from the root path (eg. `https://cdn.example.com/upload`, etc). This allows clients to talk to servers interchangeably when uploading or retrieving blobs
|
||||
|
||||
## Error Responses
|
||||
|
||||
@@ -92,7 +92,7 @@ This ensures that if a user was to copy or reuse the redirect URL it would still
|
||||
|
||||
### Get Authorization (optional)
|
||||
|
||||
The server may optionally require authorization when reteriving blobs from the `GET /<sha256>` endpoint
|
||||
The server may optionally require authorization when retrieving blobs from the `GET /<sha256>` endpoint
|
||||
|
||||
In this case the server MUST perform additional checks on the authorization event
|
||||
|
||||
@@ -101,7 +101,7 @@ In this case the server MUST perform additional checks on the authorization even
|
||||
|
||||
If the client did not send an `Authorization` header the server must respond with the appropriate HTTP status code `401` (Unauthorized)
|
||||
|
||||
Example event for retreiving a single blob:
|
||||
Example event for retrieving a single blob:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -119,7 +119,7 @@ Example event for retreiving a single blob:
|
||||
}
|
||||
```
|
||||
|
||||
Example event for retreiving multiple blobs from single server:
|
||||
Example event for retrieving multiple blobs from single server:
|
||||
|
||||
```json
|
||||
{
|
||||
|
||||
30
buds/02.md
30
buds/02.md
@@ -8,7 +8,7 @@ Blob upload and management
|
||||
|
||||
_All pubkeys MUST be in hex format_
|
||||
|
||||
Defines the `/upload`, `/list` and `DELETE /<sha256>` enpoints
|
||||
Defines the `/upload`, `/list` and `DELETE /<sha256>` endpoints
|
||||
|
||||
## Blob Descriptor
|
||||
|
||||
@@ -19,28 +19,6 @@ A blob descriptor is a JSON object containing `url`, `sha256`, `size`, `type`, a
|
||||
- `size` The size of the blob in bytes
|
||||
- `type` (optional) The MIME type of the blob
|
||||
- `uploaded` The unix timestamp of when the blob was uploaded to the server
|
||||
- `compressed` (optional) A JSON object containing the `sha256`, `size`, `library`, `version`, and `parameters` JSON object containing the compression parameters `quality` and `mode`. This field is optional and should only be included if the blob was compressed
|
||||
|
||||
Blob Descriptor example:
|
||||
```json
|
||||
{
|
||||
"sha256": "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553",
|
||||
"uploaded": 1708773959,
|
||||
"size": 123456,
|
||||
"type": "application/pdf",
|
||||
"compressed": {
|
||||
"sha256": "e2e2e1d2b9c04e2a9914c245e3f789d230e3c2d5e3a0f5c6e4b4e3f9c7d4f3e2",
|
||||
"size": 56789,
|
||||
"library": "brotli",
|
||||
"version": "1.0.7",
|
||||
"parameters": {
|
||||
"quality": 11,
|
||||
"mode": "text"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
Servers may include additional fields in the descriptor like `magnet`, `infohash`, or `ipfs` depending on other protocols they support
|
||||
|
||||
@@ -83,7 +61,7 @@ Example Authorization event:
|
||||
|
||||
The `/list/<pubkey>` endpoint MUST return a JSON array of [Blob Descriptor](#blob-descriptor) that where uploaded by the specified pubkey
|
||||
|
||||
The endpoint MUST support a `since` and `until` query parameter to limit the returned blobs by thier `uploaded` date
|
||||
The endpoint MUST support a `since` and `until` query parameter to limit the returned blobs by their `uploaded` date
|
||||
|
||||
Servers may reject a list for any reason and MUST respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection
|
||||
|
||||
@@ -116,7 +94,7 @@ Example Authorization event:
|
||||
|
||||
Servers MUST accept `DELETE` requests to the `/<sha256>` endpoint
|
||||
|
||||
Servers may reject a delete request for any reason and should respond with the aproperate HTTP `4xx` status code and an error message explaining the reason for the rejection
|
||||
Servers may reject a delete request for any reason and should respond with the appropriate HTTP `4xx` status code and an error message explaining the reason for the rejection
|
||||
|
||||
### Delete Authorization (required)
|
||||
|
||||
@@ -138,7 +116,7 @@ Example Authorization event:
|
||||
"created_at": 1708774469,
|
||||
"tags": [
|
||||
["t", "delete"],
|
||||
["x", "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553 "],
|
||||
["x", "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553"],
|
||||
["expiration", "1708858680"]
|
||||
],
|
||||
"sig": "2ba9af680505583e3eb289a1624a08661a2f6fa2e5566a5ee0036333d517f965e0ffba7f5f7a57c2de37e00a2e85fd7999076468e52bdbcfad8abb76b37a94b0"
|
||||
|
||||
78
buds/03.md
Normal file
78
buds/03.md
Normal file
@@ -0,0 +1,78 @@
|
||||
BUD-03
|
||||
======
|
||||
|
||||
User Server List
|
||||
-------------------------
|
||||
|
||||
`draft` `optional`
|
||||
|
||||
Defines a replaceable event using `kind:10063` to advertise the blossom servers a user uses to host their blobs.
|
||||
|
||||
The event MUST include at least one `server` tag containing the full server URL including the `http://` or `https://`.
|
||||
|
||||
The order of these tags is important and should be arranged with the users most "reliable" or "trusted" servers being first.
|
||||
|
||||
The `.content` field is not used.
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "e4bee088334cb5d38cff1616e964369c37b6081be997962ab289d6c671975d71",
|
||||
"pubkey": "781208004e09102d7da3b7345e64fd193cd1bc3fce8fdae6008d77f9cabcd036",
|
||||
"content": "",
|
||||
"kind": 10063,
|
||||
"created_at": 1708774162,
|
||||
"tags": [
|
||||
["server", "https://cdn.self.hosted"],
|
||||
["server", "https://cdn.satellite.earth"]
|
||||
],
|
||||
"sig": "cc5efa74f59e80622c77cacf4dd62076bcb7581b45e9acff471e7963a1f4d8b3406adab5ee1ac9673487480e57d20e523428e60ffcc7e7a904ac882cfccfc653"
|
||||
}
|
||||
```
|
||||
|
||||
## Client Upload Implementation
|
||||
|
||||
When uploading blobs clients MUST attempt to upload the blob to at least the first `server` listed in the users server list.
|
||||
|
||||
Optionally clients MAY upload the blob to all the servers or mirror the blob to the other servers if they support [BUD-04](./04.md)
|
||||
|
||||
This ensures that the blob is available in multiple locations in the case one of the servers goes offline.
|
||||
|
||||
## Client Retrieval Implementation
|
||||
|
||||
When extracting the SHA256 hash from the URL clients MUST use the last occurrence of a 64 char hex string. This allows clients to extract hashes from blossom URLs and SOME non-blossom URLs.
|
||||
|
||||
In all the following examples, the hash `b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553` should be selected
|
||||
|
||||
- Blossom URLs
|
||||
- `https://blossom.example.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf`
|
||||
- `https://cdn.example.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553`
|
||||
- Non Blossom URLs
|
||||
- `https://cdn.example.com/user/ec4425ff5e9446080d2f70440188e3ca5d6da8713db7bdeef73d0ed54d9093f0/media/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf`
|
||||
- `https://cdn.example.com/media/user-name/documents/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf`
|
||||
- `http://download.example.com/downloads/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553`
|
||||
- `http://media.example.com/documents/b1/67/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf`
|
||||
|
||||
In the context of nostr events, clients SHOULD use the author's server list when looking for blobs that are no longer available at the original URL.
|
||||
|
||||
Take the following event as an example
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "834185269f4ab72539193105060dbb1c8b2efd702d14481cea345c47beefe6eb",
|
||||
"pubkey": "ec4425ff5e9446080d2f70440188e3ca5d6da8713db7bdeef73d0ed54d9093f0",
|
||||
"content": "I've developed a new open source P2P e-cash system called Bitcoin. check it out\nhttps://cdn.broken-domain.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf",
|
||||
"kind": 1,
|
||||
"created_at": 1297484820,
|
||||
"tags": [],
|
||||
"sig": "bd4bb200bdd5f7ffe5dbc3e539052e27b05d6f9f528e255b1bc4261cc16b8f2ad85c89eef990c5f2eee756ef71b4c571ecf6a88ad12f7338e321dd60c6a903b5"
|
||||
}
|
||||
```
|
||||
|
||||
Once the client discovers that the URL `https://cdn.broken-domain.com/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf` is no longer available. It can perform the following steps to find the blob:
|
||||
|
||||
1. Get the SHA256 has from the URL
|
||||
2. Look for the authors server list `kind:10063`
|
||||
3. If found, Attempt to retrieve the blob from each `server` listed started with the first
|
||||
3. If not found, the client MAY fallback to using a well-known popular blossom server to retrieve the blob
|
||||
|
||||
This ensures clients can quickly find missing blobs using the users list of trusted servers.
|
||||
97
buds/05.md
97
buds/05.md
@@ -1,97 +0,0 @@
|
||||
BUD-05
|
||||
======
|
||||
|
||||
Deterministic File Compression
|
||||
---------------
|
||||
|
||||
`draft` `optional`
|
||||
|
||||
## Overview
|
||||
|
||||
This document outlines the implementation of a deterministic file compression method for Blossom, specifying the metadata that must be stored for each compressed blob and the endpoint for retrieving this metadata.
|
||||
|
||||
## Server Adaptation
|
||||
|
||||
File storage servers `MUST` store the following metadata for each compressed blob:
|
||||
|
||||
- File hash (sha256)
|
||||
- Uploaded timestamp (uploaded)
|
||||
- Original file size (size)
|
||||
- Original file type (type)
|
||||
- Compressed file hash (compressed.sha256)
|
||||
- Compressed file size (compressed.size)
|
||||
- Compression library (compression.library)
|
||||
- Compression library version (compression.version)
|
||||
- Compression library parameters (compression.parameters)
|
||||
|
||||
Servers `MUST` normalize the blob metadata to ensure that the same compressed blob with the same parameters will always produce the same result, thereby ensuring deterministic compression and anonymizing any potential user data.
|
||||
|
||||
## Client adaptation
|
||||
|
||||
Clients `MUST` verify the metadata of the compressed blob to ensure that the compressed blob matches the original blob. Clients `MUST` sign and publish a new authorization event with the new blob hash. Clients `MUST` upload the compressed blob to the server using the new authorization event and the 'compress' field set to false or not set.
|
||||
|
||||
## GET /metadata - Retrieve blob metadata
|
||||
|
||||
The GET /metadata/<sha256> endpoint `MUST` return a [Blob Descriptor](https://github.com/hzrd149/blossom/blob/master/buds/02.md#blob-descriptor) containing the metadata fields for the requested blob or an error object if the blob does not exist.
|
||||
|
||||
The endpoint `MUST` accept an optional file extension in the URL. ie. `.pdf`, `.png`, etc
|
||||
|
||||
Example response:
|
||||
|
||||
```json
|
||||
{
|
||||
"url": "cdn.nostrcheck.me/b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553",
|
||||
"sha256": "b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553",
|
||||
"uploaded": 1708773959,
|
||||
"size": 123456,
|
||||
"type": "application/pdf",
|
||||
"compressed": {
|
||||
"sha256": "e2e2e1d2b9c04e2a9914c245e3f789d230e3c2d5e3a0f5c6e4b4e3f9c7d4f3e2",
|
||||
"size": 56789,
|
||||
"library": "brotli",
|
||||
"version": "1.0.7",
|
||||
"parameters": {
|
||||
"quality": 11,
|
||||
"mode": "text"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
## Example Flow
|
||||
|
||||
1. Client signs authorization event and uploads blob to Server A with 'compress' field set to true
|
||||
1. Server A compresses the blob using it's preferred library and parameters
|
||||
1. Server A stores the blob metadata
|
||||
1. Server A returns [Blob Descriptor](./02.md#blob-descriptor)
|
||||
1. Client verify the metadata matches the original blob compressing and comparing the blob locally with the same returned parameters
|
||||
1. Client signs and publishes a new authorization event with the new blob hash and the 'compress' field set to false
|
||||
1. Client uploads the compressed blob to Server B using the new authorization event and the 'compress' field set to false or not set
|
||||
1. Server B verifies blob hash metadata matches `x` tag in the new authorization event
|
||||
1. Server B stores the blob metadata
|
||||
1. Server B returns [Blob Descriptor](./02.md#blob-descriptor)
|
||||
|
||||
(optional using the /mirror endpoint)
|
||||
|
||||
1. Client sends the `url` to Server C `/mirror` using the compressed authorization event
|
||||
1. Server C downloads blob from Server A or B using the url field
|
||||
1. Server C verifies downloaded blob hash matches `x` tag in authorization event (using sha256 or compressed.sha256)
|
||||
1. Server C returns [Blob Descriptor](./02.md#blob-descriptor)
|
||||
|
||||
|
||||
|
||||
## Accepted Libraries, Versions, and Parameters
|
||||
|
||||
The following libraries, versions, and parameters are accepted for deterministic compression:
|
||||
|
||||
(This is just a DEMO table, the actual table will be updated with the final list of libraries, versions, and parameters)
|
||||
|
||||
| Library | Version | Quality Range | Modes |
|
||||
|---------|---------|---------------|------------------------|
|
||||
| Brotli | 1.0.7 | 0-11 | text, font, generic |
|
||||
| Gzip | 1.10 | 0-9 | text, font, generic |
|
||||
| Zstd | 1.4.5 | 1-22 | text, font, generic |
|
||||
| Lz4 | 1.9.2 | 1-12 | text, font, generic |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user