mirror of
https://github.com/hzrd149/blossom.git
synced 2026-01-27 23:58:51 +00:00
Compare commits
1 Commits
local-cach
...
hls-videos
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2a9319e8c4 |
@@ -27,7 +27,7 @@ BUDs or **Blossom Upgrade Documents** are short documents that outline an additi
|
||||
- [BUD-08: Nostr File Metadata Tags](./buds/08.md)
|
||||
- [BUD-09: Blob Report](./buds/09.md)
|
||||
- [BUD-10: Blossom URI Schema](./buds/10.md)
|
||||
- [BUD-11: Local Blob Cache](./buds/11.md)
|
||||
- [BUD-11: HLS Video Formatting](./buds/11.md)
|
||||
|
||||
## Endpoints
|
||||
|
||||
@@ -39,7 +39,7 @@ Blossom Servers expose a few endpoints for managing blobs
|
||||
- `Authentication`: Signed [nostr event](./buds/02.md#upload-authorization-required)
|
||||
- Return a blob descriptor
|
||||
- `HEAD /upload` [BUD-06](./buds/06.md#head-upload---upload-requirements)
|
||||
- `GET /list/<pubkey>` [BUD-02](./buds/02.md#get-listpubkey---list-blobs-unrecommended) _(optional, unrecommended)_
|
||||
- `GET /list/<pubkey>` [BUD-02](./buds/02.md#get-listpubkey---list-blobs)
|
||||
- Returns an array of blob descriptors
|
||||
- `Authentication` _(optional)_: Signed [nostr event](./buds/02.md#list-authorization-optional)
|
||||
- `DELETE /<sha256>` [BUD-02](./buds/02.md#delete-sha256---delete-blob)
|
||||
|
||||
@@ -76,9 +76,7 @@ Example Authorization event:
|
||||
}
|
||||
```
|
||||
|
||||
## GET /list/pubkey - List Blobs (Unrecommended)
|
||||
|
||||
**Note:** The `/list` endpoint is optional and unrecommended. It is not necessary for all servers to implement the `/list` endpoint. Servers MAY implement this endpoint, but are not required to do so.
|
||||
## GET /list/pubkey - List Blobs (Optional)
|
||||
|
||||
The `/list/<pubkey>` endpoint MUST return a JSON array of [Blob Descriptor](#blob-descriptor) that were uploaded by the specified pubkey
|
||||
|
||||
|
||||
164
buds/11.md
164
buds/11.md
@@ -1,107 +1,123 @@
|
||||
# BUD-11
|
||||
|
||||
## Local Blob Cache
|
||||
## HLS Video Formatting
|
||||
|
||||
`draft` `optional`
|
||||
|
||||
This document defines the specification for a local blob cache server that can be hosted on `127.0.0.1:24242` to provide fast, local access to cached blobs or proxy requests to other public Blossom servers.
|
||||
Defines how to format [HLS](https://datatracker.ietf.org/doc/html/rfc8216) (HTTP Live Streaming) videos to be compatible with [BUD-01](./01.md) blob retrieval.
|
||||
|
||||
## Server Address
|
||||
## Overview
|
||||
|
||||
A local blob cache server MUST be accessible on `http://127.0.0.1:24242`. The port `24242` is chosen to align with the Nostr event kind used for Blossom authorization events.
|
||||
HLS videos consist of multiple files:
|
||||
- A master playlist (`.m3u8`) that references variant playlists
|
||||
- Variant playlists (`.m3u8`) that reference media segments
|
||||
- Media segment files (typically `.ts` files)
|
||||
|
||||
## Health Check Endpoint
|
||||
Each file MUST be uploaded as a separate blob and referenced by its SHA256 hash using the [BUD-01](./01.md#get-sha256---get-blob) `GET /<sha256>` endpoint format.
|
||||
|
||||
The server MUST respond with a 2xx HTTP status code (typically `200 OK`) on the `HEAD /` endpoint to allow local applications to easily detect if the cache server is available.
|
||||
## Relative Paths
|
||||
|
||||
The `HEAD /` endpoint MUST return a 2xx HTTP status code. The response MUST NOT include a response body per [RFC 7231](https://www.rfc-editor.org/rfc/rfc7231#section-4.3.2).
|
||||
To ensure compatibility across different Blossom servers and allow easy server switching, all URLs in HLS playlists MUST use relative paths containing the SHA256 hash of the referenced blob.
|
||||
|
||||
This endpoint enables simple health checks:
|
||||
Clients MUST NOT include the full server domain in playlist URLs. This allows the same playlist to work with any Blossom server that hosts the referenced blobs.
|
||||
|
||||
```bash
|
||||
curl -I http://127.0.0.1:24242/
|
||||
## Master Playlist Format
|
||||
|
||||
The master playlist (`.m3u8`) MUST reference variant playlists using relative paths with SHA256 hashes.
|
||||
|
||||
The master playlist MUST have the MIME type `application/vnd.apple.mpegurl` or `application/x-mpegURL`.
|
||||
|
||||
Example master playlist:
|
||||
|
||||
```m3u8
|
||||
#EXTM3U
|
||||
#EXT-X-VERSION:3
|
||||
#EXT-X-STREAM-INF:BANDWIDTH=1280000,RESOLUTION=854x480
|
||||
a1b2c3d4e5f6789012345678901234567890abcdef1234567890abcdef123456.m3u8
|
||||
#EXT-X-STREAM-INF:BANDWIDTH=2560000,RESOLUTION=1280x720
|
||||
f6e5d4c3b2a1098765432109876543210987654321fedcba0987654321fedcba.m3u8
|
||||
```
|
||||
|
||||
## Access Control
|
||||
## Variant Playlist Format
|
||||
|
||||
The server SHOULD NOT require any authentication for blob downloads or uploads. All requests MUST be served without requiring an `Authorization` header.
|
||||
Variant playlists (`.m3u8`) MUST reference media segments using relative paths with SHA256 hashes.
|
||||
|
||||
If an implementation needs to add access control, it SHOULD restrict access based on IP address to ensure requests only come from `127.0.0.1`. This ensures that local applications can freely access cached blobs without the overhead of signing authorization events.
|
||||
Each segment URL MUST be a relative path containing the SHA256 hash of the segment file. The file extension (`.ts`, `.m4s`, etc.) MAY be included for compatibility with HLS clients.
|
||||
|
||||
## Blob Retrieval
|
||||
Example variant playlist:
|
||||
|
||||
The server MUST implement the `GET /<sha256>` and `HEAD /<sha256>` endpoints as defined in [BUD-01](./01.md#get-sha256---get-blob):
|
||||
|
||||
1. The server MUST accept optional file extensions in the URL (e.g., `/<sha256>.pdf`)
|
||||
2. The server MUST set appropriate `Content-Type` headers or default to `application/octet-stream`
|
||||
|
||||
## Range Requests
|
||||
|
||||
To better support mobile devices, video files, or low bandwidth connections, implementations SHOULD support range requests ([RFC 7233 section 3](https://www.rfc-editor.org/rfc/rfc7233#section-3)) on the `GET /<sha256>` endpoint and signal support using the `accept-ranges: bytes` and `content-length` headers on the `HEAD /<sha256>` endpoint.
|
||||
|
||||
See [MDN docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests) for more details.
|
||||
|
||||
## Proxy Hints
|
||||
|
||||
The server SHOULD accept [BUD-10](./10.md) query parameters `xs` (server) and `as` (author) hints on the `GET /<sha256>` endpoint to enable proxying blob requests to other Blossom servers when the blob is not available in the local cache.
|
||||
|
||||
### Query Parameter Format
|
||||
|
||||
When a blob is not found in the local cache, the server MAY use the following query parameters to attempt retrieval from other Blossom servers:
|
||||
|
||||
- `xs=<server>` - One or more server hints where the blob may be available
|
||||
- `as=<pubkey>` - One or more author pubkeys whose server lists may contain the blob
|
||||
|
||||
### Example Request
|
||||
|
||||
```
|
||||
GET /b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553.pdf?xs=cdn.example.com&as=ec4425ff5e9446080d2f70440188e3ca5d6da8713db7bdeef73d0ed54d9093f0
|
||||
```m3u8
|
||||
#EXTM3U
|
||||
#EXT-X-VERSION:3
|
||||
#EXT-X-TARGETDURATION:10
|
||||
#EXT-X-MEDIA-SEQUENCE:0
|
||||
#EXTINF:10.000,
|
||||
b82fcf4dbcec2d8fab7d94bdd48b070aa6e74d7240b1965a0b28c128d6858477.ts
|
||||
#EXTINF:10.000,
|
||||
cd2a98d055eef5ec3aca73bd136a40340539138da73144d589d9de5a3a52149a.ts
|
||||
#EXTINF:10.000,
|
||||
128e690f89419ecbea473a490c42cac94a2293ecf1f57d60492ceafce3d5cfdb.ts
|
||||
#EXT-X-ENDLIST
|
||||
```
|
||||
|
||||
### Proxy Behavior
|
||||
## Media Segments
|
||||
|
||||
When the server receives a request with proxy hints and the blob is not in the local cache:
|
||||
Media segment files (typically `.ts` files) MUST be uploaded as separate blobs. Each segment MUST be retrievable via the [BUD-01](./01.md#get-sha256---get-blob) `GET /<sha256>` endpoint.
|
||||
|
||||
1. The server SHOULD attempt to retrieve the blob from the servers specified in the `xs` parameters
|
||||
2. If `xs` hints fail, the server MAY attempt to retrieve the blob using the `as` parameters by:
|
||||
- Fetching the author's [BUD-03](./03.md) server list (`kind:10063`)
|
||||
- Attempting to retrieve the blob from servers in the author's list
|
||||
3. If the blob is successfully retrieved, the server SHOULD:
|
||||
- Cache the blob locally for future requests
|
||||
- Return the blob to the client with appropriate headers
|
||||
4. If the blob cannot be retrieved from any hint, the server MUST return a `404 Not Found` response
|
||||
The server SHOULD set the `Content-Type` header appropriately:
|
||||
- `.ts` files: `video/mp2t` or `video/MP2T`
|
||||
- `.m4s` files: `video/iso.segment` or `video/mp4`
|
||||
|
||||
This proxy functionality allows the local cache server to act as a transparent proxy, automatically fetching and caching blobs from remote Blossom servers when needed.
|
||||
## Client Implementation
|
||||
|
||||
## CORS Headers
|
||||
When generating HLS playlists for Blossom:
|
||||
|
||||
The server MUST set the `Access-Control-Allow-Origin: *` header on all responses to ensure compatibility with web applications, as specified in [BUD-01](./01.md#cross-origin-headers).
|
||||
1. Upload each media segment as a separate blob using [BUD-02](./02.md#put-upload---upload-blob) `PUT /upload`
|
||||
2. Upload each variant playlist as a separate blob
|
||||
3. Upload the master playlist as a separate blob
|
||||
4. In all playlists, use relative paths containing only the SHA256 hash (and optional file extension) of the referenced blob
|
||||
5. When serving the master playlist, clients MAY prepend the current server's base URL to resolve relative paths
|
||||
|
||||
## Use Cases
|
||||
Example client flow:
|
||||
|
||||
A local blob cache server enables several use cases:
|
||||
1. Upload segment `segment001.ts` → get SHA256: `1a2b3c4d...`
|
||||
2. Upload segment `segment002.ts` → get SHA256: `2b3c4d5e...`
|
||||
3. Create variant playlist referencing `1a2b3c4d...ts` and `2b3c4d5e...ts`
|
||||
4. Upload variant playlist → get SHA256: `a1b2c3d4...`
|
||||
5. Create master playlist referencing `a1b2c3d4...m3u8`
|
||||
6. Upload master playlist → get SHA256: `f9e8d7c6...`
|
||||
|
||||
- **Fast Local Access**: Applications can check the local cache first before making network requests to remote Blossom servers
|
||||
- **Offline Access**: Previously cached blobs remain accessible even when network connectivity is unavailable
|
||||
- **Bandwidth Savings**: Reduces redundant downloads of frequently accessed blobs
|
||||
- **Development**: Provides a simple local server for testing Blossom applications without requiring remote server access
|
||||
## Server Implementation
|
||||
|
||||
## Implementation Notes
|
||||
When serving HLS playlists, servers MUST:
|
||||
|
||||
- The server SHOULD implement efficient blob storage, such as using the sha256 hash as the filename or storage key
|
||||
- The server SHOULD implement cache eviction policies (e.g., LRU, size-based limits) to manage storage
|
||||
- The server SHOULD validate that downloaded blobs match their sha256 hash before caching
|
||||
- The server MAY implement the `HEAD /<sha256>` endpoint to allow clients to check blob availability without downloading
|
||||
1. Return the playlist content with the appropriate `Content-Type` header (`application/vnd.apple.mpegurl` or `application/x-mpegURL`)
|
||||
2. Serve playlists via the [BUD-01](./01.md#get-sha256---get-blob) `GET /<sha256>` endpoint
|
||||
3. Support optional file extensions (e.g., `/<sha256>.m3u8`) as specified in [BUD-01](./01.md#get-sha256---get-blob)
|
||||
|
||||
## Example Implementation Flow
|
||||
When a client requests a playlist blob, the server MUST return the playlist content as-is, without modifying relative paths. The client is responsible for resolving relative paths to absolute URLs using the current server's base URL.
|
||||
|
||||
1. Client requests blob: `GET http://127.0.0.1:24242/<sha256>`
|
||||
2. Server checks local cache
|
||||
3. If found: Return blob immediately
|
||||
4. If not found and proxy hints provided:
|
||||
- Attempt to fetch from `xs` server hints
|
||||
- If that fails, attempt to fetch using `as` author hints
|
||||
- Cache the blob if successfully retrieved
|
||||
- Return the blob to the client
|
||||
5. If not found and no proxy hints: Return `404 Not Found`
|
||||
## URL Resolution
|
||||
|
||||
When a client retrieves a playlist from `https://cdn.example.com/<sha256>.m3u8`, relative paths in the playlist (e.g., `a1b2c3d4...m3u8`) MUST be resolved relative to the playlist's origin.
|
||||
|
||||
For example, if a playlist is served from `https://cdn.example.com/f9e8d7c6...m3u8` and contains a relative path `a1b2c3d4...m3u8`, the client SHOULD resolve it to `https://cdn.example.com/a1b2c3d4...m3u8`.
|
||||
|
||||
This allows the same playlist blob to work with any Blossom server, as long as all referenced blobs are available on that server.
|
||||
|
||||
## Example Complete Structure
|
||||
|
||||
```
|
||||
Master Playlist (SHA256: f9e8d7c6...)
|
||||
└─> Variant Playlist 1 (SHA256: a1b2c3d4...)
|
||||
└─> Segment 1 (SHA256: 1a2b3c4d...)
|
||||
└─> Segment 2 (SHA256: 2b3c4d5e...)
|
||||
└─> Segment 3 (SHA256: 3c4d5e6f...)
|
||||
└─> Variant Playlist 2 (SHA256: b2c3d4e5...)
|
||||
└─> Segment 1 (SHA256: 4d5e6f78...)
|
||||
└─> Segment 2 (SHA256: 5e6f7890...)
|
||||
└─> Segment 3 (SHA256: 6f789012...)
|
||||
```
|
||||
|
||||
All references between these files use relative paths containing only SHA256 hashes, making the entire HLS structure portable across different Blossom servers.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user