Complete removal of hashtag and url tags from bookmarks. (#2141)

Co-authored-by: Jon Staab <shtaab@gmail.com>

11.md

@@ -17,6 +17,7 @@ When a relay receives an HTTP(s) request with an `Accept` header of `application
   "banner": <a link to an image (e.g. in .jpg, or .png format)>,
   "icon": <a link to an icon (e.g. in .jpg, or .png format>,
   "pubkey": <administrative contact pubkey>,
+  "self": <relay's own pubkey>,
   "contact": <administrative alternate contact>,
   "supported_nips": <a list of NIP numbers supported by the relay>,
   "software": <string identifying relay software URL>,
@@ -60,6 +61,10 @@ An administrative contact may be listed with a `pubkey`, in the same format as N
 
 Relay operators have no obligation to respond to direct messages.
 
+### Self
+
+A relay MAY maintain an identity independent from its administrator using the `self` field, which MUST be a 32-byte hex public key. This allows relays to respond to requests with events published either in advance or on demand by their own key.
+
 ### Contact
 
 An alternative contact may be listed under the `contact` field as well, with the same purpose as `pubkey`.  Use of a Nostr public key and direct message SHOULD be preferred over this.  Contents of this field SHOULD be a URI, using schemes such as `mailto` or `https` to provide users with a means of contact.

18.md

@@ -1,8 +1,6 @@
-NIP-18
-======
+# NIP-18
 
-Reposts
--------
+## Reposts
 
 `draft` `optional`
 
@@ -21,9 +19,9 @@ reposted.
 
 ## Quote Reposts
 
-Mentions to [NIP-21](21.md) entities like `nevent`, `note` and `naddr` on any 
-event must be converted into `q` tags. The `q` tag ensures quote reposts are 
-not pulled and included as replies in threads. It also allows you to easily 
+Mentions to [NIP-21](21.md) entities like `nevent`, `note` and `naddr` on any
+event must be converted into `q` tags. The `q` tag ensures quote reposts are
+not pulled and included as replies in threads. It also allows you to easily
 pull and count all of the quotes for a post. The syntax follows
 
 `["q", "<event-id> or <event-address>", "<relay-url>", "<pubkey-if-a-regular-event>"]`
@@ -36,3 +34,10 @@ as a "generic repost", that can include any kind of event inside other than
 
 `kind 16` reposts SHOULD contain a `"k"` tag with the stringified kind number
 of the reposted event as its value.
+
+When reposting a replaceable event, the repost SHOULD include an `"a"` tag with
+the event coordinate (`kind:pubkey:d-tag`) of the reposted event.
+
+If the `"a"` tag is not present, it indicates that a specific version of a replaceable
+event is being reposted, in which case the `content` field must contain the full
+JSON string of the reposted event.

45.md

@@ -29,67 +29,15 @@ In case a relay uses probabilistic counts, it MAY indicate it in the response wi
 
 Whenever the relay decides to refuse to fulfill the `COUNT` request, it MUST return a `CLOSED` message.
 
-## HyperLogLog
-
-Relays may return an HyperLogLog value together with the count, hex-encoded.
-
-```
-["COUNT", <query_id>, {"count": <integer>, "hll": "<hex>"}]
-```
-
-This is so it enables merging results from multiple relays and yielding a reasonable estimate of reaction counts, comment counts and follower counts, while saving many millions of bytes of bandwidth for everybody.
-
-### Algorithm
-
-This section describes the steps a relay should take in order to return HLL values to clients.
-
-1. Upon receiving a filter, if it is eligible (see below) for HyperLogLog, compute the deterministic `offset` for that filter (see below);
-2. Initialize 256 registers to `0` for the HLL value;
-3. For all the events that are to be counted according to the filter, do this:
-    1. Read the byte at position `offset` of the event `pubkey`, its value will be the register index `ri`;
-    2. Count the number of leading zero bits starting at position `offset+1` of the event `pubkey` and add `1`;
-    3. Compare that with the value stored at register `ri`, if the new number is bigger, store it.
-
-That is all that has to be done on the relay side, and therefore the only part needed for interoperability.
-
-On the client side, these HLL values received from different relays can be merged (by simply going through all the registers in HLL values from each relay and picking the highest value for each register, regardless of the relay).
-
-And finally the absolute count can be estimated by running some methods I don't dare to describe here in English, it's better to check some implementation source code (also, there can be different ways of performing the estimation, with different quirks applied on top of the raw registers).
-
-### Filter eligibility and `offset` computation
-
-This NIP defines (for now) two filters eligible for HyperLogLog:
-
-- `{"#e": ["<id>"], "kinds": [7]}`, i.e. a filter for `kind:7` events with a single `"e"` tag, which means the client is interested in knowing how many people have reacted to the target event `<id>`. In this case the `offset` will be given by reading the character at the position `32` of the hex `<id>` value as a base-16 number then adding `8` to it.
-- `{"#e": ["<id>"], "kinds": [6]}`, the same as above, but for `kind:6` reposts.
-- `{"#p": ["<pubkey>"], "kinds": [3]}`, i.e. a filter for `kind:3` events with a single `"p"` tag, which means the client is interested in knowing how many people "follow" the target `<pubkey>`. In this case the `offset` will be given by reading the character at the position `32` of the hex `<pubkey>` value as a base-16 number then adding `8` to it.
-- `{"#E": ["<id>"], "kinds": [1111]}`, i.e. a filter for the total number of comments any specific root event has received. In this case the `offset` will be given by reading the character at the position `32` of the hex `<id>` value as a base-16 number then adding `8` to it.
-
-### Attack vectors
-
-One could mine a pubkey with a certain number of zero bits in the exact place where the HLL algorithm described above would look for them in order to artificially make its reaction or follow "count more" than others. For this to work a different pubkey would have to be created for each different target (event id, followed profile etc). This approach is not very different than creating tons of new pubkeys and using them all to send likes or follow someone in order to inflate their number of followers. The solution is the same in both cases: clients should not fetch these reaction counts from open relays that accept everything, they should base their counts on relays that perform some form of filtering that makes it more likely that only real humans are able to publish there and not bots or artificially-generated pubkeys.
-
-### `hll` encoding
-
-The value `hll` value must be the concatenation of the 256 registers, each being a uint8 value (i.e. a byte). Therefore `hll` will be a 512-character hex string.
-
-### Client-side usage
-
-This algorithm also allows clients to combine HLL responses received from relays with HLL counts computed locally from raw events. It's recommended that clients keep track of HLL values locally and add to these on each message received from relays. For example:
-
-  - a client wants to keep track of the number of reactions an event Z has received over time;
-  - the client has decided it will read reactions from relays A, B and C (the NIP-65 "read" relays of Z's author);
-  - of these, only B and C support HLL responses, so the client fetches both and merges them locally;
-  - then the client fetches all reaction events from A then manually applies each event to the HLL from the previous step, using the same algorithm described above;
-  - finally, the client reads the estimate count from the HLL and displays that to the user;
-  - optionally the client may store that HLL value (together with some "last-read-date" for relay A) and repeat the process again later:
-    - this time it only needs to fetch the new reactions from A and add those to the HLL
-    - and redownload the HLL values from B and C and just reapply them to the local value.
-
-This procedure allows the client to download much less data.
-
 ## Examples
 
+### Followers count
+
+```
+["COUNT", <query_id>, {"kinds": [3], "#p": [<pubkey>]}]
+["COUNT", <query_id>, {"count": 238}]
+```
+
 ### Count posts and reactions
 
 ```
@@ -97,7 +45,6 @@ This procedure allows the client to download much less data.
 ["COUNT", <query_id>, {"count": 5}]
 ```
 
-
 ### Count posts approximately
 
 ```
@@ -105,13 +52,6 @@ This procedure allows the client to download much less data.
 ["COUNT", <query_id>, {"count": 93412452, "approximate": true}]
 ```
 
-### Followers count with HyperLogLog
-
-```
-["COUNT", <subscription_id>, {"kinds": [3], "#p": [<pubkey>]}]
-["COUNT", <subscription_id>, {"count": 16578, "hll": "0607070505060806050508060707070706090d080b0605090607070b07090606060b0705070709050807080805080407060906080707080507070805060509040a0b06060704060405070706080607050907070b08060808080b080607090a06060805060604070908050607060805050d05060906090809080807050e0705070507060907060606070708080b0807070708080706060609080705060604060409070a0808050a0506050b0810060a0908070709080b0a07050806060508060607080606080707050806080c0a0707070a080808050608080f070506070706070a0908090c080708080806090508060606090906060d07050708080405070708"}]
-```
-
 ### Relay refuses to count
 
 ```

51.md

@@ -26,7 +26,7 @@ For example, _mute list_ can contain the public keys of spammers and bad actors
 | Mute list         | 10000 | things the user doesn't want to see in their feeds          | `"p"` (pubkeys), `"t"` (hashtags), `"word"` (lowercase string), `"e"` (threads)                     |
 | Pinned notes      | 10001 | events the user intends to showcase in their profile page   | `"e"` (kind:1 notes)                                                                                |
 | Read/write relays | 10002 | where a user publishes to and where they expect mentions    | see [NIP-65](65.md)                                                                                 |
-| Bookmarks         | 10003 | uncategorized, "global" list of things a user wants to save | `"e"` (kind:1 notes), `"a"` (kind:30023 articles), `"t"` (hashtags), `"r"` (URLs)                   |
+| Bookmarks         | 10003 | uncategorized, "global" list of things a user wants to save | `"e"` (kind:1 notes), `"a"` (kind:30023 articles)                   |
 | Communities       | 10004 | [NIP-72](72.md) communities the user belongs to             | `"a"` (kind:34550 community definitions)                                                            |
 | Public chats      | 10005 | [NIP-28](28.md) chat channels the user is in                | `"e"` (kind:40 channel definitions)                                                                 |
 | Blocked relays    | 10006 | relays clients should never connect to                      | `"relay"` (relay URLs)                                                                              |
@@ -52,7 +52,7 @@ Aside from their main identifier, the `"d"` tag, sets can optionally have a `"ti
 | ---                   | ---   | ---                                                                                          | ---                                                                                 |
 | Follow sets           | 30000 | categorized groups of users a client may choose to check out in different circumstances      | `"p"` (pubkeys)                                                                     |
 | Relay sets            | 30002 | user-defined relay groups the user can easily pick and choose from during various operations | `"relay"` (relay URLs)                                                              |
-| Bookmark sets         | 30003 | user-defined bookmarks categories , for when bookmarks must be in labeled separate groups    | `"e"` (kind:1 notes), `"a"` (kind:30023 articles), `"t"` (hashtags), `"r"` (URLs)   |
+| Bookmark sets         | 30003 | user-defined bookmarks categories , for when bookmarks must be in labeled separate groups    | `"e"` (kind:1 notes), `"a"` (kind:30023 articles)                                   |
 | Curation sets         | 30004 | groups of articles picked by users as interesting and/or belonging to the same category      | `"a"` (kind:30023 articles), `"e"` (kind:1 notes)                                   |
 | Curation sets         | 30005 | groups of videos picked by users as interesting and/or belonging to the same category        | `"e"` (kind:21 videos)                                                              |
 | Kind mute sets        | 30007 | mute pubkeys by kinds<br>`"d"` tag MUST be the kind string                                   | `"p"` (pubkeys)                                                                     |

59.md

@@ -99,6 +99,8 @@ AUTH, and refuse to serve wrapped events to non-recipients.
 
 When adding expiration tags to both `seal` and `gift wrap` layers, implementations SHOULD use independent random timestamps for each layer. Using different `created_at` values increases timing variance and helps protect against metadata correlation attacks.
 
+Since signing keys are random, relays SHOULD delete `kind 1059` events whose p-tag matches the signer of
+[NIP-09](09.md) deletions or [NIP-62](62.md) vanish requests.
 
 ## An Example
 

69.md

@@ -41,7 +41,8 @@ Events are [addressable events](01.md#kinds) and use `38383` as event kind, a p2
     ["name", "Nakamoto"],
     ["g", "<geohash>"],
     ["bond", "0"],
-    ["expiration", "1719391096"],
+    ["expires_at", "1719391096"],
+    ["expiration", "1719995896"],
     ["y", "lnp2pbot"],
     ["z", "order"]
   ],
@@ -55,7 +56,7 @@ Events are [addressable events](01.md#kinds) and use `38383` as event kind, a p2
 - `d` < Order ID >: A unique identifier for the order.
 - `k` < Order type >: `sell` or `buy`.
 - `f` < Currency >: The asset being traded, using the [ISO 4217](https://en.wikipedia.org/wiki/ISO_4217) standard.
-- `s` < Status >: `pending`, `canceled`, `in-progress`, `success`.
+- `s` < Status >: `pending`, `canceled`, `in-progress`, `success`, `expired`.
 - `amt` < Amount >: The amount of Bitcoin to be traded, the amount is defined in satoshis, if `0` means that the amount of satoshis will be obtained from a public API after the taker accepts the order.
 - `fa` < Fiat amount >: The fiat amount being traded, for range orders two values are expected, the minimum and maximum amount.
 - `pm` < Payment method >: The payment method used for the trade, if the order has multiple payment methods, they should be separated by a comma.
@@ -67,7 +68,8 @@ Events are [addressable events](01.md#kinds) and use `38383` as event kind, a p2
 - `name` [Name]: The name of the maker.
 - `g` [Geohash]: The geohash of the operation, it can be useful in a face to face trade.
 - `bond` [Bond]: The bond amount, the bond is a security deposit that both parties must pay.
-- `expiration` < Expiration\>: The expiration date of the order ([NIP-40](40.md)).
+- `expires_at` < Expires At\>: The expiration date of the event being published in `pending` status, after this time the event status SHOULD be changed to `expired`.
+- `expiration` < Expiration\>: The expiration date of the event, after this time the relay SHOULD delete it ([NIP-40](40.md)).
 - `y` < Platform >: The platform that created the order.
 - `z` < Document >: `order`.
 

BE.md

@@ -0,0 +1,137 @@
+NIP-BE
+======
+
+Nostr BLE Communications Protocol
+---------------------------------
+
+`draft` `optional`
+
+This NIP specifies how Nostr apps can use BLE to communicate and synchronize with each other. The BLE protocol follows a client-server pattern, so this NIP emulates the WS structure in a similar way, but with some adaptations to its limitations.
+
+## Device advertisement
+A device advertises itself with:
+- Service UUID: `0000180f-0000-1000-8000-00805f9b34fb`
+- Data: Device UUID in ByteArray format
+
+## GATT service
+The device exposes a Nordic UART Service with the following characteristics:
+
+1. Write Characteristic
+   - UUID: `87654321-0000-1000-8000-00805f9b34fb`
+   - Properties: Write
+
+2. Read Characteristic
+   - UUID: `12345678-0000-1000-8000-00805f9b34fb`
+   - Properties: Notify, Read
+
+## Role assignment
+
+When one device initially finds another advertising the service, it will read the service's data to get the device UUID and compare it with its own advertised device UUID. For this communication, the device with the highest ID will take the role of GATT Server (Relay), the other will be considered the GATT Client (Client) and will proceed to establish the connection.
+
+For devices whose purpose will require a single role, its device UUID will always be:
+
+- GATT Server: `FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF`
+- GATT Client: `00000000-0000-0000-0000-000000000000`
+
+## Messages
+
+All messages will follow [NIP-01](/01.md) message structure. For a given message, a compression stream (DEFLATE) is applied to the message to generate a byte array. Depending on the BLE version, the byte array can be too large for a single message (20-23 bytes in BLE 4.2, 256 bytes in BLE > 4.2). In that case, this byte array is split into any number of batches following the structure:
+
+```
+[batch index (first 2 bytes)][batch n][is last batch (last byte)]
+```
+After reception of all batches, the other device can then join them and decompress. To ensure reliability, only 1 message will be read/written at a time. MTU can be negotiated in advance. The maximum size for a message is 64KB; bigger messages will be rejected.
+
+## Examples
+
+This example implements a function to split and compress a byte array into chunks, as well as another function to join and decompress them in order to obtain the initial result:
+
+```kotlin
+fun splitInChunks(message: ByteArray): Array<ByteArray> {
+   val chunkSize = 500 // define the chunk size
+   var byteArray = compressByteArray(message)
+   val numChunks = (byteArray.size + chunkSize - 1) / chunkSize // calculate the number of chunks
+   var chunkIndex = 0
+   val chunks = Array(numChunks) { ByteArray(0) }
+
+   for (i in 0 until numChunks) {
+         val start = i * chunkSize
+         val end = minOf((i + 1) * chunkSize, byteArray.size)
+         val chunk = byteArray.copyOfRange(start, end)
+
+         // add chunk index to the first 2 bytes and last chunk flag to the last byte
+         val chunkWithIndex = ByteArray(chunk.size + 2)
+         chunkWithIndex[0] = chunkIndex.toByte() // chunk index
+         chunk.copyInto(chunkWithIndex, 1)
+         chunkWithIndex[chunkWithIndex.size - 1] = numChunks.toByte()
+
+         // store the chunk in the array
+         chunks[i] = chunkWithIndex
+
+         chunkIndex++
+   }
+
+   return chunks
+}
+
+fun joinChunks(chunks: Array<ByteArray>): ByteArray {
+   val sortedChunks = chunks.sortedBy { it[0] }
+   var reassembledByteArray = ByteArray(0)
+   for (chunk in sortedChunks) {
+         val chunkData = chunk.copyOfRange(1, chunk.size - 1)
+         reassembledByteArray = reassembledByteArray.copyOf(reassembledByteArray.size + chunkData.size)
+         chunkData.copyInto(reassembledByteArray, reassembledByteArray.size - chunkData.size)
+   }
+
+   return decompressByteArray(reassembledByteArray)
+}
+
+```
+
+## Workflows
+
+### Client to relay
+
+- Any message the client wants to send to a relay will be a write message.
+- Any message the client receives from a relay will be a read message.
+
+### Relay to client
+
+The relay should notify the client about any new event matching subscription's filters by using the Notify action of the Read Characteristic. After that, the client can proceed to read messages from the relay.
+
+### Device synchronization
+
+Given the nature of BLE, it is expected that the direct connection between two devices might be extremely intermittent, with gaps of hours or even days. That's why it's crucial to define a synchronization process by following [NIP-77](./77.md) but with an adaptation to the limitations of the technology.
+
+After two devices have successfully connected and established the Client-Server roles, the devices will use half-duplex communication to intermittently send and receive messages.
+
+#### Half-duplex synchronization
+
+Right after the 2 devices connect, the Client starts the workflow by sending the first message.
+
+1. Client - Writes ["NEG-OPEN"](/77.md#initial-message-client-to-relay) message.
+2. Server - Sends `write-success`.
+3. Client - Sends `read-message`.
+4. Server - Responds with ["NEG-MSG"](./77.md#subsequent-messages-bidirectional) message.
+5. Client -
+   1. If the Client has messages missing on the Server, it writes one `EVENT`.
+   2. If the Client doesn't have any messages missing on the Server, it writes `EOSE`. In this case, subsequent messages to the Server will be empty while the Server claims to have more notes for the Client.
+6. Server - Sends `write-success`.
+7. Client - Sends `read-message`.
+8. Server -
+   1. If the Server has messages missing on the Client, it responds with one `EVENT`.
+   2. If the Client doesn't have any messages missing on the Server, it responds with `EOSE`. In this case, subsequent responses to the Client will be empty.
+9. If the Client detects that the devices are not synchronized yet, jump to step 5.
+10. After the two devices detect that there are no more missing events on both ends, the workflow will pause at this point.
+
+#### Half-duplex event spread
+
+While two devices are connected and synchronized, it might happen that one of them receives a new message from another connected peer. Devices MUST keep track of which notes have been sent to its peers while they are connected. If the newly received event is detected as missing in one of the connected and synchronized peers:
+
+1. If the peer is a Server:
+   1. Client - It writes the `EVENT`.
+   2. Server - Sends `write-success`.
+2. If the peer is a Client:
+   1. Server - It will send an empty notification to the Client.
+   2. Client - Sends `read-message`.
+   3. Server - Responds with the `EVENT`.

C0.md

@@ -23,9 +23,9 @@ The `.content` field contains the actual code snippet text.
 - `extension` - File extension (without the dot). Examples: "js", "py", "rs"
 - `description` - Brief description of what the code does
 - `runtime` - Runtime or environment specification (e.g., "node v18.15.0", "python 3.11")
-- `license` - License under which the code is shared (e.g., "MIT", "GPL-3.0", "Apache-2.0")
+- `license` - License under which the code (along with any related data contained within the event, when available, such as the description) is shared. This MUST be a standard [SPDX](https://spdx.org/licenses/) short identifier (e.g., "MIT", "GPL-3.0-or-later", "Apache-2.0") when available. An additional parameter containing a reference to the actual text of the license MAY be provided. This tag can be repeated, to indicate multi-licensing, allowing recipients to use the code under any license of choosing among the referenced ones
 - `dep` - Dependency required for the code to run (can be repeated)
-- `repo` - Reference to a repository where this code originates
+- `repo` - Reference to a repository where this code originates. This MUST be a either standard URL or, alternatively, the address of a [NIP-34](34.md) Git repository annoucement event in the form `"30617:<32-bytes hex a pubkey>:<d tag value>"`. If a repository announcement is referenced, a recommended relay URL where to find the event should be provided as an additional parameter
 
 ## Format
 

README.md

@@ -105,6 +105,7 @@ They exist to document what may be implemented by [Nostr](https://github.com/nos
 - [NIP-A0: Voice Messages](A0.md)
 - [NIP-B0: Web Bookmarks](B0.md)
 - [NIP-B7: Blossom](B7.md)
+- [NIP-BE: Nostr BLE Communications Protocol](BE.md)
 - [NIP-C0: Code Snippets](C0.md)
 - [NIP-C7: Chats](C7.md)
 - [NIP-EE: E2EE Messaging using MLS Protocol](EE.md)