add recommendation for HLL client-side usage.

11.md

@@ -17,7 +17,6 @@ When a relay receives an HTTP(s) request with an `Accept` header of `application
   "banner": <a link to an image (e.g. in .jpg, or .png format)>,
   "icon": <a link to an icon (e.g. in .jpg, or .png format>,
   "pubkey": <administrative contact pubkey>,
-  "self": <relay's own pubkey>,
   "contact": <administrative alternate contact>,
   "supported_nips": <a list of NIP numbers supported by the relay>,
   "software": <string identifying relay software URL>,
@@ -61,10 +60,6 @@ An administrative contact may be listed with a `pubkey`, in the same format as N
 
 Relay operators have no obligation to respond to direct messages.
 
-### Self
-
-A relay MAY maintain an identity independent from its administrator using the `self` field, which MUST be a 32-byte hex public key. This allows relays to respond to requests with events published either in advance or on demand by their own key.
-
 ### Contact
 
 An alternative contact may be listed under the `contact` field as well, with the same purpose as `pubkey`.  Use of a Nostr public key and direct message SHOULD be preferred over this.  Contents of this field SHOULD be a URI, using schemes such as `mailto` or `https` to provide users with a means of contact.

34.md

@@ -10,7 +10,7 @@ This NIP defines all the ways code collaboration using and adjacent to [`git`](h
 
 ## Repository announcements
 
-Git repositories are hosted in Git-enabled servers, but their existence can be announced using Nostr events. By doing so the author asserts themselves as a maintainer and expresses a willingness to receive patches, bug reports and comments in general, unless tag `personal-fork` is included (in which case these repositories should be treated as temporary and created only such that they could be used to open a pull request to some other repository).
+Git repositories are hosted in Git-enabled servers, but their existence can be announced using Nostr events. By doing so the author asserts themselves as a maintainer and expresses a willingness to receive patches, bug reports and comments in general, unless `t` tag `personal-fork` is included.
 
 ```jsonc
 {
@@ -25,7 +25,7 @@ Git repositories are hosted in Git-enabled servers, but their existence can be a
     ["relays", "<relay-url>", ...], // relays that this repository will monitor for patches and issues
     ["r", "<earliest-unique-commit-id>", "euc"],
     ["maintainers", "<other-recognized-maintainer>", ...],
-    ["personal-fork"], // optionally indicate author isn't a maintainer
+    ["t","personal-fork"], // optionally indicate author isn't a maintainer
     ["t", "<arbitrary string>"], // hashtags labelling the repository
   ]
 }

45.md

@@ -29,15 +29,67 @@ In case a relay uses probabilistic counts, it MAY indicate it in the response wi
 
 Whenever the relay decides to refuse to fulfill the `COUNT` request, it MUST return a `CLOSED` message.
 
+## HyperLogLog
+
+Relays may return an HyperLogLog value together with the count, hex-encoded.
+
+```
+["COUNT", <query_id>, {"count": <integer>, "hll": "<hex>"}]
+```
+
+This is so it enables merging results from multiple relays and yielding a reasonable estimate of reaction counts, comment counts and follower counts, while saving many millions of bytes of bandwidth for everybody.
+
+### Algorithm
+
+This section describes the steps a relay should take in order to return HLL values to clients.
+
+1. Upon receiving a filter, if it is eligible (see below) for HyperLogLog, compute the deterministic `offset` for that filter (see below);
+2. Initialize 256 registers to `0` for the HLL value;
+3. For all the events that are to be counted according to the filter, do this:
+    1. Read the byte at position `offset` of the event `pubkey`, its value will be the register index `ri`;
+    2. Count the number of leading zero bits starting at position `offset+1` of the event `pubkey` and add `1`;
+    3. Compare that with the value stored at register `ri`, if the new number is bigger, store it.
+
+That is all that has to be done on the relay side, and therefore the only part needed for interoperability.
+
+On the client side, these HLL values received from different relays can be merged (by simply going through all the registers in HLL values from each relay and picking the highest value for each register, regardless of the relay).
+
+And finally the absolute count can be estimated by running some methods I don't dare to describe here in English, it's better to check some implementation source code (also, there can be different ways of performing the estimation, with different quirks applied on top of the raw registers).
+
+### Filter eligibility and `offset` computation
+
+This NIP defines (for now) two filters eligible for HyperLogLog:
+
+- `{"#e": ["<id>"], "kinds": [7]}`, i.e. a filter for `kind:7` events with a single `"e"` tag, which means the client is interested in knowing how many people have reacted to the target event `<id>`. In this case the `offset` will be given by reading the character at the position `32` of the hex `<id>` value as a base-16 number then adding `8` to it.
+- `{"#e": ["<id>"], "kinds": [6]}`, the same as above, but for `kind:6` reposts.
+- `{"#p": ["<pubkey>"], "kinds": [3]}`, i.e. a filter for `kind:3` events with a single `"p"` tag, which means the client is interested in knowing how many people "follow" the target `<pubkey>`. In this case the `offset` will be given by reading the character at the position `32` of the hex `<pubkey>` value as a base-16 number then adding `8` to it.
+- `{"#E": ["<id>"], "kinds": [1111]}`, i.e. a filter for the total number of comments any specific root event has received. In this case the `offset` will be given by reading the character at the position `32` of the hex `<id>` value as a base-16 number then adding `8` to it.
+
+### Attack vectors
+
+One could mine a pubkey with a certain number of zero bits in the exact place where the HLL algorithm described above would look for them in order to artificially make its reaction or follow "count more" than others. For this to work a different pubkey would have to be created for each different target (event id, followed profile etc). This approach is not very different than creating tons of new pubkeys and using them all to send likes or follow someone in order to inflate their number of followers. The solution is the same in both cases: clients should not fetch these reaction counts from open relays that accept everything, they should base their counts on relays that perform some form of filtering that makes it more likely that only real humans are able to publish there and not bots or artificially-generated pubkeys.
+
+### `hll` encoding
+
+The value `hll` value must be the concatenation of the 256 registers, each being a uint8 value (i.e. a byte). Therefore `hll` will be a 512-character hex string.
+
+### Client-side usage
+
+This algorithm also allows clients to combine HLL responses received from relays with HLL counts computed locally from raw events. It's recommended that clients keep track of HLL values locally and add to these on each message received from relays. For example:
+
+  - a client wants to keep track of the number of reactions an event Z has received over time;
+  - the client has decided it will read reactions from relays A, B and C (the NIP-65 "read" relays of Z's author);
+  - of these, only B and C support HLL responses, so the client fetches both and merges them locally;
+  - then the client fetches all reaction events from A then manually applies each event to the HLL from the previous step, using the same algorithm described above;
+  - finally, the client reads the estimate count from the HLL and displays that to the user;
+  - optionally the client may store that HLL value (together with some "last-read-date" for relay A) and repeat the process again later:
+    - this time it only needs to fetch the new reactions from A and add those to the HLL
+    - and redownload the HLL values from B and C and just reapply them to the local value.
+
+This procedure allows the client to download much less data.
+
 ## Examples
 
-### Followers count
-
-```
-["COUNT", <query_id>, {"kinds": [3], "#p": [<pubkey>]}]
-["COUNT", <query_id>, {"count": 238}]
-```
-
 ### Count posts and reactions
 
 ```
@@ -45,6 +97,7 @@ Whenever the relay decides to refuse to fulfill the `COUNT` request, it MUST ret
 ["COUNT", <query_id>, {"count": 5}]
 ```
 
+
 ### Count posts approximately
 
 ```
@@ -52,6 +105,13 @@ Whenever the relay decides to refuse to fulfill the `COUNT` request, it MUST ret
 ["COUNT", <query_id>, {"count": 93412452, "approximate": true}]
 ```
 
+### Followers count with HyperLogLog
+
+```
+["COUNT", <subscription_id>, {"kinds": [3], "#p": [<pubkey>]}]
+["COUNT", <subscription_id>, {"count": 16578, "hll": "0607070505060806050508060707070706090d080b0605090607070b07090606060b0705070709050807080805080407060906080707080507070805060509040a0b06060704060405070706080607050907070b08060808080b080607090a06060805060604070908050607060805050d05060906090809080807050e0705070507060907060606070708080b0807070708080706060609080705060604060409070a0808050a0506050b0810060a0908070709080b0a07050806060508060607080606080707050806080c0a0707070a080808050608080f070506070706070a0908090c080708080806090508060606090906060d07050708080405070708"}]
+```
+
 ### Relay refuses to count
 
 ```

69.md

@@ -41,8 +41,7 @@ Events are [addressable events](01.md#kinds) and use `38383` as event kind, a p2
     ["name", "Nakamoto"],
     ["g", "<geohash>"],
     ["bond", "0"],
-    ["expires_at", "1719391096"],
+    ["expiration", "1719391096"],
-    ["expiration", "1719995896"],
     ["y", "lnp2pbot"],
     ["z", "order"]
   ],
@@ -56,7 +55,7 @@ Events are [addressable events](01.md#kinds) and use `38383` as event kind, a p2
 - `d` < Order ID >: A unique identifier for the order.
 - `k` < Order type >: `sell` or `buy`.
 - `f` < Currency >: The asset being traded, using the [ISO 4217](https://en.wikipedia.org/wiki/ISO_4217) standard.
-- `s` < Status >: `pending`, `canceled`, `in-progress`, `success`, `expired`.
+- `s` < Status >: `pending`, `canceled`, `in-progress`, `success`.
 - `amt` < Amount >: The amount of Bitcoin to be traded, the amount is defined in satoshis, if `0` means that the amount of satoshis will be obtained from a public API after the taker accepts the order.
 - `fa` < Fiat amount >: The fiat amount being traded, for range orders two values are expected, the minimum and maximum amount.
 - `pm` < Payment method >: The payment method used for the trade, if the order has multiple payment methods, they should be separated by a comma.
@@ -68,8 +67,7 @@ Events are [addressable events](01.md#kinds) and use `38383` as event kind, a p2
 - `name` [Name]: The name of the maker.
 - `g` [Geohash]: The geohash of the operation, it can be useful in a face to face trade.
 - `bond` [Bond]: The bond amount, the bond is a security deposit that both parties must pay.
-- `expires_at` < Expires At\>: The expiration date of the event being published in `pending` status, after this time the event status SHOULD be changed to `expired`.
+- `expiration` < Expiration\>: The expiration date of the order ([NIP-40](40.md)).
-- `expiration` < Expiration\>: The expiration date of the event, after this time the relay SHOULD delete it ([NIP-40](40.md)).
 - `y` < Platform >: The platform that created the order.
 - `z` < Document >: `order`.
 

BE.md

@@ -1,137 +0,0 @@
-NIP-BE
-======
-
-Nostr BLE Communications Protocol
----------------------------------
-
-`draft` `optional`
-
-This NIP specifies how Nostr apps can use BLE to communicate and synchronize with each other. The BLE protocol follows a client-server pattern, so this NIP emulates the WS structure in a similar way, but with some adaptations to its limitations.
-
-## Device advertisement
-A device advertises itself with:
-- Service UUID: `0000180f-0000-1000-8000-00805f9b34fb`
-- Data: Device UUID in ByteArray format
-
-## GATT service
-The device exposes a Nordic UART Service with the following characteristics:
-
-1. Write Characteristic
-   - UUID: `87654321-0000-1000-8000-00805f9b34fb`
-   - Properties: Write
-
-2. Read Characteristic
-   - UUID: `12345678-0000-1000-8000-00805f9b34fb`
-   - Properties: Notify, Read
-
-## Role assignment
-
-When one device initially finds another advertising the service, it will read the service's data to get the device UUID and compare it with its own advertised device UUID. For this communication, the device with the highest ID will take the role of GATT Server (Relay), the other will be considered the GATT Client (Client) and will proceed to establish the connection.
-
-For devices whose purpose will require a single role, its device UUID will always be:
-
-- GATT Server: `FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF`
-- GATT Client: `00000000-0000-0000-0000-000000000000`
-
-## Messages
-
-All messages will follow [NIP-01](/01.md) message structure. For a given message, a compression stream (DEFLATE) is applied to the message to generate a byte array. Depending on the BLE version, the byte array can be too large for a single message (20-23 bytes in BLE 4.2, 256 bytes in BLE > 4.2). In that case, this byte array is split into any number of batches following the structure:
-
-```
-[batch index (first 2 bytes)][batch n][is last batch (last byte)]
-```
-After reception of all batches, the other device can then join them and decompress. To ensure reliability, only 1 message will be read/written at a time. MTU can be negotiated in advance. The maximum size for a message is 64KB; bigger messages will be rejected.
-
-## Examples
-
-This example implements a function to split and compress a byte array into chunks, as well as another function to join and decompress them in order to obtain the initial result:
-
-```kotlin
-fun splitInChunks(message: ByteArray): Array<ByteArray> {
-   val chunkSize = 500 // define the chunk size
-   var byteArray = compressByteArray(message)
-   val numChunks = (byteArray.size + chunkSize - 1) / chunkSize // calculate the number of chunks
-   var chunkIndex = 0
-   val chunks = Array(numChunks) { ByteArray(0) }
-
-   for (i in 0 until numChunks) {
-         val start = i * chunkSize
-         val end = minOf((i + 1) * chunkSize, byteArray.size)
-         val chunk = byteArray.copyOfRange(start, end)
-
-         // add chunk index to the first 2 bytes and last chunk flag to the last byte
-         val chunkWithIndex = ByteArray(chunk.size + 2)
-         chunkWithIndex[0] = chunkIndex.toByte() // chunk index
-         chunk.copyInto(chunkWithIndex, 1)
-         chunkWithIndex[chunkWithIndex.size - 1] = numChunks.toByte()
-
-         // store the chunk in the array
-         chunks[i] = chunkWithIndex
-
-         chunkIndex++
-   }
-
-   return chunks
-}
-
-fun joinChunks(chunks: Array<ByteArray>): ByteArray {
-   val sortedChunks = chunks.sortedBy { it[0] }
-   var reassembledByteArray = ByteArray(0)
-   for (chunk in sortedChunks) {
-         val chunkData = chunk.copyOfRange(1, chunk.size - 1)
-         reassembledByteArray = reassembledByteArray.copyOf(reassembledByteArray.size + chunkData.size)
-         chunkData.copyInto(reassembledByteArray, reassembledByteArray.size - chunkData.size)
-   }
-
-   return decompressByteArray(reassembledByteArray)
-}
-
-```
-
-## Workflows
-
-### Client to relay
-
-- Any message the client wants to send to a relay will be a write message.
-- Any message the client receives from a relay will be a read message.
-
-### Relay to client
-
-The relay should notify the client about any new event matching subscription's filters by using the Notify action of the Read Characteristic. After that, the client can proceed to read messages from the relay.
-
-### Device synchronization
-
-Given the nature of BLE, it is expected that the direct connection between two devices might be extremely intermittent, with gaps of hours or even days. That's why it's crucial to define a synchronization process by following [NIP-77](./77.md) but with an adaptation to the limitations of the technology.
-
-After two devices have successfully connected and established the Client-Server roles, the devices will use half-duplex communication to intermittently send and receive messages.
-
-#### Half-duplex synchronization
-
-Right after the 2 devices connect, the Client starts the workflow by sending the first message.
-
-1. Client - Writes ["NEG-OPEN"](/77.md#initial-message-client-to-relay) message.
-2. Server - Sends `write-success`.
-3. Client - Sends `read-message`.
-4. Server - Responds with ["NEG-MSG"](./77.md#subsequent-messages-bidirectional) message.
-5. Client -
-   1. If the Client has messages missing on the Server, it writes one `EVENT`.
-   2. If the Client doesn't have any messages missing on the Server, it writes `EOSE`. In this case, subsequent messages to the Server will be empty while the Server claims to have more notes for the Client.
-6. Server - Sends `write-success`.
-7. Client - Sends `read-message`.
-8. Server -
-   1. If the Server has messages missing on the Client, it responds with one `EVENT`.
-   2. If the Client doesn't have any messages missing on the Server, it responds with `EOSE`. In this case, subsequent responses to the Client will be empty.
-9. If the Client detects that the devices are not synchronized yet, jump to step 5.
-10. After the two devices detect that there are no more missing events on both ends, the workflow will pause at this point.
-
-#### Half-duplex event spread
-
-While two devices are connected and synchronized, it might happen that one of them receives a new message from another connected peer. Devices MUST keep track of which notes have been sent to its peers while they are connected. If the newly received event is detected as missing in one of the connected and synchronized peers:
-
-1. If the peer is a Server:
-   1. Client - It writes the `EVENT`.
-   2. Server - Sends `write-success`.
-2. If the peer is a Client:
-   1. Server - It will send an empty notification to the Client.
-   2. Client - Sends `read-message`.
-   3. Server - Responds with the `EVENT`.

README.md

@@ -105,7 +105,6 @@ They exist to document what may be implemented by [Nostr](https://github.com/nos
 - [NIP-A0: Voice Messages](A0.md)
 - [NIP-B0: Web Bookmarks](B0.md)
 - [NIP-B7: Blossom](B7.md)
-- [NIP-BE: Nostr BLE Communications Protocol](BE.md)
 - [NIP-C0: Code Snippets](C0.md)
 - [NIP-C7: Chats](C7.md)
 - [NIP-EE: E2EE Messaging using MLS Protocol](EE.md)