add recommendation for HLL client-side usage.

Co-authored-by: Jon Staab <shtaab@gmail.com>

17.md

@@ -6,9 +6,15 @@ Private Direct Messages
 
 `draft` `optional`
 
-This NIP defines an encrypted direct messaging scheme using [NIP-44](44.md) encryption and [NIP-59](59.md) seals and gift wraps.
+This NIP defines an encrypted chat scheme which uses [NIP-44](44.md) encryption and [NIP-59](59.md) seals and gift wraps.
 
-## Direct Message Kind
+Any event sent to an encrypted chat MUST NOT be signed, and MUST be encrypted as described in [NIP-59](./59.md) and illustrated below. Omitting signatures makes messages deniable in case they are accidentally or maliciously leaked, while still allowing the recipient to authenticate them.
+
+By convention, `kind 14` direct messages, `kind 15` file messages, and [`kind 7` reactions](./25.md) may be sent to an encrypted chat.
+
+## Kind Definitions
+
+### Chat Message
 
 Kind `14` is a chat message. `p` tags identify one or more receivers of the message.
 
@@ -31,7 +37,7 @@ Kind `14` is a chat message. `p` tags identify one or more receivers of the mess
 
 `.content` MUST be plain text. Fields `id` and `created_at` are required.
 
-An `e` tag denotes the direct parent message this post is replying to. 
+An `e` tag denotes the direct parent message this post is replying to.
 
 `q` tags MAY be used when citing events in the `.content` with [NIP-21](21.md).
 
@@ -39,9 +45,7 @@ An `e` tag denotes the direct parent message this post is replying to.
 ["q", "<event-id> or <event-address>", "<relay-url>", "<pubkey-if-a-regular-event>"]
 ```
 
-Kind `14`s MUST never be signed. If it is signed, the message might leak to relays and become **fully public**.
-
-## File Message Kind
+## File Message
 
 ```jsonc
 {
@@ -80,8 +84,6 @@ Kind `15` is used for sending encrypted file event messages:
 - `thumb` (optional) URL of thumbnail with same aspect ratio (encrypted with the same key, nonce)
 - `fallback` (optional) zero or more fallback file sources in case `url` fails (encrypted with the same key, nonce)
 
-Just like kind `14`, kind `15`s MUST never be signed.
-
 ## Chat Rooms
 
 The set of `pubkey` + `p` tags defines a chat room. If a new `p` tag is added or a current one is removed, a new room is created with a clean message history.
@@ -92,7 +94,7 @@ An optional `subject` tag defines the current name/topic of the conversation. An
 
 ## Encrypting
 
-Following [NIP-59](59.md), the **unsigned** `kind:14` & `kind:15` chat messages must be sealed (`kind:13`) and then gift-wrapped (`kind:1059`) to each receiver and the sender individually.
+Following [NIP-59](59.md), the **unsigned** chat messages must be sealed (`kind:13`) and then gift-wrapped (`kind:1059`) to each receiver and the sender individually.
 
 ```js
 {
@@ -127,7 +129,7 @@ Clients SHOULD randomize `created_at` in up to two days in the past in both the
 
 The gift wrap's `p` tag can be the receiver's main pubkey or an alias key created to receive DMs without exposing the receiver's identity.
 
-Clients CAN offer disappearing messages by setting an `expiration` tag in the gift wrap of each receiver or by not generating a gift wrap to the sender's public key
+Clients MAY offer disappearing messages by setting an `expiration` tag in the gift wrap of each receiver or by not generating a gift wrap to the sender's public key. This tag SHOULD be included on the `kind 13` seal as well, in case it leaks.
 
 ## Publishing
 
@@ -145,15 +147,13 @@ Kind `10050` indicates the user's preferred relays to receive DMs. The event MUS
 }
 ```
 
-Clients SHOULD publish the gift-wrapped kind 1059 events that contain the sealed kind 14 (text) or kind 15 (file) rumors to the relays listed in the recipient’s kind 10050 event. If that is not found that indicates the user is not ready to receive messages under this NIP and clients shouldn't try.
+Clients SHOULD publish the gift-wrapped `kind 1059` events that contain the sealed rumors to the relays listed in the recipient’s kind 10050 event. If that is not found that indicates the user is not ready to receive messages under this NIP and clients shouldn't try.
 
 ## Relays
 
-It's advisable that relays do not serve `kind:1059` to clients other than the ones tagged in them.
+Relays MAY protect message metadata by only serving `kind:1059` events to users p-tagged on the event (enforced using [NIP 42 AUTH](./42.md)).
 
-It's advisable that users choose relays that conform to these practices.
-
-Clients SHOULD guide users to keep `kind:10050` lists small (1-3 relays) and SHOULD spread it to as many relays as viable.
+Clients SHOULD guide users to keep `kind:10050` lists small (1-3 relays) and SHOULD spread them to as many relays as viable.
 
 ## Benefits & Limitations
 
@@ -170,12 +170,6 @@ This NIP offers the following privacy and security features:
 
 The main limitation of this approach is having to send a separate encrypted event to each receiver. Group chats with more than 100 participants should find a more suitable messaging scheme.
 
-## Implementation
-
-Clients implementing this NIP should by default only connect to the set of relays found in their `kind:10050` list. From that they should be able to load all messages both sent and received as well as get new live updates, making it for a very simple and lightweight implementation that should be fast.
-
-When sending a message to anyone, clients must then connect to the relays in the receiver's `kind:10050` and send the events there but can disconnect right after unless more messages are expected to be sent (e.g. the chat tab is still selected). Clients should also send a copy of their outgoing messages to their own `kind:10050` relay set.
-
 ## Examples
 
 This example sends the message `Hola, que tal?` from `nsec1w8udu59ydjvedgs3yv5qccshcj8k05fh3l60k9x57asjrqdpa00qkmr89m` to `nsec12ywtkplvyq5t6twdqwwygavp5lm4fhuang89c943nf2z92eez43szvn4dt`.

45.md

@@ -29,15 +29,67 @@ In case a relay uses probabilistic counts, it MAY indicate it in the response wi
 
 Whenever the relay decides to refuse to fulfill the `COUNT` request, it MUST return a `CLOSED` message.
 
+## HyperLogLog
+
+Relays may return an HyperLogLog value together with the count, hex-encoded.
+
+```
+["COUNT", <query_id>, {"count": <integer>, "hll": "<hex>"}]
+```
+
+This is so it enables merging results from multiple relays and yielding a reasonable estimate of reaction counts, comment counts and follower counts, while saving many millions of bytes of bandwidth for everybody.
+
+### Algorithm
+
+This section describes the steps a relay should take in order to return HLL values to clients.
+
+1. Upon receiving a filter, if it is eligible (see below) for HyperLogLog, compute the deterministic `offset` for that filter (see below);
+2. Initialize 256 registers to `0` for the HLL value;
+3. For all the events that are to be counted according to the filter, do this:
+    1. Read the byte at position `offset` of the event `pubkey`, its value will be the register index `ri`;
+    2. Count the number of leading zero bits starting at position `offset+1` of the event `pubkey` and add `1`;
+    3. Compare that with the value stored at register `ri`, if the new number is bigger, store it.
+
+That is all that has to be done on the relay side, and therefore the only part needed for interoperability.
+
+On the client side, these HLL values received from different relays can be merged (by simply going through all the registers in HLL values from each relay and picking the highest value for each register, regardless of the relay).
+
+And finally the absolute count can be estimated by running some methods I don't dare to describe here in English, it's better to check some implementation source code (also, there can be different ways of performing the estimation, with different quirks applied on top of the raw registers).
+
+### Filter eligibility and `offset` computation
+
+This NIP defines (for now) two filters eligible for HyperLogLog:
+
+- `{"#e": ["<id>"], "kinds": [7]}`, i.e. a filter for `kind:7` events with a single `"e"` tag, which means the client is interested in knowing how many people have reacted to the target event `<id>`. In this case the `offset` will be given by reading the character at the position `32` of the hex `<id>` value as a base-16 number then adding `8` to it.
+- `{"#e": ["<id>"], "kinds": [6]}`, the same as above, but for `kind:6` reposts.
+- `{"#p": ["<pubkey>"], "kinds": [3]}`, i.e. a filter for `kind:3` events with a single `"p"` tag, which means the client is interested in knowing how many people "follow" the target `<pubkey>`. In this case the `offset` will be given by reading the character at the position `32` of the hex `<pubkey>` value as a base-16 number then adding `8` to it.
+- `{"#E": ["<id>"], "kinds": [1111]}`, i.e. a filter for the total number of comments any specific root event has received. In this case the `offset` will be given by reading the character at the position `32` of the hex `<id>` value as a base-16 number then adding `8` to it.
+
+### Attack vectors
+
+One could mine a pubkey with a certain number of zero bits in the exact place where the HLL algorithm described above would look for them in order to artificially make its reaction or follow "count more" than others. For this to work a different pubkey would have to be created for each different target (event id, followed profile etc). This approach is not very different than creating tons of new pubkeys and using them all to send likes or follow someone in order to inflate their number of followers. The solution is the same in both cases: clients should not fetch these reaction counts from open relays that accept everything, they should base their counts on relays that perform some form of filtering that makes it more likely that only real humans are able to publish there and not bots or artificially-generated pubkeys.
+
+### `hll` encoding
+
+The value `hll` value must be the concatenation of the 256 registers, each being a uint8 value (i.e. a byte). Therefore `hll` will be a 512-character hex string.
+
+### Client-side usage
+
+This algorithm also allows clients to combine HLL responses received from relays with HLL counts computed locally from raw events. It's recommended that clients keep track of HLL values locally and add to these on each message received from relays. For example:
+
+  - a client wants to keep track of the number of reactions an event Z has received over time;
+  - the client has decided it will read reactions from relays A, B and C (the NIP-65 "read" relays of Z's author);
+  - of these, only B and C support HLL responses, so the client fetches both and merges them locally;
+  - then the client fetches all reaction events from A then manually applies each event to the HLL from the previous step, using the same algorithm described above;
+  - finally, the client reads the estimate count from the HLL and displays that to the user;
+  - optionally the client may store that HLL value (together with some "last-read-date" for relay A) and repeat the process again later:
+    - this time it only needs to fetch the new reactions from A and add those to the HLL
+    - and redownload the HLL values from B and C and just reapply them to the local value.
+
+This procedure allows the client to download much less data.
+
 ## Examples
 
-### Followers count
-
-```
-["COUNT", <query_id>, {"kinds": [3], "#p": [<pubkey>]}]
-["COUNT", <query_id>, {"count": 238}]
-```
-
 ### Count posts and reactions
 
 ```
@@ -45,6 +97,7 @@ Whenever the relay decides to refuse to fulfill the `COUNT` request, it MUST ret
 ["COUNT", <query_id>, {"count": 5}]
 ```
 
+
 ### Count posts approximately
 
 ```
@@ -52,6 +105,13 @@ Whenever the relay decides to refuse to fulfill the `COUNT` request, it MUST ret
 ["COUNT", <query_id>, {"count": 93412452, "approximate": true}]
 ```
 
+### Followers count with HyperLogLog
+
+```
+["COUNT", <subscription_id>, {"kinds": [3], "#p": [<pubkey>]}]
+["COUNT", <subscription_id>, {"count": 16578, "hll": "0607070505060806050508060707070706090d080b0605090607070b07090606060b0705070709050807080805080407060906080707080507070805060509040a0b06060704060405070706080607050907070b08060808080b080607090a06060805060604070908050607060805050d05060906090809080807050e0705070507060907060606070708080b0807070708080706060609080705060604060409070a0808050a0506050b0810060a0908070709080b0a07050806060508060607080606080707050806080c0a0707070a080808050608080f070506070706070a0908090c080708080806090508060606090906060d07050708080405070708"}]
+```
+
 ### Relay refuses to count
 
 ```

60.md

@@ -50,6 +50,7 @@ There can be multiple `kind:7375` events for the same mint, and multiple proofs
     "kind": 7375,
     "content": nip44_encrypt({
         "mint": "https://stablenut.umint.cash",
+        "unit": "sat",
         "proofs": [
             // one or more proofs in the default cashu format
             {
@@ -69,6 +70,7 @@ There can be multiple `kind:7375` events for the same mint, and multiple proofs
  * `.content` is a [NIP-44](44.md) encrypted payload:
    * `mint`: The mint the proofs belong to.
    * `proofs`: unencoded proofs
+   * `unit` the base unit the proofs are denominated in (eg: `sat`, `usd`, `eur`). Default: `sat` if omitted.
    * `del`: token-ids that were destroyed by the creation of this token. This assists with state transitions.
 
 When one or more proofs of a token are spent, the token event should be [NIP-09](09.md)-deleted and, if some proofs are unspent from the same token event, a new token event should be created rolling over the unspent proofs and adding any change outputs to the new token event (the change output should include a `del` field).
@@ -84,6 +86,7 @@ Clients SHOULD publish `kind:7376` events to create a transaction history when t
     "content": nip44_encrypt([
         [ "direction", "in" ], // in = received, out = sent
         [ "amount", "1" ],
+        [ "unit", "sat" ],
         [ "e", "<event-id-of-created-token>", "", "created" ]
     ]),
     "tags": [
@@ -93,6 +96,7 @@ Clients SHOULD publish `kind:7376` events to create a transaction history when t
 ```
 
 * `direction` - The direction of the transaction; `in` for received funds, `out` for sent funds.
+* `unit` the base unit of the amount (eg: `sat`, `usd`, `eur`). Default: `sat` if omitted.
 
 Clients MUST add `e` tags to create references of destroyed and created token events along with the marker of the meaning of the tag:
 * `created` - A new token event was created.
@@ -121,6 +125,7 @@ If Alice spends 4 sats from this token event
     "id": "event-id-1",
     "content": nip44_encrypt({
         "mint": "https://stablenut.umint.cash",
+        "unit": "sat",
         "proofs": [
             { "id": "1", "amount": 1 },
             { "id": "2", "amount": 2 },
@@ -140,6 +145,7 @@ Her client:
     "id": "event-id-2",
     "content": nip44_encrypt({
         "mint": "https://stablenut.umint.cash",
+        "unit": "sat",
         "proofs": [
             { "id": "1", "amount": 1 },
             { "id": "2", "amount": 2 },
@@ -159,6 +165,7 @@ Her client:
     "content": nip44_encrypt([
         [ "direction", "out" ],
         [ "amount", "4" ],
+        [ "unit", "sat" ],
         [ "e", "<event-id-1>", "", "destroyed" ],
         [ "e", "<event-id-2>", "", "created" ],
     ]),

61.md

@@ -51,6 +51,7 @@ Clients MUST prefix the public key they P2PK-lock with `"02"` (for nostr<>cashu
     "pubkey": "<sender-pubkey>",
     "tags": [
         [ "proof", "{\"amount\":1,\"C\":\"02277c66191736eb72fce9d975d08e3191f8f96afb73ab1eec37e4465683066d3f\",\"id\":\"000a93d6f8a1d2c4\",\"secret\":\"[\\\"P2PK\\\",{\\\"nonce\\\":\\\"b00bdd0467b0090a25bdf2d2f0d45ac4e355c482c1418350f273a04fedaaee83\\\",\\\"data\\\":\\\"02eaee8939e3565e48cc62967e2fde9d8e2a4b3ec0081f29eceff5c64ef10ac1ed\\\"}]\"}" ],
+        [ "unit", "sat" ],
         [ "u", "https://stablenut.umint.cash" ],
         [ "e", "<nutzapped-event-id>", "<relay-hint>" ],
         [ "k", "<nutzapped-kind>"],
@@ -62,6 +63,7 @@ Clients MUST prefix the public key they P2PK-lock with `"02"` (for nostr<>cashu
 * `.content` is an optional comment for the nutzap
 * `.tags`:
   * `proof` is one or more proofs P2PK-locked to the public key the recipient specified in their `kind:10019` event and including a DLEQ proof.
+  * `unit` the base unit the proofs are denominated in (eg: `sat`, `usd`, `eur`). Default: `sat` if omitted.
   * `u` is the mint the URL of the mint EXACTLY as specified by the recipient's `kind:10019`.
   * `p` is the Nostr identity public key of nutzap recipient.
   * `e` is the event that is being nutzapped, if any.
@@ -95,6 +97,7 @@ Multiple `kind:9321` events can be tagged in the same `kind:7376` event.
     "content": nip44_encrypt([
         [ "direction", "in" ], // in = received, out = sent
         [ "amount", "1" ],
+        [ "unit", "sat" ],
         [ "e", "<7375-event-id>", "<relay-hint>", "created" ] // new token event that was created
     ]),
     "tags": [

README.md

@@ -58,6 +58,7 @@ They exist to document what may be implemented by [Nostr](https://github.com/nos
 - [NIP-39: External Identities in Profiles](39.md)
 - [NIP-40: Expiration Timestamp](40.md)
 - [NIP-42: Authentication of clients to relays](42.md)
+- [NIP-43: Relay Access Metadata and Requests](43.md)
 - [NIP-44: Encrypted Payloads (Versioned)](44.md)
 - [NIP-45: Counting results](45.md)
 - [NIP-46: Nostr Remote Signing](46.md)
@@ -182,6 +183,8 @@ They exist to document what may be implemented by [Nostr](https://github.com/nos
 | `7376`        | Cashu Wallet History            | [60](60.md)                            |
 | `7516`        | Geocache log                    | [geocaching][geocaching]               |
 | `7517`        | Geocache proof of find          | [geocaching][geocaching]               |
+| `8000`        | Add User                        | [43](43.md)                            |
+| `8001`        | Remove User                     | [43](43.md)                            |
 | `9000`-`9030` | Group Control Events            | [29](29.md)                            |
 | `9041`        | Zap Goal                        | [75](75.md)                            |
 | `9321`        | Nutzap                          | [61](61.md)                            |
@@ -213,6 +216,7 @@ They exist to document what may be implemented by [Nostr](https://github.com/nos
 | `10377`       | Proxy Announcement              | [Nostr Epoxy][nostr-epoxy]             |
 | `11111`       | Transport Method Announcement   | [Nostr Epoxy][nostr-epoxy]             |
 | `13194`       | Wallet Info                     | [47](47.md)                            |
+| `13534`       | Membership Lists                | [43](43.md)                            |
 | `17375`       | Cashu Wallet Event              | [60](60.md)                            |
 | `21000`       | Lightning Pub RPC               | [Lightning.Pub][lnpub]                 |
 | `22242`       | Client Authentication           | [42](42.md)                            |
@@ -221,6 +225,9 @@ They exist to document what may be implemented by [Nostr](https://github.com/nos
 | `24133`       | Nostr Connect                   | [46](46.md)                            |
 | `24242`       | Blobs stored on mediaservers    | [Blossom][blossom]                     |
 | `27235`       | HTTP Auth                       | [98](98.md)                            |
+| `28934`       | Join Request                    | [43](43.md)                            |
+| `28935`       | Invite Request                  | [43](43.md)                            |
+| `28936`       | Leave Request                   | [43](43.md)                            |
 | `30000`       | Follow sets                     | [51](51.md)                            |
 | `30001`       | Generic lists                   | 51 (deprecated)                        |
 | `30002`       | Relay sets                      | [51](51.md)                            |