Merge bitcoin/bitcoin#32479: [27.x] Backports

1d8856dddf doc: update release notes for 27.x (fanquake)
aef658601d depends: use "mkdir -p" when installing xproto (fanquake)
a328170e09 guix: accomodate migration to codeberg (fanquake)
e0af7bffff ci: remove --enable-external-signer (fanquake)
882166642c scripted-diff: Use bpf_cflags (MarcoFalke)
1fc7b9d260 test: Add imports for util bpf_cflags (MarcoFalke)
f52ab00f69 build: use make < 3.82 syntax for define directive (Sjors Provoost)
418bee8f9d refactor: Remove spurious virtual from final ~CZMQNotificationInterface (MarcoFalke)
992c7cad46 test: Handle empty string returned by CLI as None in RPC tests (Brandon Odiwuor)

Pull request description:

  Backports
  - #32070
  - #32187
  - #32286
  - #32439
  - #32568

ACKs for top commit:
  willcl-ark:
    ACK 1d8856dddf

Tree-SHA512: ca8534c24a806a35528f0d40a201fe0c66aa101a8cbeff9ef18d7599c300e6ca47d33563cd84198f1c15d77da7a141e8fcc0a2bd0a40165da6142dbbf1dcdc6c

ci/test/03_test_script.sh

@@ -119,7 +119,7 @@ if [ -n "$ANDROID_TOOLS_URL" ]; then
   exit 0
 fi
 
-BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} --enable-external-signer --prefix=$BASE_OUTDIR"
+BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} --prefix=$BASE_OUTDIR"
 
 if [ -n "$CONFIG_SHELL" ]; then
   "$CONFIG_SHELL" -c "./autogen.sh"

contrib/guix/INSTALL.md

@@ -319,7 +319,7 @@ Source: https://logs.guix.gnu.org/guix/2020-11-12.log#232527
 Start by cloning Guix:
 
 ```
-git clone https://git.savannah.gnu.org/git/guix.git
+git clone https://codeberg.org/guix/guix.git
 cd guix
 ```
 
@@ -607,7 +607,7 @@ checklist.
    ```
    Generation 38   Feb 22 2021 16:39:31    (current)
      guix f350df4
-       repository URL: https://git.savannah.gnu.org/git/guix.git
+       repository URL: https://codeberg.org/guix/guix.git
        branch: version-1.2.0
        commit: f350df405fbcd5b9e27e6b6aa500da7f101f41e7
    ```
@@ -757,8 +757,8 @@ Please see the following links for more details:
 
 - An upstream coreutils bug has been filed: [debbugs#47940](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47940)
 - A Guix bug detailing the underlying problem has been filed: [guix-issues#47935](https://issues.guix.gnu.org/47935), [guix-issues#49985](https://issues.guix.gnu.org/49985#5)
-- A commit to skip this test in Guix has been merged into the core-updates branch:
-[savannah/guix@6ba1058](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=6ba1058df0c4ce5611c2367531ae5c3cdc729ab4)
+- A commit to skip this test is included since Guix 1.4.0:
+[codeberg/guix@6ba1058](https://codeberg.org/guix/guix/commit/6ba1058df0c4ce5611c2367531ae5c3cdc729ab4)
 
 
 [install-script]: #options-1-and-2-using-the-official-shell-installer-script-or-binary-tarball

contrib/guix/libexec/prelude.bash

@@ -50,7 +50,7 @@ fi
 # across time.
 time-machine() {
     # shellcheck disable=SC2086
-    guix time-machine --url=https://git.savannah.gnu.org/git/guix.git \
+    guix time-machine --url=https://codeberg.org/guix/guix.git \
                       --commit=d5ca4d4fd713a9f7e17e074a1e37dda99bbb09fc \
                       --cores="$JOBS" \
                       --keep-failed \

depends/packages/capnp.mk

@@ -9,7 +9,7 @@ $(package)_sha256_hash=$(native_$(package)_sha256_hash)
 # setting in depends/config.site.in, which also hardcodes "lib".
 # Without this setting, cmake by default would use the OS library
 # directory, which might be "lib64" or something else, not "lib", on multiarch systems.
-define $(package)_set_vars :=
+define $(package)_set_vars
   $(package)_config_opts := -DBUILD_TESTING=OFF
   $(package)_config_opts += -DWITH_OPENSSL=OFF
   $(package)_config_opts += -DWITH_ZLIB=OFF

depends/packages/libmultiprocess.mk

@@ -12,7 +12,7 @@ endif
 # setting in depends/config.site.in, which also hardcodes "lib".
 # Without this setting, cmake by default would use the OS library
 # directory, which might be "lib64" or something else, not "lib", on multiarch systems.
-define $(package)_set_vars :=
+define $(package)_set_vars
 $(package)_config_opts += -DCMAKE_INSTALL_LIBDIR=lib/
 $(package)_config_opts += -DCMAKE_POSITION_INDEPENDENT_CODE=ON
 ifneq ($(host),$(build))

depends/packages/xproto.mk

@@ -21,6 +21,8 @@ define $(package)_build_cmds
   $(MAKE)
 endef
 
+# mkdir detection is broken on Alpine. Set MKDIRPROG to ensure we always
+# use "mkdir -p", and avoid parallelism issues during install.
 define $(package)_stage_cmds
-  $(MAKE) DESTDIR=$($(package)_staging_dir) install
+  $(MAKE) MKDIRPROG="mkdir -p" DESTDIR=$($(package)_staging_dir) install
 endef

doc/release-notes.md

@@ -40,18 +40,26 @@ unsupported systems.
 Notable changes
 ===============
 
+External signing is not currently supported when compiling with Boost version 1.88.0 or later.
+
 ### Test
 
 - #31419 test: fix MIN macro redefinition
+- #32286 test: Handle empty string returned by CLI as None in RPC tests
+- #32336 test: Suppress upstream -Wduplicate-decl-specifier in bpfcc
 
 ### Build
 
 - #31502 depends: Fix CXXFLAGS on NetBSD
 - #31627 depends: Fix spacing issue
+- #32070 build: use make < 3.82 syntax for define directive
+- #32439 guix: accomodate migration to codeberg
+- #32568 depends: use "mkdir -p" when installing xproto
 
 ### Misc
 
 - #31623 tracing: Rename the MIN macro to TRACEPOINT_TEST_MIN in log_raw_p2p_msgs
+- #32187 refactor: Remove spurious virtual from final ~CZMQNotificationInterface
 
 
 Credits
@@ -60,7 +68,11 @@ Credits
 Thanks to everyone who directly contributed to this release:
 
 - 0xb10c
+- Brandon Odiwuor
+- fanquake
 - Hennadii Stepanov
+- MarcoFalke
+- Sjors Provoost
 
 As well as to everyone that helped with translations on
 [Transifex](https://www.transifex.com/bitcoin/bitcoin/).

src/zmq/zmqnotificationinterface.h

@@ -1,4 +1,4 @@
-// Copyright (c) 2015-2022 The Bitcoin Core developers
+// Copyright (c) 2015-present The Bitcoin Core developers
 // Distributed under the MIT software license, see the accompanying
 // file COPYING or http://www.opensource.org/licenses/mit-license.php.
 
@@ -21,7 +21,7 @@ struct NewMempoolTransactionInfo;
 class CZMQNotificationInterface final : public CValidationInterface
 {
 public:
-    virtual ~CZMQNotificationInterface();
+    ~CZMQNotificationInterface();
 
     std::list<const CZMQAbstractNotifier*> GetActiveNotifiers() const;
 

test/functional/interface_usdt_coinselection.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-# Copyright (c) 2022 The Bitcoin Core developers
+# Copyright (c) 2022-present The Bitcoin Core developers
 # Distributed under the MIT software license, see the accompanying
 # file COPYING or http://www.opensource.org/licenses/mit-license.php.
 
@@ -17,6 +17,7 @@ from test_framework.util import (
     assert_equal,
     assert_greater_than,
     assert_raises_rpc_error,
+    bpf_cflags,
 )
 
 coinselection_tracepoints_program = """
@@ -166,7 +167,7 @@ class CoinSelectionTracepointTest(BitcoinTestFramework):
         ctx.enable_probe(probe="coin_selection:normal_create_tx_internal", fn_name="trace_normal_create_tx")
         ctx.enable_probe(probe="coin_selection:attempting_aps_create_tx", fn_name="trace_attempt_aps")
         ctx.enable_probe(probe="coin_selection:aps_create_tx_internal", fn_name="trace_aps_create_tx")
-        self.bpf = BPF(text=coinselection_tracepoints_program, usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+        self.bpf = BPF(text=coinselection_tracepoints_program, usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
 
         self.log.info("Prepare wallets")
         self.generate(self.nodes[0], 101)

test/functional/interface_usdt_mempool.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-# Copyright (c) 2022 The Bitcoin Core developers
+# Copyright (c) 2022-present The Bitcoin Core developers
 # Distributed under the MIT software license, see the accompanying
 # file COPYING or http://www.opensource.org/licenses/mit-license.php.
 
@@ -19,7 +19,10 @@ from test_framework.blocktools import COINBASE_MATURITY
 from test_framework.messages import COIN, DEFAULT_MEMPOOL_EXPIRY_HOURS
 from test_framework.p2p import P2PDataStore
 from test_framework.test_framework import BitcoinTestFramework
-from test_framework.util import assert_equal
+from test_framework.util import (
+    assert_equal,
+    bpf_cflags,
+)
 from test_framework.wallet import MiniWallet
 
 MEMPOOL_TRACEPOINTS_PROGRAM = """
@@ -144,7 +147,7 @@ class MempoolTracepointTest(BitcoinTestFramework):
         node = self.nodes[0]
         ctx = USDT(pid=node.process.pid)
         ctx.enable_probe(probe="mempool:added", fn_name="trace_added")
-        bpf = BPF(text=MEMPOOL_TRACEPOINTS_PROGRAM, usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+        bpf = BPF(text=MEMPOOL_TRACEPOINTS_PROGRAM, usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
 
         def handle_added_event(_, data, __):
             events.append(bpf["added_events"].event(data))
@@ -181,7 +184,7 @@ class MempoolTracepointTest(BitcoinTestFramework):
         node = self.nodes[0]
         ctx = USDT(pid=node.process.pid)
         ctx.enable_probe(probe="mempool:removed", fn_name="trace_removed")
-        bpf = BPF(text=MEMPOOL_TRACEPOINTS_PROGRAM, usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+        bpf = BPF(text=MEMPOOL_TRACEPOINTS_PROGRAM, usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
 
         def handle_removed_event(_, data, __):
             events.append(bpf["removed_events"].event(data))
@@ -227,7 +230,7 @@ class MempoolTracepointTest(BitcoinTestFramework):
         node = self.nodes[0]
         ctx = USDT(pid=node.process.pid)
         ctx.enable_probe(probe="mempool:replaced", fn_name="trace_replaced")
-        bpf = BPF(text=MEMPOOL_TRACEPOINTS_PROGRAM, usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+        bpf = BPF(text=MEMPOOL_TRACEPOINTS_PROGRAM, usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
 
         def handle_replaced_event(_, data, __):
             events.append(bpf["replaced_events"].event(data))
@@ -278,7 +281,7 @@ class MempoolTracepointTest(BitcoinTestFramework):
         self.log.info("Hooking into mempool:rejected tracepoint...")
         ctx = USDT(pid=node.process.pid)
         ctx.enable_probe(probe="mempool:rejected", fn_name="trace_rejected")
-        bpf = BPF(text=MEMPOOL_TRACEPOINTS_PROGRAM, usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+        bpf = BPF(text=MEMPOOL_TRACEPOINTS_PROGRAM, usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
 
         def handle_rejected_event(_, data, __):
             events.append(bpf["rejected_events"].event(data))

test/functional/interface_usdt_net.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-# Copyright (c) 2022 The Bitcoin Core developers
+# Copyright (c) 2022-present The Bitcoin Core developers
 # Distributed under the MIT software license, see the accompanying
 # file COPYING or http://www.opensource.org/licenses/mit-license.php.
 
@@ -17,7 +17,10 @@ except ImportError:
 from test_framework.messages import msg_version
 from test_framework.p2p import P2PInterface
 from test_framework.test_framework import BitcoinTestFramework
-from test_framework.util import assert_equal
+from test_framework.util import (
+    assert_equal,
+    bpf_cflags,
+)
 
 # Tor v3 addresses are 62 chars + 6 chars for the port (':12345').
 MAX_PEER_ADDR_LENGTH = 68
@@ -115,7 +118,7 @@ class NetTracepointTest(BitcoinTestFramework):
                          fn_name="trace_inbound_message")
         ctx.enable_probe(probe="net:outbound_message",
                          fn_name="trace_outbound_message")
-        bpf = BPF(text=net_tracepoints_program, usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+        bpf = BPF(text=net_tracepoints_program, usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
 
         EXPECTED_INOUTBOUND_VERSION_MSG = 1
         checked_inbound_version_msg = 0

test/functional/interface_usdt_utxocache.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-# Copyright (c) 2022 The Bitcoin Core developers
+# Copyright (c) 2022-present The Bitcoin Core developers
 # Distributed under the MIT software license, see the accompanying
 # file COPYING or http://www.opensource.org/licenses/mit-license.php.
 
@@ -15,7 +15,10 @@ except ImportError:
     pass
 from test_framework.messages import COIN
 from test_framework.test_framework import BitcoinTestFramework
-from test_framework.util import assert_equal
+from test_framework.util import (
+    assert_equal,
+    bpf_cflags,
+)
 from test_framework.wallet import MiniWallet
 
 utxocache_changes_program = """
@@ -175,7 +178,7 @@ class UTXOCacheTracepointTest(BitcoinTestFramework):
         ctx = USDT(pid=self.nodes[0].process.pid)
         ctx.enable_probe(probe="utxocache:uncache",
                          fn_name="trace_utxocache_uncache")
-        bpf = BPF(text=utxocache_changes_program, usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+        bpf = BPF(text=utxocache_changes_program, usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
 
         # The handle_* function is a ctypes callback function called from C. When
         # we assert in the handle_* function, the AssertError doesn't propagate
@@ -244,7 +247,7 @@ class UTXOCacheTracepointTest(BitcoinTestFramework):
         ctx.enable_probe(probe="utxocache:add", fn_name="trace_utxocache_add")
         ctx.enable_probe(probe="utxocache:spent",
                          fn_name="trace_utxocache_spent")
-        bpf = BPF(text=utxocache_changes_program, usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+        bpf = BPF(text=utxocache_changes_program, usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
 
         # The handle_* function is a ctypes callback function called from C. When
         # we assert in the handle_* function, the AssertError doesn't propagate
@@ -333,7 +336,7 @@ class UTXOCacheTracepointTest(BitcoinTestFramework):
         ctx = USDT(pid=self.nodes[0].process.pid)
         ctx.enable_probe(probe="utxocache:flush",
                          fn_name="trace_utxocache_flush")
-        bpf = BPF(text=utxocache_flushes_program, usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+        bpf = BPF(text=utxocache_flushes_program, usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
 
         # The handle_* function is a ctypes callback function called from C. When
         # we assert in the handle_* function, the AssertError doesn't propagate
@@ -390,7 +393,7 @@ class UTXOCacheTracepointTest(BitcoinTestFramework):
         ctx = USDT(pid=self.nodes[0].process.pid)
         ctx.enable_probe(probe="utxocache:flush",
                          fn_name="trace_utxocache_flush")
-        bpf = BPF(text=utxocache_flushes_program, usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+        bpf = BPF(text=utxocache_flushes_program, usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
         bpf["utxocache_flush"].open_perf_buffer(handle_utxocache_flush)
 
         self.log.info(f"prune blockchain to trigger a flush for pruning")

test/functional/interface_usdt_validation.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-# Copyright (c) 2022 The Bitcoin Core developers
+# Copyright (c) 2022-present The Bitcoin Core developers
 # Distributed under the MIT software license, see the accompanying
 # file COPYING or http://www.opensource.org/licenses/mit-license.php.
 
@@ -17,8 +17,10 @@ except ImportError:
 
 from test_framework.address import ADDRESS_BCRT1_UNSPENDABLE
 from test_framework.test_framework import BitcoinTestFramework
-from test_framework.util import assert_equal
-
+from test_framework.util import (
+    assert_equal,
+    bpf_cflags,
+)
 
 validation_blockconnected_program = """
 #include <uapi/linux/ptrace.h>
@@ -94,7 +96,7 @@ class ValidationTracepointTest(BitcoinTestFramework):
         ctx.enable_probe(probe="validation:block_connected",
                          fn_name="trace_block_connected")
         bpf = BPF(text=validation_blockconnected_program,
-                  usdt_contexts=[ctx], debug=0, cflags=["-Wno-error=implicit-function-declaration"])
+                  usdt_contexts=[ctx], debug=0, cflags=bpf_cflags())
 
         def handle_blockconnected(_, data, __):
             event = ctypes.cast(data, ctypes.POINTER(Block)).contents

test/functional/test_framework/test_node.py

@@ -887,6 +887,8 @@ class TestNodeCLI():
             # Ignore cli_stdout, raise with cli_stderr
             raise subprocess.CalledProcessError(returncode, self.binary, output=cli_stderr)
         try:
+            if not cli_stdout.strip():
+                return None
             return json.loads(cli_stdout, parse_float=decimal.Decimal)
         except (json.JSONDecodeError, decimal.InvalidOperation):
             return cli_stdout.rstrip("\n")

test/functional/test_framework/util.py

@@ -277,6 +277,13 @@ def wait_until_helper_internal(predicate, *, attempts=float('inf'), timeout=floa
     raise RuntimeError('Unreachable')
 
 
+def bpf_cflags():
+    return [
+        "-Wno-error=implicit-function-declaration",
+        "-Wno-duplicate-decl-specifier",  # https://github.com/bitcoin/bitcoin/issues/32322
+    ]
+
+
 def sha256sum_file(filename):
     h = hashlib.sha256()
     with open(filename, 'rb') as f: