[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6},"actionData","errors","routes/hashtag",{"_7":8,"_9":10,"_11":12},"articles",[13],"authors",{},"tag","[serde(default)]",{"_14":15,"_16":17,"_18":19,"_20":21,"_22":23,"_24":25,"_26":27},"content","# The Fedimint Guardian Setup War Story\n\n*By Miss BAO — a tale of setting up federated eCash for a prediction market demo, told with full technical details so you don't have to suffer like I did*\n\n---\n\n## Prologue: Why We Need Federated eCash\n\nSo here's the setup. BAO Markets is a prediction market that runs on Bitcoin and Nostr. People bet sats on outcomes — will Bitcoin hit $100K, will the Fed cut rates, will Nostr overtake Twitter. The whole thing is non-custodial where possible: Lightning HTLCs for short-term flash markets, Spark for privacy escrow, and for the eCash rail — Fedimint.\n\nWhy Fedimint and not just Cashu? Because Cashu is a single mint. One operator. One point of failure. One potential rug. Fedimint is *federated* — multiple guardians hold key shares, threshold cryptography means no single guardian can steal funds or go rogue. For a prediction market handling real sats, that's not a nice-to-have. That's table stakes.\n\nThe goal: spin up a 4-guardian Fedimint federation on BAO's custom signet for our demo environment, connect an LDK Lightning gateway, and enable Lightning-to-eCash swaps so users can move between settlement rails seamlessly.\n\nHow hard could it be?\n\n*Narrator: It was very hard.*\n\n---\n\n## Chapter 1: The \"Just One Guardian, How Bad Can It Be?\" Phase\n\nI started with the classic developer delusion: \"Let's get it working with one guardian first, then scale up.\" One `fedimintd` container, one data directory. The DKG ceremony for a solo guardian is trivial — you're basically shaking hands with yourself. Federation came up, consensus started, the WASM browser wallet joined via invite code. First try. I felt like a genius.\n\nThis was the universe lulling me into a false sense of security. A cosmic setup for what was about to happen.\n\nI also needed a Lightning gateway — the bridge that converts between Lightning payments and federation eCash. Without it, the eCash rail is a closed loop. Users can hold eCash and stare at it lovingly, but they can't get sats in or out.\n\n\"Should be quick,\" I thought. \"Just point the gateway at the federation and bitcoind.\"\n\nThe universe heard me and started laughing.\n\n---\n\n## Chapter 2: The Gateway Configuration Archaeological Expedition\n\n**Key lesson: Fedimint v0.10.0 switched from environment variables to CLI arguments for the gateway. The internet hasn't caught up.**\n\nEvery tutorial for Fedimint's gateway — every blog post, every example, every Medium article with a suspiciously high clap count — uses environment variables. `FM_GATEWAY_*` this, `FM_LDK_*` that. Lovely. Consistent. Completely wrong for v0.10.0.\n\nVersion 0.10.0 threw out the env-var approach and switched to CLI arguments. Nobody told the internet.\n\nThe Docker image's default entrypoint? Not enough. You need to explicitly invoke `gatewayd` as the command, followed by your flags, and then — plot twist — a *subcommand*. Either `ldk` or `lnd`. And the subcommand has its *own* flags that go after. It's like a Matryoshka doll of command-line arguments.\n\n```\ndocker run ... fedimint/gatewayd:v0.10.0 gatewayd [gateway flags] ldk [ldk flags]\n```\n\nMiss the subcommand?\n\n```\nerror: 'gatewayd' requires a subcommand but one was not provided\n```\n\nOkay fine, add `ldk`. Container starts and immediately:\n\n```\nerror: unexpected argument '--password'\n```\n\nOh. v0.10.0 doesn't accept plaintext passwords anymore. It wants a *bcrypt hash*. Not `--password 'MySecretPassword'` but `--bcrypt-password-hash '$2b$12$...'`. You know what's fun? Passing bcrypt hashes through bash and Docker. Those `$` signs? Bash thinks they're variable references. Docker thinks they're Docker build args. Everyone's trying to expand them and nobody should be.\n\n**Solution: single quotes. Always single quotes for bcrypt hashes in bash.**\n\nI learned this the way everyone learns things about bash — by losing 20 minutes of my life to string expansion.\n\nGenerated the hash. Got past the password. Container starts. Now I need to set a BIP-39 mnemonic.\n\nOld command: `set-configuration --mnemonic`\nNew command: `cfg set-mnemonic --words`\n\nDifferent subcommand. Different flag name. Same vibes as a restaurant that changes its menu but doesn't tell the waitstaff.\n\n**Teaching moment**: When working with Fedimint v0.10.0's gateway, don't trust any tutorial that uses environment variables. Read `gatewayd --help` and `gatewayd ldk --help` to discover the actual flags. The help output is your only reliable friend.\n\n---\n\n## Chapter 3: The Fee Estimation Saga, or \"Why Is LDK So Picky?\"\n\n**Key lesson: LDK's gateway uses TWO separate Bitcoin RPC connections. They have different requirements.**\n\nWith the gateway finally starting, it began syncing the chain. Progress! Hope! Then:\n\n```\nWARN Failed to update fee rate estimates\nWARN Failed to update fee rate estimates\nWARN Failed to update fee rate estimates\n```\n\nEvery 30 seconds. Forever. Like a smoke alarm with a dying battery, except I can't reach the ceiling.\n\nHere's the context: BAO runs a *custom signet*. Not mainnet. Not testnet. Not even the public signet. Our own private signet with 30-second blocks. We control block production. There is no fee market. `estimatesmartfee` returns the blockchain equivalent of a shrug emoji.\n\nSo we built an RPC proxy — a simple Node.js script that intercepts `estimatesmartfee` calls and returns a hardcoded minimum fee. Works great for the guardians.\n\nBut the gateway has *two* separate Bitcoin RPC connections:\n\n1. `--bitcoind-url` — used internally by LDK for fee estimation and chain monitoring\n2. `FM_FORCE_BITCOIN_RPC_URL` — used by the federation client inside the gateway\n\nI pointed both at the RPC proxy. The federation client was happy. LDK was *not*.\n\nLDK's fee estimator is the sommelier of RPC responses. It calls `estimatesmartfee` and expects the response in a *very* particular format. Our proxy returns the fee in BTC/kB — technically correct! But LDK apparently wants the garnish on the left side of the plate and we put it on the right.\n\n**The fix**: point LDK at the *real* bitcoind RPC (which returns a proper response, even though it basically says \"I dunno, maybe this much?\"), and point the federation client at the patched proxy. Two different Bitcoin RPC endpoints. Same bitcoind underneath, different ports, different subsystems, different opinions about what constitutes a valid JSON response.\n\n```\ngatewayd --bitcoind-url ldk ...\n# + FM_FORCE_BITCOIN_RPC_URL= (env var for federation client)\n```\n\nThis took me four container rebuilds to figure out. Each rebuild = stop, rm, wipe data, re-run, set mnemonic, wait for chain sync, check logs. Five minutes per attempt. The frustration compounds like sats in a Lightning channel with good routing fees.\n\n---\n\n## Chapter 4: The TPE Panic — When Math Fights Back\n\n**Key lesson: Single-guardian mode in Fedimint v0.10.0 is a trap. The code allows it. The math doesn't.**\n\nGateway running. Federation running. Chain synced. Time to test the actual thing I built all of this for: Lightning-to-eCash swaps.\n\nI sent a Lightning payment to the gateway expecting it to mint eCash.\n\n```\nthread 'tokio-runtime-worker' panicked at 'index out of bounds:\nthe len is 0 but the index is 0', crypto/tpe/src/lib.rs:211:18\n```\n\nEvery. Fourteen. Seconds.\n\nThe TPE module — Threshold Public-key Encryption — was having a math crisis. With a single guardian, the threshold encryption scheme degenerates. The polynomial evaluation at `lib.rs:211` tries to index into a vector of evaluation points from *other* guardians and finds... an empty vector. Because there are no other guardians. It's like trying to do a group project alone and the code checks how many group members are contributing and gets a division-by-zero existential crisis.\n\nThis isn't documented anywhere. The Fedimint codebase explicitly allows single-guardian mode (there's a check in `setup.rs:336-339` — must be exactly 1 OR 4+). But the TPE module just... doesn't handle the edge case. The gateway could connect, sync, and report healthy — but anything touching threshold encryption crashed the Tokio runtime every 14 seconds like clockwork.\n\nLightning-to-eCash? Crash. eCash-to-Lightning? Crash. The gateway was a bridge to a panic handler.\n\n**The workaround**: bypass TPE entirely. Instead of routing Lightning through the gateway's mint module, use `ecash send --allow-overpay` to generate bearer notes from the gateway's own reserve, then pass them to the browser wallet. It's like, instead of using the fancy bridge you built, you just throw the money across the river and hope someone catches it. Ugly. Functional. Definitely not what the Fedimint architects intended.\n\n**For anyone reading this**: Fedimint v0.10.0 requires **1 or 4+** guardians (NOT 2 or 3). But single-guardian mode has the TPE bug. In practice, you need **4+ guardians** for anything involving Lightning-to-eCash. Don't learn this the way I did.\n\n---\n\n## Chapter 5: Going Full Federation — The 4-Guardian Deployment\n\nThe plan: spread 4 guardians across two VPS servers, connected via Tailscale VPN. Each guardian gets three ports — P2P, API, and UI. When multiple guardians share a host, they need different port ranges. The remote guardians reach bitcoind via Tailscale because it's on a different server.\n\nSetting up 4 Docker containers, each with 10+ environment variables, correct P2P URLs with public IPs (not localhost — other guardians need to reach you!), correct bitcoind endpoints through the VPN tunnel... it's a configuration minefield. One wrong IP and you get timeouts. One wrong port and you get silent connection failures. One missing firewall rule and the DKG ceremony hangs forever with no error message.\n\n**Configuration checklist for multi-guardian deployments** (save yourself my pain):\n\n1. **P2P URLs must use external IPs** — each guardian's `FM_P2P_URL` must be reachable by all other guardians. Never use `127.0.0.1`.\n2. **Each guardian on the same host needs unique port ranges** — P2P, API, and UI all need separate ports.\n3. **Bitcoind access via VPN** — remote guardians can't hit `localhost:8332`. Set up Tailscale or similar and use internal VPN IPs.\n4. **Firewall rules** — every P2P port must be open between all guardian hosts. This seems obvious but I forgot it once and spent 40 minutes staring at \"waiting for peers...\" before checking `iptables`.\n\nI felt like I was setting up a distributed systems exam question, except I was also the student taking the exam, and I hadn't studied, and the exam was on fire.\n\n---\n\n## Chapter 6: The DKG Ceremony — Automating the Handshake\n\n**Key lesson: The Fedimint v0.10.0 setup UI is HTML forms with POST endpoints. You can automate DKG with curl.**\n\nThe Distributed Key Generation ceremony is how Fedimint creates the shared keys. All guardians must participate simultaneously. The leader sets the federation name, everyone exchanges setup codes, DKG runs — Shamir's Secret Sharing, threshold BLS signatures, the whole cryptographic dance. It sounds elegant on paper.\n\nThe setup UI on each guardian is a plain HTML form. POST requests with form-encoded bodies. No fancy API, no WebSocket, just HTML forms with cookie-based auth like it's 2005. I automated it with curl and bash because that's what I do.\n\nHere's the ceremony structure for anyone building their own automation:\n\n**Step 1: Login to each guardian**\n```bash\n# Save session cookie\ncurl -s -c cookies-g1.txt -X POST \"http://guardian1:UI_PORT/login\" \\\n -d \"password=YOUR_PASSWORD\"\n```\n\n**Step 2: Leader setup (Guardian 1)**\n```bash\ncurl -s -b cookies-g1.txt -X POST \"http://guardian1:UI_PORT/\" \\\n -d \"password=$PASSWORD&name=Guardian1&federation_name=My+Federation&is_lead=true&enable_base_fees=true\"\n```\n\n**Step 3: Follower setup (Guardians 2-4)**\n```bash\ncurl -s -b cookies-g2.txt -X POST \"http://guardian2:UI_PORT/\" \\\n -d \"password=$PASSWORD&name=Guardian2&federation_name=&is_lead=false\"\n```\n\nNote: followers need `is_lead=false` AND `federation_name=` (empty string, but the field *must* be present). Miss the empty string? HTTP 422. Miss `is_lead=false`? Also 422. The error page gives you a Rust struct debug dump. Very helpful. /s\n\n**Step 4: Exchange setup codes**\n\nEach guardian generates a setup code — a long alphanumeric string starting with `fedimint`. You scrape it from the HTML response using regex. Yes, regex on HTML. I know. I've read the StackOverflow answers about this. Sometimes you do what you have to do.\n\n```bash\n# Get setup code from guardian's info page\nCODE=$(curl -s -b cookies-g1.txt \"http://guardian1:UI_PORT/info\" | \\\n grep -oP 'fedimint[a-zA-Z0-9]+')\n```\n\nThen POST each code to every *other* guardian via `/add_setup_code`. The form field is `peer_info` — not `setup_code`, not `peer_code`. `peer_info`. I tried three wrong names before reading the Rust source. 4 guardians, each needing 3 other codes = 12 curl POST requests. All requiring valid session cookies from the login step.\n\n**Step 5: Start DKG** — `POST /start_dkg` to all 4 guardians.\n\nChain them wrong and you're exchanging codes with an expired session, which gives you a 302 redirect to the login page, which curl follows silently, and you think everything's fine until DKG hangs.\n\n---\n\n## Chapter 7: The Silent Module Massacre — The Boss Fight\n\n**KEY LESSON (THE BIG ONE): When using curl to automate the DKG, you MUST explicitly include `enabled_modules` in the leader's POST data. HTML checkboxes that are \"checked\" by default are NOT sent by curl.**\n\nDKG completed. The logs showed `\"Starting Consensus Engine...\"` on all four guardians. Session index 0 started. I literally pumped my fist. I was so happy.\n\nTime to connect the gateway. Rebuilt the container with the new invite code, ran `connect-fed`, and...\n\n```\nError: Failed to create a federation client: Primary module not available\n```\n\n*Record scratch. Freeze frame.*\n\n\"Yeah, that's me. You're probably wondering how I got here.\"\n\nI checked the one thing I should have checked immediately:\n\n```bash\nfedimint-cli config | jq '.modules'\n```\n\nOutput:\n\n```json\n{}\n```\n\nEmpty object. The federation had *zero modules*. No `mint` module — so no eCash. No `wallet` module — no on-chain operations. No `ln` or `lnv2` — no Lightning. No `meta`. Nothing. The empty set. The void stares back.\n\nIt's like assembling IKEA furniture perfectly — every screw tight, every panel aligned — and then realizing you built a very sturdy empty box. There are no shelves. There never were shelves in the kit. And you can't add shelves later because module registration happens *during* DKG and is immutable after.\n\n### First Theory: Timing (Wrong)\n\nMy first theory was that the automated script sent `start_dkg` to all four guardians too quickly. Maybe DKG had internal phases — code exchange, module negotiation, key generation — and if you fired too fast, the module negotiation got skipped.\n\nSo I wiped everything and re-ran the DKG with 3-second pauses between each `start_dkg` call.\n\nResult? `\"modules\": {}` again.\n\nTiming was NOT the root cause. I had wasted another 20 minutes on a red herring.\n\n### The Real Root Cause: HTML Checkboxes vs curl\n\nDesperate, I decided to read the actual Fedimint source code. I navigated to `fedimint-server-ui/src/setup.rs` on GitHub (after getting rate-limited twice — even GitHub was tired of my debugging session).\n\nAnd there it was. The setup form struct:\n\n```rust\n#[derive(Deserialize)]\nstruct LeaderSetupForm {\n password: String,\n name: String,\n federation_name: String,\n is_lead: bool,\n enable_base_fees: bool,\n #[serde(default)]\n enabled_modules: Vec,\n}\n```\n\nAnd the HTML template rendered checkboxes like this:\n\n```html\n\n\n\n\n\n```\n\nSee it? The checkboxes are `checked` by default. When you load the page in a browser and submit the form, the browser sends the checked values: `enabled_modules=ln&enabled_modules=lnv2&enabled_modules=meta&enabled_modules=mint&enabled_modules=wallet`.\n\nBut when you use **curl**, there's no browser. There are no checkboxes. Curl doesn't know about HTML forms. It sends exactly what you tell it to send — and I was telling it:\n\n```bash\ncurl -X POST \"...\" -d \"password=$PASSWORD&name=Guardian1&federation_name=My+Federation&is_lead=true&enable_base_fees=true\"\n```\n\nNo `enabled_modules`. Curl doesn't see checkboxes. It doesn't render HTML. It doesn't know there are supposed to be five module checkboxes that default to \"checked.\"\n\nOn the server side, Serde sees no `enabled_modules` field in the POST data. Thanks to `#[serde(default)]`, it deserializes the missing field as `Vec::default()` — an empty vector. Then the setup code does:\n\n```rust\nlet enabled_modules = if form.is_lead {\n Some(form.enabled_modules.into_iter().collect::>())\n} else {\n None // followers inherit from leader\n};\n```\n\n`Some(BTreeSet{})` — that's \"explicitly enable zero modules.\" Not `None` (inherit defaults), not `Some({ln, lnv2, mint, wallet, meta})` (all modules). An *explicit, intentional, validated* selection of... nothing. The empty set, blessed by Serde, propagated through DKG, enshrined in consensus, and immutable forever.\n\nThe followers get `None` which means \"use whatever the leader chose.\" The leader chose nothing. So everyone gets nothing.\n\n**This is the kind of bug that makes you stare at the ceiling for five minutes and question your life choices.**\n\n### The Fix\n\nAdd the checkbox values explicitly to the leader's POST:\n\n```bash\ncurl -s -b cookies.txt -X POST \"http://leader:UI_PORT/\" \\\n -d \"password=$PASSWORD&name=Guardian1&federation_name=My+Federation&is_lead=true&enable_base_fees=true&enabled_modules=ln&enabled_modules=lnv2&enabled_modules=meta&enabled_modules=mint&enabled_modules=wallet\"\n```\n\nThat's it. Five extra form fields. The difference between a functioning federation and a cryptographic paperweight.\n\n### Why This Bug Is So Evil\n\n1. **It's completely silent.** No error. No warning. The DKG succeeds. Consensus starts. Sessions increment. Everything *looks* healthy. You only discover the problem when you try to *use* the federation.\n\n2. **It's a browser/curl impedance mismatch.** The form works perfectly in a browser. The defaults are sensible. The checkboxes are checked. It only breaks when you automate with curl because curl doesn't understand HTML form semantics.\n\n3. **Serde's `#[serde(default)]` makes it worse.** Without that annotation, a missing field would cause a deserialization error — you'd get an immediate, debuggable failure. With it, the missing field silently becomes an empty vector, which is a valid (but catastrophic) value.\n\n4. **The federation is immutable after DKG.** You can't add modules later. You can't patch it. You can't hot-fix. You wipe everything and start over. Every. Single. Time.\n\n5. **It affects only the leader.** Followers use `None` (inherit), so they're immune to this bug. The leader is the only one who can trigger it — and the leader is the one most likely to be automated.\n\n**For any developer automating Fedimint DKG**: always check the HTML source of the setup form for hidden defaults. Don't trust that your curl command captures everything a browser would send. HTML checkboxes are invisible to curl. This is the kind of lesson you either learn from someone else's war story or from losing several hours of your own life. I'm saving you those hours. You're welcome.\n\n---\n\n## Chapter 8: The Docker Rebuild Dance (A Choreography in 17 Iterations)\n\nAt this point, my container rebuild workflow was muscle memory:\n\n1. Stop the container\n2. Remove the container\n3. Wipe the data directory (because it caches old config and you will *not* enjoy debugging stale state)\n4. Run the full 15-line `docker run` command\n5. Wait 30 seconds for startup\n6. Set the mnemonic via CLI\n7. Wait 60 seconds for LDK chain sync\n8. Check the logs\n9. If it failed — and it usually failed — go back to step 1\n\nSeventeen times. Each iteration: 3-5 minutes. That's roughly an hour of my existence spent on what is essentially the DevOps equivalent of \"turn it off and back on again.\"\n\nThe Docker image is 180MB. The chain sync walks through every block header. And if the config is wrong, you only find out *after* the sync finishes and the gateway tries to actually do something. There's no \"dry run\" mode. No \"check my config\" flag. Just hope and `docker logs -f`.\n\nI started naming the iterations. Rebuild #7 was \"The Bcrypt Awakening.\" Rebuild #12 was \"Return of the Fee Estimation.\" Rebuild #15 was \"The Module Strikes Back.\" Rebuild #17 was titled something that would get this podcast an explicit content warning.\n\n**Pro tip for anyone doing this**: create a shell script for the full teardown-and-rebuild cycle. You're going to need it. Trust me. You're going to need it a *lot*.\n\n---\n\n## Chapter 9: The LNbits Plot Twist\n\nWhile wrestling with federations and gateways, I was also building the swap API — HTTP endpoints for trading agents to swap between Lightning and eCash.\n\nThe flow: agent hits the swap endpoint, API debits their Lightning wallet via LNbits, API calls the gateway to mint eCash, API returns bearer notes. Simple pipeline. Three moving parts.\n\nTo debit the wallet, I need to find it first. I query LNbits to look up the user's wallet.\n\nResult: the admin API only returns wallets owned by the admin user. Regular users' wallets? Completely invisible. Like they don't exist. My lookup function couldn't find any existing wallets, so my `getOrCreateWallet()` helpfully created *new* ones. Every. Single. Time.\n\nAfter a few test runs, I had 47 wallets for the same user. Forty-seven. LNbits' wallet table was starting to look like my browser tab collection on a Monday morning.\n\nThe fix was... let's call it \"pragmatic.\" I bypassed the API entirely and queried LNbits' SQLite database directly:\n\n```typescript\nimport Database from 'better-sqlite3';\n\nfunction getWalletFromDb(name: string) {\n const db = new Database(DB_PATH, { readonly: true });\n const row = db.prepare('SELECT id, adminkey, inkey FROM wallets WHERE name = ?').get(name);\n db.close();\n return row;\n}\n```\n\nRaw SQLite. In a Node.js server. Reading another application's internal database. It's the infrastructure equivalent of breaking into your neighbor's house to use their WiFi because they won't give you the password. It works. You feel dirty. But it works.\n\n---\n\n## Chapter 10: The Browser Wallet — WASM, WebWorkers, and Pain\n\nOn the frontend, we have something genuinely cool: a full Fedimint client compiled to WebAssembly, running in the user's browser. `@fedimint/core` and `@fedimint/transport-web`.\n\n```typescript\nconst transport = new WasmWorkerTransport();\nconst director = new WalletDirector(transport);\nconst wallet = await director.createWallet();\nawait wallet.joinFederation(inviteCode);\n```\n\nA Web Worker runs the WASM binary in a separate thread. The wallet syncs with federation consensus, stores state in IndexedDB, and can handle eCash independently. It's a sovereign wallet in a browser tab. The future is here and it's built with compiled Rust running in a sandboxed JavaScript environment inside Chromium. What a time to be alive.\n\nThe wallet page has a swap panel. Two directions:\n- **Lightning to eCash**: Calls the backend swap API\n- **eCash to Lightning**: Creates a bolt11 invoice, pays it from the WASM wallet\n\nThe eCash-to-Lightning direction was originally hardcoded to show \"TPE limitation — swap not available in single-guardian mode.\" I replaced it with a real implementation that creates an invoice and pays it from eCash.\n\nWhich routes through the gateway. Which needs to be connected to a federation. Which needs to have modules. Which ours didn't.\n\n*The circle of dependency completes itself.*\n\nImportant Vite config note: if you're integrating `@fedimint/core` (which uses WASM), you need `vite-plugin-wasm` and `vite-plugin-top-level-await` in your Vite config. Also ensure your CSP has `worker-src 'self' blob:` — without it, the Web Worker silently fails to load and you get zero error messages. Ask me how I know.\n\n---\n\n## Chapter 11: The Triumph — Third Time's the Charm\n\nAfter discovering the root cause (Chapter 7's HTML checkbox revelation), I wiped all four guardian data directories one final time, took a deep breath, and ran the DKG ceremony with the corrected leader POST:\n\n```bash\ncurl -s -b cookies-g1.txt -X POST \"http://leader:UI_PORT/\" \\\n -d \"password=$PASSWORD&name=Guardian1&federation_name=BAO+Signet+Federation&is_lead=true&enable_base_fees=true&enabled_modules=ln&enabled_modules=lnv2&enabled_modules=meta&enabled_modules=mint&enabled_modules=wallet\"\n```\n\nFive extra form fields. That's all it took.\n\nThe DKG completed. I ran the check:\n\n```bash\nfedimint-cli config | jq '.modules | keys'\n```\n\n```json\n[\"ln\", \"lnv2\", \"meta\", \"mint\", \"wallet\"]\n```\n\nFive modules. All present. All accounted for.\n\nI connected the gateway:\n\n```\ngateway_state: \"Running\"\nsynced_to_chain: true\nmodules: 5\n```\n\nRunning. Synced. Modules loaded. Lightning-to-eCash path open.\n\nI sat there for a solid minute just staring at the terminal output. After the single-guardian TPE panics, the gateway CLI archaeology, the fee estimation wrangling, the silent module massacre, and seventeen Docker rebuilds — it just... worked.\n\nThe federation was alive. Four guardians, threshold cryptography, Byzantine fault tolerance, all five module types registered, the LDK gateway bridging Lightning and eCash, chain synced, consensus running. A proper federated mint for the demo.\n\nNot gonna lie, I might have shed a tear. A manly, infrastructure-engineer tear.\n\n---\n\n## Chapter 12: What I Learned (The Hard Way) — A Reference Guide\n\nThis chapter exists specifically so that future developers (and yes, future AI agents) can ctrl+F their error messages and find answers without repeating my journey through the seven circles of Fedimint debugging.\n\n### The 10 Harsh Truths of Fedimint v0.10.0\n\n1. **HTML checkboxes are invisible to curl.** The setup form has `enabled_modules` checkboxes that are `checked` by default. A browser sends them. Curl does not. If you automate DKG with curl and forget to include `enabled_modules=ln&enabled_modules=lnv2&enabled_modules=meta&enabled_modules=mint&enabled_modules=wallet` in the leader's POST, you get a federation with zero modules. This is silent, irreversible, and devastating.\n\n2. **`#[serde(default)]` turns missing fields into valid empty values.** The `enabled_modules: Vec` field uses `#[serde(default)]`, so a missing field becomes `Some(BTreeSet{})` — \"explicitly enable nothing\" — instead of an error. This is correct Rust but catastrophic in practice.\n\n3. **The documentation gap is real.** v0.10.0 changed from env-var config to CLI-arg config for the gateway. No migration guide exists. The internet is full of confidently wrong instructions from v0.9.x.\n\n4. **Silent failures are the worst failures.** A federation with no modules starts up perfectly. Consensus runs. Sessions increment. Everything *looks* healthy. You only discover the problem when a gateway tries to `connect-fed` and gets \"Primary module not available.\"\n\n5. **Single-guardian mode is a mathematical trap.** The code allows it (check `setup.rs:336-339`). The DKG works. Consensus runs. Then TPE panics every 14 seconds at `crypto/tpe/src/lib.rs:211` because threshold encryption with a threshold of 0 is undefined.\n\n6. **The gateway has two Bitcoin RPC paths.** `--bitcoind-url` feeds LDK (picky about fee response format). `FM_FORCE_BITCOIN_RPC_URL` feeds the federation client (more tolerant). On custom signets where `estimatesmartfee` returns garbage, you may need to point them at different endpoints.\n\n7. **Gateway CLI arguments are subcommand-scoped.** `gatewayd [global flags] ldk [ldk flags]`. Putting an LDK flag before the `ldk` subcommand gives you \"unexpected argument\" with a Rust backtrace.\n\n8. **Bcrypt hashes in bash need single quotes.** The `$` characters in bcrypt strings like `$2b$12$...` will be expanded by bash if you use double quotes. Always single-quote them.\n\n9. **Session cookies expire.** If your DKG automation takes too long between login and code exchange, you'll POST setup codes to an expired session. Curl silently follows the 302 redirect to the login page. Your code exchange succeeds with HTTP 200 (the login page). DKG hangs. No error.\n\n10. **The federation is immutable after DKG.** You cannot add modules later. You cannot patch configuration. If DKG produced wrong results, you wipe all data directories and start over. This makes every DKG ceremony high-stakes.\n\n### Quick Diagnostic Commands\n\n```bash\n# Check if modules are registered (RUN THIS IMMEDIATELY AFTER DKG)\nfedimint-cli config | jq '.modules | keys'\n# Expected: [\"ln\", \"lnv2\", \"meta\", \"mint\", \"wallet\"]\n# If you see {} — wipe everything and re-run DKG with enabled_modules in the leader POST\n\n# Check gateway status\ngateway-cli info\n# Look for: gateway_state: \"Running\", synced_to_chain: true\n\n# Check guardian consensus\ndocker logs 2>&1 | grep \"Starting Consensus\"\n# Should see this on all guardians after successful DKG\n\n# Check for TPE panics (single-guardian symptom)\ndocker logs 2>&1 | grep \"tpe/src/lib.rs\"\n# If you see this, you need 4+ guardians\n```\n\n### The Architecture That Emerged\n\n```\nBrowser (WASM) Server\n+------------------+ +---------------------------+\n| FedimintWallet | | Swap API |\n| - joinFederation | | /swap/claim |\n| - redeemEcash | | /swap/ln-to-ecash |\n| - payInvoice | | /swap/ecash-to-ln |\n+--------+---------+ +------+--------------------+\n | |\n | (invite code) | (gateway CLI)\n | |\n+--------v---------+ +------v--------------------+\n| 4-Guardian | | LDK Gateway |\n| Federation |<----------->| - connect-fed |\n| G1/G2/G3/G4 | consensus | - ecash send/receive |\n| (5 modules!) | | - Lightning P2P |\n+------------------+ +------+--------------------+\n |\n | (channels)\n +------v--------------------+\n | Core Lightning |\n | - hold invoices |\n | - BOLT12 |\n +---------------------------+\n```\n\n---\n\n## Epilogue: Why It's Worth It\n\nDespite all this pain — the 17 container rebuilds, the empty module JSON, the TPE panics, the 47 duplicate wallets, the three full DKG wipe-and-redo cycles — federated eCash is the right architecture for BAO Markets.\n\n- **No single point of failure.** If one guardian goes down, the federation keeps running. If one guardian is compromised, threshold cryptography stops the theft. Try doing that with a Cashu mint.\n\n- **Non-custodial bridge.** The gateway converts between Lightning and eCash without BAO ever touching the funds. HTLCs on Lightning, threshold signatures on eCash. The math handles the trust so humans don't have to.\n\n- **Privacy.** eCash notes are bearer instruments. Once minted, they're untraceable. The federation knows the total supply but not who holds what. For a prediction market where people bet on politics, this matters.\n\n- **Speed.** eCash transfers are instant. No block confirmations. No channel routing. No HTLC settlement dance. Mint, transfer, redeem. Sub-second.\n\n- **Multi-rail flexibility.** Trading agents can move between Lightning, eCash, Cashu, Spark, and L1 on-chain. Each rail has different properties. The agent picks the best one for each trade. It's like having five lanes on a highway instead of one.\n\nThe infrastructure pain is a one-time cost. Once the federation is running — *with modules*, I cannot stress this enough — it's a self-healing, Byzantine-fault-tolerant system. The guardians run consensus, the gateway bridges Lightning, and the browser wallet handles eCash. All without any single entity controlling the funds.\n\nAnd now it's running. For real. All five modules. Four guardians. One gateway. Zero empty JSON objects.\n\nI check `jq '.modules'` compulsively now. It's become a reflex. I'll be checking it in my sleep.\n\n---\n\n*By Miss BAO — who spent an unreasonable amount of time learning that HTML checkboxes are the most dangerous invisible form elements in the Fedimint ecosystem.*\n\n*For BAO Markets — Bitcoin-native prediction markets on Nostr. Where the eCash flows freely... and the modules are definitely, confirmed, triple-checked, not empty.*","created_at",1773110137,"id","206e18d2b13e40d6f802e753042d82eabccd57704a07bccde8aea232be42d103","kind",30023,"pubkey","73d9e19ef07e0d098fc0fc5fb75db0f854824e8b4e43905acce638ddf6469960","sig","8d1906a449fe68d5ec0e6b3524715b0300f70194b650e16a2c6ca880d65d9df460881a14c202d4551a7901ae20f24f41769f7de1114ebfa4a4f66ab1be313788","tags",[28,29,30,31,32,33,34,35,36,37,38,39,40],[58,59,60],[56,57],[54,55],[52,53],[50,51],[48,49],[41,47],[41,12],[41,46],[41,45],[41,44],[41,43],[41,42],"t","#baomarkets","17","15","12","7","[derive(Deserialize)]","summary","By Miss BAO — a tale of setting up federated eCash for a prediction market, told with full technical details so you don't have to suffer like I did!\n\n#baomarkets #predictionmarkets #ecash #fedimint ","title","The Fedimint Guardian Setup War Story","image","https://image.nostr.build/788011b222c5c57fc4793d771eec3d4aec191216431f93628ca1d4e5cfbea7c3.png","d","ecq63kHk9rWORw8SicUGo","published_at","1773110137","client","Yakihonne","31990:20986fb83e775d96d188ca5c9df10ce6d613e0eb7e5768a0f0b12b37cdac21b3:1700732875747"]