# How encrypted Messenger works — CAT, PIN, devices, and how bluebut reads/sends it > **⭐ ARCHITECTURE (2026-07-09): whatsmeow is the ONE path. LightSpeed is DELETED.** > bluebut reads + sends Messenger DMs **only** through **whatsmeow** (the `bluebut-e2ee` daemon = > a registered encrypted-messaging device per account). The old **LightSpeed** subsystem (browser > `sync-inbox`, the persistent LS socket, `ls-keeper`, the plain-message path) is gone. The daemon > decrypts **every** inbound message and the bridge **builds the inbox from that stream** — it upserts > the thread + contact, and resolves names from the `fb_person` entity graph. > > **Accepted trade-offs of whatsmeow-only:** a thread appears only when a message arrives (no > enumeration of idle chats), **no history** before the socket connected, **no folders/unread**, and > **no plain (type-1) threads** — older/business/page/group DMs that aren't encrypted are unreachable. > Contact names come from `fb_person`, not a Messenger roster. > > A from-scratch explainer. If you only remember one thing: **to read or send an encrypted > Messenger message, you have to BE a device on the account.** Facebook's servers physically > cannot hand you the plaintext — that's the whole point of end-to-end encryption. So bluebut > registers its own device and does the crypto itself, exactly the way the maintained bridge > (mautrix-meta) does. --- ## TL;DR - Modern 1:1 Messenger DMs are **E2EE** (`thread_type=15`) — only the participating *devices* can read them. (There are also **plain** `thread_type=1` threads, but bluebut no longer reads those — see the accepted trade-offs above.) - E2EE messages use the **Signal protocol** (Meta calls it *Labyrinth*). Each *device* holds its own crypto keys. To participate, bluebut runs a **whatsmeow** client (a Go Signal/Meta client) that **registers as a device** on the account. - The **CAT** (Cryptographic Auth Token) is the **connection ticket**: it authenticates the E2EE websocket. It lasts ~24h and can only be minted from a real logged-in browser. - The **PIN / secure storage** is a **separate thing**: it protects the encrypted **history backup**. It is **NOT needed to send or receive live messages** — only to restore old history. - So: **device (register once) + CAT (refresh ~daily) = we can read and send live E2EE messages, no PIN.** Proven live in bluebut. --- ## The picture **Read it as:** Facebook relays ciphertext → the daemon's **whatsmeow socket** decrypts every message → rows land in `bluebut-db` (`fb_message`, `fb_thread`, `fb_contact`, `fb_message_reaction`). The socket needs a **CAT** minted by the **real browser** (amber, once/~24h) — and the browser is **killed right after** the CAT is read. The **keeper** cron holds the socket up. The **PIN** hangs off to the side — history backup only, never on the live path. --- ## 1. E2EE is the world we live in ``` Messenger thread ├── thread_type = 1 → PLAIN server can read it → NOT read by bluebut anymore └── thread_type = 15 → E2EE only devices can read it → Signal protocol (whatsmeow) ← our world ``` - **E2EE (type-15):** the message is encrypted **on the sender's device** and only decryptable **on the recipient's devices**. The server relays ciphertext it can't read. Facebook has defaulted new 1:1 chats to E2EE since ~2023–2024, so **the DMs you care about are type-15.** - **Plain (type-1):** the classic Messenger, where the server sees plaintext. This used to ride the LightSpeed wire. **bluebut no longer reads plain threads** — that path is deleted. If a conversation is plain, it's unreachable (an accepted trade of going whatsmeow-only). **A decrypted E2EE message is self-contained.** whatsmeow hands you the full decrypted payload — its own `thread_key`, `sender_id`, `timestamp_ms`, `message_id`, `message_type`, `text`, and (for media) the decryptable transport (see `handler.go` `DecryptedMessage`). What whatsmeow does **not** give you is the **inbox around it**: the list of idle threads, folders, unread counts, and contact *display names*. bluebut builds what it can from the stream itself — it materializes the thread + participants per message and resolves the name from the `fb_person` graph. --- ## 2. Why E2EE forces us to BE a device End-to-end encryption means the plaintext exists **only** on the endpoints (the devices), never on Facebook's servers. There is **no API** that returns the decrypted body, because Facebook literally does not have it. So there are only two ways to read an E2EE message: 1. **Scrape the rendered browser** after it has decrypted the message on-screen (brittle, localized, and needs the PIN for history). ❌ We don't do this. 2. **Become a device**: register our own E2EE device on the account, receive the ciphertext addressed to it, and decrypt it ourselves with the Signal keys. ✅ This is what bluebut does. That "own device" is the **`bluebut-e2ee` daemon** — a Go service running **whatsmeow** (the same library that talks to WhatsApp; it also speaks Meta's E2EE). One registered device per account. ``` Other person's phone bluebut-e2ee daemon ┌───────────────┐ ciphertext (E2EE) ┌─────────────────────────┐ │ encrypts for │ ───────────────────▶ │ whatsmeow device │ │ each of the │ via FB relay │ = a registered device │ │ account's │ │ decrypts with its own │ │ devices │ │ Signal session keys │ └───────────────┘ └────────────┬────────────┘ │ plaintext ▼ bridge → fb_message (is_e2ee) ``` ### What we gave up by dropping LightSpeed A decrypted E2EE message is self-contained, so whatsmeow alone gives us message **content**. What LightSpeed used to add — and what we accept losing — is everything that isn't the message body: | | whatsmeow (what we keep) | LightSpeed (what we dropped) | |---|---|---| | E2EE message body + `thread_key`/`sender`/`ts`/`id`/type/media | ✅ | — | | **Plain (type-1)** message bodies (groups, business, pages, old threads) | ❌ unreachable | ✅ | | **Thread list** of idle conversations | ❌ appears on first message | ✅ | | **Folders** (inbox / requests / spam), unread counts | ❌ | ✅ | | **Contact display names** | resolved from `fb_person` (not a Messenger roster) | ✅ (Messenger roster) | | **History** (messages before we connected) | ❌ realtime-only | ✅ (initial mailbox sync) | **bluebut chose whatsmeow-only** — a realtime encrypted-DM operator inbox — trading the idle-thread list, folders, plain messages, and history for one clean, maintained code path and no browser-held sockets. --- ## 3. The players | Thing | What it is | Lifetime | Where it lives | |---|---|---|---| | **Account** | the FB login (cookies `c_user`/`xs`/`datr`) | long | keyvault `facebook/accounts/` | | **E2EE device** | our registered whatsmeow device (has its own keys) | **register once**, reuse forever | `WA_DB` (whatsmeow store) + keyvault `bluebut/e2ee-device/` | | **Signal sessions** | the per-conversation double-ratchet keys | persist | `WA_DB` | | **CAT** | the ticket that authenticates the E2EE websocket | **~24h** | minted per connect from the browser; not stored long | | **PIN / secure storage** | encrypts the **history backup** | account-set | Facebook's servers (restored via PIN) | Two things are commonly confused — keep them apart: - **CAT = "is this account allowed to open the E2EE connection right now?"** (transport auth, short-lived) - **PIN = "can this device unlock the encrypted history backup?"** (history restore, orthogonal) They have **nothing to do with each other.** You need the CAT to connect; you never need the PIN to send/receive live. --- ## 4. The CAT — the connection ticket (deep dive) **What it is:** the `encrypted_serialized_cat` — a token Facebook mints for a logged-in Messenger web session. When you open `facebook.com/messages/`, the page embeds it in `MessengerWebInitData`. whatsmeow sends it as the bearer credential (`ClientPayload.FbCat`) when it opens the E2EE Noise websocket. It proves "this account is authenticated and this device is authorized to connect." **Why it expires (~24h):** it's derived from the web session and is session-scoped. Facebook rotates it. An expired CAT = the websocket won't open (or drops). **Why we can't refresh it headlessly:** minting a CAT requires a **real logged-in browser page** (it's baked into `MessengerWebInitData`). Facebook's anti-bot rejects a scripted/headless login (`"xs cookie was deleted"`), so you can't just curl for a fresh CAT. In the daemon, `RefreshCAT` is a deliberate hard-error stub: ```go // client.go e2ee.RefreshCAT = func(ctx context.Context) error { // The CAT expires (~24h). We can't refresh it headlessly (that needs the web session). return fmt.Errorf("CAT refresh requires a fresh box-side probe (reconnect the account)") } ``` **How bluebut mints one:** `cat-connect.js` `cdp_load`s the account's real **cloakbox** browser, drives it to `/messages/`, reads `encrypted_serialized_cat` + `appId` off the page, POSTs the daemon `/connect`, and then **`cdp_kill`s the browser** — the daemon holds the socket, so the browser is dead weight after the CAT grab (leaving it open used to strand a Chrome per account on the Messenger page). This is the **one and only browser touch** in the whole stack, **once per ~24h per account**, not per message. > ⚠️ The CAT mint MUST drive the **public** cloakbox gateway (`https://cloakbox.hostbun.cc`), not a pbox loopback — see `cloak-client.js` `DEFAULT_DRIVE_BASE`. And it MUST release the browser (`cdp_kill`) when done — see `cat-connect.js`. --- ## 5. The device — register ONCE (ICDC) Before it can connect, our whatsmeow client must be a **registered E2EE device** on the account. This is the **ICDC** flow (Identity Consistency Data Cache): ``` POST reg-e2ee.facebook.com/v2/fb_icdc_fetch ← get the account's device list POST reg-e2ee.facebook.com/v2/fb_register_v2 ← add OUR device, signed → returns WADeviceID (e.g. 1159930757:960@msgr) ``` - This is authenticated by the **CAT** (not the web session). - It happens **exactly once** per account. **It fires a "new device logged in" alert** to the account owner — expected, one-time. - The resulting `wa_device_id` is stored (`WA_DB` + keyvault `bluebut/e2ee-device/`) and **reused on every future connect** — so reconnecting **never** re-registers and never re-alerts ("register-once"). Think of it like linking a WhatsApp companion device: you scan the QR once; after that the device stays linked and just reconnects. --- ## 6. Where decryption actually happens — the Signal sessions Once registered + connected, the daemon holds **Signal double-ratchet sessions** (in `WA_DB`), one per conversation partner. This is the actual crypto: - **Sending:** our device does an **X3DH** key agreement with the recipient's device(s) using their published *prekeys*, derives a shared secret, and encrypts. Facebook relays the ciphertext. - **Receiving:** the sender encrypted a copy **for our device's key**; our whatsmeow decrypts it with the ratchet and advances the chain. - **Multi-device:** an account can have several devices (phone, web, our daemon). Each message is encrypted **separately for each device**. Our device gets its own copy — that's why *we* can read it without anyone else's keys. **The CAT is transport auth; the Signal session keys are the message crypto.** The CAT gets you *connected*; the session keys (already in `WA_DB`) do the *decryption*. That's why a reconnect (fresh CAT) resumes reading immediately — the session keys never left. --- ## 7. The PIN / secure storage — history, NOT live When you enable E2EE, Facebook also creates **"secure storage"**: an encrypted backup of your **message history** (and some key material), protected by a **PIN** (6-digit) or device-based key. A *new* device restores that history by PIN-unlock. Crucially — and this is the part people trip on: - **The PIN is only for HISTORY.** It unlocks the encrypted backup of messages sent **before your device existed**. - **The PIN is NOT part of live send/receive.** New messages flow through the Signal sessions above, which don't touch secure storage. So the dividing line for "can we read a message?" is **not the PIN — it's *when our device registered*:** ``` message sent AFTER our device registered → we have a session for it → WE CAN READ IT (no PIN) message sent BEFORE our device registered → history → needs secure-storage restore (PIN) ``` The PIN/restore path exists in whatsmeow but is **orthogonal** and not used by the daemon's live path. bluebut does **live only**, which is all that's needed for send/receive. --- ## 8. The full lifecycle ``` ┌─ ONCE per account ──────────────────────────────────────────────┐ │ 1. mint CAT (drive the real browser, read encrypted_serialized_cat) │ 2. ICDC register → wa_device_id (fires the one new-device alert) │ 3. store wa_device_id (WA_DB + keyvault) ← register-once └─────────────────────────────────────────────────────────────────┘ │ ┌─ every connect (after restart / CAT expiry) ─────────────────────┐ │ 4. mint a fresh CAT (browser) → then KILL the browser │ │ 5. daemon /connect {fbid, app_id, cat, proxy, wa_device_id} │ │ → whatsmeow opens the Noise websocket, REUSES the device │ │ → resumes the Signal sessions from WA_DB │ └─────────────────────────────────────────────────────────────────┘ │ ┌─ while connected (the standing socket) ──────────────────────────┐ │ RECEIVE: ciphertext arrives → decrypt with session → fb_message │ │ (is_e2ee=true, decrypted_by='daemon') │ │ SEND: encrypt with session → push over the socket → verified │ └─────────────────────────────────────────────────────────────────┘ ``` **Everything after step 2 that isn't a fresh CAT is browser-free.** The device + Signal keys persist; only the CAT needs the periodic browser mint. --- ## 9. What has to be running for realtime send + receive Per account, **one daemon-held socket** — the **whatsmeow socket** — living in RAM in the `bluebut-e2ee` daemon (so a daemon restart drops it and it must be re-established): | Socket | Handles | Auth | Browser? | Kept up by | |---|---|---|---|---| | **whatsmeow / E2EE** | type-15 receive (decrypt) + send | **CAT** + registered device | **yes, for the CAT only** (once/~24h, then killed) | `whatsmeow-keepalive` (`whatsmeow-reconnect` `*/5` heal + `whatsmeow-auth-refresh` `6h --force`) | And the daemon itself must be up. **It all runs in PROD** (Coolify on hostbun); local dev just talks to the public daemon (`E2EE_DAEMON_URL=https://bluebut-e2ee.blpk.cc`) — you can't hold the socket on a laptop. --- ## 10. Scenarios (the questions people actually ask) **"Can we read an E2EE message without the PIN?"** ✅ **Yes**, if it was sent **after our device registered**. Our device has the Signal session, so the daemon decrypts it directly — no PIN. (Proven: philip/pojtie have inbound E2EE messages `decrypted_by='daemon'`, zero PIN entered.) The **only** thing the PIN would add is **back-history** from before the device existed. **"What happens if we SEND without the PIN?"** ✅ Sends fine — our device establishes a fresh Signal session with the recipient and encrypts. The PIN is irrelevant to sending live messages. **"What about on refresh / reconnect?"** The device stays registered and its Signal sessions persist in `WA_DB`. Reconnecting (with a valid CAT) resumes live send/receive **with no PIN and no re-registration.** **"What about after a daemon restart / deploy?"** The in-RAM socket dies → `whatsmeow-keepalive` reconnects: it mints a fresh CAT (a brief cloakbox browser touch, then the browser is killed) and re-opens the socket, reusing the stored device. **"Why did I get a 'new device logged in' alert once?"** That's ICDC registration (step 2). It fires **once** per account, then never again (register-once). **"Why does a conversation show up as 'Thread 61583351473582' with no name?"** whatsmeow only gives numeric ids; the name is resolved from the `fb_person` entity graph. If that person isn't in `fb_person` yet, the name is blank until they're scraped in. --- ## 11. How data is saved (the data model) The flow per received message: **whatsmeow decrypts → the daemon emits a `DecryptedMessage` → the bridge `handleE2eeIngest` upserts several tables** (all in `bluebut-db`). Every message becomes a row — no type is dropped. | Table | Written per message | Key fields | Dedup key | |---|---|---|---| | **`fb_message`** | ALWAYS — one row for every message, any type | `message_type` (text·image·video·audio·file·sticker·location·view_once), `text` (body/caption), `sender_id`, `thread_key` (the peer fb_id for a 1:1), `timestamp_ms`, `is_self`, `is_e2ee=true`, `decrypted_by='daemon'`, `edited_at_ms`, `read_at_ms`, `delivered_at_ms` | `(account, message_id)` | | **`fb_message_attachment`** | for each media part | `attachment_type`, `mime_type`, `filename`, `filesize`, `preview_width/height`; `attachment_fbid` synthesized (`-`) | `(account, message_id, attachment_fbid)` | | **`fb_message_reaction`** | on a reaction event | `actor_id`, `reaction` (emoji; NULL = removed), `timestamp_ms` | `(account, message_id, actor_id)` | | **`fb_thread`** | upserted so the conversation appears | `thread_type=15`, peer-keyed, name/avatar from `fb_person`, `last_activity_timestamp_ms` | `(account, thread_key)` | | **`fb_contact`** | upserted for the sender | `id` (numeric), `name` — resolved from the `fb_person` entity graph | `(account, id)` | All writes are **merge-upserts** (`Prefer: resolution=merge-duplicates`), so re-ingesting the same message is idempotent — safe with the realtime socket's own echo of our sent copies. ### Mutation kinds (a message that changes an existing one) The daemon emits a `kind` on `DecryptedMessage`; the bridge routes the mutation kinds **before** the normal message upsert: - **`reaction`** → upsert `fb_message_reaction` on `target_message_id` (empty emoji = removal → NULL). - **`edit`** → PATCH the target `fb_message.text` + `edited_at_ms`. - Read/delivery **receipts** arrive as `events.Receipt` → stamp `read_at_ms`/`delivered_at_ms` on the named messages. ### What IS captured (verified live) - **Every message type** — text, image, video, audio, file, sticker, view-once, location, contact (nothing dropped; unknown types still get a row). - **Media bytes** — image/video/audio/file/sticker **and view-once** are `DownloadFB`-decrypted and uploaded to the `fb-media` MinIO bucket (`playable_url`), via the bridge `POST /messenger/media`. - **Reactions, edits, read + delivery receipts** — see above. ### What is NOT captured yet (honest gaps) - **Reply context** — the quoted-message id. FB's encrypted consumer content (`ExtendedTextMessage`) carries no `ContextInfo`, so this needs envelope-level work. - **Group-thread metadata** — participant/name/photo/admin changes. - **History before the socket connected** — a permanent whatsmeow limitation (realtime-only). --- ## 12. The thread model (peer-keyed, whatsmeow-only) A 1:1 E2EE thread is keyed by the **peer's numeric fb_id** (`thread_type=15`), materialized per message by `materializeE2eeThread` (writes `fb_thread` + `fb_thread_participant` for peer and self, name/avatar from `fb_person`). A thread appears when a message arrives — there's no enumeration of idle threads. The old **LightSpeed threads** (`thread_type=1`, keyed by the FB thread id) are **legacy** — they can't get new messages and were **purged from `fb_thread`**. Their `fb_message` / `fb_thread_participant` rows are **kept** so DM-dedup ("have we already messaged this person?") still sees them — they're just no longer shown in the inbox list. During the LightSpeed→whatsmeow cutover a few conversations got their messages split across two thread keys; those were merged and won't recur (LightSpeed is gone, so every message now lands on the one peer key per person). --- ## 13. How bluebut implements it (file map) | File | Role | |---|---| | `messenger/e2ee/daemon/` (Go) | the `bluebut-e2ee` daemon — holds the whatsmeow socket per account | | ` ├─ client.go` | E2EE connect flow: CAT injection, `RefreshCAT` stub, `ClientPayload` | | ` ├─ register.go` | ICDC register (`icdc_fetch` / `icdc_register`) → `wa_device_id` | | ` ├─ handler.go` | decodes EVERY inbound (text/media/reaction/edit/view-once/receipt) → `DecryptedMessage`; downloads media bytes | | ` └─ main.go` | HTTP control API (`/connect`, `/disconnect`, `/e2ee-send`, `/status`) | | `messenger/lib/cat-connect.js` | mints the CAT from the real browser + POSTs `/connect`, then `cdp_kill`s the browser | | `messenger/lib/thread-key.js` | `materializeE2eeThread` (peer-keyed thread + participants + name) · `bumpThreadActivity` | | `bin/whatsmeow-keepalive.mjs` | keep-alive: check status → mint CAT → reconnect (crons `whatsmeow-reconnect` `*/5`, `whatsmeow-auth-refresh` `6h --force`) | | `messenger/jobs/send-message.js` | the send job — **browser-free**; routes everything through the daemon's whatsmeow `/e2ee-send` | | `ui/bridge/opencli-bridge.mjs` | `handleE2eeIngest` — materializes the inbox from the stream; `/messenger/media`, `/messenger/receipt` | | `fb_message` (bluebut-db) | where decrypted messages land (`is_e2ee`, `decrypted_by`, `is_self`) | --- ## 14. Gotchas & guardrails - **Register-once is sacred.** Re-registering fires another new-device alert and can confuse the account's device list. Always reuse the stored `wa_device_id`. - **Proxy egress.** The E2EE Noise socket must egress the account's **own proxy** (same IP as the web session) — anti-detect. - **Release the browser after minting the CAT.** The mint's browser is only for reading the token; leave it open and it strands a Chrome per account on FB. `cat-connect.js` `cdp_kill`s it in a `finally`. - **The daemon holds state in RAM.** Every daemon deploy drops the socket. Watch paths already stop unrelated pushes from redeploying it; keep daemon-code changes rare. - **The bridge image needs `curl`.** Coolify's healthcheck execs `curl`/`wget` inside the container; the slim node image lacks it, so `Dockerfile.bridge` installs `curl` — without it every bridge deploy silently rolls back. --- ## 15. The open optimization — cache the CAT Right now the reconnect sweep mints a **fresh CAT on every reconnect** (a cloakbox browser touch each time, then a kill). But a CAT is valid ~24h and is **reusable** across reconnects. So the improvement: > **Persist the CAT** (with its mint timestamp). On reconnect, try the cached CAT first — if it's still valid (<~20h), connect **browser-free**; only drive the cloakbox to mint a fresh one when the cached CAT is near expiry. That would make **daemon-restart recovery browser-free and instant**, and cut cloakbox CAT mints from "every reconnect" to "once per ~20h per account" — the fewest possible browser touches while staying always-up.