sithbitd: core & chain settings

Part of the configuration reference. sithbitd is the combined mail daemon — this page covers what it is, its shared [store] (also read by account-api), and the [grpc] / [ipfs] sections that wire up chain access. Its listener protocols (SMTP/submission, IMAP/POP), authentication and security settings, and the outbound spooler each have their own page — see the sidebar.

sithbitd

The combined mail daemon: SMTP MX + submission, IMAP, POP, and the spooler workers in one process. Run exactly one sithbitd per store while [store] kind = "sqlite" — IMAP IDLE push and per-wallet SendMail ordering are in-process. The postgres and cloud stores lift that limit; see Scaling out.

Shared listener values — written once, inherited by every listener

hostname and local_domains are top level, above the first section header (a bare key written after one reads as a key of that section). Each fills in the listeners that named none of their own, so a deployment states its identity once instead of five and three times over:

Row markers follow the going-public legend.

All four are additive defaults: a config file that never mentions them behaves exactly as it did before they existed, and nothing an operator already wrote changes meaning. The shared certificate — [tls] — follows the same rule and is documented with the listener TLS settings; the shared [quota] table is documented with the outbound quotas.

Why the last two are spelled Option in the code, and why it matters here. hostname and local_domains each have a sentinel an operator would never write on purpose — the built-in "localhost", an empty list — so “wrote nothing” is recognisable at a glance. A lone bool and a lone number have none: enable_stored_passwords = true and max_message_size = 26214400 written on a section are byte-identical to that section saying nothing at all. So both are optional on the listener configs and read everywhere else through an accessor that supplies the default, and the daemon’s inheritance pass is the only code that looks at the raw field. That is what lets a shared value fill in the listeners that were silent without overwriting one an operator deliberately wrote out.

max_message_size = 0 is refused at the top level. To the SMTP listeners 0 means advertise SIZE with no fixed limit; [imap] feeds the same number to its literal cap and advertises it as APPENDLIMIT, where 0 refuses every APPEND. One shared key cannot carry two opposite meanings, so the daemon fails at startup naming the key rather than delivering both. Per section it is untouched: [smtp] max_message_size = 0 still means what it always did.

[store] — shared with account-api

REQUIRED (public). Whichever backend kind names, its own section below carries the coordinates a public deployment has to set deliberately — the bucket, table, queue prefix, URL or database id the fleet shares. The shipped defaults point at a local directory and loopback emulators, and nothing at startup notices that a reachable instance kept them.

[store.blobs] selects the mail-body blob store: kind = "local" (default, path = "blobs"), "s3" (endpoint, bucket, region, access_key, secret_key), or "azure" (endpoint, container, account, access_key).

[store.postgres] (for kind = "postgres"): url (default "postgres://postgres:postgres@127.0.0.1:5432/sithbit"). The schema is migrated idempotently at startup; the database itself must already exist. The URL’s password is redacted from logs and Debug output.

[store.aws] (for kind = "aws"): region, table (one DynamoDB table, default "sithbit"), queue_prefix (SQS queues named <prefix>-*, default "sithbit"), endpoint_url / sqs_endpoint_url (emulators), sqs_wait_time_seconds (SQS receive long-poll seconds, default 10; 0 = short polling), access_key / secret_key — omit the keys to use the ambient AWS credential chain (env, profile, IAM role) — and kms_master_key_id (a KMS key ID, alias, or ARN for at-rest encryption of the table and queues; unset, the default, keeps provider-managed SSE). The table and queues are created idempotently at startup.

[store.azure] (for kind = "azure"): account, table, queue_prefix, table_endpoint, queue_endpoint, access_key. The defaults target a local Azurite emulator (run it with --skipApiVersionCheck); a real account derives its endpoints and needs access_key. Note the Azure blob store has no emulator key fallback: Azurite blob use needs the published well-known key passed explicitly.

[store.turso] (for kind = "turso"): database (local libSQL file, default "sithbit.db"), sync_url (libsql://… / https://… — set it to run an embedded replica against a remote Turso/libSQL primary; unset = a pure-local file, on-disk-identical to kind = "sqlite"), auth_token (bearer secret for the remote; redacted from logs — unset for local dev or an unauthenticated self-hosted sqld), and sync_interval_secs (seconds between background replica→remote pulls; unset defaults to 60s for a replica). Blobs still come from [store.blobs] (local/s3) — libSQL has no object store, exactly like SQLite. The schema is migrated idempotently at startup.

[store.cloudflare] (for kind = "cloudflare"): the daemon reaches Cloudflare’s edge primitives over plain HTTP with one bearer API token — it is not hosted on Workers. Three native services back the five store traits: D1 (SQLite-over-HTTP) for accounts + mail + keyed leases, Cloudflare Queues for the job queue, and R2 (S3-compatible) for blobs.

Provisioning — auto vs prerequisite. The D1 schema (leases table included) is provisioned automatically at startup: the shared migration set is replayed over D1’s HTTP query API, idempotently (a _d1_migrations tracking table makes a restart a no-op). The D1 database and the five Cloudflare Queues themselves are a manual prerequisite — creating them is a Cloudflare management-API/dashboard step with no local emulator, so the backend does not auto-create them. Selecting kind = "cloudflare" without those ids configured fails fast at startup with a clear config error rather than an opaque runtime 404.

Local-first testing. The full shared store-conformance suite — D1 (accounts, mail, DMARC, keyed leases) and Cloudflare Queues (the job queue) — now runs with no Cloudflare account over the same production wire path the daemon uses in the field: the real ReqwestTransport and the http.rs/queue.rs request-builders and response-parsers, driven against a loopback fake that speaks Cloudflare’s public REST envelopes (D1 /query, Queues push/pull/ack/info) over a real TCP socket. Because D1’s dialect is SQLite, the fake replays the identical SQL against an in-process SQLite; queues run over stateful in-memory state behind the REST surface. R2 blobs ride the same minio-backed S3 path the other S3-shaped backends use.

This exercises the request-building and response-parsing that the earlier in-process Transport fakes bypassed, and proves the trait semantics survive a real socket. Be clear about what it does not prove: a self-written fake only shows that http.rs/queue.rs are internally consistent with the envelopes we assumed Cloudflare speaks — it cannot confirm those assumptions against the real service. A live real-Cloudflare conformance run therefore remains deferred: it needs a paid account (R2 requires dashboard enablement, and Queues requires a Workers Paid plan) and is gated behind real credentials.

[grpc] and [ipfs] — the chain pipeline

The two sections are independent halves of chain access. [grpc] alone enables everything gateway-backed — SMTP recipient verification at RCPT time (alias resolution + postage checks), at-rest sealing, alias logins — with the chain pipeline off: delivered copies stay in state received, and boot logs the verification-only posture at info. This is the MX posture: an edge that verifies postage but never writes the chain needs no [ipfs] provider. Adding [ipfs] as well enables the full pipeline (the encrypt → pin → SendMail workers), so mail actually reaches the chain. [ipfs] without [grpc] does nothing — no gateway means no chain to announce pins to — and logs a warning at boot. With neither section, the chain pipeline is disabled entirely (the dev default).

An empty [ipfs] section (plus [grpc]) is a complete chain setup: the embedded node imports mail blobs with the fixed CID profile (CIDv1, sha2-256, raw leaves, 256 KiB balanced dag-pb — byte-identical to Kubo) and stores blocks/pin manifests in ipfs.blobs.

[ipfs.repin] runs the repin-and-verify migration: an hourly sweep enumerates every fully chained copy and, per message, fetches the sealed bytes back from the legacy service, re-pins them on the active embedded/remote node, and releases the legacy pin only when the re-imported root CID matches the recorded on-chain one. On a mismatch (the legacy service imported with a different profile) the legacy pin is kept — the on-chain CID is immutable and must stay resolvable — and the sithbit.repin.outcomes{kind="mismatch"} counter grows. Watch that counter; once it stabilizes the migration is done: remove [ipfs.repin] (and, if nothing mismatched, the legacy credentials). Requires a store backend with candidate enumeration (SQLite today); sithbitd refuses the section otherwise at startup.

kind = "remote" delegates pin/unpin/fetch to a shared sithbit-ipfsd daemon over HTTP instead of running a node in-process — the multi-instance shape: the fleet pins through one node (one swarm identity, one block store) rather than each daemon embedding its own. [ipfs.blobs] and [ipfs.swarm] are ignored in this mode; they are the daemon’s to configure.

With [ipfs.swarm] configured the node also joins the IPFS DHT: it learns peers via identify, and with provide = true announces every pinned root so stock Kubo peers can discover this node as the content’s provider (pin manifests are the source of truth — a reprovide sweep reconciles the DHT records against them on every tick, immediately at startup). While the swarm runs, the node serves its pinned blocks over bitswap (/ipfs/bitswap/1.2.0): any connected peer — or one that found us via a provider record — can ipfs get the content straight out of the configured blob store. Serving is one-way: the node answers wantlists but never fetches foreign CIDs. For public retrievability the listen addresses must be reachable from the internet (an open inbound port); behind a closed firewall the DHT records carry addresses nobody can dial.

[account_keys] — at-rest keys for stored-password accounts

Off by default. With no [account_keys] section (or no root inside it) nothing changes: this is purely additive, and a config file that never mentions it behaves exactly as it did before the section existed.

At-rest sealing covers password-less accounts, whose stored bodies are encrypted to the account’s own reading key. Accounts that have ever configured a stored mail password are deliberately left out: their CRAM-MD5 and APOP logins prove knowledge of a password, never possession of the X25519 secret a sealed body needs, so such a session could never open one. Their mail is therefore stored in the clear, protected only by whatever the storage backend encrypts at rest — table- or bucket-level encryption, which the storage provider can decrypt on its own.

This section closes that gap. The operator holds one root secret; every account’s at-rest key is derived from it by a domain-separated key-derivation function keyed on the wallet, so one root covers every account with nothing extra to store. The derivation is deterministic, so a restart, a redeploy, or a second replica reproduces the same keys.

What this protects, and what it cannot. A stored-password account’s mail must be readable by the server on demand — that is what reading mail with a password means. So no key arrangement, this one or a hardware key-management service, can stop a compromised running server from reading that mail. What the root buys is that the blob store and the database no longer hold plaintext: a stolen bucket, a leaked table export, or a restored backup yields ciphertext, because the root lives outside the store. Choose where it lives accordingly — a secret manager in production rather than a file on the mail host.

The root is unrecoverable. It is as sensitive as every stored body combined, there is no escrow, and losing it makes sealed mail permanently unreadable. Back it up exactly as carefully as store.credential_key_file.

Existing mail is not migrated. Only mail delivered after the root is configured is sealed; bodies already stored in plaintext stay that way, and each is read according to its own at-rest header, so the two coexist with no flag day. The exposure shrinks as mailboxes turn over rather than closing at once.

Trailing whitespace in the root material is ignored. A secret written to a file by shell redirection carries a trailing newline while the same secret read from a cloud secret manager does not; without this, moving a root from one to the other would derive different keys and quietly orphan every sealed body. Leading whitespace is part of the secret. Operators supplying binary material whose final bytes are meaningful whitespace should encode it (base64, hex) first.

Failure behavior differs by side, deliberately. A root that is configured but cannot be loaded — an unreadable file, an unreachable secret manager, material under 32 bytes — fails startup: an operator who named a secret meant their mail sealed, and silently storing it in the clear is the one outcome they cannot detect from the outside. At read time an unresolvable key refuses that one message transiently and leaves the rest of the mailbox listable, because there the mail is already sealed and the safe answer is to serve less rather than to start wrong.