Enforcement as one owned table replaced atomically, per-rule sets, connection marks for attribution, logged drops, simulation on the same chain, and policy that outlives the daemon

Context

ADR-0003 fixed that the agent programs the host's native firewall, keeps every rule in one table it owns, and replaces that table as a unit. ADR-0011 fixed that the agent fails static, that the last acknowledged policy survives restarts and reboots, and that root on the host can always remove enforcement locally. ADR-0015 fixed the atomic apply-or-fail acknowledgement, and ADR-0018 shaped the agent's installed-policy boundary as an interface that takes each complete next policy, deferring disk persistence to the enforcement backend. ADR-0019 built the flow path that carries decisions to the control plane.

Implementing the Linux backend made a set of decisions concrete that none of those records fixed and that a later change could plausibly undo one at a time: how a rendered policy becomes a ruleset, how the ruleset is changed when only peers change, how an accepted connection is attributed to the rule that accepted it, how a dropped one is seen at all, what simulation installs, and what happens to the kernel's rules and the agent's knowledge of them when the daemon is not running.

Decision

1. One owned table, replaced in one transaction. Every rule the agent installs lives in one inet table named for this system. A full apply is one nft -f script that declares the table (a no-op when it exists), deletes it, and recreates it with the new content; the script is one kernel transaction, so the host runs the previous ruleset or the new one and never a mixture, and a refused script leaves the previous ruleset in place, which is what lets the agent acknowledge FAILED and stay on its last good version (ADR-0015). No other table is read, written, or flushed. The one base chain hooks input at priority filter + 10: a host's own filter chains run first, an accept in one base chain never skips another, and a drop in either is final, so the agent adds constraints and removes none.

2. One named set per resolved rule and address family. Each rule's peers become elements of two interval sets, one per family, named from the resolved rule id with every character outside letters, digits, and underscore replaced by an underscore, so the authored rule id reads back from nft list ruleset alone. A change that is peer membership only, the high-churn quantity ADR-0015 made cheap on the wire, is applied as set element additions and deletions in one transaction, leaving every rule and every other set untouched; any other change is a full rendering. A set update the kernel refuses falls back to one full rendering before the apply is reported failed.

3. Chain structure. Accept loopback; accept established and related traffic; then one accept per rule and family, matching source in the rule's set and the rule's protocol and ports, writing the rule's allocated value into the connection mark (decision 4), each directly preceded by an accept with the same match restricted to traffic the kernel does not track, which touches no connection state; then the terminal rule. The untracked accept exists because the mark write reads the connection mark, a packet with no connection entry has none to read, and the kernel ends the rule at that read, before its accept: without it, admitted traffic the kernel never tracks (IPv6 neighbor discovery) or that the host exempts from tracking would fall through to the terminal rule. Untracked traffic leaves no connection to attribute, so admitting it records nothing. (Amended 2026-10-17; see Amendments.) The established-and-related accept comes before any rule and, with inbound-only scope (ADR-0010), is the invariant that makes it impossible for the agent to sever its own control-plane stream: that stream is an outbound connection the agent initiated, its return traffic is established, and nothing after that line can touch it.

4. Attribution by connection mark in one claimed region; drops by log. The agent claims bits 16 through 31 of the 32-bit connection mark, mask 0xffff0000, and nothing outside it. Each resolved rule is given a stable value in the region by a persistent allocator stored beside the policy file, and each accept writes its rule's value into the region while keeping the other sixteen bits exactly as they were (ct mark set (ct mark & 0x0000ffff) | value << 16); the collector reads only the region back. The kernel holds that write as a mask and an exclusive-or, (mark & 0x0000ffff) ^ value << 16, and nft list ruleset reads the pair back as ct mark & (0x0000ffff | value << 16) | value << 16: the same write, since the bits the listed mask adds are set by the or regardless of what they held. Rendering never renumbers: a rule keeps its value across every version that carries it, a new rule takes the lowest value never handed out, and a rule that leaves the policy frees its value without losing it, because a freed value is not given to another rule until all 65534 have been handed out once, and then the value freed longest ago goes first, never one the installed policy holds. Classification resolves a connection's value through that allocation, not through the installed policy's order, because the kernel keeps an established connection's mark across policy changes: a connection admitted under an earlier version is ALLOWED by the rule that admitted it, and one whose rule has since been removed is attributed to the removed rule by its own id, never to another rule. The documented boundary is reuse: once 65534 values have been handed out over the agent's life, a connection older than the reuse of its value attributes to the value's new holder. A policy with more rules than the region can number (65534; the value 0xffff is reserved) is refused before the kernel sees it. (Amended 2026-10-05; see Amendments.) A dropped packet never becomes a tracked connection, so the terminal rule logs every packet it sees to a netlink log group before its verdict; the collector subscribes to that group and reports each logged packet as one attempt, BLOCKED under enforcement. This is the coexistence stance for the connection mark, decided: another user of the mark on the same host keeps the low sixteen bits, its marks survive every write the agent makes, and the agent's attribution resolves regardless of what those bits hold. The network-namespace suite asserts exactly that, with a foreign table marking every new connection before the agent's chain runs.

5. Simulation is the same chain with one line changed. Simulation renders exactly the enforced ruleset except the terminal rule, which logs, writes the reserved would-block value (0xffff) into the region with the same masked write, and accepts instead of dropping. The log and its counter precede the mark write, so a packet the kernel does not track is recorded before the write's read ends the rule, and then takes the chain's policy, accept: the same verdict. Everything else, sets, marks, per-rule accepts, order, is identical, and a test asserts that the two renderings of one policy differ in exactly one line. A simulation verdict is therefore evidence about exactly the ruleset enforcement would install. Logged packets under simulation are WOULD_BLOCK attempts, counted the same way enforcement counts drops; the would-block mark lets the tracked connection contribute its bytes to the same record without counting it twice. Visibility mode installs the owned table with one base chain and nothing else: no sets and no verdict. The chain hooks input at the same priority with a policy of accept and holds one rule, which counts and accepts each new connection. That rule exists because the kernel registers connection tracking in a network namespace only once a loaded ruleset references it: on a host where nothing else does, a table with no chain leaves connection tracking off and the agent observes nothing. The chain decides nothing, because an accept in one base chain never skips another, so traffic is observed, never blocked (ADR-0001). Nothing in it logs, and that is decided: the log source is enforcement and simulation only, because in visibility the agent renders no verdict for a log to witness, and what visibility reports is what connection tracking reports, as OBSERVED. The visibility table is installed by the same full apply, in one transaction, and persists like any rendered policy (decision 6). (Amended 2026-10-08; see Amendments.)

6. Rules outlive the daemon; policy persists to disk before it is applied; only an explicit command removes either. On every apply the canonical policy, with its version, is first written durably to the agent's state directory with mode 0600 under a magic line and a checksum: a temporary file is written and synced, renamed into place, and the directory holding it is synced, with the mark allocation (decision 4) written the same way just before it. Only then does the kernel transaction run, and the agent acknowledges APPLIED only when both succeeded. A persistence failure is acknowledged FAILED with the kernel untouched, so the installed policy, the persisted one, and the last acknowledgement still agree. A kernel failure after the policy was persisted is acknowledged FAILED with the file one version ahead of the kernel, and that is benign under the desired-state model: the file is the desired state, re-applying it is idempotent, the control plane answers the failure with a snapshot (ADR-0015), and a reboot inside the window restores the desired version rather than an older one. The acknowledgement enum is unchanged. (Amended 2026-10-05; see Amendments.) A daemon exit leaves the kernel rules in place. On start the daemon loads the file and re-applies it as a full rendering before it dials anything, so a reboot restores the last persisted policy, never one older than the agent last acknowledged, before the control plane is even reachable; the version it then reports in Hello is the persisted one, and the server still answers with a snapshot (ADR-0015). A file that fails its checksum loads nothing and is reported loudly; the kernel keeps whatever it holds until the next snapshot. Enforcement is removed only by innerwall-agent down, which deletes the owned table and nothing else, leaves the persisted policy on disk, and says exactly that: a daemon that is still running, or one that starts later, re-applies the policy. This closes the persistence deferral ADR-0018 recorded, and ADR-0018 is amended to say so.

Consequences

• The kernel is the truth about enforcement and the persisted file is the truth about the desired state; the two agree after every successful apply and after every start, and the file is never behind what the agent acknowledged. The one disagreement is a refused kernel transaction, which leaves the file one version ahead until the next apply, benign by decision 6. A persistence failure is a failed apply: no acknowledgement ever describes a policy without its persisted copy. (Amended 2026-10-05; see Amendments.)
• nft list ruleset is a complete debugging surface: set names carry rule ids, rule comments carry rule ids and marks, the terminal rule's comment names the mode.
• Blocked and would-block records count attempts, one per logged packet: a client that retries a refused connection is several attempts. Allowed records count tracked connections. The two are different quantities by construction and the schema keeps them apart by decision.
• The connection mark is shared by construction: the agent owns bits 16 through 31 and never writes or reads the rest. A host tool that uses the low sixteen bits coexists without configuration; one that uses the high bits conflicts, and that is the documented boundary rather than an open question.
• Simulation and enforcement share one rendering and one code path; a change to the chain structure changes both, and the one-line test says so.
• The local kill switch is honest about the trust model: it removes the kernel rules, it does not silence the daemon, and it says which of the two the operator has done.
• A rule's value is state the agent keeps, beside the policy, for as long as the agent's state directory lives. A state directory written before the allocation was persisted carries none; it is seeded from the persisted policy in canonical order, which is exactly the numbering such a directory's kernel ruleset carries.
• Every IPv4 and IPv6 rendering, delta, allocation, and persistence path, persistence failures and the persist-before-apply order included, is covered without privileges by golden and unit tests; the kernel-facing behavior (drops, attribution, attribution of connections held open across a renumbering change and across their rule's removal, admission of traffic the kernel does not track, in both modes, simulation, set updates without reload, rules surviving the daemon, restart from disk, teardown) is covered by a network-namespace suite that runs as root in its own CI job and locally through make test-netns. The suite's visibility test runs in a namespace where nothing but the owned table references connection tracking, and asserts that connections arrive in closed flow windows as OBSERVED and that nothing is dropped. (Amended 2026-10-08 and 2026-10-17; see Amendments.)

Amendments