Orleans.Lattice.Api.Replication

A transport-agnostic control facade for runtime per-tree cross-cluster replication, layered over Orleans.Lattice.Replication.

What is it?

Orleans.Lattice.Api.Replication is the control plane for turning cross-cluster replication on and off per tree at runtime. The Orleans.Lattice.Replication package ships the shipping, bootstrap, merge, and anti-entropy engine; this package adds the administrative surface an operator dashboard, a CLI, or an internal admin service needs to enable a tree under a chosen merge mode, disable it again, and inspect the per-tree replicated set - over a single surface with no wire dependency.

It is built the same way as the sibling Orleans.Lattice.Api.Backup control facade:

A transport-agnostic facade. A single control surface (ILatticeReplicationControl, a public contract in the shared Orleans.Lattice.Api.Abstractions package) exposes enable, disable, and permission-scoped config reporting over plain request / response records. It has no wire dependency, so the same surface serves an in-process consumer and a remote one.
A code-first gRPC binding (the sibling Orleans.Lattice.Api.Replication.Grpc package) that projects this facade onto a remotely callable service and typed client.
An MCP tool group (in Orleans.Lattice.Api.Mcp) that exposes the same three operations as agent tools, gated by the same access control. The config-inspect tool is offered as soon as the host registers the group (AddReplicationTools); the mutating enable and disable tools only once replication control is opted in as well (see the replication tools).

How configuration is distributed

Replication configuration is not a bespoke store. It is itself a replicated CRDT system tree, sys-replication-config, dogfooding the exact pattern the replication engine already uses for its own membership and auth system trees. The tree holds a single OR-Map, under one well-known key, keyed by target tree id; each value is a small composite record of an enablement flag (a disable-wins RwFlag) and the fixed merge mode (an MvRegister so concurrent divergent modes survive and stay detectable rather than silently overwriting one another).

Because the configuration is a converging tree, an operator flips a tree on once, on any cluster, and every enrolled peer converges to the same decision. Per-cluster propagation is not re-consented - the trust boundary is the existing peer enrolment, so authorization gates the authoring cluster only.

Installed apps author entries in the same tree. A tenant's install of an Orleans.Lattice.Apps app whose manifest declares replication intent enrols those trees through the engine's config authority as it activates, not through this facade, so the config report lists them alongside trees enabled here. Disabling such a tree here does not stop the app: its next enable, or reconcile while the app is enabled, enrols the declared tree again.

For the engine-side mechanics - the static anchor, the compiled snapshot, and the fail-closed ambiguity handling - see runtime replication configuration.

Core properties

Opt-in and absent by default. Nothing registers unless the host calls AddLatticeReplicationApi() - or, for the read-only peer status, AddLatticeReplicationStatusApi() - on the silo, and neither facade does background work until a method is called.
Fail-closed by construction. Enable and disable authorize their target tree through the existing Lattice access gate for the dedicated LatticeOperation.Replication capability, before touching engine state: an anonymous or unauthorized caller is denied with LatticeAuthorizationDeniedException and the engine is never consulted. The config read applies the same check per tree to filter what it reports (see permission-scoped discovery below). As on the data plane, a host with no authorization add-on registered runs the core no-op access gate, which allows every call.
Mode fixed at enable time. The merge mode is chosen when a tree is first enabled and cannot be changed in place; enabling an already-enabled tree under a different mode is rejected. The sanctioned way to change a mode is to disable, then re-enable under the new mode; supplying a bootstrap source cluster on that enable re-seeds a tree that already holds data from a snapshot.
Disable never purges. Disabling never deletes data already replicated to peers. Unless the static map also declares the tree, its merge-mode resolution then returns no mode, but shipping does not pause: an already-active shipper keeps shipping the tree's new local writes, and a peer that has converged on the disable drops them at its receiver-side enrollment gate (see runtime replication configuration).
Permission-scoped discovery. GetReplicationConfigAsync reports only the trees the caller is authorized to manage, so it never reveals the existence of a tree outside the caller's grant.
Both enrollment sources are reconciled. A replication-enabled host resolves a tree's merge mode from the runtime config tree and the static deployment-time replicated-tree map, which acts as a fallback floor. GetReplicationConfigAsync reports the union under the same precedence the commit path applies, so an estate enrolled purely through deployment configuration is reported as replicating rather than as empty. Each entry's Source names which one is in force. The static map always holds the sys-replication-config tree itself, which enableRuntimeConfig: true enrols under OrMap, so the report lists that tree as a Static entry to any caller authorized to manage it.

Ordering

AddLatticeReplicationApi() must be called after AddLatticeReplication(..., enableRuntimeConfig: true) (which installs the dynamic config authority): that authority is the source of truth this facade drives. Calling it first fails fast at registration with an actionable message.

Surface

The facade operations (each reached over the gRPC binding as one RPC, and over MCP as one tool):

Operation	Purpose
Enable replication	Enable a tree under a fixed merge mode, optionally bootstrapping a non-empty tree from a named source cluster.
Disable replication	Disable a tree's runtime enrollment without purging already-replicated peer data. Idempotent.
Get replication config	Report each authorized tree's enrolled state, the merge mode in force, its ambiguity status, and which enrollment source put it in force.

Peer status

ILatticeReplicationStatus is a separate, read-only contract. It reports how each replication link is doing, and leaves ILatticeReplicationControl unchanged. Register it with AddLatticeReplicationStatusApi(), after AddLatticeReplication(...): called first, it throws at registration. It reads telemetry rather than the config authority, so it does not need enableRuntimeConfig: true or AddLatticeReplicationApi().

GetPeerStatusAsync(ReplicationPeerStatusQuery) returns a paged ReplicationPeerStatusPage. The page carries the local region id, then one ReplicationPeerStatusEntry per tree, peer region and direction. Each entry holds entries and bytes behind, consecutive errors, time since last contact, in-flight count, and a derived ReplicationLinkHealth: Healthy, Lagging, Stalled or Unknown.

Cluster-wide. Peer statistics are kept per silo. The facade fans out to every active silo through an internal grain service. When the same link appears on more than one silo, it keeps the most recent contact whole. A silo that fails to answer fails the whole read rather than producing a partial page. None of this runs on the shipping or apply path.
Effective ids, the same as the config report. Each link names its tree by the effective id, which is the id GetReplicationConfigAsync uses for the same tree, so the two reports join on tree id. A default-tenant tree keeps its bare name, and an app's tree reads a/{app}/{tree}. Under an asserted non-default tenant, the caller's own trees are tenant-qualified: t/{tenant}/{name}, or t/{tenant}/a/{app}/{tree} for an app's tree. A tree filter accepts the tenant-local name or the qualified id. A continuation token only encodes rows the caller was shown.
Permission-scoped. Each tree is checked against the same Replication capability that GetReplicationConfigAsync requires. Trees the caller may not manage are left out. A tree filter the caller may not manage returns an empty page without reading any statistics.
Configurable health. LatticeReplicationStatusOptions sets the thresholds. A signal trips a bound only when it is strictly above it, and the worst signal wins. By default a link is lagging above 1,000 entries behind, 5 consecutive errors or 30 seconds without contact, and stalled above 10,000 entries, 50 errors or 5 minutes. The backlog and no-contact defaults apply to outbound links; inbound links can have their own no-contact thresholds, which are off by default. See Configuration.

The Explorer's Replication area draws its estate diagram from this report.

Reference

API reference - the registration entry points, the public options and model types, and the control and peer-status operations.
Configuration - the public options properties, their types, and defaults.
Architecture - how the control facade authorizes, delegates to the engine authority, and scopes discovery, and how the peer-status read path reads, authorizes, and pages.

Table of Contents