Table of Contents

Replication modes

This page documents Orleans.Lattice.Replication 9.9.0, in the documentation for Orleans.Lattice 9.9.0 (release line 9.9), built 2026-10-04. It is also published as markdown, with every table and list, at replication-modes.md, and llms.txt lists every page.

Every tree replicated by Orleans.Lattice.Replication declares a LatticeMergeMode at configuration time. The mode tells receivers how to merge the captured value bytes; the producer stamps it onto every emitted WalRecord so the receiver never has to guess.

There is no implicit fallback. A tree that is not declared in LatticeReplicationOptions.ReplicatedTrees (or, with runtime replication configuration enabled, enabled at runtime - see Runtime Replication Config) is not replicated. This is deliberate - the core library stores every value as opaque byte[], so the producer cannot recognise CRDT primitives by inspection. Implicit opt-in would silently fall back to last-writer-wins on bytes and risk concurrent-update data loss; explicit declaration removes the footgun.

Receivers enforce the declaration independently. The apply path re-resolves the tree's enrollment and merge mode on the receiving cluster: an inbound entry for a tree that is not replicated there is dropped, and one whose wire mode disagrees with the locally resolved mode is dead-lettered as mode_mismatch (see Replication apply).

Declaring a mode

siloBuilder.AddLatticeReplication(opts =>
{
    opts.ClusterId = "site-a";
    opts.ReplicatedTrees = new Dictionary<string, LatticeMergeMode>
    {
        ["users"] = LatticeMergeMode.LwwRegister,
        ["orders"] = LatticeMergeMode.LwwRegister,
    };
});

null and an empty dictionary both mean "no trees are replicated" - the commit-time observer short-circuits before any sink call.

Available modes

Mode Status Convergence guarantee
LwwRegister Available Last-writer-wins ordered by HybridLogicalClock; an exact HLC tie falls to the replica-invariant tombstone, expiry, and value-byte fields before OriginClusterId. Concurrent writes from different clusters silently drop the loser; safe under single-writer-per-key discipline.
OrSet Available Observed-remove set. State-based merge - concurrent active-active adds and removes from multiple clusters survive convergence with their causal dot context preserved.
PnCounter Available Positive-negative counter. Pointwise-max merge on each replica's positive and negative components - concurrent increments and decrements from multiple clusters sum correctly.
VersionVector Available Version vector. Pointwise-max merge on each replica's HybridLogicalClock entry. Late or duplicate delivery is a no-op.
MvRegister Available Multi-value register. Dot-tagged state-based merge - concurrent writes from different clusters survive convergence as a conflict set the application resolves via MvRegisterAccessor<T>.ValuesAsync().
OrMap Available Observed-remove map of (TKey, TValue) where TValue is itself a CRDT. Per-key values converge recursively through ICrdt<TValue>.MergeFrom. Requires a one-time siloBuilder.AddOrMapShape<TKey, TValue>(treeName) registration on each receiving silo so the applier can resolve the generic shape; an unregistered shape faults the apply rather than silently dropping the entry.
Sequence Available Replicated Growable Array (RGA) sequence for collaborative ordered lists / text. Each insert ships the dot-explicit triple (dot, parentDot, value) and each remove ships the tombstoned dot, so concurrent active-active inserts and deletes from multiple clusters converge on an identical ordered traversal via the descending (Counter, ReplicaId) sibling tie-break. Author values through ILattice.Sequence<T>(key). The descriptor is a global closed shape, so no per-tree registration is required.
OrFlag Available Observed-remove (enable-wins) flag. Each key carries a single presence bit that is set while at least one enable dot survives: an observed-remove (disable) dot cancels every enable dot from the same replica at or below its counter. Concurrent enable and disable from multiple clusters converge enable-wins with their causal dot context preserved. The minimal observed-remove primitive for composite-key membership rows (e.g. a tag/key secondary index). Author through ILattice.OrFlag(key). The descriptor is a global closed shape, so no per-tree registration is required.
RwFlag Available Remove-wins (disable-wins) flag - the inverse of OrFlag. Each key carries a single presence bit tracked by three grow-only dot lists (enables, disables, and the disable-dots an observed enable has tombstoned); the flag is enabled only when an enable dot survives and no live disable remains. Concurrent enable and disable from multiple clusters converge disable-wins, so ties and unobserved withdrawals fail closed. Author through ILattice.RwFlag(key). The descriptor is a global closed shape, so no per-tree registration is required.
GCounter Available Grow-only counter. Pointwise-max merge on each replica's cumulative component - the monotonic-only counter PnCounter is built from, and the natural primitive for monotone metrics, sequence / event counters, and quota consumption where decrement never happens. Concurrent increments from multiple clusters sum correctly, and late or duplicate delivery is an idempotent no-op. Author through ILattice.GCounter(key). The descriptor is a global closed shape, so no per-tree registration is required.
GSet Available Grow-only (G) set of opaque element byte arrays with value-equality by content. The merge is set union - trivially commutative, associative, and idempotent - so concurrent active-active adds from multiple clusters all survive convergence. Grow-only by design: it carries no dots and no tombstones and has no remove operation, making it the minimal set primitive for append-only workloads (tag sets, seen-ids, accumulating audiences); reach for OrSet when removal is needed. Author through ILattice.GSet(key). The descriptor is a global closed shape, so no per-tree registration is required.
RwSet Available Remove-wins observed-remove set - the set-granularity generalisation of RwFlag (an RwFlag is a single-element RwSet, exactly as OrFlag is to OrSet). Per element it keeps a set of add dots, a set of remove dots, and a set of observed-add tombstones cancelling removes; an element is a member only when it carries an add dot and no remove dot survives. Concurrent active-active add and remove of the same element from different clusters converge remove-wins with their causal dot context preserved, so a revoke is never silently resurrected by a concurrent re-add. The remove-wins counterpart of the add-wins OrSet, for membership revocation lists and blocklists where a removal must win the tie. Author through ILattice.RwSet(key). The descriptor is a global closed shape, so no per-tree registration is required.
MaxRegister Available Monotone max register - keeps the greatest totally-ordered value ever seen, paired with an explicit total-order key carried on the wire. The fold is directional max over that total order, so it is commutative, associative, and idempotent - a backwards write or a duplicate delivery is a no-op, and concurrent active-active writes from different clusters converge on the single greatest value without needing the domain comparer on the receiver. The high-water-mark primitive (a monotone gauge, a version ceiling, a max-seen reading). Author through ILattice.MaxRegister<T>(key, orderKeySelector). The descriptor is a global closed shape, so no per-tree registration is required.
MinRegister Available Monotone min register - the inverse of MaxRegister, keeping the smallest totally-ordered value ever seen, paired with an explicit total-order key carried on the wire. The fold is directional min over that total order, so it is commutative, associative, and idempotent - a backwards write or a duplicate delivery is a no-op, and concurrent active-active writes from different clusters converge on the single smallest value without needing the domain comparer on the receiver. The low-water-mark primitive (a min-seen latency floor, a first-seen timestamp). Author through ILattice.MinRegister<T>(key, orderKeySelector). The descriptor is a global closed shape, so no per-tree registration is required.

The validator accepts every defined LatticeMergeMode value; only undefined integer values fail validation (it also rejects a null, empty, or whitespace tree-id key).

When LwwRegister is the right choice

LwwRegister is the right answer for keys with overwrite-with-latest semantics, but only under single-writer-per-key discipline. Each key must have at most one authoritative cluster at any given time (e.g. routed by tenant, by shard, or by ownership token). Under this discipline, last-writer-wins is correct: there is never a genuinely-concurrent write to resolve, and the HLC order (with its deterministic tie-break) just orders the unambiguous successor.

If your workload allows concurrent writes from multiple clusters to the same key, last-writer-wins silently drops the loser - both writes return success on their respective clusters, but only one survives the merge. For those workloads, declare a typed CRDT mode and author values through the matching accessor on ILattice.

How typed CRDT modes apply on the receiver

For every typed CRDT mode, the producer-side accessor authors the matching public typed delta DTO into the single WalRecord.Delta slot at commit time. The WalRecord.Value slot is stripped on the wire from every committed CRDT-mode Set that carries a typed delta (prepared saga entries keep it) - the canonical payload travels only as the typed delta, not as a serialised post-merge snapshot. The receiver-side applier forwards the typed delta from Delta to the tree's CRDT-delta apply path, which folds it into the stored primitive with the primitive's MergeDelta operation inside a single grain turn and records a CRDT-delta revision, exactly as a locally-authored CRDT write is recorded. The merge is wrapped in a LatticeOriginContext.With(originClusterId) scope so the entry the receiver's commit appends to its own WAL carries the foreign origin, and the receiver's ship loop - which ships only locally-authored entries - filters it out: the same cycle-break semantics as LWW. Change-feed consumers that historically read Value directly on CRDT-mode entries must migrate to either reading Delta and folding it against their own prior observed state, or reading the post-merge state through the public lattice surface (ILattice.GetAsync / typed accessors).

Typed-delta merge is commutative, associative, and idempotent: late or duplicate delivery converges to the same set / counter / vector / map regardless of arrival order. The snapshot-pinned causal floor and the shadow-forward identity cache still short-circuit redundant re-delivery, but correctness does not depend on them for typed CRDT modes.

OR-Map shape registration

OrMap<TKey, TValue> is generic, so the receiver cannot infer (TKey, TValue) from the WalRecord alone. Each silo that may apply an OR-Map entry must register the concrete shape once at startup:

using Orleans.Lattice;

siloBuilder.AddOrMapShape<string, OrSet>("tags-by-user");

The registration installs a deserialiser / merger descriptor into the CrdtShapeRegistry singleton before silo activation; an apply against a tree configured for LatticeMergeMode.OrMap with no matching registration faults the apply with a clear configuration-error message rather than silently dropping the entry.

Sequence mode back-pressure hazard

Sequence mode replicates an RGA at operation granularity: every InsertAtAsync / InsertAfterAsync / RemoveAtAsync / RemoveAsync call commits one CRDT-delta WAL entry, and every WAL entry is one unit of replication ship work. This is the right granularity for convergence - the dot-explicit insert/tombstone is the minimal structural intent a receiver needs - but it makes a high-frequency editor against a single sequence key a WAL-amplification hazard: a collaborative text buffer that commits one keystroke per InsertAfterAsync generates one WAL entry, one change-feed event, and one shipped delta per keystroke. A sustained fast typist (or a programmatic bulk import that inserts character-by-character) can therefore saturate the WAL partition that key hashes to and the outbound shipper for that key.

Mitigations, in order of preference:

  • Producer-side debounce / coalescing. Buffer keystrokes in the editor for a short window (for example 50-150 ms) and commit the batch as a run of inserts under the last-resolved parent dot, or as a single multi-character element when the application's element granularity permits. Coalescing N keystrokes into one commit cuts the WAL and ship rate by N without changing the converged order. The debounce lives in the application's edit loop, not in the lattice - the lattice deliberately commits exactly what it is told so the convergence contract stays exact.
  • Coarser-grained whole-value storage under LwwRegister for cold sequences. A sequence that is read-mostly and only occasionally rewritten wholesale (a rarely-edited document, a config list rebuilt on save) does not need per-keystroke convergence. Storing it as an opaque value under LwwRegister (or rebuilding and writing the whole list on save) replaces the per-edit WAL storm with one entry per save, at the cost of last-writer-wins on concurrent whole-document writes. Reserve Sequence mode for keys that are genuinely concurrently edited at fine granularity.

The WAL saturation back-pressure surface (IWalSaturationSignal / IWalSaturationObserver) still applies: a producer that ignores the debounce guidance and drives a single hot sequence key past the per-tree WAL admission budget observes the standard saturation signal and LatticeSaturatedException, the same as any other write-amplifying workload.

Single shape per tree

A replicated tree is single-shape: every value in it is authored and shipped under the one LatticeMergeMode the tree is declared with. The mode is a property of the tree, not of the individual write - the producer stamps the declared mode onto every WalRecord, hoists it once per batch into the encoded batch header, and the receiver re-stamps it onto every decoded entry before dispatching the typed apply, overriding any mode carried on the entry itself, so a batch cannot carry a second shape, and the receiver never re-inspects the bytes to guess one.

This means a write whose shape disagrees with the declared mode cannot converge on the peer. A plain last-writer-wins write to a tree declared as a CRDT mode ships value bytes the receiver tries to decode as a typed delta; a CRDT write under the wrong mode ships a delta the receiver decodes with the wrong shape. Either way the receiver's typed apply throws while decoding the payload under the declared shape, the entry is retried, and after MaxApplyRetries it is parked on the dead-letter queue. The origin cluster's own copy stays correct, so the divergence is silent - the peer simply never receives that key.

To turn that silent, receiver-side divergence into a loud, origin-side failure, the public ILattice write surface fails fast. When a tree is declared for replication, any write whose shape does not match the declared mode throws LatticeReplicationModeMismatchException before it commits - nothing is written locally and nothing is shipped:

  • A plain last-writer-wins write (SetAsync, SetManyAsync, SetManyAtomicAsync, SetIfVersionAsync, GetOrSetAsync, DeleteAsync, DeleteRangeAsync, BulkLoadAsync, and their predicate variants) to a tree declared as any typed CRDT mode is rejected. Author the value through the matching accessor (OrSet(key), PnCounter(key), and so on) instead.
  • A CRDT write (ApplyCrdtDeltaAsync, reached through any CRDT accessor) whose mode differs from the declared mode is rejected - whether the tree is declared as a different CRDT mode or as LwwRegister.
try
{
    // 'tree' is declared for replication as a typed CRDT mode (for example
    // OrSet). A plain last-writer-wins write is rejected before it commits,
    // because the receiver could not decode the bytes under the declared shape.
    await tree.SetAsync("k", new byte[] { 1 }, cancellationToken);
}
catch (LatticeReplicationModeMismatchException ex)
{
    Console.WriteLine($"{ex.TreeId}: declared {ex.DeclaredMode}, attempted {ex.AttemptedMode}");
}

The guard is a no-op for trees that are not replicated (the merge-mode resolver returns null, so single-cluster hosts are never affected) and for writes whose shape already matches the declared mode, including a plain write to a tree declared as LwwRegister. It costs a single cached resolver reference and one per-tree dictionary read per write.

The rejection covers the direct ILattice write surface. The cross-tree atomic write builder stages writes through a separate coordinator saga; when a slice of a cross-tree batch targets a replicated tree, the same single-shape rule applies - stage the slice with the matching accessor's Stage* method rather than a plain value write.

How the mode is resolved at commit time

The commit-time observer routes every mutation through ILatticeMergeModeResolver.Resolve(treeId):

  • The default implementation reads LatticeReplicationOptions.ReplicatedTrees and caches the per-tree outcome until IOptionsMonitor.OnChange fires.
  • A host that enables runtime replication configuration (AddLatticeReplication(..., enableRuntimeConfig: true)) gets a snapshot-backed resolver instead: it answers from the runtime configuration first, falls back to ReplicatedTrees, and returns null for a tree whose runtime mode is ambiguous even when the tree is also declared statically (see Fail-closed ambiguity).
  • Hosts can replace the registration to source the mode map from elsewhere (a control plane, a feature flag system, or a permissive test stub that opts every tree in to LwwRegister).
  • A null return value means "this tree is not replicated" and the observer returns immediately, before any sink call.

The same resolver is consulted again downstream. The leaf commit-log writer stamps the resolved mode (or a CRDT write's own authored mode) onto the durable WalRecord.Mode at WAL append, so a storage replay can recover it, and the shipper resolves the mode again when it frames each batch, hoisting it once into the batch header that the receiver re-stamps onto every decoded entry - so receivers pick the correct apply algorithm without re-inspecting the value bytes.

A null resolution does not pause a shipper that is already running. It keeps draining the tree's WAL and ships new entries under an LwwRegister batch header; a receiver that still resolves a CRDT mode dead-letters them as mode_mismatch, one that resolves no mode drops them, and either way the sender advances past them (see Fail-closed ambiguity).