Table of Contents

Per-entry TTL (time-to-live)

Orleans.Lattice supports per-entry time-to-live (TTL) on writes. An entry written with a TTL is visible to every read until its absolute expiry instant, after which it becomes invisible to reads and is eventually reaped by tombstone compaction. TTL is available on the last-writer-wins SetAsync path (below) and on typed CRDT writes (Per-entry TTL on CRDT writes); both share the same absolute-expiry storage and read-filtering rules.

TTL on SetAsync

The public API is an overload on ILattice:

Task SetAsync(string key, byte[] value, TimeSpan ttl, CancellationToken cancellationToken = default);

Typed convenience overloads are provided on TypedLatticeExtensions:

var session = new User("alice", 1);
var serializer = new JsonLatticeSerializer<User>();

await lattice.SetAsync("session:42", session, TimeSpan.FromMinutes(30));
await lattice.SetAsync("session:42", session, TimeSpan.FromMinutes(30), serializer);

Behavior:

  • Absolute UTC expiry, resolved server-side. The silo handling the SetAsync call computes DateTimeOffset.UtcNow.Add(ttl).UtcTicks and stores that absolute instant on the entry. Client clock skew does not shift individual entry lifetimes - all entries expire relative to the server that accepted them.
  • Validation. SetAsync throws ArgumentOutOfRangeException if ttl is zero, negative, or large enough to overflow DateTimeOffset.MaxValue when added to DateTimeOffset.UtcNow.
  • Non-TTL writes are unaffected. Existing SetAsync(key, value) calls and bulk-load entries carry no expiry (ExpiresAtTicks == 0) and never expire on their own.
  • Overwrite semantics. A later SetAsync on the same key wins by hybrid-logical-clock timestamp regardless of whether either write carried a TTL. Writing a non-TTL value over a TTL value clears the expiry; writing a TTL value over a non-TTL value introduces one.

Per-entry TTL on CRDT writes

The same absolute-expiry model applies to typed CRDT writes. Every typed accessor's primary mutating method has a TTL overload, and the low-level ILattice.ApplyCrdtDeltaAsync seam gains one:

await tree.OrSet("cart:42").AddAsync(new byte[] { 1 }, "cluster-A", TimeSpan.FromMinutes(30), cancellationToken);

Task<HybridLogicalClock> ApplyCrdtDeltaAsync(string key, LatticeMergeMode mode, byte[] deltaBytes, TimeSpan ttl, CancellationToken cancellationToken = default);

The TTL overload is available on the primary write of all thirteen accessors: GCounter.IncrementAsync, GSet.AddAsync, MaxRegister.SetAsync, MinRegister.SetAsync, MvRegister.SetAsync, OrFlag.EnableAsync, OrMap.SetAsync, OrSet.AddAsync, PnCounter.IncrementAsync, Sequence.InsertAtAsync, RwFlag.EnableAsync, RwSet.AddAsync, and VersionVector.TickAsync.

Behavior:

  • Absolute UTC expiry, resolved server-side, exactly as for SetAsync: the handling silo computes DateTimeOffset.UtcNow.Add(ttl).UtcTicks and stores it on the entry. Validation is identical: every accessor's TTL overload throws ArgumentOutOfRangeException for a zero or negative ttl before it reads or writes anything (so a lapsed remaining life is rejected, never written as a durable entry), and the ILattice seam throws the same for one large enough to overflow.
  • Expiry converges by a max-absolute-ticks join, not last-writer-wins. When two TTL'd CRDT writes to the same key are merged, the resolved expiry is Math.Max(existingExpiry, incomingExpiry), with 0 (durable) as the semilattice bottom. This join is commutative, associative, and idempotent, so every replica converges on the same expiry regardless of the order in which it observes the writes, and a later or concurrent TTL'd write extends (never shortens) the entry's life - refresh-extends-life.
  • Why this differs from the LWW-TTL path. An LwwValue resolves its whole value - expiry included - by the single last-writer-by-HLC. A CRDT has no single winning write: both deltas fold into the converged state, so the expiry needs its own independent commutative join, applied regardless of which write wins the row-level HLC merge.
  • A durable (no-TTL) CRDT write leaves any existing expiry unchanged. Writing without a TTL contributes 0 (the bottom element), which the max-join ignores, so it neither introduces nor clears an expiry. Clearing a TTL'd entry back to durable is out of scope for this version.

Replication and cold rebuild

The resolved absolute expiry ships on the wire with each replicated CRDT delta (WalRecord.ExpiresAtTicks) and is applied verbatim by the receiver - the absolute tick is never re-resolved from a relative TTL on receive, so inter-cluster clock skew cannot shift a replicated entry's lifetime. The batched receive path threads the same per-entry absolute expiry into each coalesced dispatch item. On a cold projection rebuild, each WAL record carries the cumulative-max expiry and replay re-applies the same max(priorExpiry, recordExpiry) join, so the rebuilt entry's expiry is reconstructed independently of HLC replay order.

Read filtering, scans, counts, cursors, caching, and tombstone compaction treat a TTL'd CRDT entry exactly like a TTL'd SetAsync entry - the expiry lives on the stored row (LwwValue.ExpiresAtTicks), not on the value shape, so all the read-path and tombstone-compaction rules below apply unchanged.

Internal representation

Every stored entry carries its TTL alongside the rest of its row:

Stored with each entry Purpose
Value bytes The stored bytes (absent for a tombstone).
HLC timestamp Last-writer-wins conflict resolution.
Tombstone flag Marks a deleted entry.
Absolute expiry (UTC ticks) The instant the entry expires. 0 means "no expiry" - this is the default and keeps pre-TTL snapshots wire-compatible.
Origin cluster and vector clock Replication metadata: which cluster authored the write, and the causal frontier it carried.
Migrated flag Marks a row that arrived through a cross-shard migration, until a newer non-migrated write supersedes it.

A single expiry predicate over the stored row decides, on every read path, whether to hide an entry.

Read paths

All read entry points filter expired entries before returning results to callers:

Operation Behaviour when entry is expired
GetAsync Returns null.
GetWithVersionAsync Returns an empty VersionedValue (Value is null, Version is HybridLogicalClock.Zero) - the same result as a missing key.
ExistsAsync Returns false.
GetManyAsync Key is absent from the returned dictionary.
GetOrSetAsync Treats the expired entry as absent and proceeds to write the supplied value.
SetIfVersionAsync Treats the expired entry as HybridLogicalClock.Zero for CAS comparison.

The effective "now" is captured once per leaf call, so every key a single leaf evaluates for one GetManyAsync / CountAsync / scan request observes the same expiry instant; keys served by different leaves are each judged against their own leaf's clock read.

Scans and counts

Operation Behaviour
ScanKeysAsync / ScanEntriesAsync Expired entries are omitted.
CountAsync / CountPerShardAsync Expired entries are excluded from the count.
DeleteRangeAsync Expired entries are skipped (no tombstone is written for them - compaction will reap them).

Durable cursors

OpenKeyCursorAsync and OpenEntryCursorAsync inherit the filtering rules above: each page fetched by the cursor reads through the same expired-entry filter applied to ad-hoc scans. A long-lived cursor iterating a shard whose entries expire mid-iteration will simply stop seeing those keys on subsequent pages.

Read cache

LeafCacheGrain (the per-silo [StatelessWorker] cache described in Read Caching) stores the raw LwwValue<byte[]> including ExpiresAtTicks. On a cache hit it applies IsExpired(nowUtcTicks) before returning, so expired entries are never served from cache even if the primary leaf has not yet had them compacted.

Atomic writes

SetManyAtomicAsync and SetManyAtomicWhereAsync accept no TTL, and their writes carry none: the saga stages every entry through the non-TTL batch write. Each committed entry therefore resolves against the key's current row by the same last-writer-wins overwrite rule as a non-TTL SetAsync - where it wins, it clears any expiry the key had.

Before staging, the saga reads each key's pre-saga row, absolute expiry included, and treats a value that has already expired as absent. That capture feeds guard evaluation only - it is never used to restore a value. A guarded batch (SetManyAtomicWhereAsync) treats any key with no live pre-saga value (missing, deleted, or expired) as failing its predicate, whatever the predicate says, and a single failing key rejects the whole batch before anything is staged.

An abort never rewrites a pre-saga entry or its TTL, because it issues no rollback writes. Staged entries wait in each leaf's per-transaction pending bucket, invisible to readers; the abort records the decision in the tree's transaction registry, then broadcasts an abort terminal that drops each bucket. Every key keeps its pre-saga row, absolute expiry included, so an aborted batch neither extends nor clears any TTL.

Shard splits

Online shard splits must carry TTL metadata across two distinct code paths:

  • Shadow-forward writes during the shadow phase read the source leaf's raw LwwValue via GetRawEntryAsync (not the filtered VersionedValue) so the shadow receives the full record including ExpiresAtTicks.
  • Drain uses MergeManyAsync(batch), which carries the complete LwwValue for each entry. Expiry is preserved verbatim on the target shard.

See Shard Splitting for the full split lifecycle.

Snapshots

A snapshot copies the live entries of each source shard - 0 to ShardCount - 1 and every shard the shard map routes to - as full stored rows, HLC timestamp and absolute expiry included. An offline snapshot bulk-loads those rows into the empty destination shard; an online snapshot merges them last-writer-wins alongside the live writes it shadow-forwards. Either way every entry the snapshot copies keeps its TTL unchanged, in both modes. An online snapshot does not mirror typed CRDT deltas - with or without a TTL - so a delta applied to a source key after the copy has read past it does not reach the destination. See Snapshots.

Resize

ResizeAsync is implemented on top of the snapshot pipeline (copy to a new physical tree, then swap the alias). TTLs are preserved verbatim on every entry the copy carries, for the same reason snapshots preserve them, and the copy has the same limits as an online snapshot. See Tree Sizing for the resize phase machine and what it does not carry, and Tree Storage for sizing guidance.

Merge (MergeAsync)

Merging one tree into another flows entries through GetDeltaSinceAsync + MergeManyAsync. Conflict resolution is LWW by HLC timestamp; the winning LwwValue carries its ExpiresAtTicks unchanged. If the winner was written without a TTL, the target entry ends up with no expiry - even if the loser had one.

Tombstone compaction

Expired live entries past the configured TombstoneGracePeriod are reaped alongside regular tombstones by the reminder-driven compaction path. The grace window prevents a peer with a lagging clock from resurrecting an entry via a delayed merge: a remote replica that has not yet seen the expiry will continue to propagate the entry as "live" (see CRDT replication below), and the grace period ensures local compaction waits long enough for those stragglers to converge before permanent removal. See Tombstone Compaction.

CRDT replication invariant

Expired entries are intentionally preserved on the replication-layer code paths:

  • BPlusLeafGrain.GetDeltaSinceAsync - returns expired entries to requesting peers.
  • BPlusLeafGrain.MergeEntriesAsync / MergeManyAsync - stores expired entries received from peers.

This is required for CRDT convergence: two replicas with different views of "now" must still agree on the last-writer-wins value for a key, and that resolution needs access to the entry's HLC timestamp even after its ExpiresAtTicks has passed. User-facing read paths apply the expiry filter; the replication layer does not. Read-cache deltas (StateDelta from the primary to LeafCacheGrain) follow the same rule - the cache receives expired entries verbatim and filters them at read time.

Cross-cluster replication

A replicated last-writer-wins write carries the absolute expiry the source resolved rather than a relative TTL, so inter-cluster clock skew cannot shift it on the receiving cluster, as for a replicated CRDT delta (see Replication and cold rebuild). A peer seeded by a whole-tree snapshot bootstrap, or repaired by the anti-entropy bootstrap fallback, likewise receives each copied key with the source entry's absolute expiry (0 for a durable key), so on a last-writer-wins tree a key with a TTL on the source keeps its expiry instant on the peer. On a typed CRDT tree those two paths copy each key's full CRDT state and merge it into the peer's row without applying the carried expiry, so a key with a TTL on the source arrives on the peer as a durable entry.

Bulk load

BulkLoadAsync and the streaming bulk-load overloads do not currently accept a TTL. Entries loaded via bulk load have no expiry. Follow-on SetAsync(..., TimeSpan) calls can introduce a TTL on specific keys if needed.

Operational notes

  • Clock skew. Because expiry is resolved on the silo that accepts the write, TTL consistency depends on reasonably synchronised silo clocks - not client clocks. Two writes with the same ttl made simultaneously from different clients will expire at nearly the same instant regardless of client time, provided the silos they land on share the same wall clock.
  • Grace period. Tune LatticeOptions.TombstoneGracePeriod to be at least the worst-case clock drift plus replication delay you expect between silos. Setting it too low risks a lagging replica resurrecting an expired entry; setting it too high delays physical space reclamation.
  • CacheTtl is independent of entry TTL. CacheTtl controls how long a LeafCacheGrain may serve reads without re-polling the primary for a delta; it has no effect on an entry's own expiry. An expired entry is filtered at the cache regardless of CacheTtl.