---
title: "Durable per-key history views"
url: "https://nsta1.github.io/Orleans.Lattice/docs/lattice/history-views.html"
source: "https://github.com/NSTA1/Orleans.Lattice/blob/release/9.9/docs/lattice/history-views.md"
package: "Orleans.Lattice"
version: "9.9.0"
documents: "Orleans.Lattice 9.9.0 (release line 9.9)"
built: "2026-10-04"
all-pages: "https://nsta1.github.io/Orleans.Lattice/llms.txt"
bundle: "https://nsta1.github.io/Orleans.Lattice/docs/lattice/llms-full.txt"
---
# Durable per-key history views

Part of the [Orleans.Lattice documentation](architecture.md).

A **history view** is an opt-in, append-only materialised view that records every
revision of every key in a source tree. Rather than a bespoke per-leaf revision
store, it reuses the [materialised view](materialised-views.md) subsystem: the
view tails the source tree's write-ahead log and re-keys each mutation into a
durable revision row, so the full timeline survives independently of source
WAL garbage collection.

History starts from what the source still holds when the view is created. If no
source write-ahead-log entry has been garbage-collected yet, the maintainer's
first drain replays the log from its beginning, so earlier revisions are
recorded with their original clocks; once garbage collection has trimmed the
start of the log, the maintainer instead seeds one revision per live key from
current source state and tails forward from there. The same seed is taken when
the source's logical id already resolves to a different physical tree (after a
resize or a restore, for example): the view then starts from current state
whether or not the log has been trimmed. Revisions trimmed from the log before
the view existed are never reconstructed.

## How it works

The history projection re-keys each source mutation to `{sourceKey}/{encodedHlc}`,
where the HLC suffix is a fixed-width, chronologically sortable encoding of the
mutation's hybrid logical clock. Because distinct mutations carry distinct HLCs,
distinct revisions map to distinct view keys, so nothing folds and the complete
per-key timeline is retained.

Each revision is stored as a `HistoryRow` carrying the kind of mutation
(a set, a delete, a CRDT delta, or a range-tombstone marker), the originating
cluster, and - for last-writer-wins values - a content hash and length plus,
depending on the retention mode, the value bytes themselves. A CRDT mutation is
recorded as its author delta (the compact, doubling-free history); only one that
carries no delta - an anti-entropy or bootstrap resync that ships the full state -
is recorded as a set of that state, which the retention mode shapes like any
other set.

## Enabling history on a tree

A history view is created the **runtime** way - through `ILatticeViewFactory` -
rather than declared at startup, because only a runtime-created view can be torn
down again (the enable/disable contract). `LatticeHistoryView.Definition` builds
the accumulative definition; resolve the factory and the silo service provider
from the cluster. Create it with `CreateAsync`, which returns only once the view's
runtime registration is durable; the synchronous `Create` overload persists the
registration in the background and cannot report a failure to the caller.

```csharp verify
using Microsoft.Extensions.DependencyInjection;

var factory = client.ServiceProvider.GetRequiredService<ILatticeViewFactory>();
var source = grainFactory.GetGrain<ILattice>("orders");

// Enable history: a runtime view named "orders-history" tailing "orders".
var history = await factory.CreateAsync(
    source,
    "orders-history",
    LatticeHistoryView.Definition("orders-history", client.ServiceProvider),
    cancellationToken);

// Disable history later: deleting the runtime view stops recording and
// releases the source WAL pin.
await factory.DeleteAsync("orders-history", cancellationToken);
```

## Retention modes

Storage cost is bounded and configurable per source tree. The retention policy is
read by the maintainer at drain time and applied around the (pure) projection, so
a change takes effect for the revisions the maintainer applies after it - including
any backlog written before the change but not yet drained - and never rewrites or
rebuilds existing rows.

| Mode | LWW value bytes | Use when |
|------|-----------------|----------|
| `MetadataOnly` (default) | Stripped to a content hash and length. | The timeline and change detection matter, not past values: history reads never return the stripped bytes (the write-ahead-log fallback read applies the same rule). |
| `FullValue` | Stored verbatim per revision. | Point-in-time values must be served directly from the history view. |
| `Hybrid` | Stored verbatim when the maintainer applies the revision within a short window of its write; stripped to metadata when it applies it later (a backlog or a catch-up replay). | Point-in-time values are wanted for promptly applied revisions without paying for them on a backlog. A stored row keeps its shape, so this does not confine full values to a recent tail. |

CRDT revisions are always stored as their delta regardless of mode - the delta
*is* the compact history - except a full-state resync, which is stored as a set
(see [How it works](#how-it-works)).

Under `Hybrid` the "short window" is set by `LatticeViewOptions.HistoryHybridFullValueWindow` (default 5 minutes): a revision keeps its full value bytes when its age at the moment the maintainer applies it is within this window, and one applied later is shaped to metadata. The decision is made once, when the row is written - a stored row is never re-shaped as it ages - so under promptly drained traffic almost every revision keeps its bytes, and only an age bound limits them. The write-ahead-log fallback read applies the same rule at read time instead, so there a revision older than the window reads as metadata.

An optional **age bound** (a positive retention window) stamps each revision row
with an absolute expiry of `now + window`; the normal entry-expiry path reaps old
rows, so no separate reaper is needed. Without an age bound revisions do not
expire: pass `null` as the window to clear it (`SetHistoryRetentionAsync` rejects
a zero or negative window), and `GetHistoryRetentionAsync` reports that state as a
window of `TimeSpan.Zero`.

Both the mode and the window are live-tunable per tree:

```csharp verify
// Keep the last 30 days of full-value revisions for this tree.
await tree.SetHistoryRetentionAsync(
    HistoryRetentionMode.FullValue,
    TimeSpan.FromDays(30),
    cancellationToken);

HistoryRetentionSettings policy = await tree.GetHistoryRetentionAsync(cancellationToken);
// policy.Mode == HistoryRetentionMode.FullValue, policy.Window == 30 days.
```

A tree with no override resolves to `MetadataOnly` with no age bound.

## Reading a key's history

`ILattice.ScanEntryHistoryAsync` returns one key's revision timeline as a page of
`EntryRevision` records. When a history view is enabled for the tree it is the
primary read: a prefix scan over the view tree's `{sourceKey}/{encodedHlc}` rows,
ordered by encoded clock and paged with a continuation token, reusing the same
range-scan machinery as an ordinary entry scan. The read is side-effect-free and
never perturbs the maintainer or its source WAL pin.

```csharp verify
// Read the first page of a key's revision timeline (oldest first).
EntryHistoryPage page = await tree.ScanEntryHistoryAsync(
    "order-42",
    fromHlc: null,
    toHlc: null,
    limit: 100,
    continuation: null,
    cancellationToken);

foreach (EntryRevision revision in page.Revisions)
{
    // revision.Hlc           - the hybrid-logical-clock stamp of the revision
    // revision.Kind          - Set / Delete / CrdtDelta / RangeTombstone
    // revision.OriginClusterId - authoring cluster, or null for a local write
    // revision.ValueHash     - content fingerprint (all retention modes)
    // revision.ValuePreview  - size-bounded value bytes (FullValue / Hybrid)
    // revision.Delta         - size-bounded CRDT author delta (CrdtDelta rows)
}

// Page through the rest of the timeline with the continuation token.
if (page.Continuation is not null)
{
    EntryHistoryPage next = await tree.ScanEntryHistoryAsync(
        "order-42", null, null, 100, page.Continuation, cancellationToken);
}
```

The optional `fromHlc` / `toHlc` arguments clamp the scan to an inclusive
hybrid-logical-clock window. The returned `EntryHistoryPage` describes where the
data came from and whether it is complete:

| Field | Meaning |
|-------|---------|
| `Source` | `View` when read from the durable history view, `WalWindow` for the best-effort write-ahead-log fallback, or `None` when neither is available - and also when the access gate denies a point read of the key, which returns an empty page rather than throwing. |
| `Truncated` | Always `false` on the `View` path - the timeline is never cut off below by WAL garbage collection; it is bounded by the configured retention age and by any rebuild that collapsed it (see [the accumulative guard](#the-accumulative-guard)). `true` on the `WalWindow` fallback when garbage collection has trimmed older entries. |
| `EarliestAvailable` | On a truncated `WalWindow` read, the oldest hybrid-logical-clock still readable; `HybridLogicalClock.Zero` otherwise. |

### Fallback without a history view

For a tree that has **not** opted into a history view, the same method falls back,
best-effort, to the retained source write-ahead-log window for the key: it
enumerates surviving mutations above the current per-partition garbage-collection
trim point in offset order and reports `Source == EntryHistorySource.WalWindow`.
This window is bounded by WAL garbage collection, so it sets `Truncated` and
`EarliestAvailable` honestly when older revisions have already been trimmed - a
partial window is never presented as a full history. Enable a history view when a
durable, retention-bounded timeline is required.

The fallback also reads the retained log raw, which the history view does not. The
view holds an atomic batch's staged writes back until the batch commits, discards
them if it aborts, and skips the records compaction writes. The fallback lists a
staged write as a revision whether its batch later commits, aborts or is still in
flight, so an aborted batch's writes appear as revisions that never took effect.

The log can also hold one revision several times. A resize or snapshot copy, a
reshard migration, a leaf split's redistribution and a replication apply each
append the entry they copy again, under the clock its author stamped, and the reap
mark compaction writes when it removes a deleted or expired entry past its grace
period carries that entry's own clock too. The fallback identifies a revision by
its clock, as the history view keys its rows, and reports each one once, across
pages too, so none of these internal records shows as a change to the key. A
write at a clock of its own is always reported, whatever its record is marked.

## The accumulative guard

An ordinary materialised view is rebuilt from *current* source state when its
projection version changes or when an unconstrained range delete is observed on a
re-keyed projection (which the history projection is) - both of which would
collapse a history timeline. A history view's registration
carries an **accumulative** flag that changes exactly those two behaviours:

- **Projection-version change:** the view adopts the new version forward and keeps
  its existing rows, resuming the drain from the durable checkpoint. The worst
  case is a row-shape discontinuity at the version boundary, never data loss.
- **Unconstrained range delete:** the maintainer records a range-tombstone marker
  revision rather than rebuilding, because in an append-only log a range delete
  does not erase the fact that prior values existed. A predicate-filtered range
  delete already carries its matched keys and yields exact per-key delete
  revisions.

An explicit, operator-triggered rebuild remains the only intentional clear: it
knowingly re-derives the view from current source state (collapsing prior
revisions) and is the escape hatch for genuine view-tree corruption. The flag does
not suppress the maintainer's other rebuild triggers, and each of them collapses
the timeline the same way: falling off the source write-ahead log (the view
lagged past garbage collection), the atomic-staging backstop, a
[source-identity rebind](materialised-views.md#source-identity-rebind) after an
alias change (a resize or its undo, a shadow-cutover restore or its revert, a
schema remediation, or an administrative alias change) repoints the source,
lag-budget eviction, and
`ReconcileAsync`, whose re-derivation from current source state differs from any
timeline that still holds earlier revisions. Retention
mode and window are deliberately kept out of the projection version: they encode
live-tunable policy, not code identity, so changing them never trips a rebuild.

## Limitations

- **No reconstruction of trimmed history.** The timeline begins with whatever the
  source still holds at creation - its untrimmed write-ahead log, or a
  one-revision-per-key seed from current state once garbage collection has
  trimmed that log or when the source is already aliased to another physical
  tree; revisions trimmed before the view existed cannot be recovered.
- **Count-based retention ("keep last N per key") is not expressible** in a pure
  per-mutation projection and is out of scope for this substrate.
- **The read path is built in.** `ILattice.ScanEntryHistoryAsync` queries a key's
  timeline directly off this substrate (see "Reading a key's history" above);
  decoding element-level CRDT provenance is layered on top of the stored deltas.
- **Do not enable lag-budget eviction on a history view.** Lag-budget eviction
  rebuilds from current source state, which collapses the timeline; leave
  `MaxLagBudget` at its default of zero for accumulative views.