--- title: "Orleans.Lattice.Api.TreeAdmin" url: "https://nsta1.github.io/Orleans.Lattice/docs/lattice.api.treeadmin/README.html" source: "https://github.com/NSTA1/Orleans.Lattice/blob/release/9.9/docs/lattice.api.treeadmin/README.md" package: "Orleans.Lattice.Api.TreeAdmin" version: "9.9.0" documents: "Orleans.Lattice 9.9.0 (release line 9.9)" built: "2026-10-04" all-pages: "https://nsta1.github.io/Orleans.Lattice/llms.txt" bundle: "https://nsta1.github.io/Orleans.Lattice/docs/lattice.api.treeadmin/llms-full.txt" --- # Orleans.Lattice.Api.TreeAdmin Part of the [documentation map](../index.md). A transport-agnostic whole-tree administration control facade for [Orleans.Lattice](../../README.md). ## What is it? `Orleans.Lattice.Api.TreeAdmin` is the **whole-tree administration control plane** of a cluster. Where each existing control facade owns one responsibility - [`Orleans.Lattice.Api.Schema`](../lattice.api.schema/README.md) manages schema policy, versioning, remediation, and compliance; [`Orleans.Lattice.Api.Backup`](../lattice.api.backup/README.md) manages capture and restore; [`Orleans.Lattice.Api.Replication`](../lattice.api.replication/README.md) manages runtime replication config - this facade presents one coherent tree-administration surface for an operator dashboard, a CLI, or an internal admin service. It follows **composition over absorption**: it does not re-implement or merge the single-responsibility facades. Instead it delegates to the existing schema, tree, registry, backup / restore, view, tag-index, compaction, WAL-placement, and retention machinery, so behavior, wire format, and authorization stay with their source owners and there is no breaking change to any facade it composes. It is built the same way as the other API facades: - **A transport-agnostic facade.** A single control surface (`ILatticeTreeAdmin`, a public contract in the shared [`Orleans.Lattice.Api.Abstractions`](../lattice.api.abstractions/README.md) package) exposes tree-administration operations over plain request / response records. It has no wire dependency, so the same surface serves an in-process consumer and a remote one. - **A code-first gRPC binding** (the sibling `Orleans.Lattice.Api.TreeAdmin.Grpc` package) that projects this facade onto a remotely callable service and typed client. This package ships no transport of its own; it is the contract every binding adapts over. ## Scope of this release The facade is discoverable and surfaces the current whole-tree administration operation set: - The read-only **capability probe** composes the wrapped schema facade's own probe and reports one flag per distinct grant: `CanAdministerTree` from the caller's whole-tree `Admin` authority (the routine, non-destructive administration grant), `CanManageTreeLifecycle` from the distinct whole-tree `TreeLifecycle` authority (the irreversible or structural lifecycle grant), `CanBulkLoad` from the distinct whole-tree `BulkLoad` authority, `CanRestore` from the distinct whole-tree `Restore` authority combined with a registered backup engine, and `CanViewDiagnostics` from the caller's whole-tree read authority. Each flag is default-deny and advisory; the server still authorizes every real call on attempt. - Six read-only **diagnostics and storage-accounting operations** wrap the existing public grain surface (`ILattice`, `ILatticeAdmin`) rather than re-implementing shard fan-out: per-shard hotness, whole-tree diagnostics, shard-map topology inspection, per-shard leaf-projection digest, rolled-up tree statistics, and cluster-wide storage accounting. Each authorizes through the shared core fail-closed access gate before dialing the grain - the per-tree verbs on whole-tree `Read` authority and the cluster-wide storage summary on the distinct `Telemetry` capability. - Seven **tree lifecycle and per-tree registry configuration operations** wrap the existing registry-backed tree metadata rather than re-implementing registration or config: explicit tree creation (idempotent, with optional initial sizing), existence checks, alias assignment / resolution, per-tree configuration read / update (publish-events, projection-digest, history-retention, and the advisory WAL retained-byte ceiling), and the registry-persisted shard-map read. The mutating verbs (create, set-alias, set-config) authorize on whole-tree `Admin` authority and reject reserved system tree ids (the `_lattice_` namespace); the read verbs (exists, resolve-alias, get-config, get-shard-map) authorize on whole-tree `Read` authority. - Four **tree soft-delete lifecycle operations** wrap the existing public tree grain (`ILattice`) rather than re-implementing the soft-delete coordinator: soft-delete, recover, hard-purge, and a read of the tree's deletion status. The three mutating verbs (delete, recover, purge) authorize on whole-tree `TreeLifecycle` authority - the dedicated destructive-lifecycle capability, held separately from routine `Admin` - and reject reserved system tree ids; hard-purge additionally requires an explicit confirmation flag. The status read authorizes on whole-tree `Read` authority. - A **chunked, resumable bulk-load (tree-creation) operation** wraps the existing public tree grain (`ILattice`) rather than re-implementing tree construction: a begin / append / commit chunked protocol grafts strictly-ascending key/value entries onto an empty tree under a stable, caller-supplied operation id, so a broken sequence resumes from its last un-acknowledged chunk and re-driven chunks deduplicate. The append is a unary call per chunk (each carrying an `IReadOnlyList`), not a client-streamed RPC. All three verbs authorize on the distinct whole-tree `BulkLoad` authority and reject reserved system tree ids; begin additionally rejects a tree that is not empty. - Three **restore-into-tree operations** compose the existing public backup/restore engine (`ILatticeBackupRestoreService`) rather than re-implementing restore: restore a single backup into a tree, restore a whole backup set as one all-or-nothing unit, and revert a restore. A single-tree restore always runs as an online, reversible shadow-cutover - the backup's base chain is replayed HLC-preserving into a fresh shadow physical tree whose alias is atomically cut over - and its result carries the shadow and previous physical trees needed to revert it. The single-tree verbs authorize on the distinct whole-tree `Restore` authority and reject reserved system tree ids; the set restore applies no facade-level whole-tree gate because it spans multiple member trees, so the engine authorizes each member fail-closed. When no backup engine is registered the capability is reported absent and every restore verb is rejected. - Two **online-reshard operations** wrap the existing public tree grain (`ILattice.ReshardAsync` / `IsReshardCompleteAsync`) rather than re-implementing the reshard coordinator: trigger an online reshard that grows or shrinks a tree to a target physical-shard count, and read the tree's reshard status (whether one is in flight plus the current physical fan-out, virtual-slot space, and map version observed from its shard map, and - while a reshard runs - the `TargetShardCount` it moves to and the `StartPhysicalShardCount` it began from, read from the coordinator's persisted intent on every status read, so progress is the distance `CurrentPhysicalShardCount` has moved from `StartPhysicalShardCount` out of the distance from `StartPhysicalShardCount` to `TargetShardCount`, in either direction). A shrink can reach its target count while its last fold is still releasing the retired shard's storage, so `InProgress`, not the count, says when it is done. The trigger authorizes on whole-tree `TreeLifecycle` authority - the same destructive-lifecycle capability as soft-delete - and rejects reserved system tree ids, then propagates the core's validation (target at least 2 and at most the smaller of 4096 and the tree's virtual slot count; a target equal to the current physical shard count is a no-op; an empty tree may be re-pinned to any count in that range; a matching in-flight target is idempotent, while a different in-flight target is rejected, and so is a reshard that would move a populated tree's data while a resize is in flight). The status read authorizes on whole-tree `Read` authority. - Three **online-resize operations** wrap the existing public tree grain (`ILattice.ResizeAsync` / `UndoResizeAsync` / `IsResizeCompleteAsync`) rather than re-implementing the resize coordinator: trigger an online resize that rebuilds a tree with a new B+ node capacity (maximum keys per leaf node and maximum children per internal node), undo the most recent resize, and read the tree's resize status (whether one is in flight, whether an accepted undo is still unwinding - `UndoRequested` - plus the current effective node capacity recorded in the registry, and a durable progress measure: the `Phase` it has reached, and `CompletedUnits` of `TotalUnits`, one unit per shard the copy drains plus one for each of the three steps after it; while an undo unwinds the phase is `Undo` and no units are reported). Undo is accept-then-poll: it is admitted even while a resize phase is running, and returns within a bounded wait with `UndoRequested` set when the unwind outlasts the call, so a caller polls the status read rather than retrying. The trigger and undo authorize on whole-tree `TreeLifecycle` authority - the same destructive-lifecycle capability as soft-delete and reshard - and reject reserved system tree ids, then propagate the core's capacity validation (maximum leaf keys at least 2, maximum internal children at least 3; a matching in-flight target is idempotent, while a resize with different parameters, one requested during an in-flight reshard, or one requested while an accepted undo is still unwinding is rejected; an in-flight resize can be undone at any phase, and a completed one for as long as the pre-resize tree remains within its soft-delete window, so undo is rejected only when no resize exists to undo or the pre-resize tree has already been purged). The status read authorizes on whole-tree `Read` authority. - Two **snapshot-capture operations** wrap the existing public tree grain (`ILattice.SnapshotAsync` / `IsSnapshotCompleteAsync`) rather than re-implementing the snapshot coordinator: capture a snapshot of a source tree into a fresh destination tree (Offline takes every source shard out of service before the copy starts and returns each once it has been copied, for a point-in-time copy; Online keeps the source serving reads and writes while its last-writer-wins mutations are shadow-forwarded to the destination and the drain converges under last-writer-wins, but typed CRDT delta applies and bulk appends are not forwarded, so one that lands on a shard after the copy has read past its key is missing from the destination), and read the tree's snapshot status (whether a capture is in flight, and while it runs the `Phase` it has reached and `CopiedShardCount` of `ShardCount` source shards). This is not the backup facade: the destination is a live tree, not a durable catalogued artifact. The capture verb authorizes on whole-tree `Admin` authority over both the source and the destination tree - the capability the core already gates snapshot at, distinct from the destructive `TreeLifecycle` verbs - and rejects a reserved source or destination system tree id, then propagates the core's validation (the destination must differ from the source and must not already exist; a matching in-flight capture is idempotent, while a different in-flight capture is rejected). An undefined `TreeSnapshotMode` is rejected with `ArgumentOutOfRangeException` before either tree is resolved or authorized. The status read authorizes on whole-tree `Read` authority. Snapshot abort and baseline prune are not surfaced here: an in-flight capture is torn down by the coordinator and per-cursor baselines self-prune by TTL and deterministic cursor-close, so neither has an operator-invokable target. - Five **WAL placement operations** surface per-tree WAL-partition placement and online-move machinery through the facade rather than re-implementing it: read a tree's live WAL placement (the default provider key plus each partition's resolved provider and whether it is resolvable on this silo), audit that placement (the same per-partition view, plus whether every partition's provider key resolves on the silo that produced the audit and that silo's known provider keys), and preview a single partition's move to a target provider key (a read-only plan carrying the entries-to-copy estimate and whether the target is already current), then execute an online move of one partition to a new provider (copying the partition tail under a quiesce lease and, by default, verifying the copy before flipping the placement) and reclaim the orphaned source tail a completed move leaves behind (the move retains that tail, so it stays revertible until this irreversible reclaim). The three inspection verbs (placement, audit, plan) authorize on whole-tree `Read` authority as pure reads with no side effects; the two mutating verbs (execute, reclaim) authorize on the distinct whole-tree `TreeLifecycle` capability - the same destructive-lifecycle gate as reshard and resize, since a WAL move relocates a tree's durability boundary - and reject reserved system tree ids. - Three **orphaned-leaf operations** find and repair descent-unreachable leaves - leaves spliced into a shard's sibling chain but unreachable from its root, left behind by an interrupted split, which hold the write-ahead-log trim floor down - through the existing public tree grain (`ILattice`) rather than re-implementing the walk: audit (a pure read reporting each orphan and what the repair would do about it), survey (the audit's opt-in, read-only full key census), and repair (unsplice each orphan whose keys were all verified readable by descent, refusing any leaf it cannot prove safe). Each call is one bounded batch resumed with its `resumeFrom` token. The audit and survey authorize on whole-tree `Read` authority; the repair authorizes on the distinct whole-tree `TreeLifecycle` capability and rejects reserved system tree ids. - Six **materialised-view administration operations** surface the view registry and per-view maintenance machinery through the facade rather than re-implementing view maintenance: list the cluster's runtime-registered views (view name, source tree, aggregation shape, accumulative flag, runtime provider key, projection version), create (or rebind) a runtime view from a host-registered projection provider key and an opaque provider payload of at most 64 KiB that only that provider interprets and the API never returns, read a single view's status (source tree, aggregation shape, apply lag, active generation tree id, runtime provider key, projection version), rebuild a view from current source state via an online shadow-swap, reconcile a view against current source state (repairing drift only when detected), and drop a view (decommissioning its maintainer and deleting its backing generations). A view is authorized by the readability of its **source tree**, which the facade resolves authoritatively (silo-local view catalog first, then the cluster-wide view registry) and never trusts the caller to supply; a view whose source cannot be resolved is rejected fail-closed. Listing authorizes on the distinct cluster-wide `Telemetry` capability; status reads authorize on the source tree's whole-tree `Read` authority; create authorizes whole-tree `Admin` on the caller-supplied source before any provider code runs, and rebinding an existing view to a different source additionally requires `Admin` on the source it currently tails; the three maintenance verbs (rebuild, reconcile, drop) authorize on the source tree's whole-tree `Admin` authority. When no view subsystem is registered every view verb is rejected with an `InvalidOperationException`; the capability probe carries no view flag, so the absence is not advertised in advance. A view declared at the `AddLatticeViews` code layer is not a runtime registration: it is not listed here and cannot be dropped at runtime. - Three **tag-index administration operations** surface the tag-index registry and per-index reconcile coordinator through the facade rather than re-implementing tag-index maintenance: list the cluster's tag indexes (index name, backing tree id, shard count, covered source trees), read a single index's status (the same shape plus whether its background reconcile sweep is idle), and reconcile an index against current source state (removing orphaned membership rows, returning the reconcile counts). A tag index is authorized by its **backing membership tree** `tag-{indexName}`, whose id the facade derives authoritatively from the caller-supplied index name and never trusts the caller to supply as a tree id; an index whose backing tree is not registered is rejected fail-closed. Listing authorizes on the distinct cluster-wide `Telemetry` capability; status reads authorize on the backing tree's whole-tree `Read` authority; reconcile authorizes on the backing tree's whole-tree `Admin` authority. Reconcile writes only to the backing membership tree - the covered source trees are scanned read-only, and only counts are returned, never key or value content. Unlike the materialised-view subsystem (a separate `AddLatticeViews` opt-in a host can skip), the tag-index factory is registered unconditionally by `AddLattice`, so it is present on every functioning host; the facade still defensively guards against a null factory and would reject every tag-index verb with an `InvalidOperationException` (`FailedPrecondition` over gRPC) if that invariant were ever violated. Index definition and drop are not surfaced here: definition binds a live subject tree at the `AddLattice`/factory code layer, and there is no atomic teardown primitive that both disables the index and unregisters its reconcile schedule. - Three **compaction and history-retention operations** surface tombstone-compaction and registry-backed history-retention machinery through the facade rather than re-implementing GC: trigger an immediate per-shard tombstone-compaction pass (an operator override that bypasses the shard's ordinary compaction cooldown, returning whether the pass was accepted), read a tree's effective history-retention policy (mode plus the optional bounded window), and set a tree's history-retention policy (mode and/or window, read back as applied). Compaction reaps only settled tombstones and expired entries and retention-set is a forward-absorbed configuration change, so neither is destructive to readable state - both are mutating but non-destructive. The retention read authorizes on whole-tree `Read` authority; the compaction trigger and the retention set authorize on whole-tree `Admin` authority and reject reserved system tree ids, and the retention set rejects an undefined `TreeHistoryRetentionMode` with `ArgumentOutOfRangeException` before the tree is resolved or authorized (only a null mode clears the override). Compaction status and an ensure-scheduled verb are not surfaced here: the compaction coordinator exposes no operator-invokable status getter, and its background sweep is scheduled unconditionally, so an ensure verb would be redundant. - The **schema-management tools** (from the sibling schema facade) are surfaced under the tree-administration MCP group by delegation. - Future whole-tree operations should continue the same pattern: delegate to the owning subsystem rather than re-implementing it here. ## Core properties - **Opt-in and absent by default.** Nothing registers unless the host calls `AddLatticeTreeAdminApi()` on the silo. That method takes an optional `Action?` configure delegate; `LatticeApiTreeAdminOptions` is a public, currently-empty options type reserved for later bounding and audit-tuning knobs, so the registration front door is stable even though there is nothing to configure today. Once added the facade does no background work until a method is called. - **Composition, not absorption.** The facade owns no admin plane of its own; it wraps the schema control facade by delegation, reaches the existing public tree surface for read-only diagnostics, and uses registry-backed metadata for lifecycle and per-tree configuration. It introduces no wire or alias change to any composed facade or tree operation. - **Fail-closed by construction.** Every operation authorizes through the shared core access gate (or the wrapped facade's own gate) before doing any work - whole-tree `Read` for the per-tree diagnostics, lifecycle-read, and orphaned-leaf audit / survey verbs, whole-tree `Admin` for the mutating registry-configuration verbs, the distinct whole-tree `TreeLifecycle` capability for the destructive soft-delete / recover / purge, reshard, resize / undo-resize, WAL-move / reclaim, and orphaned-leaf repair verbs, the distinct whole-tree `BulkLoad` capability for the bulk-load verbs, the distinct whole-tree `Restore` capability for the single-tree restore and revert verbs, whole-tree `Admin` over both the source and the destination for the snapshot capture verb, whole-tree `Admin` over both the logical id and the alias target for the alias verb, the source tree's whole-tree `Read` / `Admin` authority for the materialised-view status read and create / rebuild / reconcile / drop verbs (a rebind additionally requiring `Admin` over the source the view currently tails), the distinct cluster-wide `Telemetry` capability for the storage summary and the view listing, the backing membership tree's whole-tree `Read` / `Admin` authority for the tag-index status read and reconcile verbs (with tag-index listing on the cluster-wide `Telemetry` capability), whole-tree `Read` for the history-retention read and whole-tree `Admin` for the compaction trigger and the history-retention set, and schema-management authority for a schema mutation. The gate denies only once an authorization add-on (`Orleans.Lattice.Auth`) is registered; without one the core no-op gate admits every call, so a transport binding's own authorizer is then the only barrier. - **Read-only capability probe.** A caller can ask, with no side effects, which tree-administration operations it may perform over a given tree. The probe is advisory only: it never replaces the per-operation authorization each real call still performs. ## Ordering `AddLatticeTreeAdminApi()` must be called **after** `AddLatticeSchemaApi(...)`: the facade composes the schema control facade (`ILatticeSchemaControl`) by delegation, so that facade must be registered first. Calling it out of order fails fast at registration with an actionable message. ## Surface Every `treeId` these operations accept is a **tenant-local name**: the facade resolves it to its effective, tenant-scoped id through `ITenantContextResolver.ResolveEffectiveTreeIdAsync` at the entry point and uses that one id for **both** the authorization check and the operation, so a verb can never authorize one tree and act on another. With the tenancy add-on absent - or registered, but with no active tenant asserted, which resolves the default tenant - the bare name is returned unchanged, so behaviour is byte-for-byte as before. Under an asserted, non-default active tenant an unqualified name is scoped into that tenant's `t/{tenant}/{name}` namespace, and an already-qualified, well-formed `t/` id or a `_lattice_` system-tree name passes through unchanged (a well-formed foreign `t/{other}/{name}` is left to the tenancy access gate to adjudicate). The call fails closed with a `LatticeTenantAccessDeniedException` when the asserted tenant fails validation against the caller's own membership, or when, under an asserted tenant and outside a system-origin scope, it names a `sys-` tree or a malformed `t/` id that belongs to no tenant. Responses echo the caller's own unqualified name in the field that names the tree or view a verb was addressed by; every other id a response carries is reported as the registry, routing map or backup engine stores it - a physical tree id (an alias resolution, which names the tree itself when it is not aliased, a configuration report, a shard-map inspection, and a restore's shadow and previous physical trees), a view's source and active trees, the cluster-wide storage, view and tag-index listings, and a backup set's per-member results - so under a non-default tenant those ids carry the composed `t/{tenant}/{name}` form. The reserved-id guard the mutating verbs apply then refuses any `t/{tenant}/{name}` id the active tenant does not own - including a foreign one that resolution passed through, and every `t/` id when no tenant is asserted - so a foreign tenant's tree is administered only through that tenant's own scope; the read verbs apply no such guard and leave a foreign id to the gate. Separately, the verbs that wrap a public tree-grain lifecycle operation - soft-delete, recover, purge, bulk-load append, reshard, resize, undo-resize and snapshot capture - inherit that grain's refusal of a materialised-view backing tree (a `view-` name): the call fails with `InvalidOperationException`, because a view's contents are maintained from its source tree. See [`Orleans.Lattice.Tenancy`](../lattice.tenancy/README.md). The facade operations (each reached over the gRPC binding as one RPC, except the orphaned-leaf survey, which rides the `AuditOrphanedLeaves` RPC with the request's `Survey` flag set): | Operation | Purpose | |---|---| | Probe capabilities | Report, with no side effects, which tree-administration operations the caller may perform over a tree, embedding the composed schema capabilities. | | Get shard hotness | Read a tree's per-shard read/write hotness with tree-level totals. Requires whole-tree read authority. | | Get diagnostics | Read a whole-tree diagnostic report. Both modes page through every shard's leaf chain: the default counts live keys only, so its tombstone counts read zero, while the `deep` flag also counts each leaf's tombstoned and expired entries. Requires whole-tree read authority. | | Inspect shard map | Inspect a tree's shard-map topology (physical tree id, virtual/physical shard counts, map version). Requires whole-tree read authority. | | Get projection digest | Read a single shard's leaf-projection content digest for cheap divergence detection. Requires whole-tree read authority. | | Get tree stats | Read a tree's rolled-up topology, live-key counts, and storage byte breakdown in one call. An unregistered tree reports a zeroed snapshot rather than being registered by the read. The report's `TotalTombstones` always reads zero, because the read takes the shallow diagnostic, which counts live keys only. Requires whole-tree read authority. | | Get storage usage | Read cluster-wide storage accounting (write-ahead-log, snapshot and leaf-state bytes per registered tree, with cluster totals). By default each tree's figures come from its short-lived storage-usage cache (`LatticeOptions.StorageUsageCacheTtl`), refilled from each shard root's incrementally maintained byte totals and each WAL partition without walking the leaf chain; the `deep` flag bypasses those caches and forces a fresh leaf-walk that re-measures every shard. Requires cluster telemetry authority. | | Create tree | Explicitly create (register) a tree with an optional initial sizing (shard count, max leaf keys, max internal children; each strictly positive when supplied). Idempotent: creating an already-registered tree is a no-op that preserves its existing configuration (the supplied sizing is honoured only on first registration) and reports `Created` as false. Reserved system tree ids are rejected. Requires whole-tree admin authority; with the tenancy add-on active, the create is then also admitted against the active tenant's quota (the rate and footprint checks every write gets, plus an authoritative tree-count check against `MaxTreeCount`), a breach surfacing as `LatticeQuotaExceededException`. | | Check tree exists | Report whether a tree is registered. Requires whole-tree read authority. | | Set tree alias | Point a logical tree at a physical tree. Reserved system tree ids are rejected, as is a target equal to the logical tree, a `sys-` system data tree targeted from an ordinary tree, or a target across a tenant boundary. Requires whole-tree admin authority over **both** the logical tree id and the physical tree the alias targets, since every data-plane gate downstream is evaluated against the logical id before the alias is resolved. Also refused, with `InvalidOperationException`, when the target is itself aliased (only one level of indirection is allowed), while either tree is deleted or a delete of it is in flight, and, with `LatticeTreeOwnershipDeniedException` (gRPC `PermissionDenied`, carrying the reason), when the registered [ownership guard](../lattice/tree-registry.md#ownership-bounded-aliasing) denies it - for example when an installed app owns one side and not the other. | | Resolve tree alias | Resolve the physical tree a logical tree maps to. Requires whole-tree read authority. | | Get tree config | Read a tree's registry-backed configuration (sizing, alias, and per-tree overrides). Requires whole-tree read authority. | | Set tree config | Apply a partial per-tree configuration update (publish-events, projection-digest, history-retention, WAL retained-byte ceiling), each override written only when its apply flag is set and cleared by a null value on an applied dimension. The WAL retained-byte ceiling is advisory: it never changes what a WAL garbage-collection pass may trim, so lowering it cannot over-trim, and it takes effect on the tree's next pass. An applied history-retention window or WAL ceiling must be strictly positive. Reserved system tree ids are rejected. Requires whole-tree admin authority. | | Get shard map | Read a tree's registry-persisted shard map (custom-map flag, version, virtual/physical shard counts, physical shard indices). Distinct from the diagnostics live-routing inspection. Requires whole-tree read authority. | | Get tree deletion status | Read a tree's soft-deletion status (whether deleted, when, its recovery deadline, whether a purge is in progress or complete - with the shards it has finished out of how many - and whether it can still be recovered). Answers without waiting for a shard's purge. Requires whole-tree read authority. | | Delete tree | Soft-delete a tree, opening its recovery window. Idempotent: deleting an already-deleted tree is a no-op. On an aliased tree (after a resize, restore or schema remediation) it deletes the live copy the alias targets, as one logical operation - see [Deleting an aliased tree](../lattice/tree-deletion.md#deleting-an-aliased-tree). Rejected while the tree is the source of a materialised view (drop the dependent views first), while another tree aliases it, while a resize, restore or remediation is changing its alias, when its alias targets a tree it does not own, and for reserved system tree ids. Requires whole-tree tree-lifecycle authority. | | Recover tree | Recover a soft-deleted tree within its recovery window, including the live copy behind an aliased tree. Rejected on a tree that is not deleted (including a live resized tree) and once its purge has started. Reserved system tree ids are rejected. Requires whole-tree tree-lifecycle authority. | | Purge tree | Irreversibly hard-purge a soft-deleted tree; an explicit confirmation flag is required and a false or omitted value is rejected, as is a tree that is not deleted. On an aliased tree it purges the live copy and unregisters both the copy and the logical tree. Accept-then-poll: the shard walk runs in the background and the call returns the status within a bounded wait, with the purge still in progress for a tree too large to purge in that time; a call while it runs or after it completed returns the status without error. Reserved system tree ids are rejected. Requires whole-tree tree-lifecycle authority. | | Begin bulk-load | Open a chunked, resumable bulk-load (tree-creation) session over an empty tree under a stable, idempotent operation id. The target tree must start empty (no live keys, no tombstones); a populated tree is rejected with `TreeNotEmptyException`, and a tree with a shard the emptiness probe could not sample is rejected with `InvalidOperationException` rather than counted as empty. The emptiness probe is a fresh deep diagnostic that first drops the tree's cached diagnostic reports (see `LatticeOptions.DiagnosticsCacheTtl`), so a report cached while the tree was still empty can never admit a second session onto a tree that chunks have since been grafted onto. The operation id must be non-empty and must not contain `/`. Reserved system tree ids are rejected. Requires whole-tree bulk-load authority. | | Append bulk-load chunk | Graft one strictly-ascending chunk of key/value entries onto an open session at a zero-based, monotonically increasing chunk index, returning the accepted-entry count and the next expected index. Re-sending the same chunk index with the same operation id is idempotent, so a broken stream resumes from its last un-acknowledged chunk. A chunk whose keys are not strictly ascending is rejected with `BulkLoadOrderException` before any entry is applied; an empty chunk is a no-op. Each shard remembers only the most recent chunk it applied, so resume from the last un-acknowledged chunk rather than re-driving an older one, and keep keys ascending across chunks as well - the facade checks key order only within a chunk. Reserved system tree ids are rejected. Requires whole-tree bulk-load authority. | | Commit bulk-load | Close an open bulk-load session and report the tree's observed live-key count (a fresh count, not a cached diagnostic report) for a client-side sanity check. Reserved system tree ids are rejected. Requires whole-tree bulk-load authority. | | Restore into tree | Restore a captured backup into a tree via an online, reversible shadow-cutover, returning the restore outcome (including the shadow and previous physical trees needed to revert it). Idempotent under a stable operation id. Reserved system tree ids are rejected; rejected when no backup engine is registered. Requires whole-tree restore authority. | | Restore backup set | Restore every tree in a captured backup set as a single all-or-nothing unit, each member via an atomic shadow-cutover, returning the per-member restore results this cluster applied. The engine authorizes each member's restore scope fail-closed; rejected when no backup engine is registered. Requires whole-tree restore authority per member. | | Revert tree restore | Revert a shadow-cutover restore by swapping the target tree's alias back to its pre-restore physical tree, from the restore result handed back verbatim. Idempotent. Reserved system tree ids are rejected; rejected when no backup engine is registered. Requires whole-tree restore authority. | | Reshard tree | Trigger an online reshard that grows or shrinks a tree to a target number of distinct physical shards, returning the accepted intent and the observed fan-out. A shrink folds shards together and reports complete only once the retired shards' storage is released. The target must be at least 2 and at most the smaller of 4096 and the tree's virtual slot count (an empty tree may be re-pinned to any count in that range); a target equal to the current count, or a matching in-flight target, is an idempotent no-op, while a different in-flight target is rejected, and so is a reshard that would move a populated tree's data while a resize is in flight. Reserved system tree ids are rejected. Requires whole-tree tree-lifecycle authority. | | Reshard status | Read a tree's online-reshard status - whether a reshard is in flight plus the tree's current physical shard fan-out, virtual-slot space, and shard-map version, and, while one runs, the target and starting counts its progress is measured against (see [Operation progress](#operation-progress)) - as a pure read with no side effects. The fan-out, slot space, and version come from the registry-persisted shard map, so all three read zero until a topology change first persists a custom map (see Get shard map). Requires whole-tree read authority. | | Resize tree | Trigger an online resize that rebuilds a tree with a new B+ node capacity (maximum keys per leaf node and maximum children per internal node), returning the accepted intent and the observed capacity. Maximum leaf keys must be at least 2 and maximum internal children at least 3; a matching in-flight target is idempotent, while a resize with different parameters, one requested during an in-flight reshard, or one requested while an accepted undo is still unwinding is rejected. Reserved system tree ids are rejected. Requires whole-tree tree-lifecycle authority. | | Undo tree resize | Undo the most recent resize of a tree, returning it to its pre-resize physical tree and node capacity rather than rebuilding it: an undo before the alias swap aborts the copy and discards the destination, and one after it recovers the pre-resize tree, removes the alias, restores the prior registry configuration and discards the resized tree. Works on an in-flight resize at any phase, and on a completed one while the pre-resize tree is still within its soft-delete window. Accept-then-poll: the undo is admitted even while a resize phase is running, and the call returns within a bounded wait with `UndoRequested` set on the status when the unwind outlasts it; a retry while the undo is pending is acknowledged again. Rejected when no resize exists to undo or the pre-resize tree has already been purged, and for reserved system tree ids. Requires whole-tree tree-lifecycle authority. | | Resize status | Read a tree's online-resize status - whether a resize is in flight, whether an accepted undo is still unwinding (`UndoRequested`), the tree's current effective node capacity recorded in the registry, and the resize's progress (see [Operation progress](#operation-progress)) - as a pure read with no side effects. Requires whole-tree read authority. | | Snapshot tree | Capture a snapshot of a source tree into a fresh destination tree, returning the accepted intent (destination and mode). Offline takes the source's shards out of service and returns each once it has been copied, for a point-in-time copy; Online keeps the source serving while its last-writer-wins mutations shadow-forward and converge under last-writer-wins, but typed CRDT delta applies and bulk appends are not forwarded, so one that lands after the copy has read past its key is missing from the destination. An undefined `TreeSnapshotMode` is rejected with `ArgumentOutOfRangeException` before either tree is resolved or authorized. The destination must differ from the source and must not already exist; a matching in-flight capture is idempotent, while a different in-flight capture is rejected. Reserved system tree ids are rejected. Requires whole-tree admin authority over **both** the source tree and the destination tree, since the capture creates and populates the destination. | | Snapshot status | Read a tree's snapshot status - whether a capture is in flight for the source tree and, while it runs, its phase and copied-shard progress (see [Operation progress](#operation-progress)) - as a pure read with no side effects. Requires whole-tree read authority. | | Get WAL placement | Read a tree's live WAL placement - the default provider key plus each partition's resolved provider and whether it is resolvable on this silo - as a pure read with no side effects. Requires whole-tree read authority. | | Audit WAL placement | Audit a tree's WAL placement - the per-partition view, plus whether every partition's provider key resolves on the silo that produced the audit and that silo's known provider keys - as a pure read with no side effects. Requires whole-tree read authority. | | Get WAL reclamation | Name the durable materialiser pin holding a tree's write-ahead-log floor - the leaf behind it, its WAL partition, its pin offset, the leaf's persisted checkpoint and its state - and whether that pin has wedged reclamation, as a pure read that never activates a leaf. Served by the separate `ILatticeWalReclamation` interface. See [WAL reclamation](#wal-reclamation). Requires whole-tree read authority. | | Plan WAL move | Compute a read-only preview of moving one WAL partition to a target provider key, carrying the entries-to-copy estimate, target resolvability, and whether the partition is already at the target, with no side effects. Requires whole-tree read authority. | | Execute WAL move | Execute an online move of one WAL partition to a new provider key, copying the partition tail under a quiesce lease and (by default) verifying the copy before flipping the placement, returning the move receipt. The source tail is retained, so the move stays revertible until reclaimed. A partition already pinned to the target is an idempotent no-copy repair (outcome `AlreadyAtTarget`). Reserved system tree ids are rejected. Requires whole-tree tree-lifecycle authority. | | Reclaim moved WAL source | Reclaim the orphaned source tail left behind by a completed WAL move of a partition, returning the reclaim receipt. Irreversible: once reclaimed the move can no longer be reverted. Refused when the given source key is the partition's live placement, and for reserved system tree ids; re-running a reclaim against a source that holds nothing is an idempotent no-op (outcome `NoOp`). Requires whole-tree tree-lifecycle authority. | | Audit orphaned leaves | Audit a tree for orphaned leaves - leaves spliced into a shard's sibling chain but unreachable by descent from that shard's root, left behind by a split interrupted between linking the new sibling and publishing the parent separator - and report what the repair would do about each, as a pure read with no side effects. It reaches its verdict with the same code as the repair, so its report is what the repair would do. It also reports every region it could not establish a verdict over; reporting no orphans is a verdict in its own right only when `VerdictComplete` is true. Requires whole-tree read authority. | | Survey orphaned leaves | The audit's opt-in, read-only full census: the same walk and verdict, but every key of each orphan is verified instead of stopping at the first failure, adding the nullable per-leaf `SurveyVerifiedKeyCount`, `SurveyMissingKeyCount`, and `SurveyRoutingContradictionKeyCount` (null means not surveyed, not zero; an orphan holding more than 100,000 keys is refused as `RefusedKeyCountExceeded` rather than surveyed). A pure read with no side effects. Requires whole-tree read authority. | | Repair orphaned leaves | Repair a tree by unsplicing every descent-unreachable leaf whose keys were all shown to be readable elsewhere, releasing the write-ahead-log materialiser pin that was holding the trim floor down. Fail-closed per leaf: a leaf is unspliced only when every key it holds was verified readable by descent, and any leaf that cannot be shown safe is left exactly as it was and reported as a refusal. An irreversible structural change - run the audit first. Reserved system tree ids are rejected. Requires whole-tree tree-lifecycle authority. | | List views | List the cluster's runtime-registered materialised views (view name, source tree, aggregation shape, accumulative flag, runtime provider key, projection version) as a pure read with no side effects. Requires cluster telemetry authority. | | Create view | Create (or rebind) a runtime materialised view from a host-registered projection provider key and an opaque provider payload of at most 64 KiB that only that provider interprets and the API never returns, returning the view's status. A view cannot take another view or a reserved system tree as its source. Requires whole-tree admin authority over the caller-supplied source tree, checked before any provider code runs, and - when rebinding an existing view to a different source - over the source it currently tails as well. Rejected when no view subsystem is registered. | | Get view status | Read a materialised view's status - source tree, aggregation shape, apply lag, active generation tree id, runtime provider key, and projection version - as a pure read with no side effects. The source tree is resolved authoritatively from the view registry, never supplied by the caller. Requires the source tree's whole-tree read authority. | | Rebuild view | Rebuild a materialised view from current source state via an online shadow-swap, returning the post-rebuild status. Requires the source tree's whole-tree admin authority. | | Reconcile view | Reconcile a materialised view against current source state, repairing drift only when detected, returning whether a repair was applied. Requires the source tree's whole-tree admin authority. | | Drop view | Drop a materialised view - decommission its maintainer and delete its backing generations. Rejected for a startup-declared view and for a view a Lattice add-on declares over a reserved `sys-` system tree (both would be re-created on the next silo start); idempotent for an absent view. Requires the source tree's whole-tree admin authority. | | List tag indexes | List the cluster's tag indexes (index name, backing membership tree id, shard count, covered source trees) as a pure read with no side effects. Requires cluster telemetry authority. | | Get tag index status | Read a tag index's status - backing tree id, shard count, covered source trees, and whether its background reconcile sweep is idle - as a pure read with no side effects. The backing membership tree id is derived authoritatively from the index name, never supplied by the caller. Requires the backing tree's whole-tree read authority. | | Reconcile tag index | Reconcile a tag index against current source state, removing orphaned membership rows, returning the reconcile counts (trees covered, keys scanned, membership rows scanned, orphan rows removed) and no key or value content. Writes only to the backing membership tree; covered source trees are scanned read-only. Requires the backing tree's whole-tree admin authority. | | Trigger shard compaction | Trigger an immediate tombstone-compaction pass for one shard, an operator override that bypasses the shard's ordinary compaction cooldown, returning whether the pass was accepted. Reaps only settled tombstones and expired entries - mutating but non-destructive to readable state. Reserved system tree ids are rejected. Requires whole-tree admin authority. | | Get history retention | Read a tree's effective history-retention policy (mode plus the optional bounded window) as a pure read with no side effects. Requires whole-tree read authority. | | Set history retention | Set a tree's history-retention policy (mode and/or window), read back as applied. A forward-absorbed configuration change - mutating but non-destructive to readable state. An undefined `TreeHistoryRetentionMode` is rejected with `ArgumentOutOfRangeException` before the tree is resolved or authorized (only a null mode clears the override), and a supplied window must be strictly positive; reserved system tree ids are rejected, and so are the first-party `sys-` system data trees, whose history retention the owning add-on manages. Requires whole-tree admin authority. | ## Facade method signatures The exact `ILatticeTreeAdmin` contract (published in `Orleans.Lattice.Api.Abstractions`). Every method corresponds to one gRPC RPC in the [binding](../lattice.api.treeadmin.grpc/README.md), except `SurveyOrphanedLeavesAsync`, which rides the `AuditOrphanedLeaves` RPC with the request's `Survey` flag set. | Method | Signature | |---|---| | `ProbeCapabilitiesAsync` | `Task ProbeCapabilitiesAsync(string treeId, CancellationToken cancellationToken = default)` | | `GetShardHotnessAsync` | `Task GetShardHotnessAsync(string treeId, CancellationToken cancellationToken = default)` | | `GetDiagnosticsAsync` | `Task GetDiagnosticsAsync(string treeId, bool deep = false, CancellationToken cancellationToken = default)` | | `InspectShardMapAsync` | `Task InspectShardMapAsync(string treeId, CancellationToken cancellationToken = default)` | | `GetProjectionDigestAsync` | `Task GetProjectionDigestAsync(string treeId, int shardIndex, CancellationToken cancellationToken = default)` | | `GetTreeStatsAsync` | `Task GetTreeStatsAsync(string treeId, CancellationToken cancellationToken = default)` | | `GetStorageUsageAsync` | `Task GetStorageUsageAsync(bool deep = false, CancellationToken cancellationToken = default)` | | `CreateTreeAsync` | `Task CreateTreeAsync(string treeId, int? shardCount = null, int? maxLeafKeys = null, int? maxInternalChildren = null, CancellationToken cancellationToken = default)` | | `CheckTreeExistsAsync` | `Task CheckTreeExistsAsync(string treeId, CancellationToken cancellationToken = default)` | | `SetTreeAliasAsync` | `Task SetTreeAliasAsync(string treeId, string physicalTreeId, CancellationToken cancellationToken = default)` | | `ResolveTreeAliasAsync` | `Task ResolveTreeAliasAsync(string treeId, CancellationToken cancellationToken = default)` | | `GetTreeConfigAsync` | `Task GetTreeConfigAsync(string treeId, CancellationToken cancellationToken = default)` | | `SetTreeConfigAsync` | `Task SetTreeConfigAsync(string treeId, TreeConfigurationUpdate update, CancellationToken cancellationToken = default)` | | `GetShardMapAsync` | `Task GetShardMapAsync(string treeId, CancellationToken cancellationToken = default)` | | `DeleteTreeAsync` | `Task DeleteTreeAsync(string treeId, CancellationToken cancellationToken = default)` | | `RecoverTreeAsync` | `Task RecoverTreeAsync(string treeId, CancellationToken cancellationToken = default)` | | `PurgeTreeAsync` | `Task PurgeTreeAsync(string treeId, bool confirm, CancellationToken cancellationToken = default)` | | `GetTreeDeletionStatusAsync` | `Task GetTreeDeletionStatusAsync(string treeId, CancellationToken cancellationToken = default)` | | `BeginBulkLoadAsync` | `Task BeginBulkLoadAsync(string treeId, string operationId, CancellationToken cancellationToken = default)` | | `AppendBulkLoadAsync` | `Task AppendBulkLoadAsync(string treeId, string operationId, long chunkIndex, IReadOnlyList entries, CancellationToken cancellationToken = default)` | | `CommitBulkLoadAsync` | `Task CommitBulkLoadAsync(string treeId, string operationId, CancellationToken cancellationToken = default)` | | `RestoreTreeAsync` | `Task RestoreTreeAsync(string treeId, string backupId, string? operationId = null, CancellationToken cancellationToken = default)` | | `RestoreTreeSetAsync` | `Task> RestoreTreeSetAsync(string setId, CancellationToken cancellationToken = default)` | | `RevertTreeRestoreAsync` | `Task RevertTreeRestoreAsync(TreeRestoreResult restore, CancellationToken cancellationToken = default)` | | `ReshardTreeAsync` | `Task ReshardTreeAsync(string treeId, int targetShardCount, CancellationToken cancellationToken = default)` | | `GetReshardStatusAsync` | `Task GetReshardStatusAsync(string treeId, CancellationToken cancellationToken = default)` | | `ResizeTreeAsync` | `Task ResizeTreeAsync(string treeId, int newMaxLeafKeys, int newMaxInternalChildren, CancellationToken cancellationToken = default)` | | `UndoTreeResizeAsync` | `Task UndoTreeResizeAsync(string treeId, CancellationToken cancellationToken = default)` | | `GetResizeStatusAsync` | `Task GetResizeStatusAsync(string treeId, CancellationToken cancellationToken = default)` | | `SnapshotTreeAsync` | `Task SnapshotTreeAsync(string treeId, string destinationTreeId, TreeSnapshotMode mode, int? maxLeafKeys = null, int? maxInternalChildren = null, CancellationToken cancellationToken = default)` | | `GetSnapshotStatusAsync` | `Task GetSnapshotStatusAsync(string treeId, CancellationToken cancellationToken = default)` | | `GetWalPlacementAsync` | `Task GetWalPlacementAsync(string treeId, CancellationToken cancellationToken = default)` | | `AuditWalPlacementAsync` | `Task AuditWalPlacementAsync(string treeId, CancellationToken cancellationToken = default)` | | `PlanWalMoveAsync` | `Task PlanWalMoveAsync(string treeId, int partition, string targetProviderKey, CancellationToken cancellationToken = default)` | | `ExecuteWalMoveAsync` | `Task ExecuteWalMoveAsync(string treeId, int partition, string targetProviderKey, TreeWalMoveOptions? options = null, CancellationToken cancellationToken = default)` | | `ReclaimMovedWalSourceAsync` | `Task ReclaimMovedWalSourceAsync(string treeId, int partition, string sourceProviderKey, CancellationToken cancellationToken = default)` | | `AuditOrphanedLeavesAsync` | `Task AuditOrphanedLeavesAsync(string treeId, string? resumeFrom = null, CancellationToken cancellationToken = default)` | | `SurveyOrphanedLeavesAsync` | `Task SurveyOrphanedLeavesAsync(string treeId, string? resumeFrom = null, CancellationToken cancellationToken = default)` | | `RepairOrphanedLeavesAsync` | `Task RepairOrphanedLeavesAsync(string treeId, string? resumeFrom = null, CancellationToken cancellationToken = default)` | | `ListViewsAsync` | `Task ListViewsAsync(CancellationToken cancellationToken = default)` | | `CreateViewAsync` | `Task CreateViewAsync(string viewName, string sourceTreeId, string providerKey, byte[] payload, CancellationToken cancellationToken = default)` | | `GetViewStatusAsync` | `Task GetViewStatusAsync(string viewName, CancellationToken cancellationToken = default)` | | `RebuildViewAsync` | `Task RebuildViewAsync(string viewName, CancellationToken cancellationToken = default)` | | `ReconcileViewAsync` | `Task ReconcileViewAsync(string viewName, CancellationToken cancellationToken = default)` | | `DropViewAsync` | `Task DropViewAsync(string viewName, CancellationToken cancellationToken = default)` | | `ListTagIndexesAsync` | `Task ListTagIndexesAsync(CancellationToken cancellationToken = default)` | | `GetTagIndexStatusAsync` | `Task GetTagIndexStatusAsync(string indexName, CancellationToken cancellationToken = default)` | | `ReconcileTagIndexAsync` | `Task ReconcileTagIndexAsync(string indexName, CancellationToken cancellationToken = default)` | | `TriggerShardCompactionAsync` | `Task TriggerShardCompactionAsync(string treeId, int shardIndex, CancellationToken cancellationToken = default)` | | `GetHistoryRetentionAsync` | `Task GetHistoryRetentionAsync(string treeId, CancellationToken cancellationToken = default)` | | `SetHistoryRetentionAsync` | `Task SetHistoryRetentionAsync(string treeId, TreeHistoryRetentionMode? mode, TimeSpan? window, CancellationToken cancellationToken = default)` | `DropViewAsync` and `RevertTreeRestoreAsync` return a bare `Task` (no payload); every other verb returns a result record. `SurveyOrphanedLeavesAsync` and `CreateViewAsync` are default interface methods that throw `NotSupportedException`, so an existing third-party implementation of `ILatticeTreeAdmin` stays source-compatible; the shipped facade implements both. ## WAL reclamation `ILatticeWalReclamation.GetWalReclamationAsync(string treeId, CancellationToken cancellationToken = default)` returns a `TreeWalReclamationReport` naming which pin holds a tree's WAL floor. It is a separate interface so that adding it changes no released facade; `AddLatticeTreeAdminApi` registers it on the same facade singleton, and the gRPC binding serves it as the `GetWalReclamation` RPC. Every leaf publishes a durable materialiser pin, and the WAL is trimmed only below the lowest usable pin offset. A tree that reclaims nothing because one pin can never move reads, on its trim and storage figures, exactly like a tree with nothing to reclaim. The report tells them apart: - `FloorHolder` is the pin with the lowest usable offset (`>= 0`), chosen exactly as the WAL GC pass chooses its offset floor. Only when no pin reports a usable offset is a pin at `-1` named instead (`HoldsOffsetFloor` is then false). It is null when the tree holds no pin. - `State` is read from the leaf's own persisted checkpoint without activating it, as the WAL GC's blocking-pin classifier reads it (`orleans.lattice.wal.gc.blocking_pin_state`). - `IsWedged` is true exactly when the holder carries a usable offset above a persisted checkpoint of `-1` (`NeverCheckpointed`). The durable pin store merges monotonic-max and cannot be lowered, and the GC refuses to drive a leaf with no proven checkpoint, so this does not clear on its own ([#4191](https://github.com/NSTA1/Orleans.Lattice/issues/4191), [#3258](https://github.com/NSTA1/Orleans.Lattice/issues/3258)). - The same `NeverCheckpointed` state at offset `-1` is the benign sentinel that clears once the leaf checkpoints. The metric `orleans.lattice.wal.gc.never_checkpointed_pin_offset` splits the two as `offset_usable` and `offset_absent` ([#4198](https://github.com/NSTA1/Orleans.Lattice/issues/4198)); this read is the per-tree, per-leaf view of the same discriminator, for a caller that does not scrape metrics. The wedge is keyed on the holder, never on growth: a wedged tree need not be growing. When `PinStoreReadable` is false the pin store did not answer and nothing else in the report is a measurement. ## Operation progress View rebuild and reconcile, tag-index reconcile, WAL moves and whole-tree orphaned-leaf passes are accept-then-poll through `ILatticeTreeAdminOperations`, the facade's adoption of the shared [long-running operation contract](../lattice.api.abstractions/operations.md): a start verb returns a `LatticeOperationHandle` at once and `GetOperationStatusAsync` reports the phase and the units completed (keys projected, trees probed and repaired, WAL entries copied, shards walked). The blocking `RebuildViewAsync`, `ReconcileViewAsync`, `ReconcileTagIndexAsync` and `ExecuteWalMoveAsync` are deprecated (`LATTICE0002`) and now wrap an operation. See [Tree-administration operations](operations.md) for the kinds, phases, units, result keys and migration. Resize, snapshot and reshard are accept-then-poll: the trigger returns once the coordinator accepts the intent, and the operation runs on its own, reminder-anchored. A caller follows it by polling the status read. Each status carries a coarse, durable progress measure that never runs ahead of what a resumed operation would start from: | Status | Members | Read as | | --- | --- | --- | | `TreeResizeStatus` | `Phase` (`TreeResizePhase?`), `CompletedUnits`, `TotalUnits` (`int?`) | `CompletedUnits` of `TotalUnits`: one unit per shard the copy drains, then one for each of the three steps after it. Read `UndoRequested` before `InProgress`: while an undo unwinds, `Phase` is `Undo` and `TotalUnits` is null. | | `TreeSnapshotStatus` | `Phase` (`TreeSnapshotPhase?`), `CopiedShardCount`, `ShardCount` (`int?`) | `CopiedShardCount` of `ShardCount` source shards. | | `TreeReshardStatus` | `TargetShardCount`, `StartPhysicalShardCount` (both `int?`) | `CurrentPhysicalShardCount - StartPhysicalShardCount` of `TargetShardCount - StartPhysicalShardCount` (both negative for a shrink): each split of a grow adds one physical shard, and each fold of a shrink removes one, once its routing swap has durably committed. | A fresh, deep storage-usage measure is accept-then-poll too, on the shared long-running operation contract rather than a bespoke status: `ILatticeStorageUsageOperations.StartStorageUsageRefreshAsync` returns a handle at once, and the operation reports `trees` measured of the registered tree count, then records the cluster totals. Prefer it to `GetStorageUsageAsync(deep: true)`, which holds one request open across the whole leaf walk. See [Storage usage operations](operations.md). Each progress member is null (or 0 for a count of completed work) when nothing is in flight, and null when the status comes from a build that does not report it, such as a reshard started before `StartPhysicalShardCount` was recorded. A caller then shows the phase, or the in-flight flag, without a percentage. The new members are appended `[Id]` members, so an older client simply ignores them. Tree purge follows the same pattern: `TreeDeletionStatus` reports `PurgedShardCount` of `PurgeShardCount` while `PurgeInProgress`. ## Public model types Alongside the request/response records the operations use, the facade publishes these public types: - `LatticeApiTreeAdminOptions` - the registration options type. It is currently empty (reserved for future bounding and audit-tuning knobs). - `TreeNotEmptyException` - thrown when a bulk-load session is opened against a tree that is not empty (bulk-load requires an empty tree). Carries the offending `TreeId`. - `BulkLoadOrderException` - thrown when a bulk-load chunk's keys are not in strictly ascending order; no entry is applied. Carries `TreeId`, `ChunkIndex`, `OffendingKey`, and `PrecedingKey`. - `TreeSnapshotMode` - whether a snapshot capture quiesces its source: `Offline` (every source shard is taken out of service before any is copied and returned to service once it has been copied, so a shard's reads and writes fail until its copy is done, for a point-in-time copy) or `Online` (the source keeps serving while its last-writer-wins mutations are shadow-forwarded to the destination and converge under last-writer-wins; typed CRDT delta applies and bulk appends are not forwarded). `SnapshotTreeAsync` rejects an undefined value with `ArgumentOutOfRangeException`. - `TreeWalMoveOptions` - the optional tunables for `ExecuteWalMoveAsync`: `QuiesceLeaseSeconds`, `CopyPageSize`, and `DisableVerifyAfterCopy`. Every field is zero-defaulted, so an omitted value takes the core's conventional default (a 30-second quiesce lease, 256-entry copy pages, and verify-after-copy enabled). - `TreeRestoreMode` - how a restore applied a backup: `InPlace` (direct replay, not undoable) or `ShadowCutover` (installed into a fresh shadow tree with an atomic alias cut-over, reversible - the mode `RestoreTreeAsync` always uses). - `TreeHistoryRetentionMode` - how much of each revision's value a tree's durable history retains: `MetadataOnly` (default; content hash and length only), `FullValue` (full bytes for every revision), or `Hybrid` (full bytes for a revision the history view applies within a short window of its write, set by `LatticeViewOptions.HistoryHybridFullValueWindow` rather than by the retention window, and metadata only for one applied later, such as from a backlog or a catch-up replay). A `Hybrid` row's shape is decided once, when it is written, and is never re-shaped as it ages, so the mode does not confine full values to a recent tail; see [Retention modes](../lattice/history-views.md#retention-modes), which also covers the write-ahead-log fallback read. `SetHistoryRetentionAsync` rejects an undefined value with `ArgumentOutOfRangeException`. - `TreeResizePhase` - the step a running resize has durably reached, reported by `TreeResizeStatus.Phase`, in order: `Copy` (copying the tree, shard by shard, into a destination at the new capacity while live writes are forwarded), `Swap` (pointing the tree's name at the destination), `RejectOldShards` (setting the old copy's shards to turn away requests that still reach them), `RetireOldCopy` (soft-deleting the old copy, so the resize can still be undone), and `Undo` (an accepted undo is unwinding). - `TreeSnapshotPhase` - the step a running snapshot has durably reached, reported by `TreeSnapshotStatus.Phase`: `LockSource` (an offline snapshot taking the source's shards out of service), `BeginForwarding` (an online snapshot starting to forward the source's live writes), `Copy` (copying the source's shards), and `UnlockSource` (an offline snapshot returning a copied shard to service). - `TreeWalMoveOutcome` - the classification of a WAL move or reclaim: `Moved`, `AlreadyAtTarget`, `SourceReclaimed`, or `NoOp`. - `TreeOrphanedLeafDisposition` - what the orphaned-leaf verbs did, or would do, about one descent-unreachable leaf: `Repaired`, `Repairable` (the audit's verdict that the repair would unsplice it), or one of the refusals `RefusedUnverifiedKeys`, `RefusedKeyCountExceeded`, `RefusedBlockingState`, `RefusedChainRace`, `RefusedRoutingContradiction`. - `TreeOrphanedLeafFinding` - one descent-unreachable leaf: its shard, leaf id, key range, `KeyCount`, `VerifiedKeyCount` (the verified prefix before the first failure, never a damage census), `Disposition`, first failure `UnverifiedKey`, and `IsRefusal`. `SurveyOrphanedLeavesAsync` opts into read-only full verification and adds nullable `SurveyVerifiedKeyCount`, `SurveyMissingKeyCount` and `SurveyRoutingContradictionKeyCount`; null means not surveyed, not zero. Default audit and repair still stop at the first failure. - `TreeOrphanedLeafReport` - a bounded batch: tree id, `DryRun`, `Survey`, walked leaves, positioned `Findings`, `Gaps`, `OrphanedLeafCount`, `RepairableCount`, `RepairedCount`, `RefusedCount` and `VerdictComplete`, plus `ResumeFrom` - the opaque token the next batch resumes from, null once every shard has been examined (`IsComplete`); it names a keyspace position rather than server-side state, so it never expires. The nullable `SurveyMissingKeyCount` totals missing keys in this batch only, or is null for a non-survey batch and whenever any reached region or orphan was not surveyed. Collect all batches with the same verb; a clean whole-tree verdict requires both `IsComplete` and `VerdictComplete`. Missing routed copies are not proof of data loss; repair rechecks each candidate independently. - `TreeOrphanedLeafGapReason` - why one region of the tree could not be given a verdict: `ShardSplitInProgress`, `ShardPassAlreadyRunning`, `ChainTruncated`, `ChainTruncatedUnrecoverable`, `WalkBudgetExhaustedWithoutResumePosition`, `LeafBoundsUndecidable`, or `EntryLeafUnreachable`. - `TreeOrphanedLeafGap` - one such region: its shard, the reason, and the leaf id and key the pass stopped on. - `ApiTreeAdminTypeAliases` - the stable Orleans serialization alias constants (prefix `oit.`) that the tree-administration records in `Orleans.Lattice.Api.Abstractions` carry. - `ILatticeStorageUsageOperations` - the accept-then-poll fresh storage usage, registered by `AddLatticeTreeAdminApi`: `StartStorageUsageRefreshAsync(string? operationId = null, CancellationToken cancellationToken = default)` plus the shared `ILatticeOperations` status, list and cancel verbs, scoped to callers holding cluster telemetry. See [Storage usage operations](operations.md). - `StorageUsageRefreshOperation` - the refresh's kind (`treeadmin.storage-usage-refresh`), phase (`Measuring`) and unit (`trees`) constants. - `ILatticeWalReclamation` - the read-only WAL reclamation diagnostics, registered by `AddLatticeTreeAdminApi`: `GetWalReclamationAsync(string treeId, CancellationToken cancellationToken = default)`. See [WAL reclamation](#wal-reclamation). - `TreeWalReclamationReport` - the tree, `PinStoreReadable`, `PinCount`, `PinsWithoutOffset`, the nullable `FloorHolder`, and the derived `IsWedged`. - `TreeWalFloorHolder` - the holding pin's `ConsumerId`, nullable `LeafId`, `Partition`, `PinOffset` (`-1` when it reports none), nullable `PersistedCheckpoint`, `State`, and the derived `HoldsOffsetFloor`. - `TreeWalFloorHolderState` - the holder leaf's durable state, mirroring the core WAL GC classification: `CheckpointedUncovered`, `NeverCheckpointed`, `NoDurableState`, `Unreadable`, `Orphaned`, or `CheckpointedCoverageUnknown`. - `StorageUsageRefreshResults` - the refresh's result keys, `ToResultMap`, and `TryReadSummary`, which rebuilds the cluster totals as a deep `ClusterStorageUsageSummary` with no per-tree rows. ## See also - [Tree-administration operations](operations.md) - accept-then-poll view, tag-index, WAL-move, orphaned-leaf and fresh storage-usage operations, and migrating from the deprecated blocking verbs. - [`Orleans.Lattice.Api.Schema`](../lattice.api.schema/README.md) - the schema control facade this surface composes by delegation. - [`Orleans.Lattice.Api.Abstractions`](../lattice.api.abstractions/README.md) - the shared control-surface contract package that publishes `ILatticeTreeAdmin`. - [`Orleans.Lattice.Api.Mcp`](../lattice.api.mcp/README.md) - the MCP server binding that advertises the tree-administration group.