Table of Contents

API

The public surface of Orleans.Lattice.Vector, in the order you meet it.

Namespaces: Orleans.Lattice.Vector (the in-memory core, no Orleans dependency) and Orleans.Lattice.Vector.Persistence (the durable layer).

The in-memory core

VectorIndex

The index itself. Constructed from VectorIndexOptions.

Shape and state

Member Purpose
Dimensions, Metric, Seed The immutable configuration the index was built with.
Count, Capacity Live vectors, and reserved slots.
PartitionCount, Probes The partitioning, and how many partitions a query probes.
Version, PartitionVersion(int) Mutation stamps. The per-partition stamp advances whenever a vector enters or leaves that partition, so a durable layer can persist only what moved.
PartitionSize(int) How many vectors a partition currently holds.
State, IsReady, Status Empty, Building or Ready, and a VectorIndexStatus snapshot.
CentroidsComplete Whether a restore has finished streaming its centroids.

Mutation

Member Purpose
Add(long, ReadOnlySpan<float>) Insert a vector under a key that is not already present.
Upsert(long, ReadOnlySpan<float>) Insert or replace.
Remove(long) Retire a vector. Constant time in the corpus size (one backfill, whatever the corpus holds); never a tombstone.
Contains(long), TryGetVector(long, Span<float>) Presence and retrieval.
Clear() Drop everything.
EnsureCapacity(int) Reserve ahead of a bulk load. On an untrained index the whole reservation goes to its single cell, which makes the insert run allocation-free; on a trained one it is spread evenly over the cells, a hint rather than a guarantee.
Train() Partition the corpus. Synchronous and expensive; keep it off the request path. Returns false, and leaves the index unpartitioned and answering exhaustively, when the corpus is too small to partition usefully (below MinimumTrainingCount, or resolving to fewer than two partitions).

Query

Member Purpose
Search(query, Span<VectorSearchResult>) Top-k into a caller-owned span.
Search(query, results, out VectorSearchMode) The same, plus how the answer was produced. Prefer this overload: it is the per-response honesty signal.
SelectPartitions(query, Span<int>) Which partitions this query would probe, without scoring anything. The durable layer uses it to fetch only the chunks a query needs.

Persistence seam

Member Purpose
CreateSnapshot(int maxItemsPerChunk) Plan a chunked snapshot. Nothing is copied yet.
Restore(VectorIndexHeader, VectorIndexOptions) Static. Create an index shaped by a persisted header.
ApplyChunk(ReadOnlySpan<byte>) Apply one chunk. Order-independent and idempotent.

Value types and enums

Type Purpose
VectorSearchResult(long Key, float Score) One hit.
VectorIndexStatus A snapshot of state, counts, partitioning and BytesPerVector.
VectorIndexSnapshot A snapshot plan: Header, ChunkCount, Describe(i), MeasureChunk(i), WriteChunk(i, Span<byte>).
VectorIndexHeader The 56-byte durable header (Size). Write, Read, and TryRead (which returns false rather than throwing on bytes this build cannot read: too short, the wrong marker, an unsupported version, an out-of-range field, or more centroid chunks than chunks in total).
VectorIndexChunkDescriptor Kind, partition, sequence, item count and byte count for one chunk, without rendering it.
VectorDistanceMetric Cosine or DotProduct.
VectorIndexState Empty, Building, Ready.
VectorSearchMode Exhaustive or Approximate.
VectorIndexChunkKind Centroids or Vectors.
VectorIndexFormat Format constants - HeaderMagic, ChunkMagic, Version (1), HeaderSize (56) and ChunkHeaderSize (24) - and IsSupported.
VectorIndexFormatException Thrown when a persisted form cannot be believed.
VectorSimilarity Dot, Norm, Cosine, Scale, Normalize, vectorised.
VectorIndexMemory Per-slot and total byte accounting: SideBytesPerSlot (12 - one cached float norm and one long key per slot) and Bytes(capacity, dimensions, partitionCount), which counts the vector blocks, their side arrays and the centroid block, and deliberately excludes the key-to-location map and the per-cell array headers. VectorIndexStatus.BytesPerVector is that total divided by the live count.

The durable layer

DurableVectorIndex

Orchestrates persistence and incremental maintenance over a VectorIndex.

There is no public constructor. DurableVectorIndex.OpenAsync(store, source, options, loadMode, cancellationToken) is the normal entry point; loadMode selects a full load or a lazy partial one. A caller that must keep a partially-loaded instance across a fault uses DurableVectorIndex.CreateUnloaded(store, source, options, loadMode) followed by LoadOrResumeAsync(...), which continues the interrupted load rather than restarting it.

Member Purpose
OpenAsync Static. Opens (and, for a full load, restores) an index over a store and a source.
CreateUnloaded Static. Creates an instance without reading durable state, so a caller can retry LoadOrResumeAsync on the same instance after a fault. Not usable until a load completes.
LoadOrResumeAsync Runs the durable load, resuming a previous attempt that faulted partway; a no-op once loaded. The two-token overload bounds only the resumable key-map walk with the first token and the whole load with the second. A full load throws VectorIndexRecordUnavailableException instead of discarding when a record the committed state names was missing from one read of the store but returned by another: the durable index is kept and the same instance is retried after a backoff.
LoadDiscardReason VectorIndexLoadDiscardReason: None unless a full load completed a discard, otherwise UnloadableRecord, CountMismatch, or EmbeddingSpaceChange. Retained on that instance; an explicit rebuild is not a load discard. Lazy readers refuse invalid state without deleting it and leave this value None. Generation differences that normal recovery can adopt are not discards.
LoadDiscardedManifest The committed VectorIndexManifest of the state a full load discarded - generation, vector count, and header (partition count) - or null when nothing was discarded or the discarded state had no decodable manifest. Use it to report what a discard cost, since a discarded index otherwise reads the same as a first-ever build.
LoadedKeyCount, IsLoaded, HasBankedLoadProgress Load observability: identifier mappings loaded so far, whether a load has completed, and whether an interrupted load banked progress.
KeyPrefix, Generation, LoadMode Where the index lives, which partitioning is live, and how it was opened.
Status, Count, UpdatesSinceTraining The core's status, the live vector count, and the drift signal that tells you when to retrain.
Progress A VectorIndexBuildProgress: phase, generation, vectors indexed and expected, partitions persisted and total, whether the state was restored rather than recomputed, the lifetime counts of ingest slices stopped by their wall-clock budget (SlicesDeadlined) and of those that banked nothing (SlicesDeadlinedWithoutProgress), plus EmptyDeadlinesSinceLastAdvance, IsStarvedBySource, IsReady, and IngestedFraction. IsStarvedBySource is the present-tense stall signal: true while at least one slice has been deadlined empty-handed since the build last banked anything, cleared the moment a slice banks an item. IsReady is true only when the build has finished and produced a partitioning. IngestedFraction reports 1 when the build has finished or when the expected count is unknown, deliberately, so a caller never renders a progress bar implying knowledge the index does not have. While the expected count is unknown (VectorsExpected is 0, which a failed source count also leaves), each ingest slice retries the count until it succeeds.
BuildStepAsync Does one bounded slice of build work and returns progress.
RunBuildAsync Loops BuildStepAsync to completion.
UpsertAsync, RemoveAsync Incremental maintenance.
TryGetId, TryGetKey Resolve between an external string identifier and the index's long key.
FlushAsync Persist the partitions whose stamps moved, rewriting only the chunks whose content changed. While a trained generation is part-way through committing, a flush completes that commit instead.
Search Synchronous and allocation-free, into a caller-owned span. Returns the number of hits written and reports the path through an out parameter. Under a lazy load it answers from whatever cells are already resident.
SearchAsync Query, returning a VectorSearchOutcome. Under a lazy load it fetches any cell the query would probe.
ReconcileAsync Bounded sweep against the store of record, always settling in the source's favour.
RebuildAsync, RetrainAsync RebuildAsync discards every durable trace of the index and resets the build to NotStarted, to be driven again from the store of record. RetrainAsync re-partitions the resident corpus after distribution drift - re-reading nothing - and commits the result as a fresh generation, deleting the one it supersedes; calling it again after a failed attempt resumes that commit instead of training again.

The build is a caller-driven pump, not a thread. BuildStepAsync does one bounded slice and returns; RunBuildAsync loops it. This honours the core's single-writer constraint without fencing, lets the host decide when it can afford the work, and makes resumability deterministic.

Two readiness signals, deliberately distinct

Do not conflate these:

  • Progress.Phase == Ready - the build has finished.
  • Status.State == Ready - a usable partitioning exists.

A corpus below the training minimum legitimately finishes its build with no partitioning and answers exactly by exhaustive scan. Reporting that as not-ready would be wrong; reporting it as approximate would also be wrong. Progress.IsReady is the conjunction of the two - the build has finished and a partitioning exists - so it stays false for that small corpus even though Progress.Phase is Ready.

Supporting types

Type Purpose
IVectorIndexStore The narrow async store seam: ReadAsync (one record, null when absent), ReadManyAsync (absent keys are simply missing from the result), WriteAsync, DeleteAsync (an absent key is a no-op), ScanAsync by prefix in ascending ordinal key order (an overload resumes strictly after a key already consumed, with a correct but unoptimised default implementation), and DeletePrefixAsync.
LatticeVectorIndexStore The ILattice adapter. The only type in the package that binds to Orleans. Its prefix scans push a resume point down into the tree scan, and resume a page walk abandoned by a bare Orleans response TimeoutException up to DefaultScanTimeoutResumeAttempts (2) consecutive times without banking a record.
IVectorSource, VectorSourceEntry The store-of-record seam the background build streams from.
VectorKeyDictionary The durable string-to-long identifier mapping. A monotonic allocator, never a hash. LoadAsync adopts the persisted mapping, replacing what is held in memory; a load that faulted partway continues from where it stopped on the next call (HasBankedLoadProgress reports that it will), while a fresh load also discards any buffered records. GetOrAddAsync writes a new identifier's mapping record at once; GetOrAddBufferedAsync assigns the key the same way but buffers the record until FlushPendingAsync writes every buffered record in one store write (PendingWriteCount reports how many are waiting). TryGetKey and TryGetId resolve either direction in memory without touching the store. RemoveAsync drops one identifier's mapping and its record, and ClearAsync forgets every mapping; both also discard any matching record still buffered, so a later FlushPendingAsync cannot write a dropped mapping back, and neither rewinds the counter (NextKey). Count and Ids report the current mapping.
IVectorIndexBuildObserver, VectorIndexBuildSliceTimings The build-timing seam, bound through DurableVectorIndexOptions.BuildObserver. OnSliceCompleted receives a VectorIndexBuildSliceTimings - SourceWait, KeyAssign, IndexUpsert, KeyFlush, and Consumed - so a host can publish them on its own meter; the package itself declares no instruments. It is called once for each ingest slice that returns normally, after that slice has checkpointed and written its build state, including a slice that consumed nothing; a slice that throws, including one whose checkpoint or build-state write fails, reports nothing. SourceWait, KeyAssign and IndexUpsert accumulate across the slice's items, and KeyFlush times its one batched key-map write. The four do not cover the whole slice: the source count taken while the expected count is still unknown, releasing the source enumerator, the ingest checkpoint that then persists the slice's vector chunks and build state, and the loop's own bookkeeping belong to no stage, so the four sum to less than the slice's elapsed time. An implementation must not throw, block, or retain the timings.
VectorIndexBuildPhase NotStarted, Ingesting, Training, Persisting, Ready, in that order during a build. It is not monotonic over the index's life: RebuildAsync (or a failed verification) returns it to NotStarted, and a writer that reopens an index whose build committed its trained generation but had not yet deleted the one it superseded resumes at Persisting, finishing that deletion on its next build step.
VectorIndexBuildState, VectorIndexManifest, VectorIndexPartitionState The durable build checkpoint, the commit record, and per-partition commit state. Each renders itself as a complete checksummed record with ToRecord and decodes one with TryReadRecord, which returns false rather than throwing on a record it cannot verify or act on; the manifest and partition state also report their fixed payload Size and Write that payload into a caller's span. VectorIndexPartitionState.TryReadRecord accepts only the compact form, in which every chunk lives under one epoch; a partition that a flush rewrote in part is stored in an extended form that only the durable index decodes.
VectorIndexStorageKeys, VectorIndexPersistenceFormat, VectorIndexRecord The key layout, the framing constants, and the checksummed record envelope. VectorIndexStorageKeys builds every key and prefix beneath the index's root prefix - the manifest, the build checkpoint, the identifier watermark and mapping, the retirement journal, and the per-generation centroid, partition-state and vector-chunk keys - padding generation and epoch components to CounterWidth (19), partition identifiers to PartitionWidth (5) and chunk sequences to SequenceWidth (8) so ordinal key order is numeric order; TryReadKeyMapId and TryReadRetirementKey parse a mapping or retirement key back. VectorIndexPersistenceFormat carries RecordMagic, Version (1, checked by IsSupported), RecordHeaderSize (24), ManifestPayloadSize and PartitionStatePayloadSize. VectorIndexRecord.Wrap frames a payload (Measure and Seal stamp the envelope over a payload already rendered into its final buffer), and TryUnwrap returns false for a record that is truncated, lacks the marker, declares an unsupported layout version or a length that does not match, or fails its checksum.
VectorIndexLoadMode Full load versus lazy partial load.
VectorIndexRecordUnavailableException Thrown by a full load when a record the committed state names was missing from one read of the store but returned by another. Nothing is discarded; retry LoadOrResumeAsync on the same instance after a backoff.
VectorSearchOutcome How many hits a search wrote into the caller's buffer, plus the mode that produced them.
DurableVectorIndexOptions Durable-layer configuration; see Configuration.

Identifier mapping

Index keys are long; most consumers have string identifiers. VectorKeyDictionary owns that mapping and is not a hash at any width, because a collision returns the wrong record silently and undiagnosably. It is a durable monotonic allocator: the watermark is made durable before any identifier in a block is handed out, so a crash burns the remainder of a block rather than reissuing. Keys are never recycled, and a rebuild does not rewind the counter.

The background build does not issue a store write per identifier. It assigns keys through the buffered path and writes each slice's mapping records in one batch before that slice's checkpoint, so the mapping may run ahead of the committed cells but never behind them: a committed cell whose key had no durable identifier would be unresolvable on the next load. A failed batch keeps its records for the next attempt, and because keys are never recycled a retry rewrites the same identifier to the same key. Only the per-identifier records are batched; the watermark is still written before any identifier in a new block is handed out. UpsertAsync of an identifier the index has not seen writes its mapping record immediately.

Only the forward direction is persisted; the reverse map is rebuilt in memory from the same scan, so resolving a result costs no round trip and no allocation.