Table of Contents

Class TextEmbeddingEngine

Namespace
InferenceEngine.Core.Text
Assembly
InferenceEngine.Core.dll

One-shot text embedding engine backed by the all-MiniLM-L6-v2 ONNX model. Owns BERT tokenization end-to-end: callers pass plain strings and receive an L2-normalized embedding vector (string in -> float[] out).

public class TextEmbeddingEngine : BaseInferenceEngine<string, float[]>, IInferenceEngine<string, float[]>, IDisposable
Inheritance
TextEmbeddingEngine
Implements
Inherited Members

Remarks

The embedding is produced by masked mean-pooling over the sequence dimension of the model's primary output (typically last_hidden_state / sentence_embedding), using the attention mask, followed by L2 normalization. The vector dimension is read from the model's output metadata and is never hard-coded.

Tokenization uses an uncased BERT WordPiece tokenizer (Microsoft.ML.Tokenizers), the vocab of which is downloaded and cached via LoadMiniLmTokenizerAsync(InferenceEngineOptions, ILogger, CancellationToken). If the tokenizer asset is unavailable the engine still constructs (the embedding call will then throw a clear error), so the instance is always safe to create.

Constructors

TextEmbeddingEngine(InferenceEngineOptions, ILogger<TextEmbeddingEngine>?)

Creates the engine. The ONNX model and tokenizer are resolved lazily during LoadAsync().

public TextEmbeddingEngine(InferenceEngineOptions options, ILogger<TextEmbeddingEngine>? logger = null)

Parameters

options InferenceEngineOptions

Engine options. GPU acceleration defaults on; pass UseGpuAcceleration = false for CPU.

logger ILogger<TextEmbeddingEngine>

Optional typed logger.

Properties

ModelDownloadUrls

Download URLs for the all-MiniLM-L6-v2 ONNX model.

protected override string[]? ModelDownloadUrls { get; }

Property Value

string[]

ModelName

The model file name used for cache keys and registry lookups.

protected override string ModelName { get; }

Property Value

string

Methods

GenerateEmbeddingAsync(string, CancellationToken)

Convenience helper that loads the engine and returns the embedding for a single string.

public Task<float[]> GenerateEmbeddingAsync(string text, CancellationToken ct = default)

Parameters

text string
ct CancellationToken

Returns

Task<float[]>

GetWarmupInput()

Builds correctly-typed integer dummy tensors (int64/int32 per declared element type) so the base warmup pass does not fail on default float-zero tensors.

protected override List<NamedOnnxValue> GetWarmupInput()

Returns

List<NamedOnnxValue>

LoadAsync()

Loads the MiniLM tokenizer (best-effort) and then initializes the ONNX session. A tokenizer load failure is logged as a warning and does not abort the session load; the first embedding call will surface a clear error if a tokenizer is genuinely required.

public override Task LoadAsync()

Returns

Task

PostProcessWithContext(IDisposableReadOnlyCollection<DisposableNamedOnnxValue>, object?)

Selects the embedding output (preferring sentence_embedding/pooler/pooled), masked-mean-pools a 3D [1,seq,dim] tensor (or takes the row of a 2D [1,dim] tensor), and L2-normalizes the result.

protected override float[] PostProcessWithContext(IDisposableReadOnlyCollection<DisposableNamedOnnxValue> output, object? context)

Parameters

output IDisposableReadOnlyCollection<DisposableNamedOnnxValue>

Raw ONNX outputs.

context object

Sequence length carried from pre-processing.

Returns

float[]

An L2-normalized embedding vector.

PreProcessWithContext(string)

Tokenizes the input, then assembles the model's declared integer inputs (input_ids, attention_mask, and token_type_ids/position_ids when the model declares them). Each tensor is typed to match its declared element type (long or int).

protected override (List<NamedOnnxValue> InputTensors, object? Context) PreProcessWithContext(string input)

Parameters

input string

Returns

(List<NamedOnnxValue> InputTensors, object Context)

The input tensors and the sequence length (carried as context for pooling).