Table of Contents

Class InferenceEngineOptions

Namespace
InferenceEngine.Core
Assembly
InferenceEngine.Core.dll

Configuration options for inference engine initialization.

public class InferenceEngineOptions
Inheritance
InferenceEngineOptions
Inherited Members

Properties

DeviceId

Specific GPU Device ID (default 0). Useful for multi-GPU setups.

public int DeviceId { get; set; }

Property Value

int

MIGraphXModelCacheDirectory

Directory for the MIGraphX compiled-model (.mxr) cache (Linux x64 only). When enabled, the MIGraphX execution provider persists each compiled program to disk and reloads it on later startups, cutting engine load from a ~1-2 minute kernel compile to a ~1-2 second cache load. Cache files are keyed by MIGraphX version, graph identity, GPU architecture, and input shapes, so model swaps, ROCm/MIGraphX upgrades, or a different GPU naturally miss the cache and recompile.

  • null (default): use %LocalAppData%/InferenceEngine/MIGraphXModelCache.
  • "" (empty string): disable .mxr caching (every startup recompiles).
  • non-empty: use that directory verbatim (created if missing).

Note: quantization flags (fp16/int8) are not part of the cache key — delete the directory after changing quantization settings.

public string? MIGraphXModelCacheDirectory { get; set; }

Property Value

string

MaxModelDownloadBytes

Maximum size in bytes of a single model (or tokenizer asset) download. Downloads whose Content-Length or running byte count exceeds this are aborted and the partial file is deleted. Must be positive; the default is 2 GiB. This is a safety cap against compromised or misbehaving download sources, not a tuning knob.

public long MaxModelDownloadBytes { get; set; }

Property Value

long

ModelPath

Convenience property for local model paths. Setting this creates a LocalModelProvider internally.

public string? ModelPath { get; set; }

Property Value

string

ModelProvider

Optional model provider. If not set, the engine will use its default provider (as defined by CreateDefaultModelProvider() in the specific engine class).

public IModelProvider? ModelProvider { get; set; }

Property Value

IModelProvider

OnSessionInitialized

Optional callback invoked once, on the LoadAsync thread pool thread, after the session is created (including fallback paths). Receives the fully-populated InferenceEngineInfo. Exceptions thrown by the callback are caught and logged as warnings; they do not abort the engine.

Invoked at most once per LoadAsync call. If LoadAsync is called again after the engine is already loaded, the callback does not fire.

Runs on a Task.Run thread pool thread, not the caller's synchronization context. Consumers that touch UI state must marshal to the UI thread themselves.

public Action<InferenceEngineInfo>? OnSessionInitialized { get; set; }

Property Value

Action<InferenceEngineInfo>

OnStatusChanged

Optional callback invoked on every engine status transition (Loading → Loaded or Failed), on the LoadAsync thread pool thread. Receives the full InferenceEngineStatus object model — on failure its Info carries the partial provider-selection trail and LoadError the error. Exceptions thrown by the callback are caught and logged as warnings; they do not abort the engine.

This is the recommended hook; OnSessionInitialized remains as the success-only equivalent.

public Action<InferenceEngineStatus>? OnStatusChanged { get; set; }

Property Value

Action<InferenceEngineStatus>

UseGpuAcceleration

If true, attempts to use DirectML (Windows) or CoreML (macOS). If false, forces CPU execution.

public bool UseGpuAcceleration { get; set; }

Property Value

bool

WarmupOnLoad

If true, runs a dummy inference during LoadAsync to force shader compilation/caching immediately.

public bool WarmupOnLoad { get; set; }

Property Value

bool