Class InferenceEngineOptions
- Namespace
- InferenceEngine.Core
- Assembly
- InferenceEngine.Core.dll
Configuration options for inference engine initialization.
public class InferenceEngineOptions
- Inheritance
-
InferenceEngineOptions
- Inherited Members
Properties
DeviceId
Specific GPU Device ID (default 0). Useful for multi-GPU setups.
public int DeviceId { get; set; }
Property Value
MIGraphXModelCacheDirectory
Directory for the MIGraphX compiled-model (.mxr) cache (Linux x64 only). When enabled, the MIGraphX execution provider persists each compiled program to disk and reloads it on later startups, cutting engine load from a ~1-2 minute kernel compile to a ~1-2 second cache load. Cache files are keyed by MIGraphX version, graph identity, GPU architecture, and input shapes, so model swaps, ROCm/MIGraphX upgrades, or a different GPU naturally miss the cache and recompile.
null(default): use%LocalAppData%/InferenceEngine/MIGraphXModelCache.""(empty string): disable .mxr caching (every startup recompiles).- non-empty: use that directory verbatim (created if missing).
Note: quantization flags (fp16/int8) are not part of the cache key — delete the directory after changing quantization settings.
public string? MIGraphXModelCacheDirectory { get; set; }
Property Value
MaxModelDownloadBytes
Maximum size in bytes of a single model (or tokenizer asset) download. Downloads whose Content-Length or running byte count exceeds this are aborted and the partial file is deleted. Must be positive; the default is 2 GiB. This is a safety cap against compromised or misbehaving download sources, not a tuning knob.
public long MaxModelDownloadBytes { get; set; }
Property Value
ModelPath
Convenience property for local model paths. Setting this creates a LocalModelProvider internally.
public string? ModelPath { get; set; }
Property Value
ModelProvider
Optional model provider. If not set, the engine will use its default provider (as defined by CreateDefaultModelProvider() in the specific engine class).
public IModelProvider? ModelProvider { get; set; }
Property Value
OnSessionInitialized
Optional callback invoked once, on the LoadAsync thread pool thread, after the session is created (including fallback paths). Receives the fully-populated InferenceEngineInfo. Exceptions thrown by the callback are caught and logged as warnings; they do not abort the engine.
Invoked at most once per LoadAsync call. If LoadAsync is called again after the engine is already loaded, the callback does not fire.
Runs on a Task.Run thread pool thread, not the caller's synchronization context. Consumers that touch UI state must marshal to the UI thread themselves.
public Action<InferenceEngineInfo>? OnSessionInitialized { get; set; }
Property Value
OnStatusChanged
Optional callback invoked on every engine status transition (Loading → Loaded or
Failed), on the LoadAsync thread pool thread. Receives the full
InferenceEngineStatus object model — on failure its
Info carries the partial provider-selection trail and LoadError the
error. Exceptions thrown by the callback are caught and logged as warnings; they do
not abort the engine.
This is the recommended hook; OnSessionInitialized remains as the success-only equivalent.
public Action<InferenceEngineStatus>? OnStatusChanged { get; set; }
Property Value
UseGpuAcceleration
If true, attempts to use DirectML (Windows) or CoreML (macOS). If false, forces CPU execution.
public bool UseGpuAcceleration { get; set; }
Property Value
WarmupOnLoad
If true, runs a dummy inference during LoadAsync to force shader compilation/caching immediately.
public bool WarmupOnLoad { get; set; }