Table of Contents

Class InferenceWorkerHost

Namespace
InferenceEngine.Core.Worker
Assembly
InferenceEngine.Core.dll

The child-process message loop. The consumer's worker executable calls RunAsync<TInput, TOutput>(Func<InferenceEngineOptions, BaseInferenceEngine<TInput, TOutput>>, IWorkerSerializer<TInput, TOutput>?, Stream?, Stream?, CancellationToken, ILogger?) with a factory that builds ITS engine (a normal BaseInferenceEngine<TInput, TOutput> — exactly like the headless --diag inference path). The loop reads length-prefixed JSON requests on stdin and writes responses on stdout; logging passes through on stderr. Because this process loads no Avalonia/GL, comgr's LLVM is the only LLVM in it → no Mesa/comgr collision.

public static class InferenceWorkerHost
Inheritance
InferenceWorkerHost
Inherited Members

Methods

RunAsync<TInput, TOutput>(Func<InferenceEngineOptions, BaseInferenceEngine<TInput, TOutput>>, IWorkerSerializer<TInput, TOutput>?, Stream?, Stream?, CancellationToken, ILogger?)

public static Task<int> RunAsync<TInput, TOutput>(Func<InferenceEngineOptions, BaseInferenceEngine<TInput, TOutput>> engineFactory, IWorkerSerializer<TInput, TOutput>? serializer = null, Stream? input = null, Stream? output = null, CancellationToken cancellationToken = default, ILogger? logger = null)

Parameters

engineFactory Func<InferenceEngineOptions, BaseInferenceEngine<TInput, TOutput>>

Builds the consumer's engine from the host-supplied options.

serializer IWorkerSerializer<TInput, TOutput>

Optional serializer; defaults to SystemTextJsonWorkerSerializer<TInput, TOutput>.

input Stream

Optional input stream (stdin). Tests inject a pipe; production uses console stdin.

output Stream

Optional output stream (stdout). Tests inject a pipe; production uses console stdout.

cancellationToken CancellationToken

Observed between requests.

logger ILogger

Optional logger for the worker loop itself (frame tracing, load outcomes, timings). stdout stays protocol-only; logger output goes to whatever sink the consumer configured (typically stderr, which the host proxy forwards).

Returns

Task<int>

Type Parameters

TInput
TOutput