LMRuntime.com / Public page

Getting Started

Install LocalEndpoint, supply trusted GGUF identity, create an isolated session, generate locally, and dispose owned resources.

Fast path

Run Your First Model

Install UAIX.LmRuntime.LocalEndpoint, supply a trusted SHA-256 and byte count for a local GGUF artifact, load the verified model, create an isolated session, generate text locally, and dispose the session before the model.

Local executionNo provider APINo model downloaderNo telemetry

1. Create the console app and add LocalEndpoint

terminal

dotnet new console --name LocalModelSample
cd LocalModelSample
dotnet add package UAIX.LmRuntime.LocalEndpoint

Public examples omit a package version. Resolve and pin the package through the dependency policy used by your repository. The NuGet page owns current framework and dependency metadata.

2. Obtain the model SHA-256 and byte count

The runtime does not acquire or license models. The host selects the local GGUF artifact and supplies expected identity. Prefer a digest and byte count published by the artifact source or recorded during a trusted intake process.

PowerShell

$model = Get-Item .\models\model.gguf
(Get-FileHash $model.FullName -Algorithm SHA256).Hash.ToLowerInvariant()
$model.Length

bash

sha256sum ./models/model.gguf
wc -c < ./models/model.gguf

Identity boundaryHashing a file establishes its identity for later comparison. It does not establish model provenance, license rights, architecture support, tokenizer compatibility, or production suitability.

3. Load, create a session, generate, and dispose

This example uses the public LocalEndpoint signatures and a closed-authority session context. Replace the model path, expected identity, and prepared prompt with values owned by the host application.

Program.cs

using System.Globalization;
using UAIX.LmRuntime.LocalEndpoint;

if (args.Length != 4)
{
    Console.Error.WriteLine(
        "Usage: LocalModelSample <model.gguf> <sha256> <byte-count> <prepared-prompt>");
    return 2;
}

string modelPath = Path.GetFullPath(args[0]);
string expectedSha256 = args[1];
long expectedByteCount = long.Parse(args[2], CultureInfo.InvariantCulture);
string preparedPrompt = args[3];
string trustedModelRoot = Path.GetDirectoryName(modelPath)
    ?? throw new InvalidOperationException("The model path has no parent directory.");

var runtime = new LocalGgufRuntime();
var sessionContext = new LocalUaixRuntimeContext
{
    LoadedUaixProfilePresent = true,
    LoadedUaixProfileId = "profile1",
    LoadedUaixProfileDisplayName = "Local model profile",
    LoadedUaixLoadSessionId = "load1",
    LoadedUaixUaiRelativePath = "Memories/Profiles/profile1/.uai",
    LoadedUaixSessionRelativePath = "Memories/Sessions/load1.json",
    LongTermMemoryRootId = "wiki1",
    LongTermMemoryRootRelativePath = "Profiles/profile1",
    LongTermMemoryMode = LocalUaixLongTermMemoryMode.Isolated,
    RuntimeExecutionAllowed = false,
    MemoryCanOverridePolicy = false,
    CommandExecutionAllowed = false,
    NetworkAccessAllowed = false,
    ProviderApisAllowed = false,
    WebsitePromptIntakeAllowed = false,
    TelemetryEnabled = false,
    AutoExportAllowed = false
};

_ = LocalGgufRuntime.VerifyUaixRuntimeContext(sessionContext);

using LocalGgufModel model = runtime.LoadVerifiedModel(
    modelPath,
    new LocalGgufFileExpectation
    {
        ModelSha256 = expectedSha256,
        ModelByteCount = expectedByteCount
    },
    new LocalGgufModelLoadOptions
    {
        AllowedRootDirectory = trustedModelRoot,
        RejectReparsePoints = true,
        MaximumModelBytes = expectedByteCount,
        ExecutionLimits = new LocalGgufExecutionLimits
        {
            MaximumPromptCharacters = 32_768,
            MaximumGeneratedTokens = 256,
            MaximumStopTokenCount = 32
        }
    });

using LocalGgufSession session = model.CreateSession(
    new LocalGgufSessionContext
    {
        SessionId = "session1",
        UaixRuntimeContext = sessionContext
    });

LocalGgufGenerationResult result = session.GenerateGreedy(
    new LocalGgufGenerationRequest
    {
        Prompt = preparedPrompt,
        MaximumTokens = 128,
        ResetSession = true,
        AddSpecialTokens = false,
        ParseSpecialTokens = false,
        RemoveSpecialTokens = false,
        UnparseSpecialTokens = true,
        CleanSpaces = false
    },
    CancellationToken.None);

Console.WriteLine(result.GeneratedText);
return 0;

Prompt boundary: LocalEndpoint accepts a prepared prompt. The host remains responsible for choosing the model-specific chat template and special-token behavior.

4. Run the application

terminal

dotnet run -- \
  ./models/model.gguf \
  <expected-sha256> \
  <expected-byte-count> \
  "Write one sentence about local inference."

Successful loading proves that the current file matched the supplied digest and byte count and passed the configured intake path. Successful generation proves only that the named artifact and prepared prompt worked through the executed path.

5. Keep resource and session ownership explicit

Keep LocalGgufModel alive while any child session is active.
Create a separate LocalGgufSession for each independent conversation or worker.
Use ResetSession = true for a new sequence.
Use ResetSession = false only when the next prepared prompt deliberately continues the same session state.
Dispose the session before the model. The using declarations above enforce that order.
Pass a cancellation token from the host for long-running work.

6. Choose the package required for lower-level work

Required For	Package	Owned layer
Application integration	`LocalEndpoint`	Verified loading, sessions, and generation.
GGUF inspection and validation	`Gguf`	Metadata, tensor descriptors, hashing, sharding, mapped bytes.
Tokenizer and chat-template work	`Tokenization`	Encode/decode, special tokens, templates, token budgets.
LLaMA graph/session internals	`Models.Llama`	Configuration, binding, sessions, KV cache, forward execution.
Token selection and stop handling	`Sampling`	Logit transforms, deterministic state, selection, stops.

Compare the full package family, backend boundaries, and direct dependencies →

7. Handle failures without weakening verification

Path or identity failureReconcile the trusted root, canonical path, expected digest, and expected byte count. Do not retry with weaker checks.

GGUF validation failureTreat the artifact as malformed, truncated, unsupported, or outside configured limits.

Tokenizer or tensor-binding failureReview the exact model metadata and diagnostics. A .gguf extension does not establish compatibility.

Generation limitChange prompt or output ceilings only after evaluating memory and latency constraints.

CancellationSurface cancellation separately from verification, parsing, binding, and execution failures.