Ai-OPs
ai-ops.com
Docs
/
Troubleshoot
/

A Model or Binding Isn't Running

A Model or Binding Isn't Running

A model that won't run almost always fails because of something feeding it, not the model itself. The fastest path to the root cause is to work down the chain: model, then bindings, then the tags each binding reads, then the devices those tags live on. This page follows that order.

Koios records three diagnostic fields (error code, error message, error detail) at both the model level and the binding level. For how those fields behave, how errors auto-clear, and the full status and quality legend, see Reading Status, Quality & Errors. This page focuses on what the model and binding error codes mean and how to act on them.

Start with the Bindings Tab

When a model fails, the model-level error is often generic — it tells you the model couldn't run but not which input caused it. The binding-level errors tell you the actual cause: a disabled tag, a missing range, stale history, an on-demand read timeout.

Where each error appears:

  • Model detail, Overview tab — the status hero shows the current state. A red strip below the status carries the model error message and detail whenever the model has one. The strip follows the error, not the status, so it also appears on a model that is not Failed — such as one held back by an invalid configuration.
  • Model list — each model shows a colored status icon. Hover it to read the error message. The icon is not always red: an enabled model held back by an invalid configuration shows the grey spinner instead, and the tooltip still carries the message.
  • Bindings tab — every input and output binding is a card with a colored left border. A failed binding turns red and expands to show its error message and detail inline. This is the diagnostic view that points at the specific binding that broke the model.

How the Inference Pipeline Runs

Reading the pipeline order helps you read error codes: the code tells you which stage failed. Each inference cycle runs these steps, stopping at the first failure:

  1. Validate bindings — each input tag is assigned, enabled, and running
  2. Query history — fetch historical data from the time-series database
  3. Preprocess — interpolate, check ranges, normalize, assemble the input tensor
  4. Run inference — execute the ONNX or TFLite model
  5. Write results — de-normalize predictions, check output ranges, write to output tags

For the exact tensor shapes, input depth, and normalization rules the preprocess and inference steps depend on, see Model Inference Requirements.

Model Error Codes

These apply to the model as a whole. When you see one, check the bindings for the more specific cause. An error code of 0 (None) means no active error.

File errors

CodeMeaningWhat to do
2No File Found — no file has been uploaded, or a file is present but would not loadThe error message tells you which: No model file loaded means upload an ONNX or TFLite file on the Files tab; Model file failed to load carries the underlying reason in the error detail. Re-upload the file.

Configuration errors

CodeMeaningWhat to do
6Bindings Invalid State — the model's configuration failed the load-time checkSee Model configuration is invalid below, then Model Configuration Errors for the reason-by-reason fixes.

Inference errors

CodeMeaningWhat to do
9Binding Prediction Index Missing — an output binding's Output Index doesn't match any output positionReconcile output indices against the model architecture.
11Failed to Write Predictions — results could not be written to the live data cacheRare; usually a cache connectivity issue.

System errors

CodeMeaningWhat to do
13Generic Exception — an unclassified error; the detail carries the exceptionCheck the model logs for the full stack trace. This is where inference-time failures land — an input tensor the model would not accept, a shape mismatch, or a scaling failure.
999Unlicensed — the license does not cover this modelSee Licensing Problems.

Model configuration is invalid (code 6)

Koios checks a model's configuration every time it loads it, and refuses to run one that would assemble the wrong input. When that check fails, the model reports:

  • Error code 6 — Bindings Invalid State
  • Error message Model configuration is invalid
  • Error detail — the exact reason, naming the binding or number at fault

Read the model's Error Detail first — it names the input, order, or number at fault, and on this one failure it is the only place the reason exists. Every reason, what it means, and how to correct it is on Model Configuration Errors.

Structured input models

A model can declare a structured input space, where each input binding carries its own window length, sample rate, and interpolation setting rather than sharing one model-wide window. Warm-up, retention, and the depth and staleness checks are then sized per input: one binding can be ready while another is still filling, and the model waits for all of them.

Three behaviors to know:

  • Memory Only must be off. A structured model with Memory Only enabled stops with Model configuration is invalid and the detail Structured input spaces do not support in-memory history yet — disable Memory Only for this model. Per-input in-memory buffers are not available in this release.
  • Only PCHIP interpolation is applied. If a binding requests any other interpolation method, Koios falls back to PCHIP and writes a warning to the model's log when the model loads. The selection is preserved but has no effect yet — check the model log if resampled values are not what you expect.
  • Binding orders must be unique and dense. Inputs are keyed by their binding order when the tensor is assembled, so every input needs a distinct order running 1..N. Koios assigns these from the model file; correcting a stored duplicate or gap is a binding CSV export and import. See Model Configuration Errors.

Binding Error Codes

These apply to individual input or output bindings and are the most actionable diagnostics. A binding can fail at any pipeline stage; the groups below follow the chain from tag state outward. Code 0 (None) means the binding is healthy.

Tag-state errors — fix at the tag or device, not the model

CodeMeaningWhat to do
11Binding Tag Disabled — the bound tag is stoppedEnable the tag or its parent device.
13Binding Tag Failed — the bound tag is in a failed state, including a failed or bad-quality readFix the tag's own error. See Bad, Missing, or Frozen Tag Values.
14Binding Tag Not Assigned — no tag selected for this bindingAssign a tag on the Bindings tab.
17Upstream Model Failure — this input is written by another model whose output is currently failingFix the upstream model first. This binding recovers automatically.

On-demand read timeout

CodeMeaningWhat to do
20On-Demand Read Failed — the device did not respond to the on-demand read within the timeoutConfirm the device is online and reachable, then raise the On-Demand Timeout on the model or scan group. The read timeout is frequently caused by batch-window latency (see below).

Historical-depth and staleness errors — the tag works, the history doesn't

CodeMeaningWhat to do
1Not Enough Historical Depth — not enough samples to fill the model's input windowCommon right after a model is enabled. Wait for enough scan cycles to accumulate the required depth; the detail shows how much more is needed.
2Stale History Data — the most recent point is too old to useThe tag's device has stopped collecting or data is arriving late. The detail shows how far outside the allowed window the data is.

If a binding shows Stale History Data but the tag itself looks live, the gap is in historization, not collection. See Data Is Stale, Frozen, or Has Gaps.

Range, normalization, and calibration errors

CodeMeaningWhat to do
3No Range Given — a normalization range is required but not configuredSet Range Min and Range Max on the tag, or set Parameter Source to Custom on the binding and fill in Custom Minimum and Custom Maximum. For Z-Score, set Custom Mean and Custom Std Dev.
4Invalid Range Given — the range is invalid (min ≥ max, or Z-Score std dev is zero)Correct the range values on the tag or binding.
5Value Out of Range — an input or output value exceeded the configured failure boundsThe detail shows the value and the bound it crossed. Review the failure range, or widen it if the value is expected.
21Invalid Calibration — the calibration gain is zero or produced an invalid valueA zero gain divides by zero on write-back. Set a non-zero gain on the Bindings tab.

Rate-of-change error

CodeMeaningWhat to do
19Rate of Change Exceeded — the value changed faster than the configured thresholdA safety check. The detail shows the actual rate, threshold, and direction. If the rate is expected, raise the ROC threshold under Failure in the binding drawer.

Data-structuring errors

CodeMeaningWhat to do
16Failed to Structure Input Data — the preprocessed data could not be assembled into a valid input tensorUsually a mismatch between the data shape and what the model expects. See Model Inference Requirements.

Output and write errors — after inference

CodeMeaningWhat to do
8Prediction Index Missing — the model output has no value at this binding's Output IndexVerify the output index matches the model architecture.
9Failed to Scale Prediction — the predicted value could not be de-normalized to the tag's scaleCheck the output binding's normalization range.
18General Model Failure — the model produced no prediction, so this output has no valueThe root cause is in the input bindings or the model-level error. Check those first.

Other

CodeMeaningWhat to do
999Unlicensed — the license does not cover this bindingSee Licensing Problems.

Common Scenarios

All inputs show "Tag Disabled"

Every input binding is code 11. The model depends on tags that have been stopped, individually or because their parent device was disabled. Re-enable the tags or the device.

All inputs show "Not Enough Historical Depth"

Normal right after enabling a model or restarting a device. Wait for the device to complete enough scan cycles to fill the window — model-wide (depth × sample rate) for flat and time-series models, or per input for a structured input model. No action needed.

One input shows "Upstream Model Failure"

That binding reads a tag written by another model's output, and the upstream model is failing. Fix the upstream model; this binding recovers on its own.

Running, but an output shows "Value Out of Range"

Inference produced a prediction that exceeded the output's failure bounds. Likely causes:

  • Input data has drifted outside the range the model was trained on
  • Failure bounds are set too tightly for the expected output
  • The normalization range doesn't match the training data's range

Review the output binding's failure range under Failure in the binding drawer.

"On-Demand Read Failed"

The model or its scan group is on-demand, but the device didn't answer in time. Confirm the device is powered and reachable (see Troubleshoot a Connection), then confirm the On-Demand Timeout covers the batch window plus device read time. See On-Demand Scanning.

"Input tensor shape mismatch"

The number of input bindings doesn't match what the model file expects, usually after a file with a different architecture was put in place. You don't add or remove bindings by hand — re-activate the model's intended file on the Files tab and activation rebuilds the binding set to match, keeping the tag assignments on the bindings it retains. See Managing Model Files and Model Inference Requirements.

A scan group fails and every model in it shows "On-Demand Read Failed"

When a scan group's shared on-demand read fails, all models in the group fail together. The group-level error (On-Demand Read Failed, scan-group code 2) appears on the scan group's detail page. Fix the shared device connection; every model in the group recovers on the next cycle. For scan-group health more broadly, see Service Health & Resource Alarms.

If a Model Stays Stuck

Errors clear automatically once the next inference cycle succeeds — no manual acknowledgment. If a model won't recover:

  1. Read the Bindings tab for per-binding errors; they are more actionable than the model-level error.
  2. Set the model's log level to Debug on the Logs tab and watch each pipeline stage.
  3. Confirm every input tag is enabled and running — one disabled or failed tag blocks the whole model.
  4. Check the tag devices — a down device fails all its tags, which cascades to every model using them.
  5. Toggle the model off and on with the Enabled switch.

If none of that clears it, gather logs before opening a ticket. See Collecting Diagnostics for Support.

What's Next