ConvAI Innovations421M / 322M~1 GB VRAM at Q4_K_M

Laya — VRAM & /v1/systemone setup

Written by Jakub Rusinowski · Last updated

Small encoder-based decision models (421M and 322M, ModernBERT) for fast routing and guardrails; installed with pip, it routes non-English text to a multilingual variant. Weak at reasoning.

Laya needs about 1 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.

Call Laya

Served by laya-serve / Python Router. This model does not run in Ollama.

Question types
choice noul (yes / no) score
Inputs
text
Serve with
laya-serve / Python Router
Endpoint
/v1/systemone
Languages
100+ (multilingual checkpoint)

Setup for servers other than Ollama →

Weights on Hugging Face: convaiinnovations/laya ↗

Build a request for this model Decision models guide

Other decision models: Nimble 9B · Tev1 4B · Tev1 0.8B · Winnow 12B · Winnow E4B · Decider 2B · Decider 4B · Decider 35B-A3B (NVFP4) · Decider 12B Vision · Decider 31B v2 (NVFP4) · JevK5 4B · Intern-Decision 4B · AutoJev 27B · Clef 27B · Clef-Flash 9B · Rune 26B-A4B v3 · OpenDecider-nano · OpenDecider-small · OpenDecider-small MLX 4-bit · Bosun v3.1 0.6B · Bosun v3.1 1.7B · d1-3B · d1-omni-600M · THX-01 322M · mammoth-1 27B · elephant-1 9B · Strands Decider 2B (hobson-v21) · kev 4B · kev 9B · OpenJev 27B

Hardware fit

Weights plus overhead plus the KV cache at a 8,192-token prompt, on NVIDIA RTX 4090 (24 GB). A publisher build is sized from its file; the other rows are modelled at a standard quant. Decision requests are short, so no long-context figure is shown.

QuantVRAMFit
Q4_K_M
4.83 bpw · modelled quant
1.5 GBFits
Q6_K
6.56 bpw · modelled quant
1.6 GBFits
Q8_0
8.50 bpw · modelled quant
1.7 GBFits

pip install laya; Router auto-routes non-English text to laya-multilingual (100+ languages). Encoder model: fast routing/guardrails, weak at reasoning (Index 6.0).

How it was scored

Two different suites on two different scales. Never compare the numbers across the two cards.

Decision Index 0.2.1 (snapshot 2026-09-28)

Decision Index 0.2.1Community Decision Index (not by the model's authors)
Balanced skill6.04
ECE (lower is better)
0.1402
Median compute
5.8 ms on 1x NVIDIA RTX PRO 6000 (96 GB)

Community-maintained; not affiliated with the model authors.

Source ↗

Decision models are not ranked on chat, creative or coding scores.

Specifications

Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source. Still unconfirmed: weightsGbByQuant.

Parameters
421M / 322M
Context window
512 tokens
Architecture
Fine-tune of answerdotai/ModernBERT-large
Provider
ConvAI Innovations
Licence
Apache-2.0
Practical deployment
Edge device (phone, SBC)
Specified at
Q4_K_M
System RAM
8 GB
Record updated
2026-09-30
LicenceApache-2.0Commercial use permitted

Commercial use permitted. No usage restrictions beyond attribution.

Laya — frequently asked questions

What is Laya?

Laya is a decision model: you send it a state and typed questions (choice, yes/no/unknown, or a score) and it returns one answer per question with a probability for every option. It is not a chat model.

How do I run Laya locally?

Laya does not run in Ollama. Served by laya-serve / Python Router. This model does not run in Ollama. See the setup guide for servers other than Ollama.

How much memory does Laya need?

About 1 GB for the weights plus overhead at Q4_K_M, before the prompt's KV cache. Decision prompts are short, so the cache stays small.

How accurate is Laya?

Its published score is on the community Decision Index 0.2.1, which its authors did not build. It is not a calibration guarantee: test on your own examples.