jev-on-a-laptop: parallel constrained decoding on MLX

Prefill once, broadcast the KV cache per field, evaluate all fields in one pass on Qwen 1.5B–8B models; a Hugging Face Space demo exists.

Notes

7.9x versus token-by-token decoding, 50–85% accuracy on 24 labelled cases depending on model size. Independent.

source-backed — Public repository, docs or live artifact. About this label

Destinations

Verification levels

Report a problem →

Filed under

• #replica
• #mlx
• #qwen
• #github

In collections

• Local & replica models →