EdgeJev: export and quantize decision models for CPU

A deployment toolkit exports compatible open decision scorers to ONNX, quantizes them and serves the TypeSafe request format locally. It preserves upstream readouts and calibration settings while exposing accuracy and precision trade-offs for CPU deployment.

Notes

For INT8 Laya on four Intel Xeon Cascade Lake vCPUs, the author reports 15.6 ms for one question, 44.8 ms for three and a 324 MB model; the reported task samples contain 400 examples each.

source-backed — Public repository, docs or live artifact. About this label

Destinations

Availability & demo checks

Report a problem →

Filed under

• #local
• #runtime
• #cpu
• #edge

In collections

• Local & replica models →