Janus (formerly calibre): confidence-routed fallback study

Measures on Banking77 when Jev can answer alone versus escalate to DeepSeek V4-Pro by confidence threshold, with risk-coverage calibration analysis.

Notes

Routing below 0.67 confidence gave 80.2% accuracy at $0.1033 per 500 decisions versus 78.8% for DeepSeek alone; confidence ran ahead of measured accuracy at every level.

author demo — Result shown by its author. About this label

Destinations

Try a walkthrough

• Connect Jev to your app →

  Set up the Python SDK, make a typed routing decision and keep the result under your application's control.

• Route the next step of an agent →

  Let Jev choose a proposed next step, then apply an explicit allow, review or fallback policy before any tool runs.

Verification levels

Report a problem →

Filed under

• #calibration
• #routing
• #benchmark
• #github

In collections

• Benchmarks & independent tests →