CONNECT WITH US
Tech

Tech

The AI safety check that runs on a laptop and nearly matched a 35B model

TheNewStack – Tech logo

Published on

Add as a preferred source on Google
The AI safety check that runs on a laptop and nearly matched a 35B model

Guardrail selection has typically meant choosing between a purpose-built classifier and an LLM acting as a judge. Decision models such as TypeSafe AI’s Jev have added a third option, promising the flexibility of zero-shot policies without the cost of open-ended generation.

TypeSafe launched Jev in mid-September with performance claims based on evaluations it designed and ran itself. Red Hat’s AI Safety team has now put all three approaches through the same benchmarks. It tested nine guardrail configurations on prompt injection and content safety, running each through NVIDIA’s open source NeMo Guardrails toolkit. And the first result was actually pretty close.

Qwen3.6-35B, used as an LLM judge, topped that test with 89.31% accuracy. Red Hat’s DeBERTa-based prompt-injection classifier, which has roughly 200 million parameters, finished second at 89.01%. Qwen3.6-35B is a mixture-of-experts model with about 3 billion parameters active per token; however, the 175x figure overstates the difference in inference compute.

The latency results separated the two far more clearly. Red Hat’s main results table puts DeBERTa at a median of 54.1 milliseconds, compared with 312.5 milliseconds for Qwen and 348.1 milliseconds for Jev, which reached 86.35% accuracy. On prompt injection, the small classifier matched the largest model in the test almost point for point while returning its decision in a fraction of the time.

On prompt injection, the small classifier matched the largest model in the test almost point for point while returning its decision in a fraction of the time.


Source link
Related Topics:Tech

Disclaimer

We strive to uphold the highest ethical standards in all of our reporting and coverage. We TheMorningPulse.fyi want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It's possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.