KoNA Benchmark Evaluates Selective Non-Compliance in Vision-Language Models
September 3, 2026
KoNA introduces a benchmark for testing the ability of VLMs to perform selective non-compliance at a component level rather than a query level. It evaluates models across five categories, including false premises and visual inaccessibility, to ensure they withhold compliance only on specific unsafe or unanswerable parts of a request.
HOW THIS AFFECTS YOU
●
builderYou can build more robust VLMs that remain helpful on valid parts of a complex query while ignoring invalid components.
●
researcherThis provides a more granular framework for evaluating how models handle mixed-intent multimodal inputs.