MagiC evaluates vision-language models on grounding fidelity and reasoning validity using 5,500 weakly supervised and 900 human-curated examples. The benchmark tests 15 models from 7B to 70B parameters across answer correctness and self-correction abilities.
HOW THIS AFFECTS YOU
●
builderThis provides a framework to validate the reasoning steps of your multimodal products.
●
researcherYou can use this to test if your VLM is actually grounding in visual evidence or relying on dataset biases.