OmniSafeBench-MM: the gym where AIs learn not to be fooled

Multimodal Large Language Models (MLLMs) have become very good at looking at images, reading text, and reasoning all at once. There is a catch: they remain vulnerable to "jailbreaks." These are tricks that convince the AI to ignore safety rules and say dangerous things. Until now, tests to find these flaws were scattered, limited, and hard to compare.

Enter OmniSafeBench-MM, an open-source platform that acts as a complete gym. Inside you find 13 attack methods, 15 defense strategies, and a dataset covering 9 macro-risks split into 50 precise categories. The test questions simulate real intentions: asking for advice, giving orders, stating facts.

The real novelty is how the result is measured. A simple yes or no is not enough. The evaluation has three dimensions: how severe the harm is (from mild to catastrophic), whether the response matches the query's intent, and how much detail it provides. This shows if a model is safe but useless, or useful but dangerous.

Researchers tested 18 models, 10 open and 8 closed. All show cracks under multimodal attacks. The code is on GitHub: anyone can replicate the tests and propose better defenses. It is a concrete step toward AIs that understand the world without being manipulated.

OmniSafeBench-MM fills a critical methodological gap: the lack of a unified framework for attack-defense evaluation in the multimodal domain. Previous benchmarks (JailBreakV-28K, MM-SafetyBench, HADES) suffered from asymmetric coverage of attack vectors (text-to-image or image-to-text only), absence of standardized defensive baselines, and binary evaluation metrics (safe/unsafe) insufficient for risk-utility analysis.

The toolbox architecture is modular: an Attack Engine implementing 13 techniques (e.g., gradient-based, role-play, typographic attacks, semantic manipulation) and a Defense Engine with 15 countermeasures (input sanitization, detection classifiers, refusal fine-tuning, constitutional AI). The OmniSafeBench-MM-Dataset structures 50 risk categories (violence, illegal, privacy, self-harm, etc.) via a 3-level taxonomy, mapped to 3 inquiry types: consultative (information seeking), imperative (instruction following), declarative (fact verification). This design allows testing alignment robustness against user intent.

The three-dimensional evaluation protocol is the major innovation: (1) Harmfulness Score on a 5-level ordinal scale (Negligible, Minor, Moderate, Severe, Catastrophic), based on guidelines derived from AI Safety taxonomies; (2) Intent Alignment Score (0-1) measuring semantic coherence between response and query via embedding similarity and LLM-as-judge; (3) Detail Level (Concise, Standard, Verbose) to penalize excessive refusals (over-refusal) or verbose non-informative outputs. Experiments on 18 MLLMs (LLaVA, Qwen-VL, GPT-4V, Gemini Pro, Claude 3, etc.) reveal that closed-source models resist unimodal attacks better but collapse under composite cross-modal attacks, highlighting the need for native multimodal alignment rather than post-hoc. Reproducibility is guaranteed by Docker containerization, structured logging, and a unified API for integration into safety testing CI/CD pipelines.