How Can AI Spot Collapsed Buildings from Drone Photos?

After an earthquake, knowing instantly which houses have collapsed is crucial for saving lives and distributing aid. In Japan, traditional field surveys take days, but technology can now speed up the process. A research team combined post‑quake aerial images with a deep neural network, creating an Uncertainty‑Guided Fusion (UGFM) module. The UGFM, placed inside a standard decoder, uses uncertainty estimates to merge features extracted by a Pyramid Vision Transformer v2 (PVTv2) encoder. In practice, the model "knows" when it is most confident about a collapsed building and gives those areas more weight.

The model was trained with aerial photos from Noto Prefecture, where damage was extensive. Results show a recall of 79 % and a precision of 68 % on that set. When tested on images from Mashiki in Kumamoto Prefecture, recall dropped to 66 % but precision rose to 77 %. With on‑ground survey data, overall accuracy exceeded 87 %, proving the system can quickly pinpoint the most severely damaged structures.

This technology promises to speed up the issuance of Disaster Victim Certificates (DVC), cutting response times and improving emergency management. In the future, with more data, AI could become an indispensable ally for civil protection agencies worldwide.

The Uncertainty‑Guided Fusion (UGFM) module was designed to exploit uncertainty information generated at each network level. In practice, the decoder receives not only activation maps but also a variance map indicating how certain the model is about its prediction. This variance is normalized and used as a fusion weight: regions with low uncertainty (high confidence) exert a stronger influence on the final decision, while high‑uncertainty areas are attenuated.

The Pyramid Vision Transformer v2 (PVTv2) encoder is a variant of Vision Transformers that incorporates a hierarchical resolution pyramid, allowing it to capture both local details and global context. Thanks to its pyramidal structure, PVTv2 reduces computational cost compared to traditional transformers while maintaining rich spatial feature representations.

The training dataset was built from high‑resolution aerial images (30 cm/pixel) captured by drones and aircraft after the 2022 earthquake in Noto Prefecture. Annotations of collapsed buildings were obtained through a mix of field surveys and manual image analysis, ensuring high‑quality labels. To assess generalization, an out‑of‑domain test set was created with images from Mashiki, featuring different lighting conditions and vegetation cover.

Performance metrics include recall (sensitivity), precision (specificity), and F1‑score. On the internal set, the model achieved a recall of 79 % and precision of 68 %, with an F1 around 0.73. On the external set, recall dropped to 66 % while precision rose to 77 %, indicating good discriminative ability even in unseen scenarios. Combined analysis with field survey data showed overall accuracy exceeding 87 %, confirming the method's reliability for preliminary major‑damage identification.

Operationally, the model can be integrated into rapid‑response pipelines: aerial images are uploaded, the model outputs a collapse‑probability map, and authorities can prioritize interventions based on confidence thresholds. UGFM reduces false‑positive errors, limiting unnecessary actions, and enhances territorial coverage thanks to its ability to process large data volumes in real time. Future developments may include training with multispectral data (e.g., infrared) and employing semi‑supervised learning to leverage unlabeled images, further boosting system robustness.