How to Recognize Emotions in Photos? Science Has a New Answer!

Have you ever thought that a photo can make you feel emotions? Science is working to understand how! Recently, researchers have developed a new method called Object-Scene Attention Fusion Network (OSAFN) that analyzes objects and scenes in images to predict the emotions they can evoke. This system uses two different strategies to extract semantic information: one for objects and one for scenes. Additionally, an attention module aligns this information with visual features to improve emotion representation. Results on four datasets show that OSAFN performs better than traditional methods, opening new paths for emotion analysis in images.

The Object-Scene Attention Fusion Network (OSAFN) is an advanced model that combines semantic information from objects and scenes to enhance emotion classification in images. It uses an external tool to extract object-related concepts and an Appraisal-based Chain-of-Thought (Appraisal-CoT) prompting to generate scene information. This semantic data is then aligned with visual features through attention modules. An additional innovation is the adaptive fusion strategy, which integrates the results of object and scene semantic streams. Furthermore, a polarity-aware contrastive loss helps model the hierarchical structure of emotions, improving the discrimination of fine-grained emotional categories. Tests on four datasets have shown that OSAFN outperforms traditional methods, representing a significant advancement in the field of Image Emotion Classification (IEC).