Simple Explanation (ELI5)

Imagine asking three friends what they think of a hotel review: they almost always agree. But if you ask three versions of ChatGPT, instead of clearly saying 'happy' or 'angry,' they tend to answer 'meh, neutral' to avoid making mistakes. It's as if AI is afraid to take a stance and ends up seeing everything in gray.

Key Advantages & Current Limits

🟢 Pros: High consistency among different LLM versions, useful for quickly and scalably generating synthetic data. 🔴 Cons: Strong bias toward the 'neutral' class, which distorts sentiment analysis and risks amplifying errors in future training cycles.

Future Outlook & Takeaways

In the coming years, LLMs will become increasingly present in the automatic analysis of customer feedback, but human intervention will be needed to correct their biases. Without oversight, the synthetic data generated could burden future models with distorted information.

Why Does AI See Everything as "Neutral"? The Mystery of Bias in...

Article

100%

Imagine asking three friends to read the same hotel review and tell you if the guest is happy, angry, or indifferent. You would probably agree almost every time. But what if you swap the friends for three versions of ChatGPT? Things change, and not just a little.

A recent study put GPT-3.5, GPT-4, and GPT-4-mini to the test using the HRAST dataset: over 23,000 real sentences written by actual travelers and 2,000 synthetic sentences created by AI itself. The goal? To see if machines can "read between the lines" like a human.

Result: the AIs agree strongly with each other, especially on artificial texts. But when we compare their labels with those of a human expert, agreement drops to "moderate" or "fair," particularly on sentiments and specific aspects (cleanliness, breakfast, wifi).

The main flaw? A strong "neutrality bias." Faced with ambiguous sentences, models often pick "neutral" to play it safe, like a student answering "I don't know" during an oral exam. Not only that: how you ask the question changes everything. Guiding the AI step-by-step (one-to-one prompting) works better than feeding it thousands of reviews at once (batch processing).

The authors named three AI "vices": repetition, rigid behavior, and excessive neutrality. A valuable lesson: if we use these models to analyze real feedback or generate synthetic data, we must know their blind spots. AI is not an impartial judge, but a mirror that slightly distorts reality.

Bonus

100%

The study, published in the journal *AI* (Vol. 6, p. 178), uses Inter-Annotator Agreement (IAA) as a proxy metric to quantify annotation bias. The HRAST dataset (Hotel Review Annotated Sentences) contains annotations across three dimensions: sentiment (positive/negative/neutral), topic (e.g., room, staff, location), and aspect-based sentiment analysis (ABSA).

Results show high Fleiss' Kappa and Krippendorff's Alpha coefficients among the three LLMs (often > 0.8 on synthetic data), indicating high internal consistency. However, agreement with the human annotator drops on sentiment (Kappa ~0.4-0.5) and ABSA, rating as "moderate" on the Landis & Koch scale.

Qualitative analysis reveals the "neutrality bias": models assign the "neutral" class significantly more often than humans, especially in sentences with mixed polarity or irony. "Repetition bias" emerges in the tendency to replicate lexical patterns seen during training or in the immediate context. "Behavioral bias" manifests as rigidity in adhering to annotation patterns learned via RLHF, ignoring contextual nuances.

Crucially, prompting mode heavily impacts results: manual, iterative, one-to-one prompting yields higher human-AI alignment than automated batch prompting (large-scale zero-shot/few-shot). This suggests that human-in-the-loop annotation interaction is not replaceable by mere computational scalability. Implications: synthetic datasets generated by LLMs inherit these systematic biases, risking amplification in subsequent training cycles (model collapse). For digital marketing and NLP research, caution is needed when using LLMs as ground-truth "oracles."