AI Models Fail to Understand Persian Social Etiquette

A new study reveals that leading artificial intelligence models from companies like OpenAI, Anthropic, and Meta struggle to comprehend the complex Persian social etiquette known as taarof. The research highlights a significant cultural blind spot in AI, where models correctly navigated these nuanced interactions only 34% to 42% of the time, compared to 82% for native Persian speakers.

This performance gap, detailed in a paper titled "We Politely Insist: Your LLM Must Learn the Persian Art of Taarof," demonstrates how AI systems often default to direct, Western-style communication, potentially causing cultural misunderstandings in real-world applications.

Understanding Taarof: A Complex Social Dance

Taarof is a fundamental aspect of Persian culture, involving a system of ritual politeness where literal meaning and intended meaning often differ. It governs daily interactions, from paying for a taxi to accepting a compliment.

For example, a shopkeeper might initially refuse payment by saying, "It's a gift." A culturally unaware person might accept this at face value, which would be considered rude. The proper response is to insist on paying multiple times before the shopkeeper finally accepts the money. This exchange is a core part of taarof.

The study, led by researchers from Brock University and Emory University, shows that AI models consistently fail this test. They tend to accept offers too quickly, respond to compliments directly, and make requests without the expected level of polite hesitation, all of which violate the unspoken rules of taarof.

Putting AI to the Test with TAAROFBENCH

To quantify this issue, the researchers developed TAAROFBENCH, a specialized set of scenarios designed to measure an AI's ability to navigate these situations. The benchmark was used to test models including GPT-4o, Claude 3.5 Haiku, and Llama 3.

The results were consistent across the board, with accuracy rates lingering between 34% and 42%. This poor performance highlights a critical limitation, as the researchers note, "Cultural missteps in high-consequence settings can derail negotiations, damage relationships, and reinforce stereotypes."

The Impact of Language and Context

The study found that the language of the prompt significantly influenced the AI's performance. When prompted in Persian instead of English, the models showed marked improvement. For instance, DeepSeek V3's accuracy jumped from 36.6% to 68.6%.

This suggests that prompting in Persian activates different patterns in the model's training data that are more aligned with Persian cultural norms. However, even with this improvement, the models still fell short of human-level understanding.

"Taarof, a core element of Persian etiquette, is a system of ritual politeness where what is said often differs from what is meant," the researchers write. This divergence between literal and intended meaning is where current AI models falter.

Human vs. Machine Performance

To establish a baseline, the study also included 33 human participants. They were divided into three groups: native Persian speakers, heritage speakers (raised with exposure to the language), and non-Iranians.

• Native Speakers: Achieved 81.8% accuracy, setting the benchmark for cultural fluency.
• Heritage Speakers: Scored 60% accuracy, showing a strong but not complete grasp.
• Non-Iranians: Scored 42.3% accuracy, a figure remarkably close to the performance of the AI models.

The non-Iranian participants exhibited similar error patterns to the AI. They often interpreted polite insistence as aggression and avoided responses that might seem rude from a Western perspective, demonstrating a fundamental gap in cross-cultural decoding.

Uncovering Gender Bias in AI Responses

The research also uncovered evidence of gender bias in the AI models' outputs. All tested models provided more culturally appropriate responses when interacting with a female user compared to a male user. GPT-4o, for example, scored 43.6% for female users but only 30.9% for males.

The models often justified their responses with gender stereotypes, such as stating that "men should pay" or "women shouldn't be left alone," even when taarof principles apply equally to all genders. The researchers noted that models frequently assumed a male identity and adopted stereotypically masculine behaviors.

A Path Toward Culturally Aware AI

While the study identified a significant problem, it also explored potential solutions. The researchers found that AI models can be trained to better understand cultural nuances.

Using a technique called Direct Preference Optimization (DPO), they were able to double Llama 3's performance on taarof scenarios, raising its accuracy from 37.2% to 79.5%. Other methods, like supervised fine-tuning and providing a few examples within the prompt, also led to substantial gains.

This research marks an important step in moving beyond a one-size-fits-all approach to AI development. As these systems become more integrated into our daily lives, ensuring they can navigate the rich diversity of human communication will be essential for fostering understanding rather than creating new barriers.