Safety & Society / arXiv 2026

A Model Can Know a Cultural Norm and Still Follow a False One

CultureLens pairs cultural scenarios with documented evidence and shows how a contradictory claim can override an answer a model otherwise gets right.

The Culture Within: Towards Mechanistic Understanding of Cultural Reasoning in LLMs

Samee Arif, Furkan Danisman, Angana Borah, Honglak Lee, Dehan Kong, Lajanugen Logeswaran, Moontae Lee, Zhijing Jin, Rada Mihalcea

Overview

An assistant can answer a question about local etiquette correctly until a prompt supplies a confident but misleading description of the custom. Did the model lack the knowledge, or did it choose to follow the new claim? CultureLens tests those possibilities by varying the evidence in the prompt and examining what happens when the model’s access to a stated norm is altered.

Key Findings

The country name gives the model a clue and a reason to trust a bad claim

A cultural question can ask a model to work from what it already knows, or it can supply a local norm. A true norm can help. A confidently stated false one can lead the model to give advice that conflicts with the documented account. The paper builds CultureLens to distinguish those cases. Its 9,261 records turn documented norms from the Cultural Atlas into situations, aligned and contrasting answers, and evidence that can be added or withheld. A subset includes underlying values. The experiments test both forced choices and answers in the models' own words.

Table 2 records the share of free-form answers judged aligned by two model judges. The values-based settings use a subset, so the table below focuses on the four conditions evaluated on all records.

Evidence supplied, Table 2Llama-3.1-8BGemma-3-12BQwen-3.5-27B
None43.7%40.4%51.9%
Country only58.2%63.2%74.5%
Country and documented norm92.6%93.1%95.5%
Country and contradictory norm17.6%16.2%42.9%

The wrong norm does more than cancel the benefit of naming the country. It pulls answers below the no-evidence result on all three models. The authors next look inside the model to find out whether the country information has disappeared or lost the competition for the answer.

The choice leaves a trace before the answer starts

The researchers compare the model's hidden representation at the point just before it generates an answer. Figure 1 relates the change caused by naming a country to the gain in aligned answers across countries. The country cue has less room to help where the unconditioned model already leans toward similar answers. The study relates these representation differences to country-level patterns and to distances measured with the World Values Survey. Appendix F.1 defines the relative shift for a record rr as the size of the difference between the answer-start states with and without the country name, divided by the size of the state without it.

σ(r)=∥hcountry-only(r)−hno-evidence(r)∥∥hno-evidence(r)∥.\sigma(r)=\frac{\left\|h_{\mathrm{country\text{-}only}}(r)-h_{\mathrm{no\text{-}evidence}}(r)\right\|} {\left\|h_{\mathrm{no\text{-}evidence}}(r)\right\|}.

A shift of 0.4 means that the change is 40% of the length of the starting state. The authors average these values within each country and compare countries within each model.

Figure 1 from the preprint. Across countries, a larger change in the internal answer-start representation tends to accompany a larger behavioral gain from giving the country name.
Figure 1 from the preprint. Across countries, a larger change in the internal answer-start representation tends to accompany a larger behavioral gain from giving the country name.

A separate probe uses the answer-start state to predict whether the model will resist a contradictory norm. It learns on held-out folds and compares the hidden state with a bag-of-words baseline. Its AUROC on the full prompt reaches 0.791, 0.841, and 0.917 for Llama, Gemma, and Qwen at the reported layers. The country context contributes chiefly at middle layers. A decodable signal establishes that information is present. The next experiment asks whether the model actually uses it.

Figure 2 from the preprint. Layer-by-layer probes predict whether a model will resist a contradictory norm. Comparing prompts with and without the country helps locate its contribution.
Figure 2 from the preprint. Layer-by-layer probes predict whether a model will resist a contradictory norm. Comparing prompts with and without the country helps locate its contribution.

Block the bad instruction and some of the old knowledge comes back

In Llama and Gemma, the researchers prevent later positions from attending to the tokens of the contradictory norm. They compare blocking selected middle layers, later layers, and all layers. Blocking the middle band recovers much of the all-layer effect. Blocking a similarly sized stretch later in the network has little effect. The paper also overwrites the internal representation of the norm, an intervention it can apply to Qwen. Doing so before the relevant band has more effect than doing so afterward.

Figure 3 from the preprint. The intervention changes how much the model can use the stated norm. Earlier and middle-layer interventions restore more aligned answers than later ones.
Figure 3 from the preprint. The intervention changes how much the model can use the stated norm. Earlier and middle-layer interventions restore more aligned answers than later ones.

The recovery is strongest on records the model already answered correctly from the country alone. That points to an arbitration problem. For those records, the model had enough information to answer before the false norm gained influence. Blocking the norm restores less when the country cue on its own never helped.

Authority changes which evidence wins

The authors keep the norm's wording fixed and change its framing. In Table 4, presenting it as a guide's advice reduces resistance on the tested subset. Presenting it as hearsay or following it with the user's doubt increases resistance. For Qwen the corresponding aligned rates are 5.0% for a guide and 91.0% when followed by doubt. Naming a country may also make a false claim feel more credible. That matters when an assistant receives cultural information through retrieved documents with an authoritative tone.

Learning facts and learning when to challenge a claim take different examples

The final experiment fine-tunes on country-only examples or adds counterfactual examples in which the model must correct a misleading norm before answering. With Gemma on the held-out countries in Table 5, ordinary supervised tuning raises the country-only score from 62.5% to 70.5% while reducing resistance from 15.7% to 4.1%. Adding the counterfactual training keeps country-only alignment at 70.4% and raises resistance to 87.8%. Its true-norm score falls from 99.8% under ordinary tuning to 86.9%. Selected held-out-country results from Table 5 show that trade-off.

Gemma-3-12B trainingCountry onlyTrue normContradictory norm
Untuned62.5%93.8%15.7%
Ordinary supervised tuning70.5%99.8%4.1%
Tuning with counterfactual examples70.4%86.9%87.8%

The researchers find that the counterfactual model's hidden state better distinguishes true and false norms, and its answers change less when the misleading norm is erased.

CultureLens uses one documented source and treats countries as its grouping unit. That makes the conflict controllable for the experiment. It leaves room for substantial variation within a country and disagreement over a norm in practice. The evidence is about whether a model follows or questions a supplied claim under those conditions.

Questions & Answers

The opening answers are simulated author responses based on the paper.

Q: Does the intervention read a model’s internal beliefs?

A: It changes the model’s access to a prompt span at chosen layers and measures the effect on its answer. Recovering an answer supports the idea that relevant information remained available. It does not establish a general account of what the model believes.

Q: Why might a sourced claim be especially persuasive?

A: An apparent attribution can change how the model treats a statement. The paper tests variations in how the same norm is presented. In use, a source description should be evaluated together with the source’s actual reliability and applicability.

Q: Should a model always prefer what it already knows?

A: New information can be correct, current, and more specific than what the model learned earlier. Our counterfactual training results show that stronger resistance can also reduce responsiveness to true evidence. A useful assistant needs to distinguish those situations.

Q: What can the intervention tell us about cultures or languages outside the study?

A: The interventions help explain some responses on selected English prompts and models under the paper’s cultural references and grading process. They do not establish that the same mechanism will dominate in another language or setting. A broader test needs communities with genuinely different practices, appropriate local sources, and room for more than one supported answer.

Future Ideas

A cultural advice check that keeps provenance visible

Build a travel-information prototype that can compare a retrieved etiquette claim with independent sources, note differences in audience and setting, and ask for clarification when the advice depends on the situation. The answer should preserve the source’s date and scope. Have bilingual reviewers flag cases where the same practice is described differently by different communities.

Test plain retrieval, an evidence-checking rule, and a model trained with contradictory examples. Include outdated guides, legitimate regional variations, correct new evidence, and genuine personal requests that should be respected. Measure acceptance of valid context, resistance to false claims, and how often the assistant appropriately states uncertainty.

Hold out at least one language and collect labels from members of the relevant communities. Track false resistance to verified updates separately from appropriate rejection of unsupported claims and allow an answer to explain multiple valid practices.

Sources

Original abstract

Large Language Models (LLMs) increasingly operate across culturally diverse settings, yet existing evaluations largely measure cultural behavior at the output level. We introduce CultureLens, a reference-grounded benchmark and mechanistic study of how LLMs use cultural information. CultureLens contains 9,261 scenarios across 75 countries and systematically varies country, value, true-norm, and contradictory-norm evidence. Across Llama-3.1-8B, Gemma-3-12B, and Qwen-3.5-27B, stating a contradictory norm lowers aligned answers by 32-47 percentage points, even when the model answers correctly from the country alone. Mechanistically, models default to Anglophone norms when no country is given. The failure under a contradictory norm is not one of missing knowledge, as blocking the norm inside the model restores the correct answer in most cases. Models accept a stated norm according to how it is presented and not whether it is true, which leaves them most exposed to errors in retrieved content. Supervised fine-tuning with contradictory-norm examples raises aligned answers under a contradictory norm to 85-93%, including on unseen countries, at a small cost under a true norm.