EMNLP 2026 (Main)Oral

MAPLE: Metadata Conditioned LLM Pretraining for Locale-Aware Question Answering

Anjishnu Mukherjee, Ziwei Zhu, Antonios Anastasopoulos

George Mason University

Can geographic metadata in the pretraining data help a language model give the answer that is correct for the user's country, instead of one global default?

Keeping the source URL, country and continent of each news article in the pretraining text teaches a language model to change its answer when the locale changes. With the same locale prompt at inference, the gain is modest but consistent, from 6.04% to 7.78% paired answer switching on LocalNewsQA at 1B parameters and from 21.36% to 24.13% at 3B.

Two-part diagram. Part 1 shows documents from four world regions tagged with URLs such as news.co.uk, news.co.in, newsdaily.ng and news.com/jp feeding one global model. Part 2 shows the question 'Who was the Prime Minister in 2015?' answered David Cameron under a UK URL and Narendra Modi under an India URL.
MAPLE pretrains one global model on news articles that keep their source URL, country and continent, so the same question can get different answers under different locales.

Key findings

  • Metadata pretraining helps when the answer depends on locale. With the locale metadata at inference held fixed, pretraining with metadata raises the paired margin-switch rate on the 1,700 ambiguous LocalNewsQA pairs from 6.04% to 7.78% at 1B and from 21.36% to 24.13% at 3B. Against the control with no metadata at training or inference, accuracy on ambiguous questions rises by 6.7 points at 1B and 9.2 points at 3B, and explicit questions that name the locale improve less. (Table 3, Section 5.1; Section 5.1, Appendix Table 16)
  • One global model serves four continents. At 1B, a single global model trained with metadata has lower perplexity than the metadata-free control on the Africa, America, Asia and Europe test sets and on the combined global test set. The metadata models also reach a given perplexity with fewer training tokens, at 1B and at 3B. (Figure 3, Figure 5a, Section 5.2)
  • The URL carries most of the signal. A URL-only variant nearly matches full metadata, and country or continent tags add little once the URL is present. Leaving a continent out of training raises perplexity on that continent with or without metadata, so metadata cannot replace regional data coverage. (Figure 4, Section 5.3)
  • The gains transfer to external benchmarks. At 3B, MAPLE shows statistically significant gains on GlobalOpinionQA (+8.4), NormAD (+5.1) and BLEnD (+13.5). The point estimates on GeoMLaMA, WorldValuesBench and Global-MMLU-CS are also positive. (Section 5.4, Figure 5b, Appendix Table 17)
Grouped bar chart of perplexity (lower is better) on Africa, America, Asia and Europe local test sets and on a global test set, for four models: global with metadata, global without metadata, local with metadata and local without metadata.
At 1B, the global model with metadata has lower perplexity than the global model without metadata on every continent test set and on the combined test set.
Three line charts of perplexity against training steps from 2k to 10k for five 1B variants: URL only, country only, continent only, full metadata and no metadata. Panels show the average of metadata-formatted test sets, the full-metadata test set and the no-metadata test set.
URL-only metadata closely tracks the full URL, country and continent format, while the country-only and continent-only tags trail the URL-only variant.
Scatter plot with normalized external score on the x axis and LocalNewsQA ambiguous accuracy gain in percentage points on the y axis. Green points mark MAPLE 1B and 3B base and chat models, grey points mark other models, and two frontier lines mark the up-to-1B and 3B-plus groups.
Among models of similar size, MAPLE models sit on the frontier of LocalNewsQA ambiguous-question gain against normalized score on six external benchmarks.

What we did

MAPLE is a family of decoder-only models that we trained from scratch on English news from the NOW corpus, with about 4M documents from Africa, 10M from America, 7.5M from Asia and 5.6M from Europe and a fixed budget of 41.9B training tokens. We trained 33 models at 500M and 1B parameters plus two 3B global models, and paired each metadata model with a metadata-free control on the same data, architecture and optimization. LocalNewsQA has 18,700 multiple-choice questions from 17 countries on 4 continents and 8 topic families. Of these, 17,000 are explicit items and 1,700 are ambiguous items, each paired with a contrast locale that has a different correct answer.

Talk

Video thumbnail for the MAPLE talkWatch the talk on YouTube

Abstract

Large language models can memorize competing locale-specific facts yet fail to select among them when the locale changes, defaulting instead to a single globally dominant answer. We formalize this as localized knowledge disambiguation and introduce LocalNewsQA, an 18,700-item English-news benchmark that pairs the same question across two locales and scores whether a model actually switches its answer when the locale changes. We also introduce MAPLE, a controlled family of decoder-only models pretrained with document-level geographic metadata (source URL, country, and continent) already present in the training corpus, and compare it to metadata-free controls trained on identical data with the same token budget, architecture, and optimization. In controlled experiments at 1B and 3B, with inference-time metadata fixed, pretraining with metadata in MAPLE produces measurable switching and improves accuracy on questions whose correct answer depends on locale. Ablations and external-benchmark evaluations further suggest that locale-conditioned prediction benefits from geographic provenance learned during pretraining and that these benefits strengthen at larger model sizes.

Citation

@inproceedings{mukherjee-etal-2026-maple,
    title = "{MAPLE}: Metadata Conditioned {LLM} Pretraining for Locale-Aware Question Answering",
    author = "Mukherjee, Anjishnu  and
      Zhu, Ziwei  and
      Anastasopoulos, Antonios",
    editor = "Lee, Hung-Yi  and
      Dev, Sunipa  and
      Li, Junyi Jessy  and
      Elliott, Desmond",
    booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing",
    month = oct,
    year = "2026",
    address = "Budapest, Hungary",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.emnlp-main.2053/",
    doi = "10.18653/2026.emnlp-main.2053",
    pages = "46746--46772",
    ISBN = "979-8-89176-459-0"
}