Two Feminist Books Beat Wikipedia as RAG Knowledge for Detecting Implicit Misogyny

RAGing Against Misogyny: Improving Implicit Misogyny Explanations

Ivana Crescenzi, Arianna Muti, Maarten Sap, Tommaso Caselli, Debora Nozza

cs.CL

2026-10-26

RAG over feminist literature, Wikipedia, and slang lifts implicit misogyny detection recall to 0.923 on Italian data, and RAG explanations win human preference 88-18 over the no-RAG baseline.

What problem this solves

Implicit misogyny is hate speech without the slurs: hostility carried by sarcasm, cultural references, and insinuation, deniable on its face. The paper opens with an Italian example. After a woman was murdered by her ex-boyfriend in 2023, someone proposed her sister Elena Cecchettin as 'woman of the year'; a comment under that post insisted dignity means silence. Without the murder case as background, an LLM reads it as a generic complaint about women being exploited for attention.

Prior work by the same group showed LLMs explain this category through shallow semantic associations, and moderation systems under-enforce it. The question here is direct: does retrieving background knowledge improve both detection and explanation?

Method

The framework is standard RAG; the work sits in the knowledge bases and the compression. Three sources, each targeting a type of implicit meaning:

The pipeline extracts entities with spaCy plus rules for recombining split person names, retrieves, then compresses twice. First compression is source-specific: Wikipedia sentences score +3 for a target entity, +2 for a domain keyword, +1 for sitting in the article's first three, capped at 8 sentences and 1,500 characters; slang compresses at the sense level; literature is left alone. A reranker keeps at most 3 candidates per entity, then a second pass drops sentences under 0.3 cosine similarity to the input. One multilingual sentence transformer handles all embedding steps.

Experiments run on Llama-3.1-8B-Instruct (temperature 0.3), on ImplicIT-Mis (1,120 Italian Facebook comments tied to news events) and SBIC+ (2,344 English items, less implicit, more slurs). Both datasets are all-positive, so detection is measured by recall alone.

Results

Context first. On ImplicIT-Mis, adding the news headline under which a comment was posted takes English-prompted recall from 0.771 to 0.891; Italian prompts move from 0.838 to 0.865.

RAG's effect on detection splits by dataset (best prompt setting per source):

SystemImplicIT-Mis (IT)SBIC+ (EN)
Baseline0.891 (+headline)0.966
Wikipedia0.8810.962
slang0.9160.806
feminist lit.0.9230.723

On the Italian data, slang and literature beat the baseline; on SBIC+, all three sources fall below a baseline already at 0.966. Retrieval coverage explains part of this: Wikipedia reaches 87.4% of ImplicIT-Mis but 3.4% of SBIC+; slang is the mirror image (9.7% vs 57.1%); literature stays balanced (29.6% vs 40.8%).

Explanation quality is where RAG pays off. Two annotators (the paper's authors) blind-scored 100 texts per language for faithfulness on a 0-2 scale:

SystemIT faithfulnessEN faithfulness
no RAG0.5901.345
Wikipedia0.6591.083
slang0.7651.304
feminist lit.0.9251.360

The Italian gap is significant (Wilcoxon p < 0.0001, +0.198, Cohen's d = 0.272, a small effect). In English, RAG aggregates to 1.310 against the baseline's 1.345, not significant. In head-to-head preference, RAG explanations win 88-18 in Italian (plus 33 ties, 61 where neither was picked) and 86-27 in English. Normalized per source, literature ranks first in both languages (0.408 and 0.538); Wikipedia lands last in English (0.361), below the baseline.

A failure-mode audit against a seven-class taxonomy from prior work: 21 issues resolved across 20 problem sentences in Italian, 9 of 12 in English, mostly wrong inference and sarcasm.

Why it matters

For trust-and-safety work, the finding is not the detection number but the evidence on source selection: two theory books outperformed Wikipedia. Wikipedia's coverage swings wildly across datasets; the literature base stayed useful in both languages, and for meanings that need a theoretical frame, such as denying patriarchy as a structural cause or treating female excellence as suspect, there is no encyclopedia entry to retrieve. If you build moderation for a vertical domain, curated book-length corpora deserve a slot next to Wikipedia.

One practical detail: English prompts consistently beat Italian prompts on the Italian dataset, with the largest gap on the literature source (0.923 vs 0.666). Prompt language is a free knob in multilingual systems.

The honest framing: detection results are mixed, and RAG hurts across the board on SBIC+. The contribution is explanation quality, and explanations are what moderators actually read.

Limitations

The authors list six. The heaviest: SBIC+ is not guaranteed fully implicit (it contains slurs and explicit jokes), so the English-side conclusions are discounted; human evaluation covers only 100 texts per language and should be read as diagnostic; and both annotators are the paper's own authors, blind to system identity but with modest initial preference agreement (kappa 0.286-0.333, rising to 0.702-0.925 only after adjudication).

Two more concerns from the reading. Language and dataset are confounded: the Italian corpus leans on background knowledge while the English baseline already sits at 0.966, so 'RAG works better in Italian' does not generalize, as the authors concede. And recall is the only detection metric, justified by the all-positive datasets but silent on false positives, which dominate real moderation costs. The paper's own closing examples show retrieval sometimes fires on lexical overlap alone; a preferred explanation is not necessarily a correct inference.

Terms

Source

What people are saying

Related papers

All paper explainers