ArXiv

Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologies

2026-07-16 · 15:49 UTC ·Filippos Vlahos, Guillaume Bied, Tijl De Bie ·1 min read
Authors
Filippos Vlahos, Guillaume Bied, Tijl De Bie
Categories
cs.CL, cs.CY
arXiv
https://arxiv.org/abs/2607.15146v1
PDF
https://arxiv.org/pdf/2607.15146v1

Brief

The paper evaluates political neutrality in Grokipedia versus Wikipedia by analysing 1,394 paired articles on government members across nine expert-coded ideology dimensions, using four LLM judges (Grok, Claude, Mistral, DeepSeek). All judges rate Grokipedia as less neutral; both sites portray politicians positively but with different slants—Grokipedia toward economically right-wing actors and against socially liberal ones, while Wikipedia leans favorable to socially liberal figures.

Why it matters

The authors compare Grokipedia (released late 2025) and Wikipedia using 1,394 article pairs about government members, scored along nine expert-coded ideological dimensions and judged by four LLMs: Grok, Claude, Mistral, and DeepSeek (paper posted 2026-07-16).

Key details

  • All four LLM judges rate Grokipedia as less neutral than Wikipedia; both encyclopedias portray politicians favourably overall, but Grokipedia favours economically right-wing politicians and penalises socially liberal ones, whereas Wikipedia is rated as favourably biased toward socially liberal politicians.
Source evidence

Abstract

Online encyclopedias shape political opinion and, through it, democratic discourse. In late 2025, Grokipedia was released, an encyclopedia written entirely by the LLM Grok. One motivation behind the project was to provide an unbiased alternative to Wikipedia, which has faced accusations of "left-wing" and "liberal" bias. But does an encyclopedia written by an LLM deliver greater neutrality, or does it simply embed a different ideology? We conduct a large-scale political bias study on Grokipedia and Wikipedia, analysing 1,394 article pairs describing members of government for neutrality along nine expert-coded ideology dimensions employing four LLM judges, Grok, Claude, Mistral, and DeepSeek. As the LLMs could themselves be biased, we also investigate patterns in their judgments. We find all LLM-judges, including Grok, to rate Grokipedia less neutral than Wikipedia. Both encyclopedias are rated as portraying politicians favourably overall, but towards different ideological groups. Grokipedia particularly favours economically right-wing politicians and penalises socially liberal ones, while Wikipedia is rated as favourably biased towards the latter.