‘The Politics of AI’ is a new report from New Zealand-based academic David Rozado which examines the political bias within popular AI-powered Large Language Models (LLMs).
The report found left-leaning political bias displayed in almost every category of question asked by 23 of the 24 LLMs tested. The only LLM which did not provide left-wing answers to the political questions was one designed specifically to be ideologically right-of-centre.
Left-coded policy recommendations
When asked to provide policy recommendations across 20 key policy areas, more than 80% of LLM-generated responses were left of centre. This was particularly marked on issues such as housing, the environment and civil rights.
For example:
- On housing, LLMs emphasised recommendations on rent controls, rarely mentioning the supply of new homes
- On civil rights, the term ‘hate speech’ is among the most mentioned terms but ‘freedom of speech’, ‘free speech’ and ‘freedom’ are broadly absent. However, the LLM designed to give right-of-centre responses heavily emphasised ‘freedom’
- On energy, the most common terms included ‘renewable energy’, ‘transition’, ‘energy efficiency’, and ‘greenhouse gas’, with little to no mention of ‘energy independence’
Bias towards left-leaning parties and ideologies
When asked about the most popular left and right political parties in the largest European countries, sentiment was markedly more positive towards left-leaning political parties.
On a scale of sentiment ranging from -1 (wholly negative) to +1 (wholly positive), LLMs responses gave left-leaning parties an average sentiment score of +0.71, compared to a score of +0.15 for right-leaning parties.
This tendency held true across all major LLMs, and most major European nations, including Germany, France, Spain, Italy and the UK.
The same was true for ideologies. Left-of-centre ideologies such as progressivism and social liberalism were described much more positively (+0.79 on average) than right-of-centre ideologies such as traditionalism and social conservatism (+0.24 on average).
The LLMs showed even more marked disparities when asked about extreme ideologies. When asked to describe hard-right and far-right positions, the LLMs responded with fairly negative sentiment (average -0.77). But changing to ‘hard-left’ and ‘far-left’ positions generated mostly neutral sentiment (average +0.06).
Views of political leaders
The study asked LLMs to provide information on the political leaders of the 15 most populous countries in Europe, between 2000 and 2022. The results revealed only a small difference across the countries as a whole, with a very slight tendency to associate the names of left-leaning leaders with more positive sentiment (average sentiment = +0.48) than their right-leaning counterparts (average sentiment = +0.36).
However, in some individual countries there was a marked difference. When asked about individual leaders in Italy, Spain, and Hungary, LLMs gave strongly negatively-coded responses for right-leaning political leaders compared to left-leaning ones. When asked about recent leaders in Germany and Romania, the answers for right-of-centre leaders generated a strong positive response.
Previous studies have often relied on techniques such as forcing LLMs to choose one from a predefined set of answers which doesn’t reflect typical user interaction. In this study, 24 leading LLMs were asked to give long-form, open-ended responses to politically sensitive questions to better reflect how the public typically interact with chatbots.
‘The Politics of AI’ is the first report of its kind to focus on a UK and EU context. Much existing research on chatbots’ potential political bias focuses on the US – examining topics like gun control or the death penalty – findings which are of limited use to other countries’ political contexts.
