Hackers Poison ChatGPT and Gemini Answers to Push Fake Support Numbers

Security researchers say attackers are poisoning the answers of ChatGPT, Gemini and Google AI Overview with fake contact details, sending users to scam phone numbers and phishing pages. The findings were published by Ariel Simon on Medium, as the first part of a series on AI disinformation.

What the researchers found

Ariel Simon and four colleagues built a system that analyses the answers of all three AI systems and the sources behind them, flagging false phone numbers, URLs and emails cited in answers. Testing it on Fortune 100 organisations, banks and airlines, they detected campaigns against 374 companies — among them Delta, Lufthansa, Emirates, Bank of America, Chase, Airbnb and TripAdvisor. Tens of thousands of malicious pages were involved. Exploding Topics research cited in the post found 92% of users do not verify AI answers.

How the attacks work

The payloads use Generative Engine Optimization (GEO), shaping content so a language model cites it. Posts repeat the fake number and add phrases such as "call now" or "24/7". Digits are also written with spaces, dots, emoji or Unicode characters, which spam filters fail to recognise while the model reads them as the same value after tokenisation.

The same text is copied to social networks, to PDFs on university sites and to platforms such as Google Sites. The content targets moments of panic — refunds, cancelled flights, locked accounts — to push users to call instead of checking the official site.

Takedowns can't keep up

The campaigns are automated: hundreds of posts per day per company, thousands of pages daily. Removing one page takes hours or days while a thousand new ones appear, and copies kept on archive.org stay indexed, so the poison keeps working after a takedown. LeetCode posts targeting American Airlines filled more than 10 pages of Google results within 24 hours.

How companies and vendors responded

The researchers reported the attacks through the companies' bug bounty, fraud and security channels. Most were unaware of the scale; some dismissed the issue. Google classified it as outside its vulnerability reward programme. OpenAI closed the report as not reproducible, asking for demonstrated impact on users rather than a theoretical attack. The researchers say such campaigns cost companies revenue, brand trust and legal fees.