An investigation by the BBC revealed a straightforward method by which AI chatbots are being made to disseminate false information to the public. Google and other AI companies are now trying to fix the problem.
In February, I committed an error. I heard there was an easy way to poison AI chatbots and make them spread lies on your behalf. After some digging, I learned unscrupulous companies are abusing the problem on a massive scale. So I decided to try it myself.
We uncovered examples where ChatGPT, Gemini and the AI Overviews at the top of Google Search were being manipulated to dole out biased answers on topics as serious as your health and personal finances. And in just 20 minutes, I tricked ChatGPT and Google into telling the public that I am a world-champion competitive hot-dog eater. The gag was stupid. The problem is serious.
However, the trick worked. There was a lot of criticism for both our investigation and the work of researchers who have been keeping an eye on this problem. In response to the issue, Google has revised its policies, and there are indications that other AI companies are following suit. Ultimately, it could make AI tools and the internet as a whole a little bit safer.
But until there are better systems in place, experts say you're in danger of getting fooled.
Thomas Germain/Google/BBC I tricked Google into telling the world that I was a champion hot-dog eater, but this trick is used to manipulate AI responses to more serious questions (Credit: Thomas Germain/Google/BBC) I made Google tell the world I'm a champion hot-dog-eater, but people use this trick to manipulate AI responses for more serious questions (Credit: Thomas Germain/ Google/ BBC)
Algorythmic, a search engine optimization (SEO) and artificial intelligence (AI) consulting firm founded by Lily Ray, advises, "You should assume that you are being manipulated until they have better systems in place." "We're moving towards this 'one true answer' world. Before, Google would give you 10 blue links and you would kind of do your own research. But AI just gives you one answer. It becomes so easy to just take things at face value. You must exercise caution." Google tells me that its policy update is just a "clarification" of the efforts it has been making for a while. "We've long applied our core anti-spam policies and protections to our generative AI Search features – and we've always continually upgraded our spam fighting efforts to stay ahead of emerging tactics, even before the rise of AI," a Google spokesperson says.
Essentially, Google says it hasn't changed a thing. But behind the scenes, it seems like Google and other companies are ramping up their efforts to address the problem. Despite this, there is evidence that people are still employing the same strategies to deceive the largest search engine in the world. The problem
Typically, when you ask a chatbot a question, the AI generates a response based on the data built into the model. But sometimes, tools like ChatGPT, Claude and Google's various AI products search the internet for an answer. And that's where this problem happens.
According to Ray and other search engine experts, AI tools often throw up information from a single web page or social media post. Because of this, these systems are open to false information. And it turns out manipulating what chatbots tell the public can be as simple as publishing one, well-crafted blogpost almost anywhere online. People figured this out and quickly identified a money-making opportunity.
I was able to demonstrate the problem by publishing a single article on my personal website about my hot-dog-eating prowess. The next day, AI from some the world's biggest companies were spreading my lies. But our investigation also found the same trick being used to dismiss health concerns about medical supplements or influence financial information provided by Google's AI about retirement. According to experts, this kind of manipulation is widespread and systemic. Keeping Tabs
Thomas Germain is a senior technology journalist at the BBC. He writes the column Keeping Tabs and co-hosts the podcast The Interface. His work uncovers the hidden systems that run your digital life, and how you can live better inside them.
Biased or inaccurate information like this can also lead you to make bad decisions. It can influence how you might vote or which plumber you hire.
According to Harpreet Chatha, who is in charge of the SEO consulting firm Harps Digital, "at the most fundamental level, the concern is the economic impact." "At a more serious level, you might take medical advice that makes you sicker than you were before. Legally, you might get bad information and do something that is not legal in your state or your country."
A solution?
This is not an insignificant problem. Globally, more than a billion people use AI chatbots regularly and 2.5 billion see Google's AI overviews each month. You'll have a lot of power if you can defeat that tool. But it seems Google and other companies are finally waking up to the problem.
Last week, Google updated its spam policies to officially confirm that attempts to manipulate AI responses are against the company's rules. It may sound like a small change, but it signals that Google is pro-actively looking for those who try to abuse the system and sending them a threat. If a company or website is caught breaking the rules, it could be removed from or downranked in Google's search results. And if you're not on Google, it's like you don't exist.
Google says that I'm getting this wrong and nothing has changed. "The edit to our spam policy language last week was a clarification, not any change in approach," says Google's spokesperson.
Indeed, Google detailed it's anti-spam AI efforts in 2025. But I did my hot dog experiment almost a year later, so clearly those efforts weren't working. And just this week, Ray pulled the same stunt and made Google tell people a fellow SEO specialist is good at building sandcastles.
Ray and Chatha also say that in the past few months, they've noticed some significant changes that show that Google and other businesses are trying out different solutions. For example, Ray says it looks like Google and ChatGPT might be quietly removing companies from its AI answers when it suspects they're promoting themselves. "So if you publish a list where you say you're the greatest hot-dog-eater, they're not going to include your name," says Ray. "They might still cite your article, but you're going to be removed from consideration."
I've personally noticed some examples where Google and other AI tools are adding more labels to their responses, letting you know that the chatbot isn't confident about its answers. Others have also pointed out that ChatGPT and Claude, an artificial intelligence developed by the company Anthropic, have begun explicitly stating in response to some queries that they are attempting to weed out spam. Ray says she's noticed Google adding more caveats, recommending that you go look at third-party reviews when you ask questions related to some purchasing decisions.
None of these companies would acknowledge these changes when I asked them. Anthropic and OpenAI declined to comment. Google's spokesperson didn't respond to my questions on this.
More like this:
• Is Google on the verge of destroying the Web? • The ghosts of India's TikTok
• Online sellers are selling your address; this privacy tool will assist Regardless, Chatha is sceptical changes like these will be enough. He asserts, "Google is playing whack-a-mole." "They're announcing [the policy update] to deter people, but the tactics will just move."
He's already seeing it happen. As Google cracks down on manipulative blog posts, companies are finding subtler ways of promoting themselves. "You can give a company a penalty for their website," he says, "but there's nothing stopping them from paying 20 YouTube influencers to say their product is the best." And now, Google's AI is citing YouTube videos. The cycle continues.
For the time being, the manipulators are likely to stay one step ahead. Ray says the best defence is to remember what AI actually is: a tool that confidently gives you one answer, whether it's right or wrong. Just because it looks like a giant tech company is speaking to you instead of some random website doesn't mean you should have faith.
No comments:
Post a Comment