Scammers Found a New Trick: Registering the Web Addresses AI Makes Up
AI chatbots sometimes invent web addresses that don't exist — and researchers discovered they invent the same ones over and over. Now attackers are buying those addresses and building fake sites there, waiting for AI users to arrive. Here's how to not be one of them.
You’ve probably heard that AI chatbots sometimes “hallucinate” — confidently state things that aren’t true. Usually that’s an annoyance. But security researchers at Palo Alto Networks have documented a case where it becomes a real trap — thanks to some creative thinking by scammers.
Here’s the setup. When you ask a chatbot for a link — “where do I log into my bank’s rewards program?” — it occasionally invents a web address that looks completely plausible but doesn’t exist. The researchers’ key discovery: models don’t invent randomly. Ask the same kind of question often enough and the same made-up addresses keep appearing. And an invented address that thousands of people will be confidently sent to is worth money. Scammers can simply register it — usually for a few dollars — and build a fake login page there. The AI then does the advertising for them, one helpful answer at a time. The researchers call it “phantom squatting.”
The numbers give a sense of scale. The team ran about 685,000 link requests covering 913 well-known brands and collected 2.1 million AI-generated web addresses. Around 13,000 already led to confirmed malicious sites. More unsettling: roughly 250,000 consistently-invented addresses were still unregistered — free real estate for future scams. In one documented case, researchers noticed models kept inventing a specific address resembling a national postal service’s shop. Twenty-three days later, someone registered exactly that address and installed a phishing kit that harvested card numbers and banking credentials.
What makes this attack unusual is what it doesn’t need: no scam email, no dodgy ad, no spelling trick like “arnazon.com”. The victim asks an AI a sincere question, gets a confident answer, and clicks. The trust we place in AI answers is the entire attack surface. To be fair on scope: most of those 250,000 addresses will never be weaponized — the technique pays off mainly for banks, health services, and government sites, where a convincing fake is lucrative.
What this means for you: One habit protects you: treat links from AI chatbots like directions from a helpful stranger — probably right, worth checking. When money or passwords are involved, don’t click through; go to the site the way you already know (your bookmark, the official app, or typing the main address yourself). If a chatbot gives you a login link, that’s exactly the moment to be suspicious — real services rarely need an AI to tell you where their front door is.
Sources
Source: https://unit42.paloaltonetworks.com/phantom-squatting-hallucinated-web-domains/
The US Government Wants a 30-Day Preview of New AI Models — 'Voluntarily'
A new executive order sets up a system where AI labs can give the US government early access to their models before release. It's officially voluntary — but talks with OpenAI, Anthropic, and Google suggest it may become the norm. What that means for the tools you use.
