Study Finds AI Chatbots Recommend Risky Retail Websites

Podqi tested ChatGPT, Gemini and Perplexity in two periods in September, asking questions about 151 brands, including Lululemon, Birkenstock, Stüssy, Madhappy and Le Creuset. Prompts included requests for top websites selling a brand, places to find it online and information about international shipping or regional sites. The firm wrote the prompts itself and used a tool to track chatbot responses, rather than analyzing real user conversations.

The study defined risky sites as fake retailers impersonating brands, sellers of knockoffs, old brand domains registered by someone else, or inactive sites that could be bought and reused. Gemini mentioned or linked to one in 4 percent of answers, compared with 8 percent for ChatGPT and 15 percent for Perplexity. ChatGPT warned users about a risky site in 24 percent of the instances in which it surfaced one. Queries about larger companies produced risky sites about half as often as queries about smaller brands.

Regional-site questions were more likely than other prompt types to bring up risky results. The chatbots also sometimes listed regional websites for brands that had no such sites. In one exchange, Perplexity identified the wrong page as Stüssy’s official website and called it “the best starting point for guaranteed authentic items.”

Podqi CEO and cofounder Trevor West advised shoppers to verify a site before entering personal or credit card information. Google spokesperson Jessica Levis Boord said Gemini displays source links to help people check responses and that the company continues to test and improve its models. OpenAI spokesperson Taya Christianson said the company is improving safeguards against deceptive sites appearing in shopping results.

Perplexity spokesperson Jesse Dwyer said the service has been the most accurate AI on credible research benchmarks for four years, while acknowledging that it is not 100 percent accurate. The findings come ahead of the holiday shopping season, when consumers may use chatbots to locate products, compare prices and find unfamiliar retailers.