AI Roundup: AI Scams, and a Shopping Agent 'Hallucination'

By Roy Urrico

Finopotamus aims to highlight white papers, surveys, blogs and reports that provide a glimpse into issues that are impacting credit unions and other organizations in the financial services industry.

In this roundup we focus on the reported bad and good sides of artificial intelligence with its use to deceive and direct consumers.

FBI Warns of Scammers Impersonating the IC3

In a public service announcement (PSA) the FBI warns of an ongoing fraud scheme where criminal scammers are impersonating FBI personnel facilitating Internet Crime Complaint Center (IC3) complaints to deceive and revictimize individuals.

“This scheme combines several exploitation tactics to include the targeting of previous victims, the use of artificial intelligence-generated videos to create fictitious or misleading promotional materials for their fraud schemes; and the creation of spoofed websites with the purpose of gathering personally identifiable information entered by a user into the site,” said the PSA. “Each of the tactics give victims a false sense of safety and security, while the actors impersonate or falsely affiliate themselves with government personnel or entities.”

How the Scam Works:

Complainants report scammers use a variety of methods to initiate contact with victims. Some individuals received an email or a phone call, while other victims were lured via advertisement on social media or forums. “Almost all complainants indicated the scammers claimed to have recovered the victim's lost funds or offered to assist in recovering funds. However, the claim is a ruse to revictimize those who have already lost money to scams,” said the warning.

• Scheme 1 — A recent example of the impersonation scheme variant indicates scammers create fraudulent persona profiles and fake pages on social media networking sites, impersonating FBI personnel to lure victims into "re-targeting" scams.After realizing they are scam victims, the target reports it to the FBI and files an IC3 report. An individual impersonating an FBI agent later contacts the victim on Facebook Messenger, for example, and sends a link to update the submitted IC3 report. The link may contain malicious code or be used to collect more financial data to revictimize the person.

• Scheme 2 — Another recent example of the impersonation scheme involves scammers creating AI-generated videos (commonly referred to as "deepfakes") on social media platforms, impersonating FBI personnel and promoting the FBI IC3 government website.A social media platform shared AI-generated videos depicting a senior FBI leader encouraging users to submit victim complaints on a spoofed IC3 website.“The site features similar structure and content to the FBI IC3 government website. However, the spoofed website only allows users to interact with the complaint submission, and clicking on any further links returns the user to the home page,” said the alert. The complaint structure differs from the FBI IC3 intake in that it is a singular form step requiring only name, phone number, email address, scam type, and estimated financial loss. Upon submission the site gives a reference number and indicates that someone will be in touch with the submitter.”

AI-Generated Videos

•        Scammers use AI-generated videos to create believable depictions of public figures to bolster their fraud schemes.

•        Scammers rely on fear, urgency, and high-tech tricks like AI-generated video and caller ID spoofing to appear legitimate.

•        Scammers generate videos for real-time video chats with alleged company executives, law enforcement, or other authority figures.

•        Scammers create videos for private communications to "prove" the online contact is with a "real person."

Merchants’ Fear of AI Shopping Agents Is Just a ‘Hallucination’

Merchants' fear of AI shopping agents is just a 'hallucination,' according to London-based PSE Consulting's whitepaper, The Shortlist Economy - How AI is Rewiring Buying Habits (and How Merchants Can Respond). Based on responses from 4,250 consumers across the U.S., UK, France and Germany, the research explores the emerging “shortlist economy” — an emerging market dynamic driven by AI shopping agents and chatbots — and how AI is reshaping the path to purchase.

Chris Jones, Managing Director at PSE Consulting.

 While AI is increasingly influencing which products consumers consider, it isn't replacing the brands and marketplaces they ultimately choose to buy from. "The fear that AI agents will simply cut merchants out of the transaction is, on the evidence of our research, a hallucination," said Chris Jones, Managing Director at PSE Consulting. An AI hallucination is when an AI model—like a chatbot or image generator— produces false, fabricated, or misleading information that seems credible.

“Visa and Mastercard are investing heavily in the infrastructure required to support agentic commerce, Stripe and Google are deploying standards to support agentic payments, while merchants from Amazon to Booking.com are deploying agents on their own commerce platforms,” said the white paper.

PSE’s analysis indicates that around $1.25 trillion of e-commerce spend could migrate to agentic rails by 2030, with segments such as travel and accommodation generating significant shares. “The sudden emergence of large language models (LLMs) in the past two years hasn’t happened without widespread consumer concern emerging about the role of AI at home and at work,” revealed the white paper.

“The easiest way to understand the promise of agentic commerce is as a digital personal shopper – an AI Agent that is entrusted to make purchases on behalf of its users, whether in retail, travel, health or dozens of other segments,” the report explained. “Today, large language models like ChatGPT, Claude and Gemini are built with humans in the loop, where the purchaser approves each transaction.”

Among the key findings:

• Eighty-nine percent said recognizing the seller's brand still matters when acting on an AI recommendation.

• Ninety-two percent said reviews remain a deciding factor.

• Just 14% simply follow an AI assistant's top recommendation, compared with 32% who say price is the deciding factor.

• Nine out of 10 expect their use of established marketplaces to stay the same or increase as AI adoption grows.

• Seventy-four percent of the consumers surveyed prefer an independent AI assistant. This is either a universal tool such as ChatGPT or Gemini that works across multiple platforms and providers (41%), or a specialist AI assistant focused on a specific category such as travel, finance or healthcare (33%). By contrast, just 10% prefer an AI shopping assistant embedded directly within a retail, travel or delivery platform.

• PSE Consulting’s research indicates that merchant anxiety is misplaced. Just 14% of consumers responded they simply follow an AI assistant's top recommendation - fewer than half the 32% cited price as their primary decision driver - underlining that most are using AI to compare offers, not to delegate the decision itself.

The findings suggest that while AI agents are increasingly shaping the shortlist of products consumers consider, they are not displacing the merchants, marketplaces and brands consumers ultimately trust to complete the sale.

Consumers continue to favor established marketplaces for fulfilment, even as they lean on advertising-funded AI tools to get there - 90% said their use of retail and travel marketplaces will stay the same or increase as AI adoption grows, and just one in ten expect to rely on them less.

Amazon's lawsuit against Perplexity AI - which accused the startup's Comet browser of disguising AI agents as human shoppers to buy on its site - has been interpreted as the opening shot in a war over who controls the checkout button when AI does the shopping, crystallizing a fear now spreading through merchant boardrooms. PSE Consulting's research suggests that framing overstates the threat. "Consumers are using AI to build a shortlist, not to transfer their loyalty to buying from the agent. The Amazon-Perplexity case is really a fight about who owns the discovery layer - it is not evidence that shoppers have stopped caring who they buy from."

Jones added: “That doesn't mean merchants can relax. The competitive battle has moved upstream, into the AI discovery layer itself. But the idea that agents are about to replace the merchant relationship altogether isn't what consumers are telling us.”