New data from Exploding Topics by Semrush reveals the vast majority (78%)of consumers have used AI to shop in the last 6 months, but only 1 in 3 would actually trust AI to spend for them.
While AI shopping assistants are already transforming retail – already becoming embedded by corporate giants like Amazon’s ‘Rufus’ and Walmart’s ‘Sparky’ assistants – there’s subsequent concern around liability for errors.
Target has updated its terms of service to state customers will be financially liable for any errors made by its AI assistant by Google Gemini, and Walmart’s CEO has come under fire for reportedly bragging that AI shopping assistants encourage customers to spend 35% more.
There’s undeniable controversy around ethical constraints around AI shopping assistants, and ethical AI expert, Jonny Murphy-Campbell at Resolvable, explains it’s a “subject that requires much nuance”…
AI expansion is highly beneficial for customer experience, but it doesn’t come without limitations”, explains Jonny. “AI conversations tend to circulate in extremes, but the truth is that AI usage in the shopping experience is not inherently harmful, but the execution can be if it is not done ethically and with human oversight.
“The problem consumers have with AI embedded in the shopping experience is not AI itself – the majority of customers are already leaning on generative AI to help them find recommendations and AI customer service bots to answer FAQs – the problem lies in the transparency around AI-led shopping – or lack thereof.
“The risk isn’t the technology itself, it’s the move from assistance to automation without clear safeguards in place.”
As AI shopping agents become more autonomous, Jonny explains the 5 major red flags for consumers to identify where AI-led shopping is secure or poses a risk:
The AI assistant can complete a purchase without clear approval
AI assistants can certainly streamline the process for humans, and ultimately that’s what AI is here to do – enhance the human experience by automating repetitive or arduous tasks. Humans must have visibility and full control over where their money is spent – being able to authorise the transaction even if an AI assistant takes care of the rest of the process.
Over-delegation and allowing agents to be autonomous increases the chance of unauthorised transactions, and AI without human oversight risks significant errors, bias, and security breaches. AI should be used as a tool for humans, and human oversight ensures accountability and ethical use.
The terms and conditions are not clear
Terms and conditions establish legal and ethical boundaries, and it’s a classic case of ‘check the small print’ – businesses must explicitly state the terms and conditions of AI shopping assistants.
Terms and conditions should state if the human consumer remains responsible for the actions of the shopping agent from the beginning of the process, such as if purchases are wrong or if AI hallucinates.
Disclosures must be clear insofar as they can be easily seen and understood by consumers, not merely implied; this is essential for ethical and legal terms, but also to aid consumer trust in AI integration.
There’s no clear escalation pathway
While AI agents are becoming increasingly equipped to resolve customer service queries and issues, human oversight is crucial to handle complex queries that require human judgement and human nuanced decision-making.
Customers need a clear way to escalate the problem, and without this, human trust in AI and the retail provider can be eroded, and this error becomes a reputational risk.
AI processes require defined accountability and human oversight, making automation combined with clear escalation processes essential.
Lack of transparency in recommendations
Transparent recommendations can streamline the process for consumers so long as AI explains why it has been recommended.
Some consumers are sceptical about AI usage, and providing recommendations without explanation of why it is a suggested product – like based on past purchases or preferences – can create suspicion about commercial bias or influence.
Especially where retailers use affiliate links or sponsored placements, explaining the journey to recommendation to a consumer increases trust and confidence in the process.
Lack of clarity around data privacy
Personalised AI shopping assistants rely on customer behaviour patterns to make recommendations tailored to the consumer, and transparency around that data is critical to avoid reputational risk and maintain trust.
Clear privacy settings, explicit data use in terms and conditions, visible data controls, and data on an opt-in basis are all essential to ensure consumer comfortability and loyalty, and brand reputation.
Photo by Vitaly Gariev on Unsplash
