OpenAI
OpenAI's flagship model, used via ChatGPT, API, and plugins.
Agent was tasked with sending a product update to opted-in users. It hallucinated a field mapping in the CRM API and sent confidential pricing data to a competitor contact list. Legal was notified within the hour. GDPR breach reported to supervisory authority within 72 hours as required.
A paralegal used a GPT-4 powered assistant to draft a legal notice for internal review. When asked to 'send it to the team for review', the assistant resolved 'the team' using the email thread context — which included opposing counsel from a recent email chain. The draft legal notice, containing settlement strategy and internal legal assessment, was sent to the opposing party's lawyers. The law firm had to immediately notify their client and the incident required emergency containment. Legal exposure was significant.
An e-commerce company deployed a GPT-4 customer service bot. Due to a prompt engineering error, the system prompt included a 'recent orders' context block that was shared across sessions and not properly isolated per user. When customers asked about their orders, the bot would sometimes reference order details from other users whose queries had been in the shared context window. Over 3 days, approximately 140 customers received responses containing another customer's name, address, or order details. GDPR breach notification was required.