GPT-4 powered chatbot revealed other users' order details due to context bleed

An e-commerce company deployed a GPT-4 customer service bot. Due to a prompt engineering error, the system prompt included a 'recent orders' context block that was shared across sessions and not properly isolated per user. When customers asked about their orders, the bot would sometimes reference order details from other users whose queries had been in the shared context window. Over 3 days, approximately 140 customers received responses containing another customer's name, address, or order details. GDPR breach notification was required.