Chat responses fail during long conversations
.avif)
Step the Agent Failed on:
- 3 Wait 10 seconds for a response, scroll through the chat
End-to-end checkout Flow
Full Test That Was Run:
The chat functionality appears to be non-responsive. After sending a message, waiting 10 seconds, and performing multiple scroll attempts, no response from the concierge has been received. The message shows as "Delivered" but there's no indication of the concierge typing or responding, which represents a critical failure in the core chat functionality.
The chat functionality appears to be non-responsive. After sending a message, waiting 10 seconds, and performing multiple scroll attempts, no response from the concierge has been received. The message shows as "Delivered" but there's no indication of the concierge typing or responding, which represents a critical failure in the core chat functionality.
- 1 Click on the chat icon
- 2 Simulate a long and Simple intelligent request in the chat
- 3 Wait 10 seconds for a response, scroll through the chat
- 4 Verify that the response matches what was asked
- 5 Ask a follow up Question
- 6 Scroll through the chat.
- 7 Verify that the question asked is answered well
Bug Book
These are production bugs found for our actual customers
Search results for “gaming console” show accessories instead of consoles
Multiple UI/UX inconsistencies on international pricing page
AI chat responds in English instead of French
“Best Sellers” link in header leads to a 404 page
Checkout shows incorrect price or currency for Germany shoppers
French (Belgium) users see incorrect language during checkout
.avif)
Chat responses fail during long conversations
Liked Item Missing From Favorites
Meal selection page stuck loading
Delivery date is not displayed correctly
Subscription plan displays raw template text
Rewards page shows incorrect annual redemption limit
Platform: Native Mobile App (Android)
Device: Pixel 4a
Android Version: 35
User Type: Tobacco Member
Test Result: Failed at step 5 - Points display configuration error
Afternoon time slots don’t appear when selected
Platform: Web Browser
Website: Rockefeller Center
Test Type: E2E Purchase Flow
Total cost shown is wrong at checkout
Checkout button leads to error page
Checkout completion rate, conversion rate, revenue
Checkout total is higher than expected
Revenue accuracy, pricing accuracy, discount validation, checkout conversion rate
Reviews reference a different item (dress) on shorts page
- Prevented over $400k in lost sales
- Saved 21 hours in dev time on the bug
- Automated 21 hours in dev time on the bug
Schedule A Demo
We invite you to try us out with our new Pilot Program
