The Problem

Weekly releases, monthly drops, six storefronts, one QA engineer

Vuori is a performance apparel brand on Shopify Plus, shipping weekly releases and dropping a new collection monthly across six international storefronts. QA sat on one engineer's desk.

Six months of Playwright investment still meant partial coverage. Every ticket was verified by hand. Engineering spent 15–30 minutes of QA per PR, a week on big releases. Monthly drops meant hundreds of products checked manually, and international coverage ran US-first, with other storefronts spot-checked. A 300-product drop was slow to check by hand and still missed broken add-to-cart.

What Changed

A POC pointed at the bottleneck, not a sandbox

Vuori set clear success metrics before the POC started: time saved against the manual baseline, whether tests survive a moving UI, and whether they can be generated from acceptance criteria, with a 70–80% pass rate as the target. Then they ran it against live sprint tickets, so the savings were measured against real work: PR-level validation before merge, not regression after the fact.

The first day delivered 50% automation coverage, more than six months of Playwright had produced, and the full migration took two weeks. Deployment followed a schedule: tuned in dev, merged to staging, then automated runs every two hours on production post-merge. The pass rate didn't just clear the 70–80% bar; it settled at 93–98%.

Today 365 tests run across 17 plans and 6 international release gates: a 148-test weekly regression, a bi-weekly experience suite, monthly internationalization coverage, and drop QA duplicated per drop for a clean audit trail. Roughly 24 key Google Analytics events are validated weekly across browsers and locales via the Network Log Agent, and tickets are tested automatically from Jira tickets, requirements and comments.

What it Unlocked

QA stopped being one person's job

In-sprint automation is now written into the standard release document, and QA shifted from a single engineer to the whole team, with QA contractors onboarded on a 1:1 mentorship model. Ten hours came off regression QA before each weekly merge, and monthly drops now run as 175 unattended validations per drop instead of hundreds of hand-checked products.

"We spent six months on Playwright and still had partial coverage. Spur passed that in a single day, and QA stopped being one person's job."

The catches show the depth: a Shopify auth fault where a null response silently logged shoppers out mid-session, surfaced by a flaky logout test; a denomination dropdown regression caught pre-merge; homepage category tabs failing on mobile only; and a currency fault reproduced on Safari.

On the FA26 Drop 1 sweep, 175 PDP validations ran unattended at a 96.6% pass rate. Every release and every drop, validated, with Spur as the harness underneath all of it.