ci: run the checks that only ran on a laptop; mobile in CI; retries 0
CI ran Playwright (chromium-desktop) + the dependency audit and nothing else. cargo test,
clippy, the frontend vitest suite, svelte-check and the e2e typecheck were all green on a
developer's machine and gated NOTHING.
checks.yml (new): cargo test (with a Postgres service; --test-threads bounded so the
sqlx per-test DBs can't exhaust connections) + clippy; vitest + svelte-check; e2e tsc.
e2e.yml: also run the chromium-mobile project — 22 tests (focus traps, touch targets,
safe-area, viewport reflow) that never ran in CI on a phone-first app.
playwright.config: widen webkit-iphone beyond @smoke (2 specs) to the core guest journeys
(auth/upload/feed) — iOS Safari is the PRIMARY browser here; and retries 2 → 0.
retries:0 is deliberate and against the usual advice: this repo's real bugs are races, a
race is indistinguishable from a flake from outside, and a retry resolves that ambiguity
toward "flake" every time — it took a real 3%-flaky product bug from 1-in-33 to 1-in-37000,
green. A flake here is a bug report.
Not gated: cargo fmt (the tree has never been rustfmt'd — a 112-file reformat would bury
every real diff; do it separately). WebKit widening is unverified locally (needs libavif16
via sudo), so its first CI run may surface real iOS bugs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -23,7 +23,25 @@ export default defineConfig({
|
||||
outputDir: './test-results',
|
||||
fullyParallel: false, // Single shared backend → tests TRUNCATE between, so don't run in parallel.
|
||||
forbidOnly: !!process.env.CI,
|
||||
retries: process.env.CI ? 2 : 0,
|
||||
// NO RETRIES — not even in CI. This is deliberate and it is the opposite of the usual advice.
|
||||
//
|
||||
// The standard case for retries is "the environment is flaky, the product isn't." That argument
|
||||
// does not hold here. This backend's real bugs ARE races (the last several fixes were all export
|
||||
// concurrency), and from the outside a race is indistinguishable from a flake — so a retry
|
||||
// resolves that ambiguity, silently, in favour of "flake", every single time.
|
||||
//
|
||||
// The numbers make it concrete. A test that fails 3% of runs reports a bug roughly 1 run in 33.
|
||||
// Under `retries: 2` it fails the build only when it fails three times in a row: ~1 in 37,000.
|
||||
// We had exactly such a test, and it was reporting a REAL user-facing bug (a suggestion button
|
||||
// that committed on mousedown and destroyed itself mid-click). Retries would have buried it
|
||||
// forever, and buried it GREEN, so nobody would even have seen an amber.
|
||||
//
|
||||
// Worse, a retry does not re-run the failing conditions — it runs a CLEANER environment (the
|
||||
// interfering background worker from the previous test has since finished). So the mechanism is
|
||||
// specifically good at hiding exactly the cross-test contamination this suite is prone to.
|
||||
//
|
||||
// A flake here is a bug report. Treat it as one.
|
||||
retries: 0,
|
||||
workers: 1, // One worker. Multi-worker needs per-worker isolated DBs (Phase 2+).
|
||||
reporter: [
|
||||
['list'],
|
||||
@@ -127,7 +145,12 @@ export default defineConfig({
|
||||
{
|
||||
name: 'webkit-iphone',
|
||||
use: { ...devices['iPhone 14 Pro'] },
|
||||
grep: /@smoke/,
|
||||
// The PRIMARY user of this app is a wedding guest opening a QR link in iOS Safari. Gating
|
||||
// that on `@smoke` — which exists on exactly two specs — meant the entire iOS guarantee was
|
||||
// one happy path and one join test. Every other UA here is a secondary browser and a smoke
|
||||
// check is proportionate; WebKit is not. Give it the core journeys the guest actually walks:
|
||||
// join/recover, upload, and browse the feed.
|
||||
testMatch: ['**/__smoke/**', '**/01-auth/**', '**/02-upload/**', '**/03-feed/**'],
|
||||
},
|
||||
{
|
||||
name: 'firefox-android',
|
||||
|
||||
Reference in New Issue
Block a user