Automated vs Manual Testing: Where Each Fits in QA

What automation does well (and what it doesn’t)

Strengths

• Fast, repeatable checks for missing labels, ARIA misuse, contrast flags, title/landmark gaps.

• Runs on every commit/PR to prevent regressions.

• Scales across templates and high-traffic pages.

Limits

• Can’t fully judge keyboard flows, focus order/visibility, modal traps, meaning of alt text, or screen reader experience.

• Often misses content-level issues and dynamic states (e.g., live validation, announcements).

Bottom line: Automation is your early warning system, not your verdict.

Manual testing essentials (short, high-value passes)

• Keyboard-only: Tab/Shift+Tab, Enter/Space/Arrows; open/close menus, dialogs, carousels; confirm no focus traps.

• Focus visibility: clear, high-contrast outline everywhere; no obscured focus under sticky headers.

• Screen reader smoke test (NVDA/VoiceOver): title, headings, landmarks, links list, form labels, error announcements.

• Forms: humane validation; specific errors tied to fields; first error receives focus.

• Mobile: target sizes, reflow at 320px, zoom allowed; motion respects prefers-reduced-motion.

Goal: Validate task completion and comfort on real devices—not just pass rules.

Cost, coverage, and risk (how to decide)

ApproachCost per runCoverageCatchesRisk if skippedAutomation (CI)LowBroad (templates)Code-level violations, common regressionsHigh churn of small bugsKeyboard passLowHigh (critical flows)Focus order/visibility, operabilityTraps, blocked usersScreen reader skimLowMediumSemantics, labels, announcementsMisleading UX for AT usersExpert auditHigherTargetedComplex patterns, strategy gapsStrategic debt, legal exposure

A practical hybrid plan (what to run, when)

On every PR

• Automated ruleset on key templates (home, nav, product/service, blog, forms).

• Thresholds: 0 critical, ≤ N moderate (with ticket auto-created).

Before each release (30–45 minutes)

• Keyboard pass on top flows.

• Screen reader skim on changed areas.

• Contrast spot checks on changed components/states.

• Mobile check for targets/reflow/motion.

Quarterly (or after major UI changes)

• Short expert review of components (menus, dialogs, forms, carousels) and governance.

Example CI guardrails (pseudo-config)

name: a11y-checks
on: [pull_request]

Tip: Start with a small, stable ruleset; expand only when noise is low.

Ownership: who does what

• Engineering: CI setup, thresholds, fixing code-level issues, component patterns.

• Design/Content: contrast tokens, states, headings, link text, alt text quality.

• QA/PM: schedule manual passes, accept/reject criteria, risk sign-off.

• Leadership: resource time for audits and remediation.

Common pitfalls (and quick fixes)

• Green CI, red reality: Passing automation doesn’t guarantee usable flows → add a 10–15 min keyboard/screen reader gate.

• False positives → rule fatigue: Triage and tune rules; suppress at component level with comments + tickets.

• No owners: Assign components (Nav, Dialog, Forms) to named maintainers.

• One-time audit syndrome: Schedule recurring checks; accessibility drifts without governance.

Copy-paste checklists

Automation scope (Markdown)

Manual release gate (Markdown)

Quick QA cadence (sample)

• Weekly: CI only (on PRs).

• Fortnightly: CI + 30-minute manual gate across top journeys.

• Quarterly: CI + manual gate + mini expert check on complex components.

How Adjustable helps (after structural fixes)

• Text to Speech helps users consume complex content hands-free.

• Language Translation broadens reach for global audiences.

• Accessibility Profiles offer quick presets (e.g., “Readable” or “Low Vision”).

• Text Options, Reading Ruler, Cursor Adjustment, Screen Mask aid comprehension and focus.

• Page Options (page scaling, hide images/animations, highlight links) reduce friction on busy UIs.

• Light/Dark/Colour modes and configurable toolbar/widget placement align with user preferences.

Try Adjustable

FAQs

Can automation make us compliant?
No. It’s essential for catching regressions, but manual checks are needed for flows, meaning, and comfort.

What pages should automation cover first?
Templates that drive traffic and revenue: home, nav, listings, product/service, blog, checkout/contact.

How strict should thresholds be?
Start strict on critical (0 allowed). For moderate issues, allow a small budget that generates tickets until you reach 0.

Which manual test gives the biggest return?
A keyboard-only pass across your main journey—fast, revealing, and highly correlated with conversion blockers.

Next steps

• Add a minimal a11y CI job with tight thresholds on key templates.

• Schedule a 15–30 min manual gate before releases.

• Assign component owners and a quarterly mini audit.

• Install Adjustable to improve real-world readability and task completion while you iterate.