QA Efficiency with AI: Lessons from Running a QA Team with Claude in the Loop

Ivan Ćorić

Apr 24

7 min read

Why I Started Taking AI Seriously in QA

For most of my career, the unglamorous truth about QA was that a huge share of the work wasn't testing. It was translating. Translating product specs into test cases. Translating bug reproductions into Linear tickets. Translating failed runs into sign-off docs. Translating Slack chaos into post-mortem slides.

When I started using Claude as a genuine teammate — not just a chatbot, but something plugged into Linear, GitHub, Slack, and the test-management tool — the translation tax started to disappear. What follows is five concrete ways this changed how I run QA day-to-day.

None of this is theoretical. Every example below is from real release workflows on a complex product with many moving parts — the kind of system where a single missed edge case costs a day of firefighting.

1. Treating Releases as Projects in Linear — With AI Doing the Allocation Math

The single biggest unlock was framing each release as a Linear project, not just a cycle. A project gives you a container for scope, due dates, owners, and — crucially — a place for AI to dump structured work.

The workflow I settled on:

Milestones as release gates. Every project has milestones like "pre-release quick-check," "release-gate P0/P1 smoke pack," and "post-release validation." Claude understands these as checkpoints, not just labels.
Areas as ownership zones. Each product surface gets its own area label with a clear human owner on the QA side. Claude uses those labels to filter cleanly when assigning work, so the right person gets the right slice of the backlog without me playing traffic cop.
AI-generated ticket batches. When I spun up a new QA team plan, I asked Claude to scaffold the entire test-management integration and manual-coverage backlog. In a single pass, it produced roughly twenty tickets covering:
- Test-management ↔ Linear auto-issue creation on failed runs
- Test-management ↔ CI triggers on staging deploy and nightly
- Slack failure pings and a daily digest
- A batch of manual-test-case authoring tickets, one per product area, plus a short pre-release quick-check suite and a release-gate P0/P1 smoke pack
- Dashboards for pass/flaky/MTTR per suite

What used to be a half-day of clicking "new issue" in Linear became a ten-minute conversation. And because Claude knows the label taxonomy, every ticket lands with the right type, area, priority, and due date — so the board is sortable and filterable the moment the work is created.

The tip that matters: don't just ask AI to "create tickets." Hand it your label conventions, your team IDs, your project IDs, and let it produce tickets that already fit your system. The time saved is real, but the consistency is what pays dividends two months later when you're trying to filter the board.

2. Feature Files Written by Claude, Imported into the Test-Management Tool

Gherkin-style feature files are the connective tissue between product intent and automated tests — but writing them well is tedious and deeply dependent on knowing the product. Claude turned out to be surprisingly good at this, for one reason: it can read the source.

The workflow:

I point Claude at the PR that introduces a new feature.
It reads the diff — component changes, new API endpoints, i18n keys, the works — and extracts the user-visible behavior.
It outputs a .feature file with Given/When/Then scenarios covering the happy path, edge cases, and the tricky states I'd otherwise miss on the first pass.
I paste the file into the test-management tool's feature-file import, and every scenario becomes a first-class test case, folder-tagged by area, ready for manual execution or automation.

A real example: a new multi-option UI shipped with warning dialogs and state transitions that depended on on-chain callbacks. Claude read the PR and produced scenarios for the happy paths, the confirmation dialogs, and — importantly — the post-action state transitions that turned out to be buggy in production. It caught an "action button stays enabled after a completed action" case as an explicit scenario before QA got its hands on the build.

What this replaces: the hour I used to spend writing scenarios from memory while half-reading the PR description. The coverage is better because the AI has actually looked at the code, not just the ticket summary.

3. PR Analysis for Manual Test Coverage — The Change-Detection Layer I Never Had

This is the workflow that changed how I talk to engineering.

Before Claude, a PR merging to staging triggered one of three things on my side: I'd read it carefully (rare, time-permitting), skim it (most common), or miss it entirely until something broke. There was no systematic way to know what to retest when a change landed.

Now, for every meaningful PR, Claude gives me:

A plain-English summary of what changed, grouped by behavior rather than by file. New UI surface added, critical business logic rewritten, a legacy fallback removed — in prose a product manager could read, not a file diff.
A retest list tied to the actual code paths — including edge cases the author probably didn't think about. This regularly includes false-positive scenarios, the full localization matrix when i18n keys change, and dependent flows that share a modified component.
A process-risk callout when a PR skipped the normal guardrails. A memorable example: a PR opened and self-merged in under ten minutes, no description, no QA tag, hotfix-labeled issues closed as Completed without ever reaching my queue. Claude flagged this as a process violation in the review, not just a test-coverage question.
Linked Linear context. When commit messages reference issue IDs, Claude pulls the issues and tells me what's already been hotfixed, what's still in "To be validated," and what's sitting with a blocking-release label I should be aware of.

The outcome: I catch risky merges within minutes instead of the next morning, and my retest lists are tied to actual diffs rather than my best guess. Manual coverage goes up without me doing more work.

4. Linear's AI Agent Generating Daily QA Sign-Offs (Built with Claude)

Linear now supports agent "skills" — named prompts that the AI agent uses automatically based on context. I built a QA sign-off skill for my frontend and backend teams, and Claude wrote most of it.

The setup: I described what I wanted the QA sign-off doc to look like — cycle dates, a status count table, highlighted issue sections (hotfix, blocking-release, failed, regression-not-fixed), test-run results, and a "bugs created this week" summary. Claude turned that into a production-ready skill prompt with exact API calls, document structure, label conventions, and formatting rules.

A few details that made it actually work:

Label matching is case-insensitive. Teams drift on casing over time (hotfix vs Hotfix vs HOTFIX), so every filter in the skill is case-insensitive. That one change eliminated an entire class of "why isn't this showing up?" bugs.
Blocking-release has no cycle filter. A release blocker from the previous cycle still matters. Everything else filters by current cycle.
Test runs are pulled live from the API. The skill fetches the plan named for the current cycle and attaches the latest runs, with pass/fail/skip counts.
Document creation is idempotent. The skill looks for an existing sign-off doc with a predictable title before creating a new one, so a single morning run doesn't fragment the doc.

What makes this feel like "the future" is that I don't have to trigger it manually. When a teammate leaves a simple comment on an issue or document, the agent picks up the skill automatically and produces the doc. Every morning I get an up-to-date readiness report without opening Linear myself.

The meta-lesson: letting Claude write prompts for other AI agents is one of the highest-leverage uses of AI I've found. You get better prompts than you'd write manually, and the iteration loop is fast because you can hand Claude the failure ("this ran but skipped the regression-not-fixed section") and it corrects the prompt.

5. Post-Mortems Built from Slack and GitHub in Minutes

Post-mortems used to be a day of my life I'd never get back. Scrolling Slack, cross-referencing GitHub PRs, stitching together a timeline, writing a deck, arguing about what actually happened. Multiply by every broken release.

The new flow:

Claude reads Slack. It searches the relevant channels for the window around the incident — deploy announcements, error spikes, the "who broke what" exchanges — and extracts the messages that actually matter.
Claude reads GitHub. It pulls merged PRs across the relevant repos for the incident window, plus direct-to-main commits that bypassed the PR process entirely.
Claude stability-flags the PRs. Every merge gets triaged:
- 🔴 Critical — merged from a non-main base branch, touched infra or payment-related config, or involved very large diffs
- 🟠 High — feature-flag changes, dependency upgrades, unusual authors
- 🟡 Medium — large diffs in shared components, or merges right before release cuts
Claude writes the narrative. A timeline, a root-cause hypothesis, contributing factors, and — the part engineers always push back on — specific process changes, not generic "we should communicate better" bullet points.
Claude builds the deck. Using the pptx skill, it produces a slide file with the timeline, the stability table, the root cause, and next-steps slides. I edit for tone and ship.

One post-mortem is the canonical example: a PR merged from a development branch instead of the main branch reintroduced old code, and it took down a production flow. Claude caught the pattern retrospectively, and that same detection rule now runs in the daily sign-off, so the next "merged from the wrong base branch" event gets flagged before it causes damage. The post-mortem didn't just document the failure — it hardened the daily process against it.

What This Adds Up To

If you measure QA efficiency in test cases written per hour, this is going to underwhelm. The bigger wins are structural:

Planning throughput. A release's worth of tickets goes from a half-day to fifteen minutes, with better labeling.
Coverage quality. Feature files and retest lists are derived from actual code, not my memory.
Early warning. PR analysis and stability flagging catch risky merges in minutes, not the morning after.
Sign-off discipline. A daily, auto-generated sign-off document removes the "did anyone check?" ambiguity.
Post-incident leverage. Post-mortems become hours instead of days, and the findings feed back into daily checks.

The thing I'd tell any QA lead considering this: don't treat AI as a search box. Treat it as a junior teammate who never forgets your label conventions, has read every PR, and will draft the tedious artifacts you'd rather not write. You stay in the loop on judgment calls — what's actually a blocker, what's a stylistic nit, whether to hold a release. The AI just removes everything between you and those judgment calls.

That's the shift. Not "AI does QA," but QA stops doing the work that was never really QA in the first place.

Written from real day-to-day QA workflows. Tools referenced: Linear, a test-management platform, GitHub, Slack, Claude, and Linear's AI Agent with custom skills.