06. QA, Code Review & PR Workflow — Vibe Coding Series

Code passes lint. Tests are green. CI is happy. I confidently merge to main.

The next day, a senior on the team reviews it and calls me: “Hey, this auth middleware has 3 security issues. One — it doesn’t validate the iss claim in the JWT. Two — the error messages are too detailed, attackers can figure out which emails are registered. Three — no rate limiting on the login endpoint.”

All 3 issues: AI didn’t write tests for them. Lint didn’t catch them. CI didn’t find them. But someone with experience spotted them in 5 minutes.

This is the most important lesson of Phase 5: automated QA and AI review are the first filter, not the last.

---

Context

In article 05, I implemented all 8 auth tasks for TaskFlow using single-agent and multi-agent workflows, committing after each task, with HITL checkpoints before merging. This article moves into Phase 5: before code hits main, it needs to go through a structured QA and review process.

---

Automated QA layer

Automated QA doesn’t replace human review — it eliminates the basic issues so human reviewers can focus on what actually matters.

Static analysis: ESLint + TypeScript strict

With AI-generated code, TypeScript strict mode is non-negotiable. AI tends to write code that “works” but isn’t type-safe — and type errors often hide real logic bugs.

The config I use for TaskFlow:

// tsconfig.json
{
  "compilerOptions": {
    "strict": true,
    "noUncheckedIndexedAccess": true,
    "exactOptionalPropertyTypes": true,
    "noImplicitReturns": true,
    "noFallthroughCasesInSwitch": true
  }
}

// eslint.config.js
export default [
  {
    rules: {
      "@typescript-eslint/no-explicit-any": "error",
      "@typescript-eslint/no-floating-promises": "error",
      "@typescript-eslint/no-unused-vars": "error",
      "no-console": "warn",
      "eqeqeq": ["error", "always"]
    }
  }
]

noUncheckedIndexedAccess is especially useful with AI code — it forces you to handle the case where an array index might be undefined, something AI routinely skips.

Security scan: AI-assisted SAST

Traditional SAST tools like Semgrep or Snyk work well in CI. But with AI-generated code, I add one more step: using AI to review code from a security perspective before pushing.

The prompt I use after AI finishes a feature:

Review this code from a security perspective. Focus on:
1. Input validation — is all user input validated?
2. Authentication/Authorization — are permissions checked correctly?
3. Information disclosure — do error messages reveal sensitive information?
4. Injection — any SQL injection, command injection, or path traversal?
5. Cryptography — any weak algorithms or hardcoded secrets?

For each issue found, point to the specific line and explain why it's a problem.

[paste code]

This isn’t a replacement for Semgrep — it’s a complementary step to catch issues that rule-based scanners miss.

Test coverage threshold

With AI-generated code, coverage thresholds are an important safety net. AI tends to write tests for the happy path but skip edge cases.

// vitest.config.ts
{
  "coverage": {
    "thresholds": {
      "lines": 80,
      "functions": 80,
      "branches": 70
    },
    "exclude": ["**/*.test.ts", "src/types/**"]
  }
}

Branch coverage at 70% is the threshold I’ve found practical — enough to catch important paths without being so strict that AI has to write tests for every minor edge case.

---

AI-assisted code review

Using AI to review AI code

It sounds circular, but it’s actually effective — as long as you use the right prompt. AI reviewing AI isn’t about “AI checking itself,” it’s about using AI as a knowledgeable reviewer to catch common patterns.

The most effective prompt pattern I’ve used:

You are a senior engineer reviewing this PR. The code was written by an AI agent.

Context:
- Project: TaskFlow — task management SaaS
- Feature: User authentication (JWT + refresh token)
- Tech stack: Node.js, TypeScript, PostgreSQL

Review the code below and create review comments in this format:
[CRITICAL] — must fix before merge
[SUGGESTION] — improvement worth doing but not blocking
[QUESTION] — needs clarification

Focus on:
- Security: authentication, authorization, input validation
- Performance: N+1 queries, missing indexes, unnecessary computation
- Maintainability: naming, complexity, coupling

[paste diff or code]

This typically produces 5–10 quality comments. Not all of them are right — AI sometimes flags false positives — but it saves significant time compared to reading every line manually.

Auto-generating a review checklist from spec

One trick I find really useful: using AI to generate a review checklist from the task’s acceptance criteria.

From these acceptance criteria, create a review checklist to verify the implementation:

Acceptance criteria:
- POST /auth/login accepts { email, password }
- Returns 200 + { accessToken, refreshToken } or 401
- Saves refresh token to store
- Rate limit: max 5 attempts in 15 minutes

Checklist format:
- [ ] [check name]: [how to verify]

AI will generate a checklist like:

• [ ] Login with correct email/password → verify response has accessToken and refreshToken
• [ ] Login with wrong password → verify 401 response, no token
• [ ] Login with non-existent email → verify 401, error message doesn’t reveal whether email exists
• [ ] Try logging in 6 times in a row → verify 6th attempt gets 429 Too Many Requests
• [ ] Refresh token saved to database → verify with direct query

This checklist doubles as both test cases and a review guide for the human reviewer.

---

PR workflow

PR template with AI-generated summary

With AI-generated code, PR descriptions are often left blank or written carelessly. I use a mandatory template and AI to generate the summary:

<!-- .github/pull_request_template.md -->
## Summary
<!-- AI-generated: paste output from the prompt below -->

## Changes
- [ ] Feature implementation
- [ ] Tests added/updated
- [ ] Documentation updated

## Security checklist
- [ ] Input validation complete
- [ ] No hardcoded secrets
- [ ] Error messages don't reveal sensitive information
- [ ] Rate limiting (if applicable)

## Test coverage
- Unit tests: X%
- Integration tests: covered / not covered
- Manual testing: [describe what was manually tested]

## AI involvement
- [ ] Code generated by AI agent
- [ ] AI security review done
- [ ] Human final review done

## Breaking changes
<!-- Any backward-incompatible changes? -->

To generate the Summary section, I use this prompt:

Summarize this PR in 3–5 sentences for the reviewer. Include:
- What feature is being implemented
- The main approach (no implementation details needed)
- What the reviewer should pay special attention to

[paste git diff --stat and a few key changed files]

Branch strategy with multi-agent

When using multi-agent (like in article 05), each agent works on its own branch. Before creating a PR, I merge the agent branches into an integration branch:

# Create integration branch
git checkout -b feature/auth-integration

# Merge each agent branch in dependency order
git merge feature/auth-utilities    # Tasks 1, 2
git merge feature/user-schema       # Task 3
git merge feature/auth-endpoints    # Tasks 5, 6, 7, 8

# Resolve conflicts if any, run tests
npm test

# Create PR from integration branch to main

The reason not to PR directly from each agent branch: reviewers would have to review 4–5 separate PRs and figure out the relationships themselves. One integration branch brings everything together and is much easier to review.

Squash vs. merge: keeping history clean

With AI-generated code, commit history tends to be noisy — AI often creates commits like “fix”, “update”, “refactor” with no context. I use squash merge when merging PRs into main:

# Squash merge: collapse all PR commits into 1
git merge --squash feature/auth-integration
git commit -m "feat(auth): implement JWT authentication with refresh token

- hashPassword/verifyPassword utilities (bcrypt, salt 12)
- JWT access token (15min) + refresh token (7 days)
- Register, login, refresh, logout endpoints
- Auth middleware for protected routes
- Rate limiting on login endpoint (5 req/15min)"

That final commit message I write by hand — this is where human judgment matters. AI can draft it, but you need to review and make sure it accurately describes what actually changed.

---

Real-world example: TaskFlow auth PR

After implementing all 8 auth tasks in article 05, here’s the process I went through:

Step 1: Automated checks

npm run lint          # ESLint + TypeScript check
npm run test          # Vitest with coverage
npm run test:security # Semgrep scan

Result: 2 TypeScript errors (AI used any in 2 places), coverage 84%, Semgrep clean.

Fixed the 2 TypeScript errors, re-ran — everything passed.

Step 2: AI security review

Pasted the full diff into Claude with the security review prompt above. Results:

• [CRITICAL] Login endpoint doesn’t validate email format before querying the database
• [CRITICAL] Register endpoint error message reveals “Email already exists” — should return a generic message
• [SUGGESTION] Add httpOnly flag to cookies if switching to cookie-based auth later
• [QUESTION] Is the refresh token rotated after each use?

Fixed the 2 critical issues, noted the suggestion and question to discuss with the team.

Step 3: Create PR

Used the template, AI generated the summary, filled in the checklist. PR description is clear — reviewers know exactly where to focus.

Step 4: Human final review

Reviewer (me, after a day away) re-read with fresh eyes. Found one more thing: the refresh token store has no index on the userId column — will be slow when calling deleteAllUserTokens. Added a migration to create the index.

Step 5: Squash merge

Wrote the consolidated commit message, merged to main.

Total time: ~45 minutes for the entire QA + review process. Without this process, those issues would have gone to production.

---

Pitfalls

Using AI to review AI without a human final check. The AI reviewer and AI author share the same blind spots — both were trained on the same data, the same patterns. Issues the AI author missed are often the same issues the AI reviewer misses. Human final check doesn’t mean reading every line, but it has to happen.

PRs that are too large. When using multi-agent, the temptation is to bundle everything into one big PR. A PR with 50+ changed files won’t get a thorough review — reviewers will just approve to move on. Keeping PRs under 200–300 lines of change is a practical threshold for quality reviews.

---

Summary

• Automated QA (lint, type check, coverage) eliminates basic issues so human reviewers can focus on what matters
• AI security review catches common patterns — use a structured prompt, not just “review this code”
• PR template + AI-generated summary helps reviewers understand context faster
• Squash merge keeps history clean when AI generates noisy commits
• Human final check is mandatory — AI reviewer and AI author share the same blind spots

---

What’s Next

Code has passed QA, been reviewed, and merged to main. The final step: shipping it to production safely — feature flags, canary deployment, and observability so you know immediately when something goes wrong.

→ Article 07: Deploy, Rollback & Observability

← Article 05: Execution with AI Agents