Agentic QA glossary: flows, mapping sessions, smoke suites, self-healing tests
"Agentic QA" gets used loosely enough that the same word means different things depending on which tool's docs you're reading. This glossary defines the terms as QA Reef uses them, and — where it matters — notes how another well-known platform, QA Wolf, uses the same or a related term, sourced to its public docs and blog rather than guessed at. The goal is a shared vocabulary you can use regardless of which tool you end up choosing.
Flow
A flow is a named, ordered recording of steps performed in a browser: navigate, click, type, drag, scroll. It's the atomic unit both QA Reef and QA Wolf build on — QA Reef's flows are JSON step records that generate Playwright test code; QA Wolf's flows are .flow.ts files, a plain ES module whose default export is flow(name, target, callback) from its @qawolf/flows package, with one test() block per logical step. Different formats, same underlying idea: a flow is what you replay or turn into code, not the code itself.
Mapping session
A mapping session is the live, recorded browser session used to author a flow. It happens one of two ways: you drive the app directly and every click/type/drag gets recorded, or you describe a goal in plain English and an agent drives it — reading the screen, deciding the next action, and recording what it did. QA Wolf's public materials describe an equivalent "Mapping AI" capability that builds a test from a recorded session, operating against the DOM. QA Reef's mapping session can do the same, or fall back to an OCR-driven operator when there's no reliable DOM to map from in the first place.
Automate job / Automation AI
The step that turns a mapped flow into real test code. QA Reef calls this an Automate job: a deterministic code generator walks the recorded steps into Playwright API calls, and a cheap model is asked only for a title and up to three assertions — never full step logic — with every model output stamped with the exact model id that proposed it. QA Wolf's public materials describe a comparable "Automation AI" that can write Playwright code from a prompt, autonomously or guided — a wider scope than QA Reef's current deliberately narrow approach. Source: QA Wolf's own blog on multi-agent system design, qawolf.com/blog.
Smoke suite
A smoke suite is a small set of fast, high-confidence flows — usually the handful of paths that, if broken, mean the app is fundamentally down (can a user log in, can they reach the dashboard, does the core action complete) — run frequently, often on every deploy, to catch major breakage before a slower, fuller regression suite runs. It's a triage layer, not full coverage: a green smoke suite means "probably safe to keep going," not "nothing is broken."
Self-healing test
A self-healing test is one whose automation adapts when part of the page changes, instead of failing outright the moment a selector stops matching. QA Wolf's public engineering writeups describe this as evaluating "DOM diffs, network traces, console errors, and fixture state" to re-resolve what changed — explicitly DOM-based, not vision-based (qawolf.com/blog). QA Reef does not implement self-healing today: recorded steps become Playwright locators at generation time, and there's no automatic re-resolution of a broken locator yet — a real, current gap, not a design choice we're presenting as a feature.
Dark control
A dark control is a gate — a paywall, a permission check, a feature flag — tested on its negative path: verifying it correctly shows nothing to someone who shouldn't have access, not just that it correctly shows the right thing to someone who should. Most test suites only ever check the happy path (the door opens for the right person); a dark-control test proves the door stays shut for everyone else, and that "shut" isn't just "we never checked." This is a QA Reef-specific naming convention, not a term we're attributing to QA Wolf or any other tool.
Evidence / trace
The record a run leaves behind beyond a pass/fail label: a screenshot, a Playwright trace, and — for agent-driven steps — the OCR'd text an operator acted on and the model that decided the action. The point of evidence is that a failure (or a suspicious pass) is auditable after the fact without re-running anything.
PASS / FAIL / UNMEASURED
A three-state verdict, instead of the usual two. UNMEASURED covers a check that didn't get a clean enough look at what it was testing to report a real result — a page that never loaded, a viewport that drifted mid-check, a network hop that never resolved. Forcing that case into FAIL (the common default) makes an infrastructure hiccup look like a regression; forcing it into PASS is worse. UNMEASURED is QA Reef's explicit third bucket, and it's guaranteed never to silently collapse into either of the other two. Full background: why a QA verdict needs a third state.
Runner
The thing that actually executes a browser action or a test. Depending on context this might mean the Playwright process running a generated test locally, or — a distinct, narrower meaning — QA Wolf's own coordinate-based "Runner" tool, a screenshot/act loop documented in its CLI's bundled Agent Skill for an external agent (like a coding assistant) to drive a browser through. QA Wolf's own docs are explicit that this Runner takes an on-demand screenshot per call rather than running a continuous vision model — worth knowing, since "Runner" and "agent that watches the screen continuously" get conflated in casual conversation but aren't the same thing.
Open-core
A licensing model where part of a product's codebase is open source (usually under MIT or Apache-2.0) and the rest — typically the hosted platform, proprietary integrations, or a managed service layer — stays closed. QA Wolf's open-core boundary sits low: Playwright/Appium underneath, plus its own local flow-runtime CLI and packages, while Mapping AI, Automation AI, the runner grid, and the human QA service are closed. QA Reef is building its boundary higher: the automation core itself (the operator, the LLM adapter, evidence storage) is going into an MIT-licensed packages/core, not just the transport layer beneath it.
Related: Why we read the screen with OCR instead of trusting selectors · QA Wolf vs QA Reef, in depth · Docs