Kiro builds from the spec. Karate reads .kiro/specs and grades every task against real test runs.
The report above is public, regenerated by CI, and needs no licence or install.
The category-level gap
Kiro is AWS's flagship spec-driven development IDE. It turns an idea into a spec folder, .kiro/specs/<feature>/{requirements,design,tasks}.md, with EARS acceptance criteria, then drives an agent to implement it. This is a real step up from vibe coding: the intent is written down before the code exists.
But the workflow records completion the only way it can: an agent ticks - [x] in tasks.md. The tick records that the agent finished; it doesn't record which tests exercised the acceptance criteria that task named, or whether they passed. That's what a checkbox is for, in any spec-driven tool. The checkbox is a claim, not evidence.
So the question every team standardising on spec-driven development eventually asks: the tasks are all ticked and the build is green, but did anything check?
How it works
Karate joins what Kiro says should be true to what your tests actually proved. Three steps, all deterministic.
Step 1
Point Karate at .kiro/specs. Nothing is copied, re-authored, or reformatted. Every EARS acceptance criterion in the folder becomes a row in the traceability matrix, and the ticked checkboxes in tasks.md become completion claims to grade.
Step 2
Requirements-verification traceability, computed from execution: each criterion is graded on exercised vs. claimed coverage. A criterion counts as verified only when a passing test actually exercised it, not when a test file mentions it and not when a task claims it.
Step 3
Every criterion a completed task claims but no passing check backs is reported as claimed done, not verified. A high-criticality one is a release blocker. The confidence-to-ship score is computed by versioned code, so a checkbox cannot move it.
This is the Trust & Governance pillar applied to spec-driven development. Kiro's spec folder is the cleanest requirement source we ingest: the criteria are already numbered, already structured, and already joined to tasks. An enterprise standardising on Kiro gets a governance layer for AI-built software without changing how its teams work.
The worked example
The public kiro-demo kit is a small lending service with a spec folder authored in the exact format Kiro emits. It runs in about half a second, no browser, nothing to install but the engine.
What the dashboard says
12 / 12
scenarios passed
API operation coverage: 100%. Tasks 1 to 5 in tasks.md ticked. A coverage tool would call this finished.
What the evidence says
NOT READY
release verdict
1 criterion marked done by task 5 but not verified: the checkbox is a claim, not evidence.
.kiro/specs/…/tasks.md
- [x] 3. Implement the decline gate
- [x] 4. Implement decision routing
- [x] 5. Implement APR pricing
- Apply the existing-customer discount
- _Requirements: 4.1, 4.2, 4.3_
- [ ] 6. Publish the decision eventKarate traceability matrix
criterion 4.2 existing-customer discount
claimed by: task 5 [x]
exercised by: nothing
status: CLAIMED DONE, NOT VERIFIED
criticality: high → release blockerThe criterion is implemented in the service and exercised by no test. That gap is invisible to a green dashboard and to the agent that wrote the code. A green build is not evidence that the work is done.
The Karate Power for Kiro
A Power is Kiro's extension format: POWER.md + mcp.json + steering files. Install the Karate Power from GitHub and Kiro can ask for the verdict itself, over MCP.
# In Kiro's chat:
> Which completed tasks aren't actually verified?
# Kiro calls Karate's MCP tool and reports the verdict in its own words:
"Task 5 is marked complete, but criterion 4.2 (the existing-customer
discount) has no test exercising it. The release verdict is NOT READY.
I'll write the missing check now."
# Kiro adds an existing-customer case, re-runs, and the criterion goes covered.
# Kiro authors the intent. Karate proves it.Point Kiro's Power installer at the GitHub URL or a local folder. It wires Kiro to your Karate server over MCP.
The transport is MCP, the same integration Claude Code, Cursor, and Copilot use. Kiro joins the list of supported MCP clients.
The engine and the verdict computation run inside your network. No cloud control plane, no telemetry. Bring your own LLM.
Not just APIs. Karate Agent drives a real browser too, so an acceptance criterion about what your user sees gets the same evidence: API, browser and business-rule checks, one verdict. See Karate Agent's UI automation →
FAQ
Kiro records that a task is done when an agent ticks its checkbox in tasks.md. The tick doesn't record which tests exercised the acceptance criteria the task named, or whether they passed. Karate adds that record: it reads .kiro/specs as the requirement source, joins every criterion to real test runs, and reports each one a completed task claims but no passing check backs.
Point Karate at your project's .kiro/specs folder. It is read in place, nothing is copied or reformatted, and every acceptance criterion becomes a row in a requirements-verification traceability matrix. Write Karate tests as usual (API and UI), and each run grades every criterion as exercised or merely claimed. Install the Karate Power and Kiro itself can drive the loop over MCP. See testing AI-generated code for the broader pattern.
A Power is Kiro's extension format: a POWER.md file, an mcp.json, and steering files. The Karate Power installs from a GitHub URL or a local folder and wires Kiro to a self-hosted Karate server over MCP, so Kiro can ask for the release verdict and act on it.
With the Karate Power installed, yes. Ask Kiro which completed tasks aren't actually verified and it calls Karate's MCP tool, reports the verdict in its own words, and can then write the missing check until the criterion goes covered. Kiro authors the intent, Karate proves it.
The pattern is general: a completion checkbox records a claim, and Karate connects that claim to evidence from test runs. Karate's traceability engine reads requirements from structured spec files, and .kiro/specs support means Kiro's format works in place today. If your tool emits requirements and acceptance criteria as files, the same join applies. See Trust & Governance.
Karate is self-hosted. The engine, the test runs, and the verdict computation all execute inside your infrastructure, with no cloud control plane and no telemetry. AI requests go only to the model provider you choose; pick a self-hosted model and nothing leaves your network. See self-hosted AI testing.
Add deterministic evidence to your Kiro specs: exercised vs. claimed coverage, requirements-verification traceability, and a release verdict a checkbox can't influence.