Kiro generates the spec and drives an agent to build it. Karate is the evidence layer underneath: it reads .kiro/specs in place as the requirement source, grades every completed task against real test runs, and computes a release verdict a checkbox cannot influence. Complementary by construction.
The report above is public, regenerated by CI, and needs no licence or install.
The category-level gap
Kiro is AWS's flagship spec-driven development IDE. It turns an idea into a spec folder, .kiro/specs/<feature>/{requirements,design,tasks}.md, with EARS acceptance criteria, then drives an agent to implement it. This is a real step up from vibe coding: the intent is written down before the code exists.
But the workflow records completion the only way it can: an agent ticks - [x] in tasks.md. Nothing checks whether any test actually exercises the acceptance criteria that task named. That's not a Kiro flaw; no spec-driven tool verifies its own completion signal. The checkbox is a claim, not evidence.
So the question every team standardising on spec-driven development eventually asks: the tasks are all ticked and the build is green, but did anything check?
How it works
Karate joins what Kiro says should be true to what your tests actually proved. Three steps, all deterministic.
Step 1
Point Karate at .kiro/specs. Nothing is copied, re-authored, or reformatted. Every EARS acceptance criterion in the folder becomes a row in the traceability matrix, and the ticked checkboxes in tasks.md become completion claims to grade.
Step 2
Requirements-verification traceability, computed from execution: each criterion is graded on exercised vs. claimed coverage. A criterion counts as verified only when a passing test actually exercised it, not when a test file mentions it and not when a task claims it.
Step 3
Every criterion a completed task claims but no passing check backs is reported as claimed done, not verified. A high-criticality one is a release blocker. The confidence-to-ship score is computed by versioned code, so a checkbox cannot move it.
This is the Trust & Governance pillar applied to spec-driven development. Kiro's spec folder is the cleanest requirement source we ingest: the criteria are already numbered, already structured, and already joined to tasks. An enterprise standardising on Kiro gets a governance layer for AI-built software without changing how its teams work.
The worked example
The public kiro-demo kit is a small lending service with a spec folder authored in the exact format Kiro emits. It runs in about half a second, no browser, nothing to install but the engine.
What the dashboard says
12 / 12
scenarios passed
API operation coverage: 100%. Every task in tasks.md ticked. A coverage tool would call this finished.
What the evidence says
NOT READY
release verdict
1 criterion marked done by task 5 but not verified: the checkbox is a claim, not evidence.
.kiro/specs/…/tasks.md
- [x] 3. Implement eligibility rules
- [x] 4. Implement term validation
- [x] 5. Implement APR pricing
- [x] 6. Wire up the quote endpointKarate traceability matrix
criterion 5.2 APR tier boundaries
claimed by: task 5 [x]
exercised by: nothing
status: CLAIMED DONE, NOT VERIFIED
criticality: high → release blockerThe criterion is implemented in the service and exercised by no test. That gap is invisible to a green dashboard and to the agent that wrote the code. A green build is not evidence that the work is done.
The Karate Power for Kiro
A Power is Kiro's extension format: POWER.md + mcp.json + steering files. Install the Karate Power from GitHub and Kiro can ask for the verdict itself, over MCP.
# In Kiro's chat:
> Which completed tasks aren't actually verified?
# Kiro calls Karate's MCP tool and reports the verdict in its own words:
"Task 5 is marked complete, but criterion 5.2 (APR tier boundaries)
has no test exercising it. The release verdict is NOT READY.
I'll write the missing check now."
# Kiro writes the test, re-runs, and the criterion goes covered.
# Kiro authors the intent. Karate proves it.Point Kiro's Power installer at the GitHub URL or a local folder. It wires Kiro to your Karate server over MCP.
The transport is MCP, the same integration Claude Code, Cursor, and Copilot use. Kiro joins the list of supported MCP clients.
The engine and the verdict computation run inside your network. No cloud control plane, no telemetry. Bring your own LLM.
FAQ
No spec-driven development tool verifies its own completion signal. Kiro records that a task is done when an agent ticks its checkbox in tasks.md; nothing in that workflow checks whether any test exercises the acceptance criteria the task named. Karate adds that layer: it reads .kiro/specs as the requirement source, joins every criterion to real test runs, and reports each one a completed task claims but no passing check backs.
Point Karate at your project's .kiro/specs folder. It is read in place, nothing is copied or reformatted, and every acceptance criterion becomes a row in a requirements-verification traceability matrix. Write Karate tests as usual (API and UI), and each run grades every criterion as exercised or merely claimed. Install the Karate Power and Kiro itself can drive the loop over MCP. See testing AI-generated code for the broader pattern.
A Power is Kiro's extension format: a POWER.md file, an mcp.json, and steering files. The Karate Power installs from a GitHub URL or a local folder and wires Kiro to a self-hosted Karate server over MCP, so Kiro can ask for the release verdict and act on it.
With the Karate Power installed, yes. Ask Kiro which completed tasks aren't actually verified and it calls Karate's MCP tool, reports the verdict in its own words, and can then write the missing check until the criterion goes covered. Kiro authors the intent, Karate proves it.
The gap is category-level: any tool that records completion with a checkbox has a completion signal nothing verifies. Karate's traceability engine reads requirements from structured spec files, and .kiro/specs support means Kiro's format works in place today. If your tool emits requirements and acceptance criteria as files, the same join applies. See Trust & Governance.
No. Karate is self-hosted and bring-your-own-LLM. The engine, the test runs, and the verdict computation all execute inside your infrastructure. There is no cloud control plane and no telemetry. See self-hosted AI testing.
Add deterministic evidence to your Kiro specs: exercised vs. claimed coverage, requirements-verification traceability, and a release verdict a checkbox can't influence.