Most API governance stops at the document, because that is all a document linter can see. Karate grades the document and the execution evidence around it, deterministically.
No model. No network. The same spec and the same rules always produce the same grade, and anyone can re-derive it.
Is this operation claimed by a requirement?
Or does it exist because someone added it once?
Was it ever exercised by a run?
Documented is not the same as tested.
Did anything other than our own mock answer it?
A stand-in answering your test is not evidence about the service.
The Ceiling
A conformant API and a governed API are not the same thing, and neither can be established by reading a document.
Karate runs all of these. They are worth having. They are not the interesting part.
These need execution data. A ruleset can only ever ask questions the document can answer about itself, which is why a linter will call a spec fine when nobody has ever called half of it.
The Rulepack
Every finding carries a severity, a message, and the exact JSON pointer plus the line it lands on.
A rule whose selector cannot pin a position says nothing rather than guessing at one. You never chase a finding to a line that has nothing to do with it.
Same document, same rulepack, same findings. No model in the loop, so there is nothing to take on trust and nothing that drifts between runs.
You cannot game the grade by declaring a security scheme. An operation that requires auth should declare its 401 and 403, so declaring the scheme raises the findings that come with it.
Already running a style guide? Keep it. The linting page covers how this fits alongside the rules you already have, and Spectral ruleset import is shipping this quarter.
The Grade
The lint is scored across five dimensions into a single index, a ship verdict, and a named maturity level.
Structure, naming, and consistency across the declared surface.
Declared schemes, transport, and the error responses that auth implies. Bundled rules only, and deliberately not a security posture grade: it reads the document, not your running system.
Descriptions, contacts, licensing, and examples that validate.
Is it required by anyone, was it ever run, and did something other than a mock answer.
How far the paired runs went. This is contract testing as a graded dimension.
A dimension with no evidence behind it reads not measured. It never scores zero, and it never quietly disappears from the mean. Without a run behind it, proven-ness says so, instead of scoring full marks for never having been asked.
What was not measured is reported, not renormalised away. Zero of zero reads not checked, never passed.
How The Number Is Built
A cap binds a dimension. A gate decides the verdict and never touches the number.
Each dimension is scored from its findings.
A binding rule holds a dimension down, and names itself.
A weighted harmonic mean of the dimensions that had evidence.
The verdict is decided. The index is not altered to fit it.
The maturity level follows from what the evidence supports.
One unmet rule is holding this dimension at a fifth of the score the rules alone would have given it.
Score the rules alone would give
49%
Capped at 20% by security-scheme-declared
20%
The hatched area is the score the dimension would have reached. Keeping it visible is the point: you can see what a single unmet rule is costing you, and which rule to fix first. Declaring that scheme is seven lines, and it will raise fresh findings of its own, because an operation that requires auth then owes you a 401 and a 403.
READY
No gate is failing
CONDITIONAL
Ship with the findings acknowledged
NOT_READY
A gate is failing outright
Maturity
You cannot write your way to the top of this ladder. That is the join no spec linter can make.
Something other than your own mock has answered, and the two were compared.
The operations were actually called by a run, not merely described.
Operations are claimed by requirements, so you know who needs them.
The document meets the standard the organisation set for it.
There is a document, and it says the basic things a document should.
The starting point. Nothing has been established either way.
See It For Yourself
Our own reference kit grades badly, and we publish it anyway. That is more useful to you than a demo tuned to score well.
One error-severity finding stands unwaived, and one more gate is open. 31 rules ran. One dimension had no evidence to grade.
The contract dimension carries a declared weight and could not be measured, so it reads not measured. It is not scored zero, and it is not silently dropped. The remaining weights are renormalised across the measured dimensions, and the report states plainly that it did so, because a mean has to be taken over something and hiding that would be the dishonest part.
Design scores 96%. A document linter would call this spec fine. Proven-ness scores 34% at the heaviest declared weight of the five, and it is asking whether operations are claimed by a requirement, whether anything ever ran them, and whether anything but a mock answered. The gap between 96% and 34% is the argument of this page, measured.
No model produced any number above. The payload carries that fact explicitly, so nobody has to take our word for it, and anyone can re-derive the grade from the same document and the same rulepack.
Two Different Questions
Is it well designed, properly documented, and actually exercised? That is this page. The unit is the API.
You are here
Requirements joined to tests joined to a verdict. The unit is the release, and the audience is whoever signs off on it.
Release governanceMost tools tell you the API matches its description. Karate tells you whether anyone needed it, whether anything ran it, and whether your stand-in can actually replace it.