Test Data Generation

Test data that knows the answer.

Boundary, pairwise, and covering-array decks generated from your executable business rules. Every row carries the computed expected result, an audit of how it was reached, and the requirement it proves.

Two things this is not: it is not masking, subsetting, or environment provisioning, the classic TDM job. And it is not fake data. Faker and Mockaroo manufacture plausible inputs; every row of ours carries the computed answer.

rating-deck.json
// boundary row: 17|18 learned from the rules
{
  "driver_age": 17,
  "vehicle_class": "sport",
  "expected_result": {
    "eligible": false,
    "reason": "age below minimum for class"
  },
  "requirements": ["REQ-114"]
},
{
  "driver_age": 18,
  "vehicle_class": "sport",
  "expected_result": {
    "eligible": true,
    "premium": 2417.50
  },
  "requirements": ["REQ-114", "REQ-201"]
}

The Problem

Every tool generates the question. The answer stays your problem.

The testing literature has a name for it: the oracle problem. Tools got very good at producing inputs. What the system should answer for each input still lives in hand-kept spreadsheets and someone's head.

TDM platforms Provision, subset, and mask records and environments. Necessary work, and none of it computes an expected result.
Synthetic generators Manufacture plausible inputs at volume. The oracle stays in your test case.
Pairwise and combinatorial tools Optimize which input rows to run, constraints included, and tools like PICT and ACTS do it well and free. Still no expected results.
Model-based testing The honest near-miss: MBT tools do derive expected results and requirement links, from a model you hand-build and maintain beside the system, forever.
Karate Partitions learned from your executable rules running, every row with its computed answer and requirement link, in one framework-agnostic deck. No second model to maintain.

Most test-data tools stop at the input. Every row in our deck includes the rulebook-computed answer, and the requirement it proves.

The Partitions

Learned, not declared

Boundary value analysis and equivalence partitioning are decades old. What made them expensive was declaring the partitions by hand, and keeping the declarations honest as the rules changed.

The classic way

A test designer reads the spec, decides that ages 17 and 18 sit on a boundary, and writes it down. If the underwriting table moves the boundary to 21 next quarter, the declaration is now quietly wrong, and no tool will tell you.

The Karate way

The partitions are learned by observing the executable rules run. Equivalence classes and boundaries are discovered, not declared, including the ones buried in lookup tables that no static reading of the spec would surface. When the rules change, the next run learns the new boundaries.

This is also why we do not call it model-based testing. There is no flowchart to hand-build, no state model to keep in sync beside the system, no second artifact to drift. The rules are executable, so the rules are the model. What you maintain is the one artifact your business already owns: the logic itself.

The Deck

What every row carries

A deck is a set of test rows in the canonical data-driven-testing shape. Four things travel on every row.

01

The inputs

Chosen deliberately: boundary values, equivalence-class representatives, pairwise and covering-array combinations.

02

The expected result

Computed by running the rulebook on this row's inputs. The column every data-driven-testing tutorial tells you to fill in by hand.

03

The audit trail

A plain-English account of how the answer was reached, so a reviewer can adjudicate a disagreement without reading code.

04

The requirement ids

Every row names the requirements it exercises, so a passing row is evidence against a requirement, not just a green line.

Pairwise and covering arrays, pruned for feasibility

The generator produces boundary and equivalence-class decks, pairwise decks, and higher-strength covering arrays, with the strength selected by the business criticality you assign. Combinations your rules make infeasible are pruned rather than generated and wasted, so the deck is the minimal set that actually earns the coverage it claims.

Framework-Agnostic

A deck your framework already knows how to run

The deck is plain JSON or CSV: input columns plus an expected-result column. That is the exact table every data-driven-testing tutorial teaches, for every framework. Karate generates it; anything can run it, with zero Karate in the loop.

Playwright

import deck from './rating-deck.json';

for (const row of deck) {
  test(`rates ${row.case_id}`,
    async ({ request }) => {
    const res = await request
      .post('/quote', { data: row.inputs });
    expect((await res.json()).premium)
      .toBe(row.expected_result.premium);
  });
}

Postman

// collection runner + rating-deck.csv
const json = pm.response.json();

pm.test("premium matches the deck",
  () => {
  pm.expect(json.premium).to.eql(
    Number(pm.iterationData
      .get("expected_premium")));
});

REST-assured

// rows from rating-deck.json
@Test(dataProvider = "deck")
public void rates(Row row) {
  given().body(row.inputs())
  .when().post("/quote")
  .then().body("premium",
    equalTo(row.expected("premium")));
}

Decks are served as JSON from the API and MCP surface today, and one-command CSV and JSON file export is shipping this quarter. Teams that will never run Karate still get the deck. The deck is the on-ramp; what follows is why the traceability is the upsell.

The Upsell

Rows that light up the traceability matrix

Because every row names the requirements it exercises, running the deck is not just a pass count. It is evidence, joined to intent.

Run the deck inside Karate and each passing row becomes real execution evidence against the requirement it names: the traceability matrix updates, coverage is graded, and a requirement whose rows fail turns up in the release verdict with the failing input attached. The same deck that your Playwright suite runs as data becomes, on our rails, part of the answer to "is this release safe to ship?"

FAQ

Common questions

What is test data management vs test data generation?

TDM provisions: it subsets databases, masks sensitive fields, and delivers environments. Generation designs the cases. Karate generates, and adds the piece the market leaves out: the computed expected result and the requirement link on every row. If you need masking or subsetting, that is a different product, and we will say so.

How do I generate expected results for test cases?

Make the rules executable. The rulebook is an executable model of your decision logic, so producing a row means running the rules on its inputs and recording the answer, with an audit trail. No spreadsheet, no hand-kept oracle.

What is pairwise testing?

A combinatorial technique: a small set of rows that covers every pair of input values at least once, because most defects come from the interaction of one or two inputs. Karate generates pairwise decks and stronger covering arrays, prunes infeasible combinations, and attaches the expected result the dedicated pairwise tools cannot supply.

Can I use this with Playwright or Postman without Karate?

Yes. The deck is plain JSON or CSV in the shape those tools already consume for data-driven testing. Generate the deck, hand it to your framework, and run it with zero Karate in the loop.

How is this different from Faker or Mockaroo?

They make plausible inputs. We make deliberate cases with computed answers. Plausible data fills a database; a deck with expected results proves behavior.

The deck knows the answer.

Generated from the rules your business already owns, runnable by any framework, and traceable to the requirements it proves.