Conformance Test Kit

Getting started#

Install from npm:

npm install @openinherit/conformance

Or clone from the repository:

git clone https://github.com/openinherit/standard.git
cd standard/packages/conformance

What’s included#

Quick start#

Using the Python runner#

pip install jsonschema
python runners/run-tests.py

Using the bash runner#

The bash runner works with any validator that accepts a file path and outputs JSON with a valid field:

chmod +x runners/run-tests.sh
./runners/run-tests.sh "inherit validate --json"

Invalid fixture catalogue#

Each invalid fixture targets a specific validation rule:

Extension-level fixtures#

Some fixtures in estate/invalid/ are Level 1 valid (they pass the base bundled schema) but contain errors that only surface when the relevant extension schema is also loaded. These have "valid": true in their expected results because the conformance kit ships only the base schema. The note field describes the extension-level error.

If your validator loads extension schemas alongside the base schema, these fixtures should fail. If it validates against the base schema only, they should pass.

Expected result format#

Each .expected.json contains:

{ "valid": true, "schemaMode": "estate" }

or for invalid documents:

{ "valid": false, "schemaMode": "estate", "minErrors": 1 }

Your validator must return a JSON object with at least a valid boolean field. The minErrors field is advisory — use it to verify your validator produces specific errors, not just the right boolean.

Building your own runner#

Read manifest.json to enumerate all tests. For each entry:

1. Load the document at file
2. Validate it using your implementation
3. Load the expected result at expected
4. Assert result.valid === expected.valid

Example: Node.js runner#

import { readFileSync } from 'node:fs';

const manifest = JSON.parse(readFileSync('manifest.json', 'utf-8'));
let passed = 0;
let failed = 0;

for (const test of manifest.tests) {
  const doc = JSON.parse(readFileSync(test.file, 'utf-8'));
  const expected = JSON.parse(readFileSync(test.expected, 'utf-8'));

  const result = yourValidator(doc);

  if (result.valid === expected.valid) {
    passed++;
  } else {
    failed++;
    console.error(`FAIL: ${test.file} — expected valid=${expected.valid}, got ${result.valid}`);
  }
}

console.log(`${passed} passed, ${failed} failed`);

Coverage#

The test fixtures cover:

• Valid and invalid estate documents across multiple jurisdictions (England, Scotland, New York, Singapore, Dubai)
• Multi-currency and multi-script name support
• Religious succession rules (Islamic, Hindu, Jewish)
• Referential integrity validation (Level 2)
• Extension data validation
• Catalogue documents (asset-only mode)

Known lint false positives#

Running jsonschema lint against the bundled schemas produces orphan_definitions warnings. These are false positives — the lint tool cannot resolve #-fragment references across $id boundaries in bundled schemas. Validators resolve them correctly at runtime. Suppress with --exclude orphan_definitions when linting bundled schemas.