133 of 646 real-world JSON Schemas contain keywords that do nothing
Every schema in the SchemaStore catalogue was validated against its own declared meta-schema and linted. The most common defect is one that raises no error anywhere: validation keywords placed beside a $ref, where the specification says they are ignored.
Keywords next to $ref are silently discarded
In draft-07 — which 592 of these 646 schemas declare — $ref replaces the entire schema object. Every sibling keyword is ignored. Write {"$ref": "#/$defs/user", "minProperties": 3} and the minProperties constraint does absolutely nothing.
No validator warns you. The schema is valid, it validates documents, and one of your rules is quietly absent.
133 of 646 schemas (21%) do this at least once. Including tsconfig.json.
| Schema | Sites where a keyword sits beside $ref |
|---|---|
| Dein Config | 10 |
| Drupal Recipe | 10 |
| Code Climate | 10 |
| cloudify | 10 |
| coffeelint.json | 10 |
| Detekt Config (detekt.yml) | 10 |
| Foundry VTT - Base package Manifest | 10 |
| GitHub Workflow | 10 |
The catalogue is still overwhelmingly draft-07
| Declared draft | Schemas | Share |
|---|---|---|
| draft-07 | 592 | 92% |
| 2020-12 | 31 | 5% |
| draft-04 | 17 | 3% |
| 2019-09 | 4 | 1% |
| none declared | 2 | 0% |
draft-07 was published in 2018. 2020-12 has been out for years and remains a small minority.
What else the audit turned up
1 schema fails to validate against its own declared meta-schema.
A number of schemas use editor vocabulary that is not JSON Schema at all — markdownDescription, defaultSnippets, deprecationMessage, allowTrailingCommas. VS Code reads them; a generic validator ignores them. That is expected, not a bug, but it is worth knowing which of your constraints are editor hints rather than enforced rules.
A handful contain genuine keyword typos — misspellings that a validator silently skips.
The validator reports this
Our schema validator flags every one of these: $ref siblings, draft mismatches, unknown keywords and unchecked formats. It says what it is not enforcing rather than returning a bare green tick. Browse the audited schemas in the schema registry.
Standards referenced
The rule that a sibling keyword beside $ref is ignored comes from the pre-2019-09 drafts, where $ref replaced the whole schema object.
- JSON Schema Core, draft 2020-12 §8.2.3.1 — Direct References with "$ref" — JSON Schema: A Media Type for Describing JSON Documents, A. Wright, H. Andrews, B. Hutton, G. Dennis, Eds., 2022.
- JSON Schema Core, draft 2020-12 §8.1.1 — "$schema" — JSON Schema: A Media Type for Describing JSON Documents, A. Wright, H. Andrews, B. Hutton, G. Dennis, Eds., 2022.
- RFC 6901 §5 — JSON String Representation — JavaScript Object Notation (JSON) Pointer, P. Bryan, Ed., K. Zyp, M. Nottingham, Ed., 2013.
Only standards bodies and peer-reviewed venues are cited here.
How this was measured
All 646 schemas in the SchemaStore catalogue that resolve to a file were parsed, validated against their declared meta-schema, and linted. Results were cross-checked against Ajv, which agreed on every schema our engine fully supports.
Every figure on this page came from executing the parser named, not from documentation. The collection harness is public — see the error registry method notes.