# JSON Query Syntax Reference

The Resource Registry API provides a powerful JSON-based Query-By-Example (QBE) engine. This syntax allows clients to perform deep topological matching, graph traversal, and response projection without writing raw backend SQL.

The engine is universally accessible across all collections via their respective execution endpoints and read-only mirrors:
- **Contexts:** `QUERY /contexts` and `QUERY /access/contexts`
- **Schema Types:** `QUERY /types` and `QUERY /access/types`
- **Instances:** `QUERY /instances` and `QUERY /access/instances`
- **Query Templates:** `QUERY /query-templates` and `QUERY /access/query-templates`

This document serves as the formal reference manual for constructing these JSON queries.

## Core Principles

- **Query-By-Example:** The structure of the JSON query mimics the structure of the desired output. You nest objects (e.g., `consistsOf`, `isRelatedTo`) exactly as they appear in a standard representation.
- **Mandatory Root `type`:** Every JSON query must explicitly declare the `"type"` keyword at its root. This instructs the query engine on which collection/class to scan.
  - When querying Contexts, the type is structurally fixed (e.g., `"type": "Context"`).
  - When querying Schema Types, the type is one of the queryable Type-definition categories: `"type": "ResourceSchema"`, `"FacetSchema"`, `"IsRelatedToSchema"`, `"ConsistsOfSchema"`, or `"PropertySchema"` (not the concrete schema name being defined — e.g. to find the `EService` schema definition, query `{"type": "ResourceSchema", "name": "EService"}` with `?depth=0`, not `{"type": "EService"}`; without `depth=0` the match also includes every descendant of `EService`).
  - When querying Instances, the type is the specific schema name (e.g., `"type": "EService"` or `"type": "HostingNode"`).
  - When querying Templates, the type is `"type": "QueryTemplate"`.
- **Mandatory Nested `type`:** Because the engine matches the query tree exactly against the database schema, the `"type"` keyword is **strictly mandatory at every level that describes an instance**: the root, every element of a `consistsOf` or `isRelatedTo` array, their `source` or `target` objects, and the `_source` object. Omitting it returns `400 Bad Request`. It is not used for embedded values, such as the elements of a List of embedded values or a nested **`Property`** value, which are addressed by field name only. Written inside a nested value, or at the end of a dotted path (`"authorization": { "type": "SecurityScheme" }`, `"authorization.type": "SecurityScheme"`), `type` is a predicate on the Type of that value: it matches the named Type and its subtypes, as the `type` of a level does, and an unknown Type name returns `400 Bad Request`. *(This is unrelated to `depth`-based `name` matching within Type-definition queries — see below — which is a distinct mechanism scoped to the Types collection only.)*
- **System Keywords (`_` Prefix):** To avoid namespace collisions with user-defined property fields (Open-World paradigm), all query operators, traversal commands, and projection directives strictly begin with an underscore `_` (e.g., `_emit`, `_and`, `_eq`, `_source`).

## Depth-Based Type-Name Matching (Type-Definition Queries Only)

This section applies exclusively to `QUERY /types` (and its `/access` mirror), i.e. queries whose root `"type"` is one of the Type-definition categories (`ResourceSchema`, `FacetSchema`, `IsRelatedToSchema`, `ConsistsOfSchema`, `PropertySchema`). It does not apply to Instance queries and Query Template execution, where `polymorphic` instead controls entry-point subclass expansion, nor to Context and Query Template definition queries, which have no Type hierarchy to expand (see [`polymorphic`](../11-api-query-parameters.html#polymorphic)); the Types collection deliberately reuses `depth` instead, for consistency with `GET /types/{type-name}` within the same collection (see [`depth`](../11-api-query-parameters.html#depth)).

Type definitions form a flat collection: each one is distinguished only by its own `name` and its declared `extendedTypes`. Consequently, an equality match against a Type name — the query's own `name` field, or the `source`/`target` name inside a Relation Type definition — is resolved against the **descendant closure of that name up to `depth` levels** (the same Type-hierarchy walk that powers `GET /types/{type-name}?depth=N`), not just the literal name. `depth=0` restricts the match to the exact name only; a positive `depth=N` includes descendants up to N levels down; `depth=-1` includes all descendants recursively. On `QUERY /types`, `depth` defaults to `-1` (polymorphic-equivalent matching, consistent with the `polymorphic=true` default on Instance queries and Query Template execution — see the [`depth`](../11-api-query-parameters.html#depth) section for the full default-value rationale).

*Example — `depth`-based `source`/`target` matching on a Relation Type:*
```json
{
  "type": "IsRelatedToSchema",
  "source": "Service",
  "target": "Service"
}
```
With the default `depth=-1` (or an explicit `?depth=-1`), this matches `Uses` even though `Uses` is declared between two `EService` instances (a subtype of `Service`), because `EService` is a descendant of `Service`. With an explicit `?depth=0`, only Relation Type definitions declared with the *literal* source and target name `Service` are matched.

*Example — `depth`-based `name` matching combined with a structural constraint:*
```json
{
  "type": "FacetSchema",
  "name": "Facet",
  "properties": { "name": { "_contains": "name" } }
}
```
`Facet` is the abstract base schema and declares no `properties` of its own. With the default `depth=-1` (or an explicit `?depth=-1`), the `name` match expands to `Facet` and every registered descendant (e.g. `ContactFacet`), and any other constraint in the same query — such as this `properties` check — is then evaluated independently against each candidate in that expanded set, using the collection operators described under [Collection and Map Operators](#collection-and-map-operators): `properties` is a list of embedded values, so the nested path `properties.name` denotes the names of all its elements. Result: every Facet Type definition (base or descendant) that declares a property named `name`, e.g. `ContactFacet`. With an explicit `?depth=0`, this query returns an **empty array** instead (the literal `Facet` record does not have a property named `name`).

## Response Projection (`_emit`)

By default, a query returns the full representation of the matched instances. To optimize payloads and return only specific fields (GraphQL style), use the `_emit` block.

The `_emit` object acts as an alias map: the keys are the fields to return, and the values are the custom aliases you want returned in the final JSON response.

A field, in `_emit` as in a predicate, is named by its dot-separated path (`metadata.createdBy`); one element of a list is named by its position, starting from zero (`domain[0]`); the Type of an element is named `type`, as in its representation, and `type` at the end of a path names the Type of that nested value (`metadata.type` is `Metadata`). Any other form of name returns `400 Bad Request`, and so do a name containing `@`, the reserved names `in` and `out`, and any name starting with `in_` or `out_`. A projected field that has no value is returned as `null`.

A projected embedded value is returned as a read returns it, a whole `metadata` with its identity fields redacted for the roles that see them so; an `_emit` key that names an instance rather than a value returns `400 Bad Request`.

### Sensitive Metadata Predicates

Every query is evaluated against the caller-visible logical representation produced by privacy authorization. This privacy view is established before and independently of response shaping: `metadataPolicy`, `_emit`, and other projection controls determine which fields are returned, not which privacy-transformed values predicates and operators consume. `IS-M`, `Infra-M`, and `IS-Admin` therefore compare the stored values of `metadata.createdBy` and `metadata.lastUpdateBy`; every other role compares the same `"[REDACTED]"` strings that a direct read or `_emit` projection exposes. This rule applies to direct, nested, and logical predicates and to projection, and consequently to result counts and pagination. A real identity never matches a value hidden from the caller, while `"[REDACTED]"` may match it. Referencing a sensitive field does not itself return `403 Forbidden`.

The engine must not evaluate a predicate against the stored secret and then remove or redact matching rows. Result membership, counts, and pagination boundaries can all disclose whether a hidden value matched. Privacy transformation is logically applied before query evaluation, even when the physical execution plan optimizes the operation without materializing an intermediate representation.

### Canonical `_emit` Precedence Rules
This subsection is **normative** and serves as the single source of truth for `_emit` behavior across direct JSON queries and stored query-template execution:
1. **Projection-first behavior:** When `_emit` is present, the response is strictly projection-based (selected fields/aliases) and will not return the full instance representation.
2. **Metadata/context expansion disabled:** Structural inclusion parameters such as `metadataPolicy`, `contextsPolicy`, and `membershipScope` are completely ignored and have no effect when `_emit` is used (a `membershipScope=global` from a role not entitled to it still returns `403 Forbidden`). `contexts` is computed on every read and is not a stored field, so it can be neither projected by `_emit` nor used in a predicate; either use returns `400 Bad Request`. A property of type **`Encrypted`** can be projected by `_emit`: as in a read, its value is returned encrypted with the key of the Context of the request. It is projected as a whole: an `_emit` key that names a field inside it returns `400 Bad Request`. It cannot be used in any predicate (`400 Bad Request`): its caller-visible form is a ciphertext that differs per row and per Context, so a comparison has no meaning, and allowing one would let a client test candidate secrets until a row matches. `metadata.createdBy` and `metadata.lastUpdateBy` differ: for a caller who sees them as `"[REDACTED]"` they are one constant value, so predicates on them are evaluated against that constant (see [Sensitive Metadata Predicates](#sensitive-metadata-predicates)).
3. **Privacy masking still enforced:** If `_emit` projects sensitive fields like `metadata.createdBy` or `metadata.lastUpdateBy` (under any alias), their values are strictly masked as `"[REDACTED]"` for roles without full metadata visibility (`Ctx-M`, `IS-Editor`, `IS-Contrib`, `IS-Member`).
4. **Alias-agnostic enforcement:** Privacy masking is applied based on the true source of the projected field, regardless of the custom alias name chosen by the client.
5. **Scope of application:** These rules apply uniformly to all query engines (`QUERY /{collection}`, `QUERY /query-templates/{name}`, their respective `/access` mirrors, and the corresponding `POST .../query` compatibility fallbacks).

### Projecting Context Fields
```json
{
  "type": "Context",
  "effectiveState": "suspended",
  "_emit": {
    "id": "ContextUUID",
    "name": "ContextName",
    "path" : "ContextPath"
  }
}
```

### Projecting Type Schema Fields
```json
{
  "type": "ResourceSchema",
  "_emit": {
    "id": "UUID",
    "name": "Name",
    "description": "Description"
  }
}
```

### Projecting Instance Fields
```json
{
  "type": "EService",
  "_emit": {
    "id": "ServiceID",
    "type": "ElementType"
  }
}
```

### Deep Projection
You can place `_emit` blocks at any nested level of the query to project fields from related elements into a flat response object.

```json
{
  "type": "EService",
  "consistsOf": [
    {
      "type": "IsIdentifiedBy",
      "target": {
        "type": "SoftwareFacet",
        "_emit": {
          "name": "SoftwareName",
          "version": "SoftwareVersion"
        }
      }
    }
  ]
}
```

### Projecting QueryTemplate Fields
```json
{
  "type": "QueryTemplate",
  "_emit": {
    "name": "TemplateName",
    "description": "Description",
    "version": "Version"
  }
}
```

## Computed Values

A projection can return a value computed from the fields of one element. A computing operator is a key of the query level of that element, next to `_emit` or in its place, and like `_emit` it makes the response projection-based.

| Operator | Computes |
|---|---|
| `_sum` | Sum of numbers; concatenation when an operand is a string or a `separator` is given |
| `_minus` | Difference |
| `_multiply` | Product |
| `_divide` | Quotient |
| `_mod` | Remainder |
| `_bitrshift`, `_bitlshift` | Bitwise right and left shift |
| `_bitand`, `_bitor`, `_bitxor` | Bitwise AND, OR, and XOR |
| `_arrayconcat` | Concatenation of collections |

The value of the operator is an object with these members:

- `values` (mandatory): a non-empty array of operands, applied from left to right. An operand is the name of a field of the element, under the field-name rule of `_emit`, or a nested computation in the form `{ "<operator>": [ operands ] }`, which is evaluated first.
- `as` (mandatory): the alias under which the result is returned.
- `separator` (optional, `_sum` only): a string placed between consecutive operands, which makes the result their concatenation.

Any other member, and a `separator` on an operator other than `_sum`, returns `400 Bad Request`.

An operand whose field is absent or `null` does not contribute to the result. An operand that names `metadata.createdBy` or `metadata.lastUpdateBy` contributes `"[REDACTED]"` for the roles that see those fields so. A missing or mistyped `values`, `as`, or `separator`, an empty `values`, an operand that is neither a field name nor a nested computation, an operand that names an **`Encrypted`** property, and a computing operator inside a logical operator return `400 Bad Request`.

```json
{
  "type": "MemoryFacet",
  "_sum": {
    "values": [
      { "_minus": [ "size", "used" ] },
      "unit"
    ],
    "separator": " ",
    "as": "SpaceLeft"
  }
}
```

For a **Facet** instance with `size` 1024, `used` 512, and `unit` `GB`, the row is `{ "SpaceLeft": "512 GB" }`.

## Logical Operators

The engine supports explicitly grouping constraints using `_and`, `_or`, and `_not`. Their operand shapes are fixed:

| Operator | Operand | Empty operand |
|----------|---------|---------------|
| `_and` | A JSON array of condition objects; all must match. An object operand is invalid (`400 Bad Request`) | `[]` is vacuously true (no filter) |
| `_or` | A JSON array of condition objects; at least one must match | `[]` returns `400 Bad Request` |
| `_not` | A single condition object, which must not match. An array operand is invalid (`400 Bad Request`) | `{}` returns `400 Bad Request` |

An empty relationship array (`"consistsOf": []` or `"isRelatedTo": []`) adds no constraint: it does not mean "no relationship".

A condition that adds no constraint (for example `{"tags": {"_containsAll": []}}`) is true: `_and` ignores it, it makes `_or` true, and it makes `_not` false.

A logical operator combines conditions on the properties of the object it appears in. The relationship fields (`consistsOf`, `isRelatedTo`, `source`, `target`, `_source`) and `_emit` cannot appear inside it: a condition that carries one of them returns `400 Bad Request`.

### `_or` (Match ANY condition)
Use `_or` when you need to match across distinct fields (for simple multi-value matching on the same field, prefer the `_in` operator).
```json
{
  "type": "SoftwareFacet",
  "_or": [
    { "name": "gcat" },
    { "group": "data-catalogue" }
  ]
}
```

### `_and` (Match ALL conditions)
Normally, providing multiple fields in a JSON object acts as an implicit AND. However, you can use `_and` for complex groupings.
```json
{
  "type": "EService",
  "_and": [
    { "id": "1a50a46d-2210-4a17-bd2f-22450b109264" },
    { "metadata": { "createdBy": "luca.frosini" } }
  ]
}
```

### `_not` (Match when the condition does NOT hold)
```json
{
  "type": "StateFacet",
  "_not": {
    "value": "down"
  }
}
```

## Comparison Operators

Providing a non-null scalar directly (for example, `"name": "gcat"`) is shorthand for `_eq`. A direct `null` (for example, `"name": null`) matches a property whose value is null; to match a non-null value, negate it: `"_not": { "name": null }`. An operator object may contain more than one compatible operator, which are combined with logical AND. Operands use native JSON types and are never coerced; inside an operator object, null operands are invalid. A shape, type, cardinality, bound-order, or regular-expression violation returns `400 Bad Request`. So does a JSON array given directly as the value of a property (for example, `"extendedTypes": ["Service"]`), because a collection is compared through the collection operators, and a comparison operator placed directly in an instance object, because an operator is always the value of a property.

In the tables below, examples are property fragments to be placed inside an otherwise valid query. “Ordered scalar” means a number, `String`, or `Date`; booleans, collections, and maps are not ordered. Numeric comparisons are algebraic, generic strings lexicographical, and Dates chronological after strict parsing with `yyyy-MM-dd HH:mm:ss.SSS Z`. For a property the Type definition does not declare, two `_between` string bounds that both parse as Dates are ordered chronologically, any other pair of strings lexicographically.

### Scalar and Presence Operators

| Operator | Exact operand and property domain | Null / empty semantics | Valid fragment | Invalid fragment |
|---|---|---|---|---|
| `_eq` | One non-null scalar compatible with the property. | Use a direct `null` to test null. | `"state": {"_eq": "ready"}` | `"state": {"_eq": null}` |
| `_ne` | One non-null scalar compatible with the property. | Use a direct `null` to test null. | `"state": {"_ne": "down"}` | `"state": {"_ne": ["down"]}` |
| `_gt` | One compatible ordered scalar. | Null is invalid. | `"load": {"_gt": 10}` | `"active": {"_gt": false}` |
| `_gte` | One compatible ordered scalar. | Null is invalid. | `"load": {"_gte": 10}` | `"load": {"_gte": "10"}` |
| `_lt` | One compatible ordered scalar. | Null is invalid. | `"name": {"_lt": "M"}` | `"name": {"_lt": null}` |
| `_lte` | One compatible ordered scalar. | Null is invalid. | `"when": {"_lte": "2026-10-05 12:00:00.000 +0200"}` | `"when": {"_lte": "2026-10-05"}` |
| `_between` | A two-element JSON array `[lower, upper]` of non-null values from the same ordered type family, with `lower <= upper`. Bounds are inclusive. | Empty, one-element, or overlong arrays are invalid. | `"load": {"_between": [10, 30]}` | `"load": {"_between": [30, 10]}` |
| `_like` | One JSON string on a String property; `%` means any sequence and `_` one character. | Empty string is a valid literal pattern; null is invalid. | `"name": {"_like": "Luk%"}` | `"name": {"_like": 12}` |
| `_matches` | One syntactically valid regular-expression string on a String property. | Empty expression is valid; null is invalid. | `"email": {"_matches": "^[^@]+@[^@]+$"}` | `"email": {"_matches": "["}` |
| `_isDefined` | The literal JSON boolean `true`; valid for every property type. | Takes no value beyond the required marker; `false` and null are invalid. | `"name": {"_isDefined": true}` | `"name": {"_isDefined": false}` |
| `_isNotDefined` | The literal JSON boolean `true`; valid for every property type. | Takes no value beyond the required marker; `false` and null are invalid. | `"legacy": {"_isNotDefined": true}` | `"legacy": {"_isNotDefined": null}` |

### Collection and Map Operators

**Lists of embedded values:** When a List or Set property holds embedded values (for example the `properties` list of a Type definition), nesting a field name under the property addresses the collection of the values that field takes across all the elements. Collection operators then apply to that nested path: `"properties": {"name": {"_contains": "version"}}` matches when at least one element of `properties` has `name` equal to `version`. A constraint cannot relate two fields of the same element.

A scalar comparison on a collection can never match, so the validator rejects it with `400 Bad Request` whenever the Type definition declares the property as a List or Set: this covers a plain value (`"properties": {"name": "version"}`) and every scalar operator. The check applies to every query endpoint, to the saving of a Query Template (on the query produced by its defaults), and to its execution. A property the Type definition does not declare has no known type: the check cannot apply to it, and a scalar comparison on a stored collection simply matches nothing.

Array operands for `_in`, `_containsAll`, and `_containsAny` may contain at most 1,000 elements. Every element must be non-null and compatible with the compared scalar or the collection's declared element type; mixed incompatible domains are invalid.

| Operator | Exact operand and property domain | Empty-collection semantics | Valid fragment | Invalid fragment |
|---|---|---|---|---|
| `_in` | A JSON array applied to a scalar property; the property must equal at least one element. | `[]` is valid and always false. | `"state": {"_in": ["ready", "down"]}` | `"state": {"_in": "ready"}` |
| `_contains` | One non-null scalar applied to a List or Set property. | An empty stored collection does not match. | `"tags": {"_contains": "blue"}` | `"tags": {"_contains": ["blue"]}` |
| `_containsAll` | A JSON array applied to a List or Set property; every operand element must occur in the property. | `[]` is valid and vacuously true. | `"tags": {"_containsAll": ["blue", "green"]}` | `"tags": {"_containsAll": "blue"}` |
| `_containsAny` | A JSON array applied to a List or Set property; at least one operand element must occur in the property. | `[]` is valid and always false. | `"tags": {"_containsAny": ["blue", "green"]}` | `"tags": {"_containsAny": ["blue", null]}` |
| `_containsKey` | One non-null JSON string applied to a Map property. | An empty stored map does not match. | `"labels": {"_containsKey": "env"}` | `"labels": {"_containsKey": 3}` |
| `_containsValue` | One non-null scalar compatible with the Map's declared value type. | An empty stored map does not match. | `"weights": {"_containsValue": 3}` | `"weights": {"_containsValue": [3]}` |

An operator applied to the wrong property domain is invalid even when its JSON shape is otherwise correct. Unknown `_`-prefixed operators are syntax errors, never custom property names.

## Inverse Traversal (`_source`)

Standard queries traverse the graph forwards (e.g., from a **Resource** instance to its **ConsistsOf** instances to the **Facet** instances). 
However, if your query's entry point (root node) is a **Facet** instance, you can traverse *backwards* to find the **Resource** instance that owns it by using the `_source` keyword. A query whose entry point is a **Relation** instance reaches its endpoints with `source` and `target`.

*(Note: `_source` is strictly permitted only on the entry-point object of a JSON query whose entry point is a **Facet** instance. You cannot use it deep within a forward-traversal chain: on any other object, the entry point of a **Relation** query included, it returns `400 Bad Request`).*

### Example: Finding the EService that owns a specific StateFacet
```json
{
  "type": "StateFacet",
  "value": "down",
  "_source": {
    "type": "ConsistsOf",
    "source": {
      "type": "EService",
      "_emit": {
        "id": "ServiceUUID"
      }
    }
  }
}
```

## Query Templates (Variables)

```eval_rst
When the JSON query is saved as a ``QueryTemplate`` (via ``PUT /query-templates/{name}``), you can insert template variables prefixed with \$. These variables are dynamically substituted at execution time based on the JSON body provided to ``QUERY /query-templates/{name}``.
```

The declaration schema, accepted JSON value domains, defaults, requiredness, null and coercion rules, and validation phases are normative in the [`TemplateVariable` Contract](../06-data-representation-instances-envelopes.html#templatevariable-contract). Substitution recursively visits the complete query value tree and replaces only a string node whose entire value equals a declared variable name. It does not perform partial string interpolation, rescan inserted values, or substitute object keys.

The execution algorithm is deterministic: validate the body as a JSON object, ignore and warn about undeclared input keys, resolve each declared variable from a supplied value or its permitted default, validate the resolved native JSON value, perform recursive whole-node substitution, validate the resulting query, and execute it. Any missing `required=true` value or invalid resolved value returns `400 Bad Request` before query execution.

**Target Limitation:** The Query Template mechanism is structurally restricted to **Instances**. Therefore, the root `"type"` in a saved template must always specify a valid **Entity** or **Relation** type (e.g., **Resource**, **EService**). Query templates cannot be used to save queries targeting Contexts, Type Schemas, or other Query Templates.

```json
{
  "type": "EService",
  "consistsOf": [
    {
      "type": "ConsistsOf",
      "target": {
        "type": "StateFacet",
        "value": "$targetState"
      }
    }
  ]
}
```
*(Execution Payload)*
```json
{
  "$targetState": "ready"
}
```
