Reference: JQ Expressions for Digital Twin Models and Adapters

Practical JQ expression patterns for device inputs, adapter envelopes, route conditions, and payload mappings to normalize telemetry into what's defined in the digital twin models.

Key Concepts

You can write JQ expressions inside a digital twin adapter as placeholders ${ ... } to compute target values in payload-mapping and to evaluate route conditions.

Common ways to use JQ expressions:

  • Device input: Raw payloads posted by devices.
  • Envelope: Declares reference endpoints and example payload shapes, for example you can define timeObserved mapping.
  • Routes: Evaluate conditions including endpoints, headers, body, and select a payload mapping.
  • Payload mappings: Transform, convert units, rename keys, and normalize to the DTDL model schema.
  • Output: The normalized JSON output that must satisfy digital twin model validation including types, ranges, units.
  • Mapping function support: Arithmetic and floor functions are accepted. Functions like toInteger and number are not supported.
  • Endpoint matching: Uses segment-based conditions with endpoint(n) for example: ${endpoint(1) == 'home' and endpoint(2) == 'sonnen' and endpoint(3) == 'status'} instead of unsupported wildcard patterns.
  • Integer vs double schema:
    • For schema: "integer", ensure your mapping emits integral numerics (for example, "${(.velocity_kph / 1.609) | floor}").
    • For schema: "double", fractional outputs are accepted; use floor only if you want whole-number storage as double for example, 68.0.
  • Soft conversion type: Numeric-like strings and whole-number doubles are accepted when they match the model type (for example, integer). Casting helpers like number() and toInteger remain unsupported in route expressions; rely on arithmetic and floor, or adopt schema: "double" to preserve fractions.
    • Pass-through (mph, schema "integer"): {"speed": "60"} and {"speed": 60.0} are stored as 60. {"speed": "60.2"} is rejected unless mapping coerces to an integer for example, with floor.
    • Metric route (kph → mph): {"velocity_kph": 110}68; {"velocity_kph": "110"}68 because the mapping floor emits an integer. Keep arithmetic inputs numeric to avoid expression errors; prefer 110 over "110" where possible.
    • Rounding remains explicit: Soft conversion does not auto-round 68.35 to 68. Use floor for integer schema, or switch the model to schema: "double" to preserve fractions.
  • Envelope and time handling: If timeObserved is not provided, the platform may use receivedTime. Use fromdateformat, todateformat, and related functions for time conversions in envelope or payload mappings.
Note

Data validation happens against your DTDL model. If a property is declared as integer, then the normalized output must be an integer value, not a string or float. See the integer fix pattern below.

For more information, see DTMI Validation Extension Reference

Device input examples

Typical incoming payloads with unit specific schemas:

Standard USA units miles per hour:

{
  "speed": 60
}

European metric units kilometers per hour:

{
  "velocity_kph": 110
}

Nested payload from a sensor:

{
  "telemetry": { "temp_c": 22.4, "humidity": 48 }
}

Array of samples:

{
  "samples": [ { "kph": 30 }, { "kph": 50 }, { "kph": 0 } ]
}

These examples show a DTDL model where temperature is an integer within range of 0–100 and humidity is optional.

When both values are present and the temperature is within the allowed range and of the correct type, it's valid and the full input value is ingested:
{ "temperature": 60, "humidity": 45 }
When a value is absent then only the valid data is ingested. In this example temperature is missing so the humidity is ingested and the temperature value is not updated.
{
"humidity": 45
}
When a value is null it's ignored and only the valid value is ingested:
{
		“temperature”: null
 		“humidity”: 45
	}

Envelope mapping

The envelope.json declares a referenceEndpoint and a referencePayload. Optionally, an envelope mapping can set time-observed:

{
  "referenceEndpoint": "telemetry/automotive/standard",
  "referencePayload": {
    "dataFormat": "JSON",
    "data": { "speed": 65 }
  }
}
Note

The special identifier receivedTime may be provided by the platform when the device omits time. If the envelope mapping is specified and contains a timeObserved then receivedTime is used as timeObserved value.

Route condition patterns

Route conditions are rules or expressions used to determine which mapping or processing rule should be applied to an incoming message or request. If a route condition evaluates to true, the associated mapping or transformation rule is triggered and applied.

In this example, the digital twin adapter route condition defines two routes for processing incoming device messages, based on the format of the data for example if metric units are received on the 3rd segment of the endpoint path, /vehicle/speed/metric-units/ then

[
  {
    "description": "European metric to mph; convert then floor (no explicit cast).",
    "condition": "${endpoint(3) == \"metric-units\"}",
    "payloadMapping": {
      "$.speed": "${(.velocity_kph / 1.609) | floor}"
    },
    "referencePayload": {
      "dataFormat": "JSON",
      "data": { "velocity_kph": 104 }
    }
  },
  {
    "description": "USA standard units passthrough.",
    "condition": "*",
    "payloadMapping": { "$.speed": "$.speed" }
  }
]
  • endpoint(n) selects the n-th path segment (0- or 1-based depending on the adapter). In the normalizing units of measurement scenario, endpoint(3) is used so that /vehicle/speed/metric-units/ matches the third segment metric-units.
  • Place more specific conditions before the "*" catch-all.
  • payloadMapping:
    • Takes the field velocity_kph speed in kilometers per hour from the payload, and converts it to miles per hour: .velocity_kph / 1.609
    • Applies floor to round down to the nearest integer floor does not cast, so result may be a float.
    • The result is assigned to the output field speed.
  • referencePayload:
    • Demonstrates the expected input for this route: {"velocity_kph": 104}
  • Input endpoint: /vehicle/speed/metric-units/device123
  • Input payload: { "velocity_kph": 104 } Output mapping: { "speed": 64 } (since 104 / 1.609 = 64.64, floor is 64)

Payload mapping examples

Common JQ expressions for payload mappings:

  • Pass-through: "$.speed": "$.speed"
  • Rename key: "$.speed": "${.velocity_kph}"
  • Unit conversion: "$.mph": "${.kph / 1.609}"
  • Floor / ceil / round: "${.x | floor}", "${.x | ceil}", "${.x | round}"
  • Coalesce/default (version dependent): "${ if .value? then .value else 0 end }"
  • Type conversion:
    • To number: "${.value | tonumber}" (supports tonumber in most builds)
    • To string: "${.value | tostring}"
    Note

    In an IoT digital twin adapter, casting functions like toInteger or number are not accepted in inbound-routes. You can use arithmetic with floor defined for an integer schema or use schema: "double" and rounding formats for downstream data ingestion. inbound-routes must be valid JSON; expressions belong inside quoted strings "${ ... }".
  • Nested extraction: "${.telemetry.temp_c}"
  • Array map: "${[ .samples[] | .kph / 1.609 | floor ]}"
  • Conditional: "${ if .kph > 0 then .kph / 1.609 else 0 end }"
Note

Depending on your adapter integration, expressions inside quotes "${...}" may be serialized as strings. When the engine supports unquoted expressions for example, ${...} as a raw value, its recommended that form to emit a JSON number rather than a string.

Nuances: Integer types, strings, and computed numbers

When a DTDL property is schema: "integer", the normalized output must be an integer type. Two common failure modes when computing values:

  1. Stringification: An expression wrapped in quotes can yield "68" (string), failing integer validation.
  2. Float-like numbers: Arithmetic produces 68.0; some validators treat this as non-integer even if mathematically integral.

Fix patterns:

  • Use floor only for integer schema: "${(.velocity_kph / 1.609) | floor}" to produce an integral numeric that satisfies integer typing.
  • Alternative: switch the model property to schema: "double" to preserve fractional precision, or apply floor in mapping while storing a whole-number as a double. Round/format in APEX/SQL as needed.

Digital Twin Adapter Mapping Examples

This example normalizes kilometers per hour (KPH) to miles per hour (MPH) using floor (no cast).

{
  "description": "European auto uses metric units; convert to mph and floor to whole number.",
  "condition": "${endpoint(3) == \"metric-units\"}",
  "payloadMapping": {
    "$.speed": "${(.velocity_kph / 1.609) | floor}"
  },
  "referencePayload": {
    "dataFormat": "JSON",
    "data": { "velocity_kph": 104 }
  }
}

Example: Default catch-all pass-through

{
  "description": "English units passthrough.",
  "condition": "*",
  "payloadMapping": {
    "$.speed": "$.speed"
  }
}

Example: Nested telemetry and coalesce default

{
  "description": "Extract nested temp; default to 0 when missing.",
  "condition": "${endpoint(2) == \"env\"}",
  "payload-mapping": {
    "$.room_temp_c": "${ if .telemetry.temp_c? then .telemetry.temp_c else 0 end }"
  }
}

Example: Array normalization

{
  "description": "Normalize kph samples to mph (whole-number mph via floor).",
  "condition": "${.samples?}",
  "payload-mapping": {
    "$.speeds": "${ [ .samples[] | .kph / 1.609 | floor ] }"
  }
}

Output expectations vs. model validation

Outputs must satisfy digital twin model schemas and constraints. For the automotive unit model in model.json, see Scenario: Normalizing Units of Measurement using a Digital Twin Adapter:

  • name: speed
  • schema: integer
  • unit: milePerHour
  • minimum: 0, maximum: 100

Normalized speed must be an integer within 0,100. A computed string "68" or float number 68.0 will fail validation.

Limitations and tips

  • Filter availability varies: Most core jq filters (floor, ceil, round, tonumber, tostring, map, select, add)
  • Type emission: Expressions embedded in quoted strings may serialize as strings. Prefer raw unquoted expressions if supported to emit numeric types.
  • Null handling: Operations on null may produce null. Use if .x? then ... else ... end for defensive defaults.
  • Precision: Floating-point arithmetic can introduce rounding artifacts; apply round and floor as needed before casting.
  • Integer cast: In observed tests, toInteger and number were not accepted in inbound-routes at adapter creation time. Prefer arithmetic + floor for integer schema or use schema: "double" and round the format downstream.

Applying the integer fix to the automotive scenario

The scenario creates a model with an integer speed and routes metric unit payloads to the same digital twin model. Without an explicit cast, the computed value can be rejected by validation due to type drift (string or float-like number).

Fix used in the scenario:

"$.speed": "${(.velocity_kph / 1.609) | floor}"

Why it is needed:

  • floor ensures the value is whole-numbered, satisfying integer typing.
  • Alternative: use schema: "double" to preserve fractional precision and round and format downstream if needed.
Note

When using integer schema, prefer arithmetic plus floor. For double schema, you may omit floor to keep fractional MPH or include it to store a whole number MPH as a double.

Quick reference snippets

Supported examples:

  • Condition: endpoint match: "condition": "${endpoint(3) == \"metric-units\"}"
  • Mapping: pass-through: "$.speed": "$.speed"
  • Mapping: kph → mph (integer schema): "$.speed": "${(.kph / 1.609) | floor}"
  • Mapping: nested: "$.room_temp_c": "${.telemetry.temp_c}"
  • Mapping: default: "$.value": "${ if .value? then .value else 0 end }"
  • Array transform: "$.list": "${ [ .arr[] | .x | tonumber ] }"

Integer vs Double Model Behavior

This section summarizes how the model schema type affects adapter mapping and validation.

  • schema: "integer"
    • Type: Must be an integer JSON number that's integral. Strings like 68 or float-like results 68.0 can fail integer validation.
    • Mapping: Arithmetic is allowed; floor is supported. Functions like toInteger and number are not supported.
    • Pattern: Use "${(.velocity_kph / 1.609) | floor}" to coerce to integral MPH. This passed validation and telemetry was accepted HTTP 202.
    • Range: Minimum and maximum values are enforced, for example, 0–100
  • schema: "double"
    • Type: Any JSON number integral or fractional is accepted if within the range; no auto-rounding performed by the platform.
    • Mapping: You may preserve fractional precision, for example "${.velocity_kph / 1.609}", or also apply floor to produce a whole number MPH stored as double for example: 68.0
    • Range: If defined, minimum and maximum enforced for example: 0–100

Observed Telemetry Results

The following combinations were validated end-to-end (HTTP/1.1 202 Accepted):

  • Integer model + floor mapping: "${(.velocity_kph / 1.609) | floor}" → accepted
  • Double model + raw mapping: "${.velocity_kph / 1.609}" → fractional mph accepted
  • Double model + floor mapping: "${(.velocity_kph / 1.609) | floor}" → whole number mph accepted and stored as double

Example curl (placeholders)

Double model, raw value that's fractional mph as expected:

curl -i -X POST \
  -u "european-auto-raw:secret-or-certificate-ocid" \
  -H "Content-Type: application/json" \
  "https://device-host/telemetry/automotive/metric-units" \
  -d '{ "velocity_kph": 110 }'

Double model, floor a whole number MPH, that's stored as double:

curl -i -X POST \
  -u "european-auto-dfloor:secret-or-certificate-ocid" \
  -H "Content-Type: application/json" \
  "https://device-host/telemetry/automotive/metric-units" \
  -d '{ "velocity_kph": 110 }'

Expected HTTP response for each case:

HTTP/1.1 202 Accepted
content-type: text/plain

Accepted

Downstream in APEX or SQL considerations

  • With double, fractional speed is preserved. Use SQL FLOOR/ROUND for whole-number display:
    SELECT FLOOR(speed) AS speed_mph FROM ...
  • With integer, ensure mapping outputs integral values (e.g., via floor) to satisfy validation.

Authorization Username and Quotes

In OCI IoT, using the Basic authentication username equals the instance external-key. If the instance is created with embedded quotes in the external key, those quotes become part of the required username and must be sent literally. This often causes shell quoting problems.

Best practice: Create digital twin instances with external keys that do not include quotes for example, american-auto When you must authenticate with quoted usernames, construct an Authorization: Basic ... header rather than using -u to avoid quoting errors.