5 Using Telemetry Streaming
This chapter explains how to use Telemetry Streaming to ingest and query data.
Topics:
- Ingesting Metric Data
This section explains the different ways you can ingest metric data into Telemetry Streaming. - Querying Metric Data
This section explains the different ways you can query metric data in Telemetry Streaming.
5.1 Ingesting Metric Data
This section explains the different ways you can ingest metric data into Telemetry Streaming.
For clients outside of Oracle AI Database, Telemetry Streaming integrates with ORDS to allow REST API calls through ORDS endpoints to ingest data into Telemetry Streaming. For SQL clients, Telemetry Streaming provides PL/SQL packages to ingest data.
Parent topic: Using Telemetry Streaming
5.1.1 Using PL/SQL to Ingest Metric Data
Any existing Oracle user can be enabled as an ingest user for a workspace. However, a user can be added as an ingest user for only one workspace.
The DBMS_TELEMETRY_INGEST
PL/SQL package enables an ingest user to
ingest data into Telemetry Streaming either as a single sample ingest data or as
CLOB
data.
Table 5-1 DBMS_TELEMETRY_INGEST
Package
Function | Description |
---|---|
|
For single metric sample ingestion |
|
For metric ingestion as |
See Also:
DBMS_TELEMETRY_INGEST in
Oracle AI Database PL/SQL Packages and Types Reference for more
information about the DBMS_TELEMETRY_INGEST
PL/SQL package.
Parent topic: Ingesting Metric Data
5.1.2 Using REST APIs with ORDS to Ingest Metric Data
If you want an external client to push metric data through ORDS into Telemetry Streaming, you can use the REST APIs through ORDS endpoints. ORDS employs the OAuth2 authorization mechanism to ingest data using ORDS endpoints. Therefore, you must authenticate the requests using the client credentials (Client ID and Client Secret pair), which is created by the Telemetry ORDS administrator or the workspace administrator.
The client credentials flow in OAuth2 is a two-legged process. You use the client credentials to return an access token, which is then used to authenticate the API calls to the ORDS endpoints and ingest data.
To secure the ORDS APIs, Telemetry Streaming ensures that:
-
All the REST APIs are authenticated using the OAuth2 Client Credentials protocol.
-
A workspace administration user (who is an existing database user) can get the OAuth credentials (
client-id, client-secret
) for the ORDS administration user.Note:
A workspace administration user should enable ORDS using
DBMS_TELEMETRY_ADMIN.ENABLE_WORKSPACE_ORDS
before fetching the ORDS admin credentials. -
Only an ORDS user with administrative privileges can create the ORDS ingest and ORDS query users.
The following section provides the REST API documentation for the ingest REST APIs.
Note:
-
Ensure that ORDS is installed and Telemetry Streaming is REST enabled before using REST APIs.
-
To make ORDS Ingest API calls, you need an OAuth2 access token to use for authorization. The token can be derived using the token URL by providing the Client ID and Client Secret of the ORDS ingest user.
See Also:
Installing Oracle REST Data Services for more information about the ORDS installation.
Ingesting Data Using REST API
Once an ORDS ingest user is created using the ORDS admin user, it can be used to ingest data through the following API endpoints.
Table 5-2 REST API Summary
API Type | Summary |
---|---|
POST
|
Ingests data into Telemetry Streaming using the ORDS API endpoint (Telemetry protocol) |
POST
|
Ingests data into Telemetry Streaming using the ORDS API endpoint (Line protocol) |
Ingest Data (Telemetry protocol) | |
---|---|
METHOD: |
POST |
PATH: |
|
USER: |
ORDS Ingest User |
SUMMARY: |
Ingests data into Telemetry Streaming using the ORDS API endpoint (Telemetry protocol) |
DATA: |
|
HTTP CODE: |
|
EXAMPLE: |
|
Ingest Data (Line protocol) | |
---|---|
METHOD: |
POST |
PATH: |
|
USER: |
ORDS Ingest User |
SUMMARY: |
Ingests data into Telemetry Streaming using the ORDS API endpoint (Line protocol) |
DATA: |
|
HTTP CODE: |
|
EXAMPLE: |
|
Ingest Example Using cURL
- Add a new ingest user.
Use the following cURL command to add a new user named
INGEST1
with the roleINGEST
using workspace ORDS admin access token:Request:curl -k --request POST -i \ -H "Authorization: Bearer {admin user token}" \ -H "Content-Type: application/json" \ --data '{ "user_name": "<USERNAME>", "role": "<ROLE>" }' \ -v <ADD_USER_URL>
Response:{"client_id":<client_id>, "client_secret":<client_secret key>}
Example:curl -k --request POST -i \ -H "Authorization: Bearer moVgM6Vi6TqJB0xbYezicA" \ -H "Content-Type: application/json" \ --data '{ "user_name": "INGEST1", "role": "INGEST" }' \ -v http://example.com:8085/ords/wksp1/add_user
Response:{"client_id":"aBcDefg1hij2k3l..","client_secret":"mNoPq45rst6u7xyz.."}
- Get the access token using the client ID:client secret.
Request:
curl -k --request POST \ --user $INGEST_USER_CLIENT_ID:$INGEST_USER_CLIENT_SECRET \ --data "grant_type=client_credentials" <TOKEN_URL>
Response:{"access_token":<token>, "token_type":<token type>, "expires_in":<time in seconds>}
Example:curl -k --request POST \ --user aBcDefg1hij2k3l..:mNoPq45rst6u7xyz.. \ --data "grant_type=client_credentials" <http://example.com:8085/ords/wksp1/oauth/token>
Response:{"access_token":"B1o38ikZ55tNMXsuPlksDQ","token_type":"bearer","expires_in":3600}
- Ingest the data.
Request:
curl -k --request POST -i -H "Authorization: Bearer {ingest user token}" \ --header "Content-Type: application/json" \ --data @<file_name> -v <INGESTION URL>
Response:{"metrics_data_size":<num>,"metrics_ingested":<num>, "metrics_given":<num>,"metrics_metadata":<num>,"ingest_duration_ms":<time in ms>, "ingest_format":<format>,"ingest_method":<method>,"ingest_user_name":<ingest user name>}
Example (using the default Telemetry protocol):curl -k --request POST -i -H "Authorization: Bearer B1o38ikZ55tNMXsuPlksDQ" \ --header "Content-Type: application/json" \ --data @data.json -v https://example.com:8000/ords/workspace1/ingest1/ingest
Response:{"metrics_data_size":79,"metrics_ingested":1, "metrics_given":1,"metrics_metadata":1,"ingest_duration_ms":10, "ingest_format":"telemetry","ingest_method":"ords","ingest_user_name":"ingest1"}
Example (using Line protocol):curl -k --request POST -i -H "Authorization: Bearer B1o38ikZ55tNMXsuPlksDQ" \ --header "Content-Type: application/json" \ --data @data.json -v https://example.com:8000/ords/workspace1/ingest1/ingestlp
Response:{"metrics_data_size":79,"metrics_ingested":1, "metrics_given":1,"metrics_metadata":1,"ingest_duration_ms":10, "ingest_format":"line","ingest_method":"ords","ingest_user_name":"ingest1"}
Parent topic: Ingesting Metric Data
5.2 Querying Metric Data
This section explains the different ways you can query metric data in Telemetry Streaming.
You can use PromQL or SQL to query the time series data that is ingested in Telemetry Streaming.
- Using PromQL to Query Metric Data
Learn about the supported PromQL queries and how to use them to query in Telemetry Streaming. - Using REST API with ORDS for Querying
This section describes the REST APIs used for querying. - Using SQL to Query Metric Data
Learn how you can use SQL for querying in Telemetry Streaming.
Parent topic: Using Telemetry Streaming
5.2.1 Using PromQL to Query Metric Data
Learn about the supported PromQL queries and how to use them to query in Telemetry Streaming.
To query the data using PromQL, you can use either of the following:
- PL/SQL APIs using the
DBMS_TELEMETRY_QUERY
packageSee Also:
DBMS_TELEMETRY_QUERY in Oracle AI Database PL/SQL Packages and Types Reference for more information about the
DBMS_TELEMETRY_QUERY
PL/SQL package. - ORDS REST endpoints
Querying using REST API is explained later in this section.
In Oracle AI Database 26ai, Release 23.26.0, only a subset of PromQL constructs are supported.
See Also:
PromQL Querying Basics to get started with building queries using PromQL
Parent topic: Querying Metric Data
5.2.1.1 Supported PromQL Queries for
DBMS_TELEMETRY_QUERY
The following types of PromQL queries are supported in
DBMS_TELEMETRY_QUERY
for each query category.
Range/Instant PromQL Queries
- Vector
Selector
node_cpu_seconds_total{cpu="1"}
- Functions
(rate/irate)
rate(node_cpu_seconds_total{cpu="1"}[3m])
- Aggregate: sum, avg, count, max, min, stddev, stdvar, group,
quantile, topk, bottomk,
count_values
sum(node_cpu_seconds_total) min(node_boot_time_seconds{device!="eth0"}) stddev(node_disk_info{major!="11"}) group(container_spec_cpu_period{job="cadvisor"}) quantile(0.95,node_arp_entries{device=~"eth0"}) topk(5,node_boot_time_seconds{job="node"}) count_values("values",node_cpu_online)
- Aggregate by: sum, avg, count, max, min, stddev, stdvar, group,
quantile, topk, bottomk,
count_values
avg by(cpu)(node_cpu_online{job=~"node"}) count(go_gc_gogc_percent{job="node"}) stdvar by(bios_date)(node_disk_info{job="node"}) group by(address)(container_spec_cpu_period{job="cadvisor"}) quantile by(adminstate)(0.80,go_gc_gogc_percent{job="node"}) bottomk by(tags)(node_boot_time_seconds{job=~"node"}) count_values by(tags)("cpu",node_cpu_load)
Label Queries
- Fetch all the distinct keys in the tags column or for a given tag key, fetch all the distinct values of that corresponding key.
Series Queries
- Fetch a range of time series that matches a specific criteria, such
as a label filter or metric
name.
node_cpu_seconds_total{cpu="1"}
See Also:
PromQL Querying for more information about PromQL querying
Parent topic: Using PromQL to Query Metric Data
5.2.1.2 Using Supported PromQL Query Operators
The following shows the usage of the supported PromQL queries for
DBMS_TELEMETRY_QUERY
in the initial version.
Range Queries
Range queries in PromQL are queries that span over a time range.
select dbms_telemetry_query.promql_range(promql_query, start_time_epoch, end_time_epoch, step_size_seconds, fetch_sql[optional]) from dual;
Note:
Thefetch_sql
value can be either 0 or 1. If it is 1, the SQL conversion of the PromQL query
is returned; if it is 0 [default value], the result after executing the SQL is
returned as JSON.
The range queries can be of 4 types:
- Simple expression
queries
select dbms_telemetry_query.promql_range('node_cpu_load{cpu="1"}',1389312400,1389312600,10) from dual;
- Simple aggregate queries (sum, avg, count, count_values, max, min,
std deviation, std variance, group, quantile, top_k,
bottom_k)
select dbms_telemetry_query.promql_range('sum(node_cpu_load{cpu="1"})',1389312400,1389312600,10) from dual;
- Simple aggregate-by queries (sum, avg, count, count_values, max,
min, std deviation, std variance, group, quantile, top_k,
bottom_k)
select dbms_telemetry_query.promql_range('sum by(tag)(node_cpu_load{cpu="1"})',1389312400,1389312600,10) from dual;
- Functions
(Rate/IRate)
select dbms_telemetry_query.promql_range('irate(node_cpu_load{cpu="1"}[5m])',1389312400,1389312600,10) from dual;
Instant Queries
Instant queries in PromQL request time series data for a particular timestamp.
select dbms_telemetry_query.promql_instant(promql_query, instant_time_epoch, fetch_sql[optional]) from dual;
Note:
Thefetch_sql
value can be either 0 or 1. If it is 1, the SQL conversion of the PromQL query
is returned; if it is 0 [default value], the result after executing the SQL is
returned as JSON.
Instant queries can be of four types.
- Simple expression
queries
select dbms_telemetry_query.promql_instant('node_cpu_load{cpu="1"}',1389312600) from dual;
- Simple aggregate queries (sum, avg, count, count_values, max, min,
std deviation, std variance, group, quantile, top_k,
bottom_k)
select dbms_telemetry_query.promql_instant('sum(node_cpu_load{cpu="1"})',1389312600) from dual;
- Simple aggregate-by queries (sum, avg, count, count_values, max,
min, std deviation, std variance, group, quantile, top_k,
bottom_k)
select dbms_telemetry_query.promql_instant('sum by(tag)(node_cpu_load{cpu="1"})',1389312600) from dual;
- Functions
(Rate/IRate)
select dbms_telemetry_query.promql_instant('rate(node_cpu_load{cpu="1"}[3m])',1389312600) from dual;
Label Queries
Label queries in PromQL fetch distinct keys or distinct values.
select dbms_telemetry_query.promql_label(promql_query, start_time_epoch, end_time_epoch, fetch_sql[optional]) from dual;
Note:
Thefetch_sql
value can be either 0 or 1. If it is 1, the SQL conversion of the PromQL query
is returned; if it is 0 [default value], the result after executing the SQL is
returned as JSON.
Label queries are of two types.
- Fetches all the distinct keys in the tags (if the promql query is
empty)
select dbms_telemetry_query.promql_label(' ',1389312400,1389312600) from dual;
- Fetches all the distinct values that a particular key can take in
the
tags
select dbms_telemetry_query.promql_label('cpu',1389312400,1389312600) from dual;
Series Queries
Series queries in PromQL fetch the metadata (metric name and tags) of the time series that match the given PromQL query.
select dbms_telemetry_query.promql_series(promql_query, start_time_epoch, end_time_epoch, fetch_sql[optional]) from dual;
Note:
Thefetch_sql
value can be either 0 or 1. If it is 1, the SQL conversion of the PromQL query
is returned; if it is 0 [default value], the result after executing the SQL is
returned as JSON.
select dbms_telemetry_query.promql_series('node_cpu_load{cpu="1"}',1389312400,1389312600) from dual;
See Also:
PromQL Querying for more information about PromQL querying
Parent topic: Using PromQL to Query Metric Data
5.2.2 Using REST API with ORDS for Querying
This section describes the REST APIs used for querying.
Once an ORDS query user is created using the ORDS workspace administrator, the user can query data or view query statistics through the following REST API endpoints.
Note:
Ensure that ORDS is installed and Telemetry Streaming is REST enabled before using REST APIs.
See Also:
Installing Oracle REST Data Services for more information about the ORDS installation.
Table 5-3 REST API Summary
API Type | Summary |
---|---|
POST
|
Query Telemetry Streaming data using ORDS API endpoint |
Query Data | |
---|---|
METHOD: |
POST |
PATH: |
|
QUERY TYPE: |
|
USER: |
ORDS Query User |
SUMMARY: |
Query Telemetry Streaming data using ORDS API endpoint |
Range Query | |
---|---|
DATA: |
|
HTTP CODE: |
|
EXAMPLE |
|
Instant Query | |
---|---|
DATA: |
|
HTTP CODE: |
|
EXAMPLE |
|
Series Query | |
---|---|
DATA: |
|
HTTP CODE: |
|
EXAMPLE |
|
Label Query | |
---|---|
DATA: |
|
HTTP CODE: |
|
EXAMPLE |
|
Query Example Using cURL
- Add a new query user.
Request:
curl -k --request POST -i \ -H "Authorization: Bearer {admin user token}" \ -H "Content-Type: application/json" \ --data '{ "user_name": "<USERNAME>", "role": "<ROLE>" }' \ -v <ADD_USER_URL>
Response:{"client_id":<client_id>, "client_secret":<client_secret key>}
Example:curl -k --request POST -i \ -H "Authorization: Bearer moVgM6Vi6TqJB0xbYezicA" \ -H "Content-Type: application/json" \ --data '{ "user_name": "QUERY1", "role": "QUERY" }' \ -v http://example.com:8085/ords/wksp1/add_user
Response:{"client_id":"fymGQ8uO8zuPTiBWrw8CEw..","client_secret":"fPJ3RcJlR_XEmlrC276dEg.."}
- Generate an access token for the query user.
Generate an OAuth token using the
client_id
andclient_secret
.Request:curl -k --request POST \ --user $QUERY_USER_CLIENT_ID:$QUERY_USER_CLIENT_SECRET \ --data "grant_type=client_credentials" <TOKEN_URL>
Response:{"access_token":<token>, "token_type":<token type>, "expires_in":<time in seconds>}
Example:curl -X POST \ --user fymGQ8uO8zuPTiBWrw8CEw..:fPJ3RcJlR_XEmlrC276dEg.. \ --data "grant_type=client_credentials" \ http://example.com:8085/ords/wksp1/oauth/token
Response:{"access_token":"GVCvNXcqRG9OBZM98zHPcQ","token_type":"bearer","expires_in":3600}
- Run the PromQL Range/PromQL Instant/PromQL Series/PromQL Label
query using the access token.
Use the generated token (for example,
GVCvNXcqRG9OBZM98zHPcQ
) to query the PromQL endpoint as theQUERY1
user.The following examples demonstrate the query requests and responses for each query type.
PromQL Range:
Example:curl -k --request POST -i -H "Authorization: Bearer GVCvNXcqRG9OBZM98zHPcQ" \ --header "Content-Type: application/json" \ --data '{ "promql_query": "met1", "start_time": 1759752400, "end_time": 1759752700, "step_size": 10}' \ -v http://example.com:8085/ords/wksp1/QUERY1/promql_range
Response:{ "status" : "success", "data" : { "resultType" : "matrix", "result" : [ { "metric" : { "__name__" : "met1", "tag1" : "val1" }, "values" : [ [ 1759752590, "10.000000" ] ] } ] } }
PromQL Instant
Example:curl -k --request POST -i -H "Authorization: Bearer GVCvNXcqRG9OBZM98zHPcQ" \ --header "Content-Type: application/json" \ --data '{ "promql_query": "met1", "point_time" : 1751460370}' \ -v http://example.com:8085/ords/wksp1/QUERY1/promql_point
Response:{ "status" : "success", "data" : { "resultType" : "vector", "result" : [ { "metric" : { "__name__" : "met1", "tag1" : "val1" }, "value" : [ 1759752700, "10.000000" ] } ] } }
PromQL Series
Example:curl -k --request POST -i -H "Authorization: Bearer GVCvNXcqRG9OBZM98zHPcQ" \ --header "Content-Type: application/json" \ --data '{ "promql_query": "met1", "start_time": 1759752400, "end_time": 1759752700}' \ -v http://example.com:8085/ords/wksp1/QUERY1/promql_series
Response:{ "status" : "success", "data" : [ { "__name__" : "met1", "tag1" : "val1" } ] }
PromQL Label
Example:curl -k --request POST -i -H "Authorization: Bearer GVCvNXcqRG9OBZM98zHPcQ" \ --header "Content-Type: application/json" \ --data '{ "promql_query": "tag1", "start_time": 1759752400, "end_time": 1759752700}' \ -v http://example.com:8085/ords/wksp1/QUERY1/promql_label
Response:{ "status" : "success", "data" : [ "val1" ] }
Parent topic: Querying Metric Data
5.2.3 Using SQL to Query Metric Data
Learn how you can use SQL for querying in Telemetry Streaming.
Telemetry Streaming is built on Oracle AI Database, and so it supports SQL to query the time series data. The advantage of using SQL is its ability to construct complex queries that can give better insight into the data.
A workspace query user can access metrics data using SQL on the
TM$<WORKSPACE_NAME>.TELEMETRY_METRICS
view of the
workspace.
select metric_name, metric_tags, metric_value, metric_time from TM$WORKSPACE1.TELEMETRY_METRICS.
See Also:
Table -8 in Tables and Views for more information about the workspace administration views
Parent topic: Querying Metric Data