Refining Data
Additional refinement of data is essential to accelerating your ability to share data from the Data Exchange, enrich the data for discovery, and for prepping language enriched data for Generative AI. After you have refined your data for these areas, you can review examples for querying JSON or Language data.
JSON Data Ready Automatically
When sharing data from Data Exchange, JSON data is prepped automatically on top of the entities in the industry model schema. The JSON Views that provide this data are created automatically in your model schema with a #JSON suffix on the end. For example, a standardized entity of PERSON has a corresponding JSON View PERSON#JSON created by default.
Preparing Language Data
Data Exchange is able to create Language Views that provide semantic enrichment for any entity in the industry model schema for which you have synced data. The Language Views created appear in your model schema with a #LANGUAGE suffix on the end. For example, a standardized entity of PERSON has a Language View PERSON#LANGUAGE created as part of this routine.
To prepare Language Data on top of your Model Schema:
After you have mapped and standardized your data, you can run the steps below to connect your data relationships and prep language data.
-
Run the
REFRESH_DYNAMIC_RELATED_ENTITIESprocedure to dynamically connect data relationships to simplify querying your language data. This procedure must be rerun whenever new data has arrived in your model, and is useful to run after completingSYNC_COLLECTIONto keep the graph relationships up to date. This process ensures the data in the Materialized ViewDYNAMIC_RELATED_ENTITIESis up-to-date in your Collection.SET SERVEROUTPUT ON EXEC {your_model_schema}.DE_COLLECTION.REFRESH_DYNAMIC_RELATED_ENTITIES; -
Run the
REFRESH_LANGUAGE_VIEWSprocedure to create Language Views for your industry model data. All language views created have a#LANGUAGEsuffix on the end and a unifying view namedALL#LANGUAGEis created to simplify patterns for querying data. This procedure only needs to be rerun if you change or add mapping views in your Collection.SET SERVEROUTPUT ON EXEC {your_model_schema}.DE_AI.REFRESH_LANGUAGE_VIEWS;