A APIs for Model Pipelines
All the listed APIs/Functions can be utilized by users in Model Pipelines.
Table A-1 Python Paragraph Scripting
| API Type | Functionality | Prerequisite | Notebook Execution Script | Notebook Script Input | Minimum Supported Version |
|---|---|---|---|---|---|
| Set User | Set the user for a session | - |
%python from mmg.constants import * user = "SAUSER" set_user(user) |
user - username to be set | 8.1.1 |
| Get User | Get the user that has been set | Set User |
%python from mmg.constants import * get_user() |
- | 8.1.1 |
| Attach Workspace | Attach the workspace for a session | - |
%python from mmg.workspace import attach_workspace workspace = "CS" attach_workspace(workspace) |
workspace- workspace to be attached | 8.1.1 |
| Get Workspace | Get the workspace that has been attached | Attach Workspace |
%python from mmg.workspace import get_workspace get_workspace() |
8.1.1 | |
| List Workspaces | Lists all workspaces available to a user | Set User |
%python from mmg.workspace import list_workspaces list_workspaces() |
|
8.1.1 |
| Check AIF | Check if AIF is set or not | Set User |
%python from mmg.workspace import check_aif check_aif() |
|
8.1.1 |
| List All Data Sources | List all data sources available in a workspace for a set user. The default workspace chosen is the one that is attached. | Set User, Attach Workspace |
%python from mmg.workspace import list_datasources list_datasources(order=None) |
order - None ensures that all the data sources will be listed for the attached workspace | 8.1.1 |
| List All Data Sources of Workspace | List all Data Sources of a specific Workspace that is not the attached workspace for a set user. | Set User |
%python from mmg.workspace import list_datasources workspace="HELLO NEW" list_datasources(workspace,order=None) |
workspace - Name of workspace for which data sources should be listed order - None ensures that ALL data sources will be listed for that workspace |
8.1.1 |
| Fetch First Order Data Source | This method will fetch the data source related to attached workspace with default order 1. | Set User, Attach Workspace |
%python from mmg.workspace import list_datasources list_datasources() |
order - 1 by default, if not specified |
8.1.1 |
| Fetch First Order Data Source of Workspace | This method will fetch the data source with default order 1 related to specified workspace. | Set User |
%python from mmg.workspace import list_datasources workspace="CS" list_datasources(workspace) |
workspace - Name of workspace for which data source should be listed order- 1 by default if not specified |
8.1.1 |
| Fetch Nth Order Data Source | This method will fetch the data source related to attached workspace with given order | Set User, Attach Workspace |
%python from mmg.workspace import list_datasources list_datasources(order=2) |
order - Order of data source to be fetched | 8.1.1 |
| Fetch Nth Order Data Source of Workspace | This method will fetch the data source with given order related to specified workspace. | Set User |
%python from mmg.workspace import list_datasources workspace="CS" list_datasources(workspace,order=2) |
workspace - Name of workspace for which data source should be listed order = Order of data source to be fetched |
8.1.1 |
| Get Connection Object | Returns Oracle/ SQL Alchemy Connection Object to Data Source of order 1 in attached workspace. |
Set User, Attach Workspace
|
%python import cx_Oracle from mmg.workspace import get_connection conn=get_connection() if not isinstance(conn,cx_Oracle.connect): print("Not Connection Object") print(conn) |
8.1.2.0 | |
| Get Connection Object To Specific Data Source | Returns Oracle/ SQL Alchemy Connection Object to specified Data Source | Set User |
%python import cx_Oracle from mmg.workspace import get_connection datasource="DS001" conn=get_connection(datasource) if not isinstance(conn,cx_Oracle.connect): print("Not Conn Object") print(conn) |
datasource - Name of datasource for which the connection object has to be returned | 8.1.2.0 |
| Get Connection Object To Data Source Using Order | Returns Oracle/ SQL Alchemy Connection Object to Data Source of specified order in attached workspace. | Set User, Attach Workspace |
%python import cx_Oracle from mmg.workspace import get_connection datasource=2 conn=get_connection(datasource) if not isinstance(conn,cx_Oracle.connect): print("Not Conn Object") print(conn) |
datasource - Order number of datasource in attached workspace for which the connection object has to be returned | 8.1.2.0 |
| Test Connection Object | This script can be used to test the connection object to data source |
Get Connection Object OR Get Connection Object To Specific Data Source OR Get Connection Object To Data Source Using Order |
%python query="SELECT table_name FROM user_tables" cur = conn.cursor() cur.execute(query) print(cur.fetchall()) if cur: cur.close() |
8.1.2.0 |
Table A-2 PYTHON LINEPARSER APIs (for MMG version 8.1.0)
| API Type | Functionality | Notebook Execution Script | Note |
|---|---|---|---|
| Attach Workspace | Attach the workspace for a session |
%python mmg.attachWorkspace(workspace) #eg- mmg.attachWorkspace("CS") |
workspace - workspace to be attached (String) Sets 'mmg' python variable which contains the information about the attached workspace. Output- Print a boolean in one line on the status of workspace attachment and another line of boolean on status of AIF enabled and attached. |
| List Workspaces | Lists all workspaces available to a user |
%python mmg.listWorkspaces() |
Sets 'ofs_wsList' python variable which is the list of all available workspaces. Output- Print the list of workspaces. |
| Check AIF | Check if AIF is set or not |
%python print(aif__isAttached) |
Output- Boolean on the status of AIF. |
| Get Connection Object | Returns cx_Oracle Connection Object |
%python mmg.getConnection(schema_name) #eg- mmg.getConnection("OFSAA") |
schema_name = name of the schema for which connection is required (String). Sets 'conn' python variable which is the cx_Oracle connection object to be used for firing queries. |
| Publish | Fetch and sets the Notebook JSON dump |
%python mmg.publish() |
Sets 'emf_para' python variable which is the JSON dump for the notebook component. Output- Print the set 'emf_para' value. |
| Register | Register the Notebook |
%python mmg.register(register) #eg- mmg.register({"modelname":"model1",\ # "modeldescription":"Model description"}) #eg- mmg.register(emf_para) {After doing mmg.publish()} |
register - JSON Stringify object for RegisterNotebookBean Object. Sets 'publishedNotebookId' python variable which is the studio notebook id for the published version. Output- Prints the published notebook id. |
| Load Flat File | - |
%python mmg.loadFlatFile() #eg- mmg.loadFlatFile() Incomplete |
|
| Profile Data | - |
%python mmg.profileData(code,title) #eg- mmg.profileData(ABC,XYZ) |
Table A-3 MMG Library Python APIs
| Index | API Type | Functionality | Prerequisite | Notebook Execution Script |
|---|---|---|---|---|
| 1 | Set User | Set the user for a session | %python from mmg.constants import * user =
"SAUSER" set_user(user) |
|
| 2 | Get User | Get the user that has been set |
Set User |
%python from mmg.constants import *
get_user() |
| 3 | Attach Workspace | Attach the workspace for a session | %python from mmg.workspace import
attach_workspace workspace = "CS"
attach_workspace(workspace)
|
|
| 4 | Get Workspace | Get the workspace that has been attached |
Attach Workspace |
%python from mmg.workspace import
get_workspace get_workspace()
|
| 5 | Get Base URL | Get MMG Studio Service Base URL | %python from mmg.constants import *
get_mmg_studio_service_url()
|
|
| 6 | List Workspaces | Lists all workspaces available to a user |
Set User |
%python from mmg.workspace import
list_workspaces list_workspaces() |
| 7 | Check AIF | Check if AIF is set or not |
Set User |
%python from mmg.workspace import check_aif
check_aif() |
| 8 | List All Data Sources | List all data sources available in a workspace for a set user. The default workspace chosen is the one that is attached. |
Set User, Attach Workspace |
%python from mmg.workspace import
list_datasources list_datasources(order=None) |
| 9 | List All Data Sources of Workspace | List All Data Sources of a specific Workspace that is not the attached workspace for a set user |
Set User |
%python from mmg.workspace import
list_datasources workspace="HELLO NEW"
list_datasources(workspace,order=None) |
| 10 | Fetch First Order Data Source | This method will fetch the data source related to attached workspace with default order 1 | Set User, Attach Workspace | %python from mmg.workspace import
list_datasources list_datasources()
|
| 11 | Fetch First Order Data Source of Workspace | This method will fetch the data source with default order 1 related to specified workspace | Set User | %python from mmg.workspace import
list_datasources workspace="CS"
list_datasources(workspace) |
| 12 | Fetch Nth Order Data Source | This method will fetch the data source related to attached workspace with given order | Set User, Attach Workspace | %python from mmg.workspace import
list_datasources list_datasources(order=2) |
| 13 | Fetch Nth Order Data Source of Workspace | This method will fetch the data source with given order related to specified workspace | Set User | %python from mmg.workspace import
list_datasources workspace="CS"
list_datasources(workspace,order=2)
|
| 14 | Get Connection Object | Returns Oracle/ SQL Alchemy Connection Object to Data Source of order 1 in attached workspace | Set User, Attach Workspace | %python import cx_Oracle from mmg.workspace
import get_connection conn=get_connection() if not
isinstance(conn,cx_Oracle.connect): print("Not Conn Object")
print(conn) |
| 15 | Get Connection Object To Specific Data Source | Returns Oracle/ SQL Alchemy Connection Object to specified Data Source | Set User | %python import cx_Oracle from mmg.workspace
import get_connection datasource="DS"
conn=get_connection(datasource, conn_type=None) if not
isinstance(conn,cx_Oracle.connect): print("Not Conn Object")
print(conn) |
| 16 | Get Connection Object To Data Source Using Order | Returns Oracle/ SQL Alchemy Connection Object to Data Source of specified order in attached workspace | Set User, Attach Workspace | %python import cx_Oracle from mmg.workspace
import get_connection datasource=2
conn=get_connection(datasource, conn_type=None) if not
isinstance(conn,cx_Oracle.connect): print("Not Conn Object")
print(conn) |
| 17 | Get Connection Object | Returns Oracle/ SQL Alchemy Connection Object to Data Source of order 1 in attached workspace | Set User, Attach Workspace | %python import oracledb from mmg.workspace import
get_conn conn=get_conn() if not
isinstance(conn,oracledb.Connection): print("Not Conn Object")
print(conn) |
| 18 | Get Connection Object To Specific Data Source | Returns Oracle/ SQL Alchemy Connection Object to specified Data Source | Set User | %python import oracledb from mmg.workspace import
get_conn datasource="DS" conn=get_conn(datasource,
conn_type=None) if not isinstance(conn,oracledb.Connection):
print("Not Conn Object") print(conn) |
| 19 | Get Connection Object To Data Source Using Order | Returns Oracle/ SQL Alchemy Connection Object to Data Source of specified order in attached workspace | Set User, Attach Workspace | %python import oracledb from mmg.workspace import
get_conn datasource=2 conn=get_conn(datasource, conn_type=None)
if not isinstance(conn,oracledb.Connection): print("Not Conn
Object") print(conn) |
| 20 | Get Objective Hierarchy | Returns the complete objective hierarchy including pipeline name | %python print(objectiveId) |
|
| 21 | Fetch FTPSHARE Path | Get complete ftpshare path on the server | %python from mmg.datasets import fetch_ftpshare
fetch_ftpshare() |
|
| 22 | Get Pipeline Name | Get the Pipeline Name to which the current notebook is attached | Attach Workspace | %python from mmg.constants import get_pipeline_name
get_pipeline_name() |
| 23 | Get Pipeline ID | Get ID of Pipeline to which the current notebook is attached | Attach Workspace | %python from mmg.constants import get_pipeline_id
get_pipeline_id() |
| 24 | Get Pipeline Version | Get version of Pipeline to which the current notebook is attached | Attach Workspace | %python from mmg.constants import get_pipeline_version
get_pipeline_version() |
| 25 | Get MISDATE | Return MISDATE from pipeline execution if set otherwise takes default value provided | Attach Workspace | %python
misdate=ds.date_picker(name='$FICMISDATE$',format='yyyy-MM-dd',
default_value='2000-01-01', label='FICMISDATE')
print(misdate) |
| 26 | Get BATCHRUNID | Returns BATCHRUNID if set from pipeline, otherwise takes default value | %python
batchrunid=ds.textbox(name='$BATCHRUNID$',default_value='Batch_auto',label='BATCHRUNID')
print(batchrunid) |
|
| 27 | Get TASKID | Returns TASKID if set from pipeline, otherwise takes default value | %python
taskid=ds.textbox(name='$TASKID$',default_value='task1',label='TASKID')
print(taskid) |
Miscellaneous Helper Python Scripts
Table A-4 Miscellaneous Helper Python Scripts
| Index | Name | Description | Prerequisite | Script |
|---|---|---|---|---|
| 1 | Create Connection Object Using Wallet Alias | This script can be used to test the connection to any Oracle database using wallet alias. Make sure to close connection object after use. | %python wallet_alias = "WALLET_ALIAS_NAME" import
oracledb oracledb.init_oracle_client() conn =
oracledb.connect(dsn=wallet_alias |
|
| 2 | Test Connection Object | This script can be used to test the connection object to data source | Get Connection Object OR Get Connection Object To Specific Data Source OR Get Connection Object To Data Source Using Order | %python query="SELECT table_name FROM user_tables" cur =
conn.cursor() cur.execute(query) print(cur.fetchall()) if cur:
cur.close() |
| 3 | Close Connection Object | Connection Object should be created using any of the above methods | %python query="SELECT table_name FROM user_tables" cur =
conn.cursor() cur.execute(query) print(cur.fetchall()) if cur:
cur.close() |
|
| 4 | Close Cursor | Cursor Object should be created using any of the above methods | %python if cur: cur.close() |
|
| 5 | Check environment variables |
For example: Check wallet alias value of MMG_LIB_WALLET_ALIAS |
%python import os
print(os.environ['MMG_LIB_WALLET_ALIAS']) |
|
| 6 | Save Dataframe to a File | Save dataframe to a file stored on server | %python from mmg.datasets import fetch_ftpshare,
save_dataframe from mmg.workspace import get_workspace from
mmg.constants import get_pipeline_version
misdate=ds.date_picker(name='$FICMISDATE$', format='yyyy-MM-dd',
default_value='2000-01-01', label='FICMISDATE')
batchrunid=ds.textbox(name='$BATCHRUNID$',
default_value='Batch_auto', label='BATCHRUNID')
taskid=ds.textbox(name='$TASKID$', default_value='task1',
label='TASKID') filename="df.csv" filepath=fetch_ftpshare() +
"/mmg/" + get_workspace() + "/pipelines/" + objectiveId + "/" +
get_pipeline_version() + "/output/" + misdate + "/" + batchrunid
+ "/" + taskid + "/" + filename res=save_dataframe(df,filepath)
message=res["messages"] if res["status"]=="ERROR": raise
Exception(message) else: print(message) |
Python Linepraser APIs (for MMG version 8.1.0)
Table A-5 Python Linepraser APIs (for MMG version 8.1.0)
| Index | API Type | Functionality | Notebook Execution Script | Note |
|---|---|---|---|---|
| 1 | Attach Workspace | Attach the workspace for a session | %python mmg.attachWorkspace(workspace) #eg-
mmg.attachWorkspace("CS") |
workspace = workspace to be attached (String) Sets 'mmg' python variable which contains the information about the attached workspace Output- Print a boolean in one line on the status of workspace attachment and another line of boolean on status of AIFenabled and attached |
| 2 | List Workspaces | Lists all workspaces available to a user | %python mmg.listWorkspaces() |
Sets 'ofs_wsList' python variable which is the list of all available workspaces Output- Print the list of workspaces |
| 3 | Check AIF | Check if AIF is set or not | %python print(aif__isAttached) |
Output- Boolean on the status of AIF |
| 4 | Get Connection Object | Returns cx_Oracle Connection Object | %python mmg.getConnection(schema_name) #eg-
mmg.getConnection("OFSAA") |
schema_name = Name of the schema for which connection is required (String) Sets 'conn' python variable which is the cx_Oracle connection object to be used for firing queries |
| 5 | Publish | Fetch and sets the Notebook JSON dump | %python mmg.publish() |
Sets 'emf_para' python variable which is the JSON dump for the notebook component Output- Print the set 'emf_para' value |
| 6 | Register | Register the Notebook | %python mmg.register(register) #eg-
mmg.register({"modelname":"model1",\ # "modeldescription":"Model
description"}) #eg- mmg.register(emf_para) {After doing
mmg.publish()} |
register = JSON Stringify object for RegisterNotebookBean Object Sets 'publishedNotebookId' python variable which is the studio notebook id for the published version. Output- Prints the published notebook ID |
| 7 | Load Flat File | %python mmg.loadFlatFile() #eg- mmg.loadFlatFile()
Incomplete |
||
| 8 | Profile Data | %python mmg.profileData(code,title) #eg-
mmg.profileData(ABC,XYZ) |
code = title = |