mlm_insights.core.readers.interfaces package¶
Submodules¶
mlm_insights.core.readers.interfaces.data_reader module¶
- class mlm_insights.core.readers.interfaces.data_reader.DataReader(name: str, engine: str, **kwargs: Any)¶
Bases:
ABCAbstract Base Class for all Insights readers. The child class must implement read and make_schema methods.
Note: Do not derive from this class directly. Instead, use Execution Engine specific reader base classes.
- classmethod create(config: Dict[str, Any]) DataReader¶
- get_engine() str¶
Returns the name of engine which was set during initialization.
Returns¶
- engine_name: str
Name of engine, eg ‘dask’, ‘spark’
- get_name() str¶
Returns the name of reader which was set during initialization.
Returns¶
- reader_name: str
Name of the reader
- abstract read(schema_provider: SchemaProvider, **kwargs: Any) Any¶
Read the data from the data-source and return engine specific data frame.
Parameters¶
- schema_providerSchemaProvider
Engine specific schema provider. Use it to call get_engine_schema() to get engine specific schema to pass to the respective read methods
- kwargs:
Key-value pair for dynamic arguments
Returns¶
- dataframe: Any
Engine specific DataFrame