mlm_insights.tests.test_types.metric_based_tests package

Subpackages

Submodules

mlm_insights.tests.test_types.metric_based_tests.common module

mlm_insights.tests.test_types.metric_based_tests.common.get_feature_tag(feature_name: str) → Dict[str, str]

mlm_insights.tests.test_types.metric_based_tests.test_is_complete module

class mlm_insights.tests.test_types.metric_based_tests.test_is_complete.TestIsComplete(user_defined_tags: ~typing.Dict[str, str] = <factory>, feature_name: str | None = None, threshold_value: int | float | None = None)

Bases: MetricBasedTestBase

A metric-based test which tests whether completion percentage of a feature is greater than the threshold value (in percentage), using ‘Count’ metric.

Configuration

None

Example - Using Insights Test Builder API

test_builder = InsightsTestBuilder() 
    .with_context(test_context=get_test_context()) 
    .add_test(TestIsComplete(
        feature_name = 'feature_a',
        threshold_value=90.0)
test_runner = test_builder.build()
test_result = test_runner.run()

Example - Using Insights Configuration

{
    "test_config": {
        "feature_tests": [
            {
                "feature_name": "feature_a",
                "tests": [
                    {
                        "test_name": "TestIsComplete",
                        "threshold_value": 95.0
                    }
                ]
            }
        ]
}
execute(test_context: TestContext, *args: Any, **kwargs: Any) → TestResult

Execute the test and return the TestResult. It uses TestContext to query required metric data.

Parameters

test_context : TestContext, args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestResult

feature_name: str = None
threshold_value: int | float | None = None
validate(test_context: TestContext, *args: Any, **kwargs: Any) → TestValidationResult

Pre validated the test before executing. It can be used to validate all its parameters and test_context.

Parameters

test_context : TestContext args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestValidationResult

mlm_insights.tests.test_types.metric_based_tests.test_is_matching_inference_type module

class mlm_insights.tests.test_types.metric_based_tests.test_is_matching_inference_type.InferenceType(value)

Bases: Enum

Enumeration describing the various supported types for evaluating inference type of a feature

BOOLEAN = 'Boolean'
FLOAT = 'Float'
INTEGER = 'Integer'
NONE = 'None'
STRING = 'String'
class mlm_insights.tests.test_types.metric_based_tests.test_is_matching_inference_type.TestIsMatchingInferenceType(user_defined_tags: ~typing.Dict[str, str] = <factory>, feature_name: str | None = None, threshold_value: ~mlm_insights.tests.test_types.metric_based_tests.test_is_matching_inference_type.InferenceType | None = None)

Bases: MetricBasedTestBase

A metric-based test which tests if all the values in a feature match a data-type specified by the threshold value. This test is based on metric

TypeMetric.

The threshold value can either be provided in test config or validated against the inference type in reference profile.

Configuration

feature_name: str

Name of the feature to be tested.

threshold_value: Optional[InferenceType]

Asserts that the feature values are of the data type specified by threshold_value. If not provided, the test uses REFERENCE profile.

Accepted values for threshold_value: Integer, String, Float, Boolean, None (Test fails if threshold_value is None and no reference profile is provided)

Example - Using Insights Test Builder API

test_builder = InsightsTestBuilder()
    .with_context(test_context=get_test_context())
    .add_test(TestIsMatchingInferenceType(
        feature_name = 'feature_a',
        threshold_value=InferenceType.INTEGER)
test_runner = test_builder.build()
test_result = test_runner.run()

Example - Using Insights Configuration

{
    "test_config": {
        "feature_tests": [
            {
                "feature_name": "feature_a",
                "tests": [
                    {
                      "test_name": "TestIsMatchingInferenceType",
                      "threshold_value": "Integer"
                    }
                ]
            }
        ]
}
execute(test_context: TestContext, *args: Any, **kwargs: Any) → TestResult

Execute the test and return the TestResult. It uses TestContext to query required metric data.

Parameters

test_context : TestContext, args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestResult

feature_name: str = None
threshold_value: InferenceType | None = None
validate(test_context: TestContext, *args: Any, **kwargs: Any) → TestValidationResult

Pre validated the test before executing. It can be used to validate all its parameters and test_context.

Parameters

test_context : TestContext args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestValidationResult

mlm_insights.tests.test_types.metric_based_tests.test_is_negative module

class mlm_insights.tests.test_types.metric_based_tests.test_is_negative.TestIsNegative(user_defined_tags: ~typing.Dict[str, str] = <factory>, feature_name: str | None = None, METRIC_KEY: str = 'IsNegative')

Bases: MetricBasedTestBase

A metric-based test which tests if all the values in a feature are negative using IsNegative metric

Configuration

None

Example - Using Insights Test Builder API

test_builder = InsightsTestBuilder()
    .with_context(test_context=get_test_context())
    .add_test(TestIsNegative(feature_name = 'feature_a')
test_runner = test_builder.build()
test_result = test_runner.run()

Example - Using Insights Configuration

{
    "test_config": {
        "feature_tests": [
            {
                "feature_name": "feature_a",
                "tests": [
                    {
                      "test_name": "TestIsNegative"
                    }
                ]
            }
        ]
}
METRIC_KEY: str = 'IsNegative'
execute(test_context: TestContext, *args: Any, **kwargs: Any) → TestResult

Execute the test and return the TestResult. It uses TestContext to query required metric data.

Parameters

test_context : TestContext, args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestResult

feature_name: str = None
validate(test_context: TestContext, *args: Any, **kwargs: Any) → TestValidationResult

Pre validated the test before executing. It can be used to validate all its parameters and test_context.

Parameters

test_context : TestContext args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestValidationResult

mlm_insights.tests.test_types.metric_based_tests.test_is_non_zero module

class mlm_insights.tests.test_types.metric_based_tests.test_is_non_zero.TestIsNonZero(user_defined_tags: ~typing.Dict[str, str] = <factory>, feature_name: str | None = None)

Bases: MetricBasedTestBase

A metric-based test which tests if all the values in a feature are non-zero using IsNonZero metric

Configuration

None

Example - Using Insights Test Builder API

test_builder = InsightsTestBuilder()
    .with_context(test_context=get_test_context())
    .add_test(TestIsNonZero(feature_name = 'feature_a')
test_runner = test_builder.build()
test_result = test_runner.run()

Example - Using Insights Configuration

{
    "test_config": {
        "feature_tests": [
            {
                "feature_name": "feature_a",
                "tests": [
                    {
                      "test_name": "TestIsNonZero"
                    }
                ]
            }
        ]
}
execute(test_context: TestContext, *args: Any, **kwargs: Any) → TestResult

Execute the test and return the TestResult. It uses TestContext to query required metric data.

Parameters

test_context : TestContext, args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestResult

feature_name: str = None
validate(test_context: TestContext, *args: Any, **kwargs: Any) → TestValidationResult

Pre validated the test before executing. It can be used to validate all its parameters and test_context.

Parameters

test_context : TestContext args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestValidationResult

mlm_insights.tests.test_types.metric_based_tests.test_is_positive module

class mlm_insights.tests.test_types.metric_based_tests.test_is_positive.TestIsPositive(user_defined_tags: ~typing.Dict[str, str] = <factory>, feature_name: str | None = None, METRIC_KEY: str = 'IsPositive')

Bases: MetricBasedTestBase

A metric-based test which tests if all the values in a feature are positive using IsPositive metric

Configuration

None

Example - Using Insights Test Builder API

test_builder = InsightsTestBuilder()
    .with_context(test_context=get_test_context())
    .add_test(TestIsPositive(feature_name = 'feature_a')
test_runner = test_builder.build()
test_result = test_runner.run()

Example - Using Insights Configuration

{
    "test_config": {
        "feature_tests": [
            {
                "feature_name": "feature_a",
                "tests": [
                    {
                      "test_name": "TestIsPositive"
                    }
                ]
            }
        ]
}
METRIC_KEY: str = 'IsPositive'
execute(test_context: TestContext, *args: Any, **kwargs: Any) → TestResult

Execute the test and return the TestResult. It uses TestContext to query required metric data.

Parameters

test_context : TestContext, args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestResult

feature_name: str = None
validate(test_context: TestContext, *args: Any, **kwargs: Any) → TestValidationResult

Pre validated the test before executing. It can be used to validate all its parameters and test_context.

Parameters

test_context : TestContext args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestValidationResult

mlm_insights.tests.test_types.metric_based_tests.test_no_new_category module

class mlm_insights.tests.test_types.metric_based_tests.test_no_new_category.TestNoNewCategory(user_defined_tags: ~typing.Dict[str, str] = <factory>, feature_name: str | None = None, threshold_value: ~typing.List[str] | None = None)

Bases: MetricBasedTestBase

A metric-based test which asserts that no new categories are added in a feature. This test is based on metric

TopKFrequentElements, which considers only Top K elements.

Configuration

feature_name: str

Name of the feature under test.

threshold_value: Optional[List[str]]

Asserts that no other categories but the one in threshold_value appears. If not provide it’ll use REFERENCE profile

Example - Using Insights Test Builder API

test_builder = InsightsTestBuilder()
    .with_context(test_context=get_test_context())
    .add_test(TestNoNewCategory(
        feature_name = 'feature_a',
        threshold_value=['cat_a', 'cat_b'])
test_runner = test_builder.build()
test_result = test_runner.run()

Example - Using Insights Configuration

{
    "test_config": {
        "feature_tests": [
            {
                "feature_name": "feature_a",
                "tests": [
                    {
                      "test_name": "TestNoNewCategory",
                      "threshold_value": ["cat_a", "cat_b"]
                    }
                ]
            }
        ]
}
execute(test_context: TestContext, *args: Any, **kwargs: Any) → TestResult

Execute the test and return the TestResult. It uses TestContext to query required metric data.

Parameters

test_context : TestContext, args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestResult

feature_name: str = None
threshold_value: List[str] | None = None
validate(test_context: TestContext, *args: Any, **kwargs: Any) → TestValidationResult

Pre validated the test before executing. It can be used to validate all its parameters and test_context.

Parameters

test_context : TestContext args:

Additional arguments, implemented as per child class.

kwargs:

Key-value pair for dynamic arguments.

Returns

TestValidationResult

Module contents