he context of which the profile is uploaded, e.g.:from zenml.integrations.whylogs.steps import get_whylogs_profiler_step

train_data_profiler = get_whylogs_profiler_step(dataset_id="model-2")

test_data_profiler = get_whylogs_profiler_step(dataset_id="model-3")

The step can then be inserted into your pipeline where it can take in a pandas.DataFrame dataset, e.g.:

from zenml import pipeline

@pipeline

def data_profiling_pipeline():

data, _ = data_loader()

train, test = data_splitter(data)

train_data_profiler(train)

test_data_profiler(test)

data_profiling_pipeline()

As can be seen from the step definition , the step takes in a dataset and returns a whylogs DatasetProfileView object:

@step

def whylogs_profiler_step(

dataset: pd.DataFrame,

dataset_timestamp: Optional[datetime.datetime] = None,

) -> DatasetProfileView:

...

You should consult the official whylogs documentation for more information on what you can do with the collected profiles.

You can view the complete list of configuration parameters in the SDK docs.

The whylogs Data Validator

The whylogs Data Validator implements the same interface as do all Data Validators, so this method forces you to maintain some level of compatibility with the overall Data Validator abstraction, which guarantees an easier migration in case you decide to switch to another Data Validator.

All you have to do is call the whylogs Data Validator methods when you need to interact with whylogs to generate data profiles. You may optionally enable whylabs logging to automatically upload the returned whylogs profile to WhyLabs, e.g.:

import pandas as pd

from whylogs.core import DatasetProfileView

from zenml.integrations.whylogs.data_validators.whylogs_data_validator import (

WhylogsDataValidator,

from zenml.integrations.whylogs.flavors.whylogs_data_validator_flavor import (

WhylogsDataValidatorSettings,

from zenml import step

whylogs_settings = WhylogsDataValidatorSettings(

enable_whylabs=True, dataset_id="<WHYLABS_DATASET_ID>"

@step(

settings={