_operator

@step(step_operator=step_operator.name)def step_on_spark(...) -> ...:

...

Additional configuration

For additional configuration of the Spark step operator, you can pass SparkStepOperatorSettings when defining or running your pipeline. Check out the SDK docs for a full list of available attributes and this docs page for more information on how to specify settings.

PreviousAzureML

NextDevelop a Custom Step Operator

Last updated 15 days ago