 us know!

Configuration at pipeline or step levelWhen running your ZenML pipeline with the Sagemaker orchestrator, the configuration set when configuring the orchestrator as a ZenML component will be used by default. However, it is possible to provide additional configuration at the pipeline or step level. This allows you to run whole pipelines or individual steps with alternative configurations. For example, this allows you to run the training process with a heavier, GPU-enabled instance type, while running other steps with lighter instances.

Additional configuration for the Sagemaker orchestrator can be passed via SagemakerOrchestratorSettings. Here, it is possible to configure processor_args, which is a dictionary of arguments for the Processor. For available arguments, see the Sagemaker documentation . Currently, it is not possible to provide custom configuration for the following attributes:

image_uri

instance_count

sagemaker_session

entrypoint

base_job_name

env

For example, settings can be provided in the following way:

sagemaker_orchestrator_settings = SagemakerOrchestratorSettings(

processor_args={

"instance_type": "ml.t3.medium",

"volume_size_in_gb": 30

They can then be applied to a step as follows:

@step(settings={"orchestrator.sagemaker": sagemaker_orchestrator_settings})

For example, if your ZenML component is configured to use ml.c5.xlarge with 400GB additional storage by default, all steps will use it except for the step above, which will use ml.t3.medium with 30GB additional storage.

Check out this docs page for more information on how to specify settings in general.

For more information and a full list of configurable attributes of the Sagemaker orchestrator, check out the SDK Docs .

S3 data access in ZenML steps