thon script can be found here .

Export: job -> S3Data from within the job (e.g. produced by the training process, or when preprocessing large data) can be exported as well. The structure is highly similar to that of importing data. Copying data to S3 can be configured with output_data_s3_mode, which supports EndOfJob (default) and Continuous.

In the simple case, data in /opt/ml/processing/output/data will be copied to S3 at the end of a job:

sagemaker_orchestrator_settings = SagemakerOrchestratorSettings(

output_data_s3_mode="EndOfJob",

output_data_s3_uri="s3://some-results-bucket-name/results"

In a more complex case, data in /opt/ml/processing/output/data/metadata and /opt/ml/processing/output/data/checkpoints will be written away continuously:

sagemaker_orchestrator_settings = SagemakerOrchestratorSettings(

output_data_s3_mode="Continuous",

output_data_s3_uri={

"metadata": "s3://some-results-bucket-name/metadata",

"checkpoints": "s3://some-results-bucket-name/checkpoints"

Enabling CUDA for GPU-backed hardware

Note that if you wish to use this orchestrator to run steps on a GPU, you will need to follow the instructions on this page to ensure that it works. It requires adding some extra settings customization and is essential to enable CUDA for the GPU to give its full acceleration.

PreviousGoogle Cloud VertexAI Orchestrator

NextTekton Orchestrator

Last updated 19 days ago