File size: 48,464 Bytes

---
library_name: pytorch
license: apache-2.0
tags:
- foundation
- android
pipeline_tag: image-segmentation

---

![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/sam/web-assets/model_demo.png)

# Segment-Anything-Model: Optimized for Mobile Deployment
## High-quality segmentation mask generation around any object in an image with simple input prompt


Transformer based encoder-decoder where prompts specify what to segment in an image thereby allowing segmentation without the need for additional training. The image encoder generates embeddings and the lightweight decoder operates on the embeddings for point and mask based image segmentation.

This model is an implementation of Segment-Anything-Model found [here](https://github.com/facebookresearch/segment-anything).


This repository provides scripts to run Segment-Anything-Model on Qualcomm® devices.
More details on model performance across various devices, can be found
[here](https://aihub.qualcomm.com/models/sam).


### Model Details

- **Model Type:** Semantic segmentation
- **Model Stats:**
  - Model checkpoint: vit_l
  - Input resolution: 720p (720x1280)
  - Number of parameters (SAMDecoder): 5.11M
  - Model size (SAMDecoder): 19.6 MB

| Model | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
|---|---|---|---|---|---|---|---|---|
| SAMDecoder | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 7.455 ms | 0 - 32 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 6.857 ms | 6 - 9 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.so) |
| SAMDecoder | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | ONNX | 11.054 ms | 1 - 66 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.onnx) |
| SAMDecoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | TFLITE | 5.215 ms | 0 - 37 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 4.318 ms | 4 - 23 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.so) |
| SAMDecoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 7.82 ms | 2 - 57 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.onnx) |
| SAMDecoder | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 5.108 ms | 0 - 38 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 4.824 ms | 4 - 46 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | ONNX | 8.303 ms | 2 - 50 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.onnx) |
| SAMDecoder | SA7255P ADP | SA7255P | TFLITE | 53.0 ms | 0 - 31 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | SA7255P ADP | SA7255P | QNN | 49.808 ms | 4 - 11 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 7.476 ms | 0 - 28 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | SA8255 (Proxy) | SA8255P Proxy | QNN | 6.857 ms | 4 - 6 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | SA8295P ADP | SA8295P | TFLITE | 10.548 ms | 0 - 30 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | SA8295P ADP | SA8295P | QNN | 9.556 ms | 0 - 10 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 7.459 ms | 0 - 35 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | SA8650 (Proxy) | SA8650P Proxy | QNN | 6.875 ms | 4 - 7 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | SA8775P ADP | SA8775P | TFLITE | 10.483 ms | 0 - 32 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | SA8775P ADP | SA8775P | QNN | 9.676 ms | 0 - 8 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | QCS8275 (Proxy) | QCS8275 Proxy | TFLITE | 53.0 ms | 0 - 31 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | QCS8275 (Proxy) | QCS8275 Proxy | QNN | 49.808 ms | 4 - 11 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 7.463 ms | 0 - 29 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 6.851 ms | 4 - 7 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | QCS9075 (Proxy) | QCS9075 Proxy | TFLITE | 10.483 ms | 0 - 32 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | QCS9075 (Proxy) | QCS9075 Proxy | QNN | 9.676 ms | 0 - 8 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | QCS8450 (Proxy) | QCS8450 Proxy | TFLITE | 9.166 ms | 0 - 38 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.tflite) |
| SAMDecoder | QCS8450 (Proxy) | QCS8450 Proxy | QNN | 8.229 ms | 4 - 44 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 7.424 ms | 4 - 4 MB | FP16 | NPU | Use Export Script |
| SAMDecoder | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 14.715 ms | 13 - 13 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMDecoder.onnx) |
| SAMEncoderPart1 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 208.174 ms | 12 - 78 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 175.417 ms | 14 - 16 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.so) |
| SAMEncoderPart1 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | ONNX | 165.892 ms | 24 - 127 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.onnx) |
| SAMEncoderPart1 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | TFLITE | 147.8 ms | 11 - 658 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 126.997 ms | 12 - 28 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.so) |
| SAMEncoderPart1 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 122.647 ms | 24 - 694 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.onnx) |
| SAMEncoderPart1 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 144.144 ms | 10 - 664 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 141.645 ms | 12 - 659 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | ONNX | 108.212 ms | 23 - 668 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.onnx) |
| SAMEncoderPart1 | SA7255P ADP | SA7255P | TFLITE | 1173.603 ms | 0 - 644 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | SA7255P ADP | SA7255P | QNN | 1103.265 ms | 4 - 12 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 209.255 ms | 12 - 69 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | SA8255 (Proxy) | SA8255P Proxy | QNN | 179.225 ms | 12 - 14 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | SA8295P ADP | SA8295P | TFLITE | 242.469 ms | 12 - 640 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | SA8295P ADP | SA8295P | QNN | 207.255 ms | 0 - 11 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 208.157 ms | 12 - 69 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | SA8650 (Proxy) | SA8650P Proxy | QNN | 177.435 ms | 13 - 16 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | SA8775P ADP | SA8775P | TFLITE | 249.667 ms | 12 - 656 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | SA8775P ADP | SA8775P | QNN | 211.757 ms | 1 - 9 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | QCS8275 (Proxy) | QCS8275 Proxy | TFLITE | 1173.603 ms | 0 - 644 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | QCS8275 (Proxy) | QCS8275 Proxy | QNN | 1103.265 ms | 4 - 12 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 207.823 ms | 12 - 76 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 177.311 ms | 12 - 15 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | QCS9075 (Proxy) | QCS9075 Proxy | TFLITE | 249.667 ms | 12 - 656 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | QCS9075 (Proxy) | QCS9075 Proxy | QNN | 211.757 ms | 1 - 9 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | QCS8450 (Proxy) | QCS8450 Proxy | TFLITE | 232.379 ms | 12 - 993 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.tflite) |
| SAMEncoderPart1 | QCS8450 (Proxy) | QCS8450 Proxy | QNN | 223.767 ms | 12 - 971 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 174.73 ms | 12 - 12 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart1 | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 180.556 ms | 38 - 38 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart1.onnx) |
| SAMEncoderPart2 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 663.719 ms | 12 - 112 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.tflite) |
| SAMEncoderPart2 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 733.441 ms | 13 - 15 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.so) |
| SAMEncoderPart2 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | ONNX | 749.685 ms | 18 - 220 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.onnx) |
| SAMEncoderPart2 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 545.405 ms | 12 - 28 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.so) |
| SAMEncoderPart2 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 591.244 ms | 24 - 1422 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.onnx) |
| SAMEncoderPart2 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 473.547 ms | 11 - 1142 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.tflite) |
| SAMEncoderPart2 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 527.677 ms | 12 - 1112 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart2 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | ONNX | 460.831 ms | 36 - 1406 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.onnx) |
| SAMEncoderPart2 | SA7255P ADP | SA7255P | QNN | 1885.474 ms | 4 - 11 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart2 | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 656.701 ms | 11 - 93 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.tflite) |
| SAMEncoderPart2 | SA8255 (Proxy) | SA8255P Proxy | QNN | 731.619 ms | 12 - 15 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart2 | SA8295P ADP | SA8295P | TFLITE | 707.412 ms | 12 - 1172 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.tflite) |
| SAMEncoderPart2 | SA8295P ADP | SA8295P | QNN | 786.133 ms | 0 - 11 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart2 | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 684.922 ms | 12 - 104 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.tflite) |
| SAMEncoderPart2 | SA8650 (Proxy) | SA8650P Proxy | QNN | 737.991 ms | 12 - 14 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart2 | SA8775P ADP | SA8775P | TFLITE | 702.795 ms | 0 - 1144 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.tflite) |
| SAMEncoderPart2 | SA8775P ADP | SA8775P | QNN | 740.054 ms | 2 - 8 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart2 | QCS8275 (Proxy) | QCS8275 Proxy | QNN | 1885.474 ms | 4 - 11 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart2 | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 653.785 ms | 12 - 103 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.tflite) |
| SAMEncoderPart2 | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 734.787 ms | 12 - 14 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart2 | QCS9075 (Proxy) | QCS9075 Proxy | TFLITE | 702.795 ms | 0 - 1144 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.tflite) |
| SAMEncoderPart2 | QCS9075 (Proxy) | QCS9075 Proxy | QNN | 740.054 ms | 2 - 8 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart2 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 633.697 ms | 12 - 12 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart2 | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 745.618 ms | 53 - 53 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart2.onnx) |
| SAMEncoderPart3 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 669.851 ms | 12 - 110 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.tflite) |
| SAMEncoderPart3 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 741.286 ms | 12 - 15 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.so) |
| SAMEncoderPart3 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | ONNX | 748.764 ms | 0 - 183 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.onnx) |
| SAMEncoderPart3 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 545.914 ms | 12 - 27 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.so) |
| SAMEncoderPart3 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 580.246 ms | 36 - 1433 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.onnx) |
| SAMEncoderPart3 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 428.668 ms | 11 - 1141 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.tflite) |
| SAMEncoderPart3 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 576.105 ms | 12 - 1112 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart3 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | ONNX | 459.599 ms | 35 - 1408 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.onnx) |
| SAMEncoderPart3 | SA7255P ADP | SA7255P | QNN | 1886.342 ms | 4 - 10 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart3 | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 664.029 ms | 12 - 111 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.tflite) |
| SAMEncoderPart3 | SA8255 (Proxy) | SA8255P Proxy | QNN | 741.521 ms | 12 - 15 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart3 | SA8295P ADP | SA8295P | TFLITE | 707.193 ms | 12 - 1175 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.tflite) |
| SAMEncoderPart3 | SA8295P ADP | SA8295P | QNN | 784.205 ms | 0 - 11 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart3 | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 672.362 ms | 12 - 105 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.tflite) |
| SAMEncoderPart3 | SA8650 (Proxy) | SA8650P Proxy | QNN | 737.027 ms | 12 - 15 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart3 | SA8775P ADP | SA8775P | TFLITE | 703.475 ms | 0 - 1144 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.tflite) |
| SAMEncoderPart3 | SA8775P ADP | SA8775P | QNN | 738.516 ms | 1 - 9 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart3 | QCS8275 (Proxy) | QCS8275 Proxy | QNN | 1886.342 ms | 4 - 10 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart3 | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 666.947 ms | 12 - 103 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.tflite) |
| SAMEncoderPart3 | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 738.236 ms | 12 - 15 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart3 | QCS9075 (Proxy) | QCS9075 Proxy | TFLITE | 703.475 ms | 0 - 1144 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.tflite) |
| SAMEncoderPart3 | QCS9075 (Proxy) | QCS9075 Proxy | QNN | 738.516 ms | 1 - 9 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart3 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 644.658 ms | 12 - 12 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart3 | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 751.206 ms | 51 - 51 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart3.onnx) |
| SAMEncoderPart4 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 674.618 ms | 12 - 104 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.tflite) |
| SAMEncoderPart4 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 746.674 ms | 12 - 14 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.so) |
| SAMEncoderPart4 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | ONNX | 736.999 ms | 24 - 228 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.onnx) |
| SAMEncoderPart4 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 545.535 ms | 12 - 28 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.so) |
| SAMEncoderPart4 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 585.741 ms | 35 - 1436 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.onnx) |
| SAMEncoderPart4 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 472.497 ms | 11 - 1140 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.tflite) |
| SAMEncoderPart4 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 530.712 ms | 12 - 1113 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart4 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | ONNX | 439.426 ms | 36 - 1409 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.onnx) |
| SAMEncoderPart4 | SA7255P ADP | SA7255P | QNN | 1884.573 ms | 0 - 9 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart4 | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 653.028 ms | 12 - 103 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.tflite) |
| SAMEncoderPart4 | SA8255 (Proxy) | SA8255P Proxy | QNN | 744.778 ms | 12 - 15 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart4 | SA8295P ADP | SA8295P | TFLITE | 705.387 ms | 12 - 1174 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.tflite) |
| SAMEncoderPart4 | SA8295P ADP | SA8295P | QNN | 782.232 ms | 0 - 11 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart4 | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 656.723 ms | 12 - 112 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.tflite) |
| SAMEncoderPart4 | SA8650 (Proxy) | SA8650P Proxy | QNN | 749.36 ms | 14 - 16 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart4 | SA8775P ADP | SA8775P | TFLITE | 711.855 ms | 0 - 1145 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.tflite) |
| SAMEncoderPart4 | SA8775P ADP | SA8775P | QNN | 740.717 ms | 1 - 9 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart4 | QCS8275 (Proxy) | QCS8275 Proxy | QNN | 1884.573 ms | 0 - 9 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart4 | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 660.012 ms | 12 - 104 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.tflite) |
| SAMEncoderPart4 | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 746.363 ms | 12 - 14 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart4 | QCS9075 (Proxy) | QCS9075 Proxy | TFLITE | 711.855 ms | 0 - 1145 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.tflite) |
| SAMEncoderPart4 | QCS9075 (Proxy) | QCS9075 Proxy | QNN | 740.717 ms | 1 - 9 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart4 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 623.063 ms | 12 - 12 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart4 | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 734.372 ms | 51 - 51 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart4.onnx) |
| SAMEncoderPart5 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 663.865 ms | 12 - 104 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.tflite) |
| SAMEncoderPart5 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 745.203 ms | 12 - 15 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.so) |
| SAMEncoderPart5 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | ONNX | 738.266 ms | 24 - 216 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.onnx) |
| SAMEncoderPart5 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 544.125 ms | 12 - 27 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.so) |
| SAMEncoderPart5 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 588.519 ms | 23 - 1423 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.onnx) |
| SAMEncoderPart5 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 427.751 ms | 12 - 1142 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.tflite) |
| SAMEncoderPart5 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 580.371 ms | 12 - 1111 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart5 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | ONNX | 462.137 ms | 24 - 1397 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.onnx) |
| SAMEncoderPart5 | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 674.516 ms | 12 - 109 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.tflite) |
| SAMEncoderPart5 | SA8255 (Proxy) | SA8255P Proxy | QNN | 727.73 ms | 12 - 15 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart5 | SA8295P ADP | SA8295P | TFLITE | 706.687 ms | 12 - 1172 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.tflite) |
| SAMEncoderPart5 | SA8295P ADP | SA8295P | QNN | 784.357 ms | 0 - 10 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart5 | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 655.632 ms | 12 - 103 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.tflite) |
| SAMEncoderPart5 | SA8650 (Proxy) | SA8650P Proxy | QNN | 735.971 ms | 16 - 18 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart5 | SA8775P ADP | SA8775P | TFLITE | 703.846 ms | 0 - 1145 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.tflite) |
| SAMEncoderPart5 | SA8775P ADP | SA8775P | QNN | 740.709 ms | 1 - 8 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart5 | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 671.243 ms | 12 - 104 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.tflite) |
| SAMEncoderPart5 | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 742.409 ms | 12 - 14 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart5 | QCS9075 (Proxy) | QCS9075 Proxy | TFLITE | 703.846 ms | 0 - 1145 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.tflite) |
| SAMEncoderPart5 | QCS9075 (Proxy) | QCS9075 Proxy | QNN | 740.709 ms | 1 - 8 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart5 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 624.498 ms | 12 - 12 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart5 | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 731.48 ms | 53 - 53 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart5.onnx) |
| SAMEncoderPart6 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 676.887 ms | 12 - 104 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.tflite) |
| SAMEncoderPart6 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 745.987 ms | 12 - 15 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.so) |
| SAMEncoderPart6 | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | ONNX | 742.01 ms | 12 - 209 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.onnx) |
| SAMEncoderPart6 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 544.172 ms | 12 - 28 MB | FP16 | NPU | [Segment-Anything-Model.so](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.so) |
| SAMEncoderPart6 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 589.508 ms | 32 - 1433 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.onnx) |
| SAMEncoderPart6 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 430.345 ms | 11 - 1139 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.tflite) |
| SAMEncoderPart6 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 569.23 ms | 12 - 1111 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart6 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | ONNX | 460.98 ms | 36 - 1407 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.onnx) |
| SAMEncoderPart6 | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 669.101 ms | 12 - 113 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.tflite) |
| SAMEncoderPart6 | SA8255 (Proxy) | SA8255P Proxy | QNN | 746.045 ms | 12 - 15 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart6 | SA8295P ADP | SA8295P | TFLITE | 708.764 ms | 12 - 1174 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.tflite) |
| SAMEncoderPart6 | SA8295P ADP | SA8295P | QNN | 783.365 ms | 0 - 11 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart6 | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 633.429 ms | 12 - 104 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.tflite) |
| SAMEncoderPart6 | SA8650 (Proxy) | SA8650P Proxy | QNN | 749.732 ms | 12 - 14 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart6 | SA8775P ADP | SA8775P | TFLITE | 711.885 ms | 0 - 1144 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.tflite) |
| SAMEncoderPart6 | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 674.605 ms | 12 - 114 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.tflite) |
| SAMEncoderPart6 | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 741.375 ms | 12 - 14 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart6 | QCS9075 (Proxy) | QCS9075 Proxy | TFLITE | 711.885 ms | 0 - 1144 MB | FP16 | NPU | [Segment-Anything-Model.tflite](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.tflite) |
| SAMEncoderPart6 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 641.457 ms | 12 - 12 MB | FP16 | NPU | Use Export Script |
| SAMEncoderPart6 | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 736.709 ms | 51 - 51 MB | FP16 | NPU | [Segment-Anything-Model.onnx](https://huggingface.co/qualcomm/Segment-Anything-Model/blob/main/SAMEncoderPart6.onnx) |




## Installation


Install the package via pip:
```bash
pip install "qai-hub-models[sam]"
```


## Configure Qualcomm® AI Hub to run this model on a cloud-hosted device

Sign-in to [Qualcomm® AI Hub](https://app.aihub.qualcomm.com/) with your
Qualcomm® ID. Once signed in navigate to `Account -> Settings -> API Token`.

With this API token, you can configure your client to run models on the cloud
hosted devices.
```bash
qai-hub configure --api_token API_TOKEN
```
Navigate to [docs](https://app.aihub.qualcomm.com/docs/) for more information.



## Demo off target

The package contains a simple end-to-end demo that downloads pre-trained
weights and runs this model on a sample input.

```bash
python -m qai_hub_models.models.sam.demo
```

The above demo runs a reference implementation of pre-processing, model
inference, and post processing.

**NOTE**: If you want running in a Jupyter Notebook or Google Colab like
environment, please add the following to your cell (instead of the above).
```
%run -m qai_hub_models.models.sam.demo
```


### Run model on a cloud-hosted device

In addition to the demo, you can also run the model on a cloud-hosted Qualcomm®
device. This script does the following:
* Performance check on-device on a cloud-hosted device
* Downloads compiled assets that can be deployed on-device for Android.
* Accuracy check between PyTorch and on-device outputs.

```bash
python -m qai_hub_models.models.sam.export
```
```
Profiling Results
------------------------------------------------------------
SAMDecoder
Device                          : Samsung Galaxy S23 (13)
Runtime                         : TFLITE                 
Estimated inference time (ms)   : 7.5                    
Estimated peak memory usage (MB): [0, 32]                
Total # Ops                     : 845                    
Compute Unit(s)                 : NPU (845 ops)          

------------------------------------------------------------
SAMEncoderPart1
Device                          : Samsung Galaxy S23 (13)
Runtime                         : TFLITE                 
Estimated inference time (ms)   : 208.2                  
Estimated peak memory usage (MB): [12, 78]               
Total # Ops                     : 584                    
Compute Unit(s)                 : NPU (584 ops)          

------------------------------------------------------------
SAMEncoderPart2
Device                          : Samsung Galaxy S23 (13)
Runtime                         : TFLITE                 
Estimated inference time (ms)   : 663.7                  
Estimated peak memory usage (MB): [12, 112]              
Total # Ops                     : 572                    
Compute Unit(s)                 : NPU (572 ops)          

------------------------------------------------------------
SAMEncoderPart3
Device                          : Samsung Galaxy S23 (13)
Runtime                         : TFLITE                 
Estimated inference time (ms)   : 669.9                  
Estimated peak memory usage (MB): [12, 110]              
Total # Ops                     : 572                    
Compute Unit(s)                 : NPU (572 ops)          

------------------------------------------------------------
SAMEncoderPart4
Device                          : Samsung Galaxy S23 (13)
Runtime                         : TFLITE                 
Estimated inference time (ms)   : 674.6                  
Estimated peak memory usage (MB): [12, 104]              
Total # Ops                     : 572                    
Compute Unit(s)                 : NPU (572 ops)          

------------------------------------------------------------
SAMEncoderPart5
Device                          : Samsung Galaxy S23 (13)
Runtime                         : TFLITE                 
Estimated inference time (ms)   : 663.9                  
Estimated peak memory usage (MB): [12, 104]              
Total # Ops                     : 572                    
Compute Unit(s)                 : NPU (572 ops)          

------------------------------------------------------------
SAMEncoderPart6
Device                          : Samsung Galaxy S23 (13)
Runtime                         : TFLITE                 
Estimated inference time (ms)   : 676.9                  
Estimated peak memory usage (MB): [12, 104]              
Total # Ops                     : 573                    
Compute Unit(s)                 : NPU (573 ops)          
```


## How does this work?

This [export script](https://aihub.qualcomm.com/models/sam/qai_hub_models/models/Segment-Anything-Model/export.py)
leverages [Qualcomm® AI Hub](https://aihub.qualcomm.com/) to optimize, validate, and deploy this model
on-device. Lets go through each step below in detail:

Step 1: **Compile model for on-device deployment**

To compile a PyTorch model for on-device deployment, we first trace the model
in memory using the `jit.trace` and then call the `submit_compile_job` API.

```python
import torch

import qai_hub as hub
from qai_hub_models.models.sam import Model

# Load the model
model = Model.from_pretrained()
decoder_model = model.decoder
encoder_splits[0]_model = model.encoder_splits[0]
encoder_splits[1]_model = model.encoder_splits[1]
encoder_splits[2]_model = model.encoder_splits[2]
encoder_splits[3]_model = model.encoder_splits[3]
encoder_splits[4]_model = model.encoder_splits[4]
encoder_splits[5]_model = model.encoder_splits[5]

# Device
device = hub.Device("Samsung Galaxy S23")

# Trace model
decoder_input_shape = decoder_model.get_input_spec()
decoder_sample_inputs = decoder_model.sample_inputs()

traced_decoder_model = torch.jit.trace(decoder_model, [torch.tensor(data[0]) for _, data in decoder_sample_inputs.items()])

# Compile model on a specific device
decoder_compile_job = hub.submit_compile_job(
    model=traced_decoder_model ,
    device=device,
    input_specs=decoder_model.get_input_spec(),
)

# Get target model to run on-device
decoder_target_model = decoder_compile_job.get_target_model()
# Trace model
encoder_splits[0]_input_shape = encoder_splits[0]_model.get_input_spec()
encoder_splits[0]_sample_inputs = encoder_splits[0]_model.sample_inputs()

traced_encoder_splits[0]_model = torch.jit.trace(encoder_splits[0]_model, [torch.tensor(data[0]) for _, data in encoder_splits[0]_sample_inputs.items()])

# Compile model on a specific device
encoder_splits[0]_compile_job = hub.submit_compile_job(
    model=traced_encoder_splits[0]_model ,
    device=device,
    input_specs=encoder_splits[0]_model.get_input_spec(),
)

# Get target model to run on-device
encoder_splits[0]_target_model = encoder_splits[0]_compile_job.get_target_model()
# Trace model
encoder_splits[1]_input_shape = encoder_splits[1]_model.get_input_spec()
encoder_splits[1]_sample_inputs = encoder_splits[1]_model.sample_inputs()

traced_encoder_splits[1]_model = torch.jit.trace(encoder_splits[1]_model, [torch.tensor(data[0]) for _, data in encoder_splits[1]_sample_inputs.items()])

# Compile model on a specific device
encoder_splits[1]_compile_job = hub.submit_compile_job(
    model=traced_encoder_splits[1]_model ,
    device=device,
    input_specs=encoder_splits[1]_model.get_input_spec(),
)

# Get target model to run on-device
encoder_splits[1]_target_model = encoder_splits[1]_compile_job.get_target_model()
# Trace model
encoder_splits[2]_input_shape = encoder_splits[2]_model.get_input_spec()
encoder_splits[2]_sample_inputs = encoder_splits[2]_model.sample_inputs()

traced_encoder_splits[2]_model = torch.jit.trace(encoder_splits[2]_model, [torch.tensor(data[0]) for _, data in encoder_splits[2]_sample_inputs.items()])

# Compile model on a specific device
encoder_splits[2]_compile_job = hub.submit_compile_job(
    model=traced_encoder_splits[2]_model ,
    device=device,
    input_specs=encoder_splits[2]_model.get_input_spec(),
)

# Get target model to run on-device
encoder_splits[2]_target_model = encoder_splits[2]_compile_job.get_target_model()
# Trace model
encoder_splits[3]_input_shape = encoder_splits[3]_model.get_input_spec()
encoder_splits[3]_sample_inputs = encoder_splits[3]_model.sample_inputs()

traced_encoder_splits[3]_model = torch.jit.trace(encoder_splits[3]_model, [torch.tensor(data[0]) for _, data in encoder_splits[3]_sample_inputs.items()])

# Compile model on a specific device
encoder_splits[3]_compile_job = hub.submit_compile_job(
    model=traced_encoder_splits[3]_model ,
    device=device,
    input_specs=encoder_splits[3]_model.get_input_spec(),
)

# Get target model to run on-device
encoder_splits[3]_target_model = encoder_splits[3]_compile_job.get_target_model()
# Trace model
encoder_splits[4]_input_shape = encoder_splits[4]_model.get_input_spec()
encoder_splits[4]_sample_inputs = encoder_splits[4]_model.sample_inputs()

traced_encoder_splits[4]_model = torch.jit.trace(encoder_splits[4]_model, [torch.tensor(data[0]) for _, data in encoder_splits[4]_sample_inputs.items()])

# Compile model on a specific device
encoder_splits[4]_compile_job = hub.submit_compile_job(
    model=traced_encoder_splits[4]_model ,
    device=device,
    input_specs=encoder_splits[4]_model.get_input_spec(),
)

# Get target model to run on-device
encoder_splits[4]_target_model = encoder_splits[4]_compile_job.get_target_model()
# Trace model
encoder_splits[5]_input_shape = encoder_splits[5]_model.get_input_spec()
encoder_splits[5]_sample_inputs = encoder_splits[5]_model.sample_inputs()

traced_encoder_splits[5]_model = torch.jit.trace(encoder_splits[5]_model, [torch.tensor(data[0]) for _, data in encoder_splits[5]_sample_inputs.items()])

# Compile model on a specific device
encoder_splits[5]_compile_job = hub.submit_compile_job(
    model=traced_encoder_splits[5]_model ,
    device=device,
    input_specs=encoder_splits[5]_model.get_input_spec(),
)

# Get target model to run on-device
encoder_splits[5]_target_model = encoder_splits[5]_compile_job.get_target_model()

```


Step 2: **Performance profiling on cloud-hosted device**

After compiling models from step 1. Models can be profiled model on-device using the
`target_model`. Note that this scripts runs the model on a device automatically
provisioned in the cloud.  Once the job is submitted, you can navigate to a
provided job URL to view a variety of on-device performance metrics.
```python
decoder_profile_job = hub.submit_profile_job(
    model=decoder_target_model,
    device=device,
)
encoder_splits[0]_profile_job = hub.submit_profile_job(
    model=encoder_splits[0]_target_model,
    device=device,
)
encoder_splits[1]_profile_job = hub.submit_profile_job(
    model=encoder_splits[1]_target_model,
    device=device,
)
encoder_splits[2]_profile_job = hub.submit_profile_job(
    model=encoder_splits[2]_target_model,
    device=device,
)
encoder_splits[3]_profile_job = hub.submit_profile_job(
    model=encoder_splits[3]_target_model,
    device=device,
)
encoder_splits[4]_profile_job = hub.submit_profile_job(
    model=encoder_splits[4]_target_model,
    device=device,
)
encoder_splits[5]_profile_job = hub.submit_profile_job(
    model=encoder_splits[5]_target_model,
    device=device,
)

```

Step 3: **Verify on-device accuracy**

To verify the accuracy of the model on-device, you can run on-device inference
on sample input data on the same cloud hosted device.
```python
decoder_input_data = decoder_model.sample_inputs()
decoder_inference_job = hub.submit_inference_job(
    model=decoder_target_model,
    device=device,
    inputs=decoder_input_data,
)
decoder_inference_job.download_output_data()
encoder_splits[0]_input_data = encoder_splits[0]_model.sample_inputs()
encoder_splits[0]_inference_job = hub.submit_inference_job(
    model=encoder_splits[0]_target_model,
    device=device,
    inputs=encoder_splits[0]_input_data,
)
encoder_splits[0]_inference_job.download_output_data()
encoder_splits[1]_input_data = encoder_splits[1]_model.sample_inputs()
encoder_splits[1]_inference_job = hub.submit_inference_job(
    model=encoder_splits[1]_target_model,
    device=device,
    inputs=encoder_splits[1]_input_data,
)
encoder_splits[1]_inference_job.download_output_data()
encoder_splits[2]_input_data = encoder_splits[2]_model.sample_inputs()
encoder_splits[2]_inference_job = hub.submit_inference_job(
    model=encoder_splits[2]_target_model,
    device=device,
    inputs=encoder_splits[2]_input_data,
)
encoder_splits[2]_inference_job.download_output_data()
encoder_splits[3]_input_data = encoder_splits[3]_model.sample_inputs()
encoder_splits[3]_inference_job = hub.submit_inference_job(
    model=encoder_splits[3]_target_model,
    device=device,
    inputs=encoder_splits[3]_input_data,
)
encoder_splits[3]_inference_job.download_output_data()
encoder_splits[4]_input_data = encoder_splits[4]_model.sample_inputs()
encoder_splits[4]_inference_job = hub.submit_inference_job(
    model=encoder_splits[4]_target_model,
    device=device,
    inputs=encoder_splits[4]_input_data,
)
encoder_splits[4]_inference_job.download_output_data()
encoder_splits[5]_input_data = encoder_splits[5]_model.sample_inputs()
encoder_splits[5]_inference_job = hub.submit_inference_job(
    model=encoder_splits[5]_target_model,
    device=device,
    inputs=encoder_splits[5]_input_data,
)
encoder_splits[5]_inference_job.download_output_data()

```
With the output of the model, you can compute like PSNR, relative errors or
spot check the output with expected output.

**Note**: This on-device profiling and inference requires access to Qualcomm®
AI Hub. [Sign up for access](https://myaccount.qualcomm.com/signup).



## Run demo on a cloud-hosted device

You can also run the demo on-device.

```bash
python -m qai_hub_models.models.sam.demo --on-device
```

**NOTE**: If you want running in a Jupyter Notebook or Google Colab like
environment, please add the following to your cell (instead of the above).
```
%run -m qai_hub_models.models.sam.demo -- --on-device
```


## Deploying compiled model to Android


The models can be deployed using multiple runtimes:
- TensorFlow Lite (`.tflite` export): [This
  tutorial](https://www.tensorflow.org/lite/android/quickstart) provides a
  guide to deploy the .tflite model in an Android application.


- QNN (`.so` export ): This [sample
  app](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/sample_app.html)
provides instructions on how to use the `.so` shared library  in an Android application.


## View on Qualcomm® AI Hub
Get more details on Segment-Anything-Model's performance across various devices [here](https://aihub.qualcomm.com/models/sam).
Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)


## License
* The license for the original implementation of Segment-Anything-Model can be found
  [here](https://github.com/facebookresearch/segment-anything/blob/main/LICENSE).
* The license for the compiled assets for on-device deployment can be found [here](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/Qualcomm+AI+Hub+Proprietary+License.pdf)



## References
* [Segment Anything](https://arxiv.org/abs/2304.02643)
* [Source Model Implementation](https://github.com/facebookresearch/segment-anything)



## Community
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
* For questions or feedback please [reach out to us](mailto:[email protected]).