Segmentation models perform a pixel-wise classification by classifying the pixels into different classes. The classified pixels correspond to different objects or regions in the image. These models have a wide variety of use cases across multiple domains. When used with satellite and aerial imagery, these models can help to identify features such as building footprints, roads, water bodies, crop fields, and so on.
Generally, every segmentation model must be trained from scratch using a dataset labeled with the objects of interest. This can be an arduous and time-consuming task. Segment Anything Model (SAM) by Meta AI is aimed at creating a foundational model that can be used to segment (as the name suggests) anything using zero-shot learning and generalize across domains without additional training.
SAM is trained on the Segment Anything 1-Billion mask dataset (SA-1B), which comprises a diverse set of 11 million images and over 1 billion masks. This makes the model highly robust in identifying object boundaries and differentiating between various objects across domains, even though it might have never seen them before. Use this model to extract masks of various objects in any image.
License requirements
To complete this workflow, the following are the license requirements:
- ArcGIS Desktop—ArcGIS Image Analyst extension for ArcGIS Pro
- ArcGIS Enterprise—ArcGIS Image Server
- ArcGIS Online—ArcGIS Image for ArcGIS Online
Model details
This model has the following characteristics:
- Input—8-bit, 3-band imagery.
- Output—Feature class containing masks of various objects in the image.
- Compute—This workflow is compute-intensive, and a GPU with minimum CUDA compute capability of 6.0 is recommended. This model requires a GPU with at least 8 GB of GPU memory.
- Applicable geographies—The model is expected to work globally.
- Architecture—This model is based on the open-source Segment Anything Model (SAM) by Meta AI.
Access and download the model
Download the Segment Anything Model (SAM) pretrained model from ArcGIS Living Atlas of the World. Alternatively, access the model directly from ArcGIS Pro using the Detect Objects Using Deep Learning tool, or consume it in ArcGIS Image for ArcGIS Online.
- Browse to ArcGIS Living Atlas of the World.
- Sign in with your ArcGIS Online credentials.
- Search for Segment Anything Model and open the item page from the search results.
- Click the Download button to download the model.
You can use the downloaded .dlpk file directly in ArcGIS Pro, or upload and use it in ArcGIS Enterprise. Additionally, you can fine-tune the pretrained model if necessary.
Release notes
The following are the release notes:
| Date | Description |
|---|---|
| May 2023 |
First release of Segment Anything Model (SAM) |