Use the model

You can use the Prompt Based Segmentation model in the Classify Pixels Using Deep Learning tool available in the Image Analyst toolbox in ArcGIS Pro.

Complete the following steps to use the Prompt Based Segmentation pretrained model:

1. Download the model and add the imagery layer in ArcGIS Pro.

   

2. Zoom to an area of interest.
3. Click the Analysis tab and browse to Tools.

   

4. Click the Toolboxes tab in the Geoprocessing pane, select Image Analyst Tools, and browse to the Classify Pixels Using Deep Learning tool under Deep Learning.

   

5. On the Parameters tab, set the variables as follows:
   1. Input Raster—Select the imagery.
   2. Output Classified Raster—Set the output raster that will contain the binary classification results.
   3. Model Definition—Select the pretrained model .dlpk file.
   4. Arguments (optional)—Change the values of the arguments if required.
      • prompt—Text that describes the feature to be segmented. For visual prompting, an image URL or a local path can be provided.
      • threshold—The pixel classifications with the probability value higher than this threshold are included in the result. The allowed values range from 0 to 1.0. This threshold value will be effective only when return_probability_raster is set to False.
      • batch_size—Number of image tiles processed in each step of the model inference. This depends on the memory of your graphic card.
      • return_probability_raster—If True, the output classified raster will be a continuous magnitude raster indicating probability value at each pixel.
      • padding—Number of pixels at the border of image tiles from which predictions are blended for adjacent tiles. Increase its value to smooth the output while reducing edge artifacts. The maximum value of the padding can be half of the tile size value.
      • test_time_augmentation—Performs test time augmentation while predicting. If True, predictions of flipped and rotated variants of the input image will be merged into the final output.
      • merge policy—Policy for merging predictions (mean, min, or max). Applicable when test_time_augmentation is True.
   
6. On the Environments tab, set the variables as follows:
   1. Processing Extent—Select Current Display Extent or any other option from the drop-down menu.
   2. Cell Size—Set the value appropriately.

      Select the cell size in meters in such a way that maximizes the visibility of the objects of interest throughout the chosen extent. Consider a larger cell size for detecting larger objects and a smaller cell size for detecting smaller objects. For example, set the cell size for cloud detection to 10 meters, while for car detection, set it to 0.30 meters (30 centimeters). For further information regarding cell size, refer to the provided resource.

   3. Processor Type—Select CPU or GPU as needed.

      If GPU is available, it is recommended that you select GPU and set GPU ID to the GPU to be used.

   
7. Click Run.

   Once processing is complete, the output layer is added to the map.