Detect Objects Using Deep Learning

Detect Objects Using Deep Learning The Detect Objects Using Deep Learning tool runs a trained deep learning model on an input raster to produce a feature class containing the objects it finds. The features can be polygons around the objects found, or points at the centers of the found objects.

Note:

This functionality is currently only supported in Map Viewer Classic (formerly known as Map Viewer). It will be available in a future release of the next-generation Map Viewer (formerly known as Map Viewer Beta).

If you do not see this tool in Map Viewer Classic, contact your organization administrator. You may not have image analysis privileges, available with the ArcGIS Image for ArcGIS Online license.

Workflow diagram

Detect Objects Using Deep Learning workflow

Examples

  • Given an image of a neighborhood, detect the location of every roof with a solar panel.
  • Given an image of wilderness in northern Canada, detect the location of every glacial lake.

Usage notes

The input deep learning model for this tool must be a deep learning package (.dlpk) item stored in your portal. You can generate a .dlpk item using the Train Deep Learning Model geoprocessing tool in ArcGIS Pro or the ArcGIS REST API raster analysis tool.

The input .dlpk item must include an Esri model definition file (.emd). See the sample .emd file below.

{
    "Framework" :"TensorFlow",
    "ModelConfiguration": "ObjectDetectionAPI",
    
    "ModelFile": ".\\CoconutTreeDetection.model",
    "ModelType": "ObjectDetection",
    "ImageHeight": 850,
    "ImageWidth": 850,
    "ExtractBands": [0,1,2],
    "ImageSpaceUsed": "MAP_SPACE"
    "Classes": [
    {
        "Value": 0,
        "Name": "CoconutTree",
        "Color": [0, 255, 0]
    }
    ]
}

Check the box to turn on Non Maximum Suppression to identify and remove duplicate features from the object detection.

If Use current map extent is checked, only the pixels that are visible in the current map extent will be analyzed. If unchecked, the entire input imagery layer will be analyzed.

The parameters for this tool are listed in the following table:

ParameterExplanation
Choose image used to detect objects

The input image that will be used to classify objects.

Choose deep learning model used to detect objects

The input deep learning package (.dlpk) item.

The deep learning package contained the Esri model definition JSON file (.emd), the deep learning binary model file, and optionally, the Python raster function to be used.

Specify deep learning model arguments

The function arguments are defined in the Python raster function referenced by the input model. This is where you list additional deep learning parameters and arguments for refinement, such as a confidence threshold for adjusting sensitivity.

The names of the arguments are populated by the tool from reading the Python module.

Remove duplicate features from the output (optional)

Specifies whether nonmaximum suppression will be performed in which duplicate found objects are identified and the duplicated feature with the lower confidence value is removed.

  • Unchecked—All objects that are detected will be in the output feature class. This is the default.
  • Checked—Duplicate objects that are detected will be removed.

Confidence score field

The field in the feature service that will contain the confidence scores output by the object detection method.

This parameter is required when the Non maximum suppression parameter is checked.

Class value field

The class value field in the output feature service. If not specified, the tool will use the standard class value fields Classvalue and Value. If these fields do not exist, all features will be treated as the same object class.

This parameter is required when the Non maximum suppression parameter is checked.

Maximum overlap ratio

The maximum overlap ratio for two overlapping features, which is defined as the ratio of intersection area over union area. The default is 0.

This parameter is required when the Non maximum suppression parameter is checked.

Processing mode

Specifies how all raster items in an image service will be processed.

  • Process as mosaicked image—All raster items in the image service will be mosaicked together and processed. This is the default.
  • Process all raster items separately—All raster items in the image service will be processed as separate images.
.

Result layer name

The name of the layer that will be created in My Content and added to the map. The default name is based on the tool name and the input layer name. If the layer already exists, you will be prompted to provide another name.

You can specify the name of a folder in My Content where the result will be saved using the Save result in drop-down box.

Tip:

Click Show Credits before you run your analysis to check how many credits will be consumed.

Environments

Analysis environment settings are additional parameters that affect a tool's results. You can access the tool's analysis environment settings by clicking the gear icon Analysis Environments at the top of the tool pane.

This tool honors the following Analysis Environments:

  • Extent—Specifies the area to be used for analysis.
  • Cell size—The cell size to use in the output layer.
  • Recycle interval of processing workers—Defines how many image sections to process before restarting worker processes.
  • Parallel processing factor—Controls the raster processing CPU or GPU instances.
  • Number of retries on failures—Defines how many retries a worker process will attempt when there is random failure processing a job.

Similar tools and raster functions

Use the Detect Objects Using Deep Learning tool to detect objects in an image. Other tools may be useful in solving similar problems.

Map Viewer Classic analysis tools and raster functions

Use the Classify Objects Using Deep Learning tool to detect the location of objects in an image. Use the Classify Pixels Using Deep Learning tool to classify each pixel in an image.

ArcGIS Pro analysis tools and raster functions

The Detect Objects Using Deep Learning geoprocessing tool is available in the Image Analyst toolbox. Other tools in the Deep Learning toolset perform deep learning workflows.

ArcGIS Enterprise developer resources

If you are working in ArcGIS REST API, use the Detect Objects Using Deep Learning operation.

If you are working in ArcGIS API for Python, perform deep learning tasks ArcGIS for Python API website using the arcgis.learn module.