An overview of the Deep Learning toolset

The Deep Learning toolset contains tools that detect specific features in an image and classify pixels in a raster dataset.

Deep learning is a type of machine learning artificial intelligence that detects features in imagery using multiple layers in neural networks in which one or more unique features in the image can be extracted to each layer. The tools in the Deep Learning toolset use GPU processing to perform analysis in a timely manner.

These ArcGIS AllSource tools consume the models that have been trained to detect specific features in third-party deep learning frameworks—such as TensorFlow, CNTK, and PyTorch—and output features or class maps.

The following table lists the deep learning tools and provides a brief description of each:

ToolDescription

Classify Objects Using Deep Learning

Runs a trained deep learning model on an input raster and an optional feature class to produce a feature class or table in which each input object or feature has an assigned class or category label.

This tool requires a model definition file containing trained model information. The model can be trained using the Train Deep Learning Model tool or by a third-party training software such as TensorFlow, PyTorch, or Keras. The model definition file can be an Esri model definition JSON file (.emd) or a deep learning model package, and it must contain the path to the Python raster function to be called to process each object and the path to the trained binary deep learning model file.

Classify Pixels Using Deep Learning

Runs a trained deep learning model on an input raster to produce a classified raster, with each valid pixel having an assigned class label.

This tool requires a model definition file containing trained model information. The model can be trained using the Train Deep Learning Model tool or by a third-party training software such as TensorFlow, PyTorch, or Keras. The model definition file can be an Esri model definition JSON file (.emd) or a deep learning model package, and it must contain the path to the Python raster function to be called to process each object and the path to the trained binary deep learning model file.

Compute Accuracy For Object Detection

Calculates the accuracy of a deep learning model by comparing the detected objects from the Detect Objects Using Deep Learning tool to ground truth data.

Detect Change Using Deep Learning

Runs a trained deep learning model to detect change between two rasters.

This tool requires a model definition file containing trained model information. The model definition file can be an Esri model definition JSON file (.emd) or a deep learning model package, and it must contain the path to the Python raster function to be called to process each object and the path to the trained binary deep learning model file.

Detect Objects Using Deep Learning

Runs a trained deep learning model on an input raster to produce a feature class containing the objects it finds. The features can be bounding boxes or polygons around the objects found or points at the centers of the objects.

This tool requires a model definition file containing trained model information. The model can be trained using the Train Deep Learning Model tool or by a third-party training software such as TensorFlow, PyTorch, or Keras. The model definition file can be an Esri model definition JSON file (.emd) or a deep learning model package, and it must contain the path to the Python raster function to be called to process each object and the path to the trained binary deep learning model file.

Export Training Data For Deep Learning

Converts labeled vector or raster data to deep learning training datasets using a remote sensing image. The output is a folder of image chips and a folder of metadata files in the specified format.

Extract Features Using AI Models

Runs one or more pretrained deep learning models on an input raster to extract features and automate the postprocessing of the inferenced outputs.

Non Maximum Suppression

Identifies duplicate features from the output of the Detect Objects Using Deep Learning tool as a postprocessing step and creates a new output with no duplicate features. The Detect Objects Using Deep Learning tool can return more than one bounding box or polygon for the same object, especially as a tiling side effect. If two features overlap more than a given maximum ratio, the feature with the lower confidence value will be removed.

Train Deep Learning Model

Trains a deep learning model using the output from the Export Training Data For Deep Learning tool.

Train Using AutoDL

Trains a deep learning model by building training pipelines and automating much of the training process. This includes data augmentation, model selection, hyperparameter tuning, and batch size deduction. Its outputs include performance metrics of the best model on the training data, as well as the trained deep learning model package (.dlpk file) that can be used as input for the Extract Features Using AI Models tool to predict on new imagery.

Tools in the Deep Learning toolset

Related topics