The Deep Learning toolset contains tools that detect specific features in an image and classify pixels in a raster dataset.
Deep learning is a type of machine learning artificial intelligence that detects features in imagery using multiple layers in neural networks in which one or more unique features in the image can be extracted to each layer. The tools in the Deep Learning toolset use GPU processing to perform analysis in a timely manner.
These ArcGIS AllSource tools consume the models that have been trained to detect specific features in third-party deep learning frameworks—such as TensorFlow, CNTK, and PyTorch—and output features or class maps.
The following table lists the deep learning tools and provides a brief description of each:
| Tool | Description |
|---|---|
Runs a trained deep learning model on an input raster and an optional feature class to produce a feature class or table in which each input object or feature has an assigned class or category label. This tool requires a model definition file containing trained model information. The model can be trained using the Train Deep Learning Model tool or by a third-party training software such as TensorFlow, PyTorch, or Keras. The model definition file can be an Esri model definition JSON file (.emd) or a deep learning model package, and it must contain the path to the Python raster function to be called to process each object and the path to the trained binary deep learning model file. | |
Runs a trained deep learning model on an input raster to produce a classified raster, with each valid pixel having an assigned class label. This tool requires a model definition file containing trained model information. The model can be trained using the Train Deep Learning Model tool or by a third-party training software such as TensorFlow, PyTorch, or Keras. The model definition file can be an Esri model definition JSON file (.emd) or a deep learning model package, and it must contain the path to the Python raster function to be called to process each object and the path to the trained binary deep learning model file. | |
Calculates the accuracy of a deep learning model by comparing the detected objects from the Detect Objects Using Deep Learning tool to ground truth data. | |
Runs a trained deep learning model to detect change between two rasters. This tool requires a model definition file containing trained model information. The model definition file can be an Esri model definition JSON file (.emd) or a deep learning model package, and it must contain the path to the Python raster function to be called to process each object and the path to the trained binary deep learning model file. | |
Detects ground control points in a mosaic dataset. | |
Runs a trained deep learning model on an input raster to produce a feature class containing the objects it finds. The features can be bounding boxes or polygons around the objects found or points at the centers of the objects. This tool requires a model definition file containing trained model information. The model can be trained using the Train Deep Learning Model tool or by a third-party training software such as TensorFlow, PyTorch, or Keras. The model definition file can be an Esri model definition JSON file (.emd) or a deep learning model package, and it must contain the path to the Python raster function to be called to process each object and the path to the trained binary deep learning model file. | |
Converts labeled vector or raster data to deep learning training datasets using a remote sensing image. The output is a folder of image chips and a folder of metadata files in the specified format. | |
Runs one or more pretrained deep learning models on an input raster to extract features and automate the postprocessing of the inferenced outputs. | |
Identifies duplicate features from the output of the Detect Objects Using Deep Learning tool as a postprocessing step and creates a new output with no duplicate features. The Detect Objects Using Deep Learning tool can return more than one bounding box or polygon for the same object, especially as a tiling side effect. If two features overlap more than a given maximum ratio, the feature with the lower confidence value will be removed. | |
Trains a deep learning model using the output from the Export Training Data For Deep Learning tool. | |
Trains a deep learning model by building training pipelines and automating much of the training process. This includes data augmentation, model selection, hyperparameter tuning, and batch size deduction. Its outputs include performance metrics of the best model on the training data, as well as the trained deep learning model package (.dlpk file) that can be used as input for the Extract Features Using AI Models tool to predict on new imagery. |