Deep learning capabilities are available in ArcGIS AllSource for imagery and point clouds through several tools and capabilities.
Model training
Before you can use a deep learning model to identify features or objects in an image, point cloud, or other dataset, you must first train it to recognize those objects. Training a deep learning model involves many of the same steps as training a traditional machine learning classification model. You must collect and provide training samples and input data; then train the model so it learns to recognize those features or objects.
Training data preparation
The Label Objects for Deep Learning pane is used to collect and generate labeled imagery datasets to train a deep learning model for imagery workflows. You can interactively identify and label objects in an image, and export the training data as the image chips, labels, and statistics required to train a model. If you have existing labeled vector or raster data, you can use the Export Training Data For Deep Learning tool to generate the training data needed for the next step.
The Prepare Point Cloud Training Data tool creates data for training and validating a convolutional neural network for point cloud classification. This tool creates many overlapping blocks of uncompressed HDF5 files used to train a point cloud. For more information about preparing and training point cloud data, see Train a deep learning model for point cloud classification.
Model preparation and training
The Train Deep Learning Model tool trains a deep learning model for imagery workflows using prepared training data. Various model types and arguments are available to configure the training process.
The Train Point Cloud Classification Model tool trains a deep learning model for point cloud classification. For more information about preparing and training point cloud data, see Train a deep learning model for point cloud classification.
Model inferencing
Model inferencing refers to the process of extracting information from an image or point cloud using a trained model. The options for model inferencing in ArcGIS AllSource are as follows:
- Detect objects—Generate bounding boxes around objects or features in an image to identify their location. Use the Detect Objects Using Deep Learning tool.
- Classify objects—Generate labels for features in an image to identify their class or category. Use the Classify Objects Using Deep Learning tool.
- Classify pixels—Generate a classified raster where each pixel belongs to a class or category. Use the Classify Pixels Using Deep Learning tool.
- Classify point clouds—Generate a classified point cloud where points are classified into specific classification codes. Use the Classify Point Cloud Using Trained Model tool. See Classify a point cloud with deep learning to learn more.
Exploratory analysis
The Object Detection exploratory analysis tool uses a trained deep learning model to recognize objects displayed in the current map or scene. Each identified feature is represented by a point feature with a location in the coordinate system of the map, attributes detailing the orientation and extent of the object, and its confidence value. The tool can work with any trained Faster R-CNN model, as well as YOLO, Single Shot Detector (SSD), and RetinaNet models, and is designed for on-demand detection of objects in the active view.
Model results
You can review the results of a deep learning model at two stages: after training the model and after running an inferencing tool.
Model training results
When training a deep learning model for imagery, the output from the Train Deep Learning Model tool includes a file named model_metric.html. This file contains information about the trained model, such as the learning rate, training and validation loss, and the average precision score.
When training a deep learning model for point clouds, the output from the Train Point Cloud Classification Model tool includes results in the messages section of the tool results window. The detailed report contains training loss, validation loss, and accuracy at each epoch, as well as precision, recall, and f1_score for the saved deep learning model. The tool also generates a CSV table that contains the precision, recall, and F1 score for each class code per epoch. For more information about reviewing the training results, see Assess point cloud training results.
Model inferencing results
After using a deep learning model, review the results and assess the accuracy of the model.
Use the Attributes pane to review the results from the object-based inferencing (Classify Objects Using Deep Learningtool or Detect Objects Using Deep Learning tool) or from exploratory analysis (interactive Objection Detection tool).
You can use the Compute Accuracy For Object Detection tool to generate a table and report for accuracy assessment after performing object detection.