Classify point clouds using deep learning

ArcGIS Pro allows you to use machine learning classification methods to classify point clouds. Deep learning is a type of machine learning that relies on multiple layers of nonlinear processing for feature identification and pattern recognition described in a model. The process typically requires a user to provide manual training locations of known assets. For example, if the need is to classify stop signs the user would first need to train the software to find examples of stop signs with known examples of the stop sign. Once enough training examples have been provided, the classification can be run on new point cloud data. ArcGIS has tools for training deep learning models and running pretrained models on point clouds.

The benefits of using deep learning to classify point cloud data include:

• Accuracy—Deep learning models can learn complex patterns and features in point cloud data. Additional training can be supplied to teach the model how to find variations of the trained target.
• Efficiency—Deep learning models can process large volumes of point cloud data that otherwise may not be provided by traditional classification techniques, providing an unsupervised method to classify point clouds.
• Automation—Deep learning allows for automated classification, reducing the need for manual intervention.
• Flexibility—Deep learning models can learn and adapt to new data, making them more flexible and adaptable than traditional classification methods.

If it’s not possible to train your own model, you can find various pretrained deep-learning models in Living Atlas that work with point clouds. These models can be used directly on lidar with minimal setup. ArcGIS pretrained deep learning models eliminate the need for huge volumes of training data, massive compute resources, and extensive artificial intelligence (AI) knowledge. Using a pretrained model eliminates a large amount of preprocessing to classify your lidar.

Workflows

With the following general steps, you can integrate deep learning models with ArcGIS Pro for point classification. For more details, reference the links throughout this document.

There are three general steps in using deep learning to classify point clouds:

1. Prepare training data—The first step to using deep learning with point clouds is to prepare the point cloud data for training. The Prepare Point Cloud Training Data geoprocessing tool generates training data and validates a convolutional neural network for point cloud classification.
2. Train a model—Use the Train Point Cloud Classification Model geoprocessing tool to train a deep learning model for point cloud classification.
3. Use the model—Use the trained model to run the Classify Point Cloud Using the Trained Model geoprocessing tool.

Considerations

It’s important to consider the following points before starting the process of training and implementing deep learning workflows for classifying point clouds:

• Install deep learning libraries.
• GPU—NVIDIA GPU with CUDA Compute Capability (CC). Required and recommended versions of CC are listed on the Deep Learning Libraries Installer.
• Minimum dedicated GPU RAM is 8 GB. This is more than the minimum requirement for image-based deep learning tools because point cloud processing requires more memory. For additional information on GPU requirements, see Deep learning frequently asked questions.
• ArcGIS 3D Analyst extension license.

Required software

You'll need ArcGIS Pro Standard with ArcGIS 3D Analyst to classify point clouds using deep learning. You may also want to use ArcGIS Online or ArcGIS Enterprise to share point clouds as 3D scene services.

Explore the following resources to learn more about using deep learning in ArcGIS.

ArcGIS help documentation

Reference material for ArcGIS products:

• Deep learning and point clouds
• Rule based classification techniques in ArcGIS Pro
• Train a deep learning model for point cloud classification
• Assess point cloud training results
• Classify a point cloud with deep learning in ArcGIS Pro
• Classification toolset in 3D Analyst
• Deep learning frequently asked questions—ArcGIS Pro
• Pretrained deep learning models for point clouds

ArcGIS blogs, articles, stories, and technical papers

Supplemental guidance about concepts, software functionality, and workflows:

• Pretrained deep learning models
• Deep learning models in ArcGIS
• Vegetation encroachment analysis in 3D using deep learning
• Pretrained deep learning models as of September 2022
• Classify a point cloud with deep learning in ArcGIS Pro
• How to fine-tune a pretrained deep learning model
• Benefits of deep learning and GIS
• Introducing pretrained deep learning models
• Pretrained deep learning models in Living Atlas

Videos

Esri-produced videos that clarify and demonstrate concepts, software functionality, and workflows:

• Introduction to deep learning
• Deep dive into deep learning
• How to use ArcGIS 3D Analyst for lidar classification and feature extraction
• Leverage deep learning models
• How you can integrate deep learning with ArcGIS using Python
• How to work with massive lidar, point cloud dataset using deep learning in ArcGIS Pro
• Deep-learning related Esri videos

Tutorials

Guided, hands-on lessons based on real-world problems:

• Classifying powerlines using deep learning

ArcGIS solutions

Industry-specific configurations for ArcGIS:

• 3D buildings solution
• Tree management solution

Esri training

Authoritative learning resources focusing on key ArcGIS skills:

• Deep learning in ArcGIS

Esri community

Online places for the Esri community to connect, collaborate, and share experiences:

• Ask the 3D community questions.