Compute Accuracy For Pixel Classification (Image Analyst)

Available with Image Analyst license.

Summary

Computes a confusion matrix, based on errors of omission and commission, and the Intersection over Union (IoU) score. The accuracy is computed between the output from the Classify Pixels Using Deep Learning tool and the ground truth data.

The tool is only valid for pixel classification models, not other models used with the Classify Pixels Using Deep Learning tool.

Usage

• This tool calculates the user and producer accuracy for each class as well as an overall kappa index of agreement. These accuracy rates range from 0 to 1 in which 1 represents 100 percent accuracy. The following is an example of a confusion matrix:

  Confusion matrix example

  	c_1	c_2	c_3	Total	U_Accuracy	Kappa
  c_1	49	4	4	57	0.8594	0
  c_2	2	40	2	44	0.9091	0
  c_3	3	3	59	65	0.9077	0
  Total	54	47	65	166	0	0
  P_Accuracy	0.9074	0.8511	0.9077	0	0.8916	0
  Kappa	0	0	0	0	0	0.8357

• User accuracy is a measure of the pixels that are correctly classified out of all the pixels that are classified as a given class. The remainder of the pixels classified into this class belong to a different class, or are identified as a different class, according to the reference information; they are false positives.

  User accuracy is also referred to as errors of commission, or type 1 error. The data to compute this error rate is read from the rows of the table.

  The Total row shows the number of points that should have been identified as a given class according to the reference data.

• Producer accuracy is a false negative in which pixels of a known class are classified as something other than that class. An example is when the classified image identifies a pixel as forest, but it should be impervious. In this case, the impervious class is missing pixels according to the reference data.

  Producer accuracy is also referred to as errors of omission, or type 2 error. The data to compute this error rate is read from the columns of the table.

  The Total column shows the number of points that were identified as a given class according to the classified map.

• Kappa index of agreement provides an overall assessment of the accuracy of the classification.

• Intersection over Union (IoU) is the area of overlap between the predicted segmentation and the ground truth divided by the area of union between the predicted segmentation and the ground truth. The mean IoU value is computed for each class.

Parameters

Dialog

Label	Explanation	Data Type
Input Raster	The input classified raster.	Mosaic Layer; Raster Layer; Image Service; String; Raster Dataset; Mosaic Dataset
Ground Truth Data	The input classification image or other thematic GIS reference data. The input can be a raster or feature class. Typical data is a classification image of a single band of integer data type. If using polygons as input, only use those that are not used as training samples. They can also be GIS land-cover data in shapefile or feature class format.	Mosaic Layer; Raster Layer; Feature Layer
Output Confusion Matrix	The output file name of the confusion matrix in table format. The format of the table is determined by the output location and path. By default, the output will be a geodatabase table. If the path is not in a geodatabase, specify the .dbf extension to save it in dBASE format.	Table
Number of Random Points (Optional)	The total number of random points that will be generated. The actual number may exceed but never fall below this number, depending on sampling strategy and number of classes. The default number of randomly generated points is 500.	Long
Sampling Strategy (Optional)	Specifies the sampling scheme that will be used. Stratified random—Randomly distributed points will be created in each class in which each class has a number of points proportional to its relative area. This is the default Equalized stratified random—Randomly distributed points will be created in each class in which each class has the same number of points. Random—Randomly distributed points will be created throughout the image.	String
Minimum Point Distance (Optional)	The minimum distance between the reference points. The default is 0.	Double

Python

ComputeAccuracyForPixelClassification(in_raster, in_ground_truth_data, out_confusion_matrix, {num_random_points}, {sampling}, {min_point_distance})

Name	Explanation	Data Type
in_raster	The input classified raster.	Mosaic Layer; Raster Layer; Image Service; String; Raster Dataset; Mosaic Dataset
in_ground_truth_data	The input classification image or other thematic GIS reference data. The input can be a raster or feature class. Typical data is a classification image of a single band of integer data type. If using polygons as input, only use those that are not used as training samples. They can also be GIS land-cover data in shapefile or feature class format.	Mosaic Layer; Raster Layer; Feature Layer
out_confusion_matrix	The output file name of the confusion matrix in table format. The format of the table is determined by the output location and path. By default, the output will be a geodatabase table. If the path is not in a geodatabase, specify the .dbf extension to save it in dBASE format.	Table
num_random_points (Optional)	The total number of random points that will be generated. The actual number may exceed but never fall below this number, depending on sampling strategy and number of classes. The default number of randomly generated points is 500.	Long
sampling (Optional)	Specifies the sampling scheme that will be used. STRATIFIED_RANDOM—Randomly distributed points will be created in each class in which each class has a number of points proportional to its relative area. This is the default EQUALIZED_STRATIFIED_RANDOM—Randomly distributed points will be created in each class in which each class has the same number of points. RANDOM—Randomly distributed points will be created throughout the image.	String
min_point_distance (Optional)	The minimum distance between the reference points. The default is 0.	Double

Code sample

ComputeAccuracyForPixelClassification example 1 (Python window)

This example generates a confusion matrix using 500 stratified random points.

import arcpy
arcpy.env.workspace = r"C:\Data\MyProject.gdb"

arcpy.ia.ComputeAccuracyForPixelClassification("BGYRre_ps16_ClassifyPixel", 
    "Classified_202412301648436631322", "BGYNIR_ps16_ComputeAccura", 500, 
    "STRATIFIED_RANDOM", 0)

ComputeAccuracyForPixelClassification example 2 (stand-alone script)

This example generates a confusion matrix using 500 random points.

# Import system modules
import arcpy
from arcpy.ia import *

# Check out the ArcGIS Image Analyst extension license
arcpy.CheckOutExtension("ImageAnalyst")

# Set local variables
in_raster=r"C:\DL\MyProject.gdb\BGYClassifyPixel"
in_ground_truth=r"C:\DL\MyProject.gdb\Classified_2024"
out__matrix=r"C:\DL\MyProject.gdb\BGYN_ComputeAccura" 
num_random_points=500
sampling="RANDOM"
min_point_distance=10

# Execute 
arcpy.ia.ComputeAccuracyForPixelClassification(in_raster, in_ground_truth, 
    out__matrix, num_random_points, sampling, min_point_distance)

Environments

Extent, Output Coordinate System