Standardize Field (Data Management)

Summary

Standardizes values in fields by converting them to values that follow a specified scale. Standardization methods include z-score, minimum-maximum, absolute maximum, and robust standardization.

Illustration

Standardize the values of a field using one of four methods.

Usage

Caution:

This tool modifies the input data. See Tools that modify or update the input data for more information and strategies to avoid undesired data changes.

• There are four standardization methods: Z-Score, Minimum-maximum, Absolute maximum, and Robust standardization.

  • The Z-Score method measures the difference between a value and the mean of all values in the field using standard deviations, also known as the standard score.
    • Potential application—Assess the significance of a value in relation to the distribution of values in a field. For example, a county's voter participation can be evaluated in the context of other counties across the country, helping identify typical voter participation patterns and counties with significantly high and low voter participation.
    • Consideration—This method expects a normal distribution. This method is not recommended if the distribution of the data is highly skewed.
    • Equation—, where x^' is the standardized value, x is the original value, x̄ is the mean (average), and σ_x is the standard deviation.
  • The Minimum-maximum method preserves the relationships among the original data values while converting the values to a scale between user-specified minimum and maximum values.
    • Potential application—A real estate assessor may want to scale characteristics of homes, such as the number of rooms in a house or the age of the house in years, to the same scale before using these characteristics in a model, such as the Forest-based Classification and Regression tool.
    • Consideration—This method is prone to influence by outliers, or extreme values, in the data.
    • Equation—, where x^' is the standardized value, x is the original value, min(x) is the minimum of the data, max(x) is the maximum of the data, a is the user-specified minimum, and b is the user-specified maximum.
  • The Absolute maximum method compares the difference between a value and the maximum absolute value in a distribution by dividing each value by the maximum absolute value in the field.
    • Potential application—This method is useful when working with data that has a stable and logical maximum and you want to compare each value to this maximum. For example, the number of votes in a county cannot contain more votes than the number of voting-age people in the county. The county with the highest proportion of votes becomes this maximum, and all other counties are assessed in relation to the absolute maximum voter participation.
    • Consideration—The output scale is between -1 and 1. Larger positive values correspond to values close to 1, and larger negative values correspond to values close to -1.
    • Equation—, where x^' is the standardized value, x is the original value, and max(|x|) is the maximum of the absolute values of the data.

  • The Robust standardization method standardizes the values in the specified fields using a robust variant of the z-score. This variant uses median and interquartile range in place of mean and standard deviation.
    • Potential application—A real estate assessor is attempting to estimate home values in a city, and an exclusive neighborhood with extremely high home values results in outliers in the data. The assessor uses robust standardization to mitigate the impact of these outliers in the distribution of home values for the city.
    • Consideration—With its use of median and interquartile range, this can be an effective method when attempting to mitigate the influence of outliers in the distribution.
    • Equation—, where x^' is the standardized value, x is the original value, median(x) is the median of the data, and IQR(x) is the interquartile range of the data.

• If multiple fields are provided, the specified standardization method is applied across all fields.

• The tool modifies the input data and appends the newly created standardized fields to the input table or feature class.

• For each selected field, summary statistics are provided in the geoprocessing message results. These include the maximum, minimum, sum, mean, standard deviation, median, skewness, and kurtosis.

• In the z-score standardization, the standard deviation is estimated by dividing the squared deviation from the mean by N, for N records, and taking the square root. Some implementations of standard deviation divide by (N-1), which causes differences in the estimated standard deviation.

Parameters

Dialog

Label	Explanation	Data Type
Input Table	The table containing the field with the values that will be standardized.	Table View; Raster Layer; Mosaic Layer
Field to Standardize	The fields containing the values that will be standardized. For each field, an output field name can be specified. If no output field name is provided, the tool will create an output field name using the field name and selected method.	Value Table
Standardization Method (Optional)	Specifies the method that will be used to standardize the values contained in the specified fields. Z-Score—The standard score (z-score), which is the number of standard deviations above or below the mean, will be used. The z-score formula calculates the difference between the value and the mean of the values in the column, divided by the standard deviation of the values in the column. This is the default. Minimum-maximum—The values will be converted to a scale between the user-specified minimum and maximum values. Absolute maximum—Each value in the column will be divided by the maximum absolute value in the column. Robust standardization— A robust variant of the z-score formula will be used to standardize the values in the specified fields. This variant uses median and interquartile range in place of mean and standard deviation.	String
Minimum Value (Optional)	The value that will be used by the Minimum-maximum method of the Standardization Method parameter to specify the minimum value in the scale of the provided output values.	Double
Maximum Value (Optional)	The value that will be used by the Minimum-maximum method of the Standardization Method parameter to specify the maximum value in the scale of the provided output values.	Double

Derived Output

Label	Explanation	Data Type
Updated Input Table	The table that contains the new encoded fields.	Table View

Python

arcpy.management.StandardizeField(in_table, fields, {method}, {min_value}, {max_value})

Name	Explanation	Data Type
in_table	The table containing the field with the values that will be standardized.	Table View; Raster Layer; Mosaic Layer
fields [[input_field, output_field],...]	The fields containing the values that will be standardized. For each field, an output field name can be specified. If no output field name is provided, the tool will create an output field name using the field name and selected method.	Value Table
method (Optional)	Specifies the method that will be used to standardize the values contained in the specified fields. Z-SCORE—The standard score (z-score), which is the number of standard deviations above or below the mean, will be used. The z-score formula calculates the difference between the value and the mean of the values in the column, divided by the standard deviation of the values in the column. This is the default. MIN-MAX—The values will be converted to a scale between the user-specified minimum and maximum values. MAXABS—Each value in the column will be divided by the maximum absolute value in the column. ROBUST— A robust variant of the z-score formula will be used to standardize the values in the specified fields. This variant uses median and interquartile range in place of mean and standard deviation.	String
min_value (Optional)	The value that will be used by the MIN-MAX method of the method parameter to specify the minimum value in the scale of the provided output values.	Double
max_value (Optional)	The value that will be used by the MIN-MAX method of the method parameter to specify the maximum value in the scale of the provided output values.	Double

Derived Output

Name	Explanation	Data Type
updated_table	The table that contains the new encoded fields.	Table View

Code sample

StandardizeField example 1 (Python window)

The following Python window script demonstrates how to use the StandardizeField function.

arcpy.management.StandardizeField("County_VoterTurnout", 
       "voter_turnout voter_turnout_Z_SCORE", "Z-SCORE")

StandardizeField example 2 (stand-alone script)

The following stand-alone script demonstrates how to use the StandardizeField function.

# Import system modules
import arcpy

# Set the workspace and input features.
arcpy.env.workspace = r"C:\\Standardize\\MyData.gdb"
inputFeatures = "County_VoterTurnout"

# Set the input fields that will be standardized
fields = "votes_total;rawdiff_dem_vs_gop;pctdiff_dem_vs_gop"

# Set the standardization method.
method = "ROBUST"

# Run the Standardize Field tool
arcpy.management.StandardizeField(inputFeatures, fields, method)

Environments

Extent