Create Space Time Cube From Defined Locations (Space Time Pattern Mining)

Summary

Structures panel data or station data (defined locations where geography does not change but attributes are changing over time) into a netCDF data format by creating space-time bins. For all locations, the trend for variables or summary fields is evaluated.

Learn more about how Create Space Time Cube From Defined Locations works

Illustration

Create Space Time Cube From Defined Locations tool illustration

Usage

  • The data structure created by the tool can be thought of as a 3D cube composed of space-time bins with the x and y dimensions representing space and the t dimension representing time.

    Space-time bins in a 3D cube

  • Every bin has a fixed position in space (x,y location if the inputs are points, and a fixed set of vertices if the inputs are polygon locations) and in time (t). Bins covering the same defined location area (either x,y or vertices) share the same location ID. Bins encompassing the same duration share the same time-step ID.

    Locations in the space-time cube

  • Each bin in the space-time cube has a LOCATION_ID, time_step_ID, and COUNT value, as well as Variables or Summary Fields parameter values that were included when the cube was created. Bins associated with the same physical location will share the same location ID and together will represent a time series. Bins associated with the same time-step interval will share the same time-step ID and together will comprise a time slice.

  • The input features can be points or polygons and should represent defined or fixed locations with associated attributes that have been collected over time. This type of data is commonly called panel or station data. The field containing the event time stamp must be of type date.

    Note:

    If the input features are stored in a file geodatabase and contain true curves (stored as arcs as opposed to stored with vertices), polygon shapes will be distorted when they're stored in the space-time cube. To determine whether the input features contain true curves, run the Check Geometry tool with the OGC option for the Validation Method parameter. If you receive an error message indicating that the selected option does not support nonlinear segments, true curves exist in the dataset and can be eliminated and replaced with vertices using the Densify tool with the Angle option for the Densification Method parameter before creating the space-time cube.

  • The input features can be repeating shapes contained in the same feature class or one set of features with a related table containing the attributes recorded over time.

  • The tool will fail if the parameters specified result in a cube with more than 2 billion bins.

  • This tool requires projected data to accurately measure distances.

  • If the Temporal Aggregation parameter is checked, the resulting space-time cube will contain a count value for each bin reflecting the number of events that occurred at the associated location within the associated time-step interval.

  • Output from this tool is a netCDF representation of the input features as well as messages summarizing the cube characteristics. The messages are written at the bottom of the Geoprocessing pane during tool processing. To access the messages, hover over the progress bar and click the pop-out button Pop-out button, or expand the messages section in the Geoprocessing pane. You can also access the messages for a previous run of the tool through the geoprocessing history. You can use the netCDF file as input to other tools such as the Emerging Hot Spot Analysis tool or the Local Outlier Analysis tool. See Visualize the space-time cube for strategies to view cube contents.

  • Provide a field of type date for the Time Field parameter. If the input is repeating shapes, this field should contain the time stamp associated with each feature. If a Related Table parameter value is provided, this field will be the time stamp associated with each record in the table. If the field is high precision (containing millisecond values), the time stamp of each space-time bin will include seconds only, and any milliseconds will be ignored.

  • The Time Step Interval parameter defines how the time span of the data will be partitioned. If the Temporal Aggregation parameter is unchecked, set the Time Step Interval parameter to the existing structure of the data. For example, if you have census data that has been collected every five years, the input should be 5 years. Check the Temporal Aggregation parameter to aggregate temporally. For example, if you have sensor data that has been recording every five minutes, you can aggregate using one-day intervals. Time-step intervals are always fixed durations, and the tool requires a minimum of 10 time steps.

    Note:

    A number of time units appear in the Time Step Interval parameter drop-down list; however, the tool only supports years, months, weeks, days, hours, minutes, and seconds.

  • If the space-time cube could not be created, the tool may have been unable to structure the input data provided into 10 time-step intervals. If you receive an error message, review the time stamps of the input to ensure they include a range of values (at least 10). The range of values must span at least 10 seconds, as this is the smallest time increment that the tool uses. Ten time-step intervals are required by the Mann-Kendall statistic.

  • The Reference Time parameter can be a date and time value or solely a date value; it cannot be solely a time value. The expected format is determined by the computer's regional time settings.

  • The trend analysis performed on the aggregated variables or summary field values is based on the Mann-Kendall statistic.

  • The following statistical operations are available for the aggregation of attributes with this tool: sum, mean, minimum, maximum, standard deviation, and median.

  • Null values present in any of the summary field records will result in those features being excluded from the output cube. If there are null values present in the input features, it is recommended that you run the Fill Missing Values tool first. If, after running the Fill Missing Values tool, null values are still present and having the count of points in each bin is part of your analysis strategy, consider creating separate cubes: one for the count (without summary fields) and one for the summary fields. If the set of null values is different for each summary field, you can also create a separate cube for each summary field.

  • When filling empty bins with spatial neighbors, the tool estimates based on the closest 8 nearest neighbors. A minimum of 4 of those spatial neighbors must have values to fill the empty bin using this option.

  • When filling empty bins with space-time neighbors, the tool estimates based on the closest 8 nearest neighbors. Additionally, temporal neighbors are used for each of those bins found to be spatial neighbors by going backward and forward one time step. A minimum of 13 space time neighbors are required to fill the empty bin using this option.

  • When filling empty bins with temporal trend, the first two time periods and last two time periods at a given location must have values in their bins to interpolate values at other time periods for that location.

  • The temporal trend fill type uses the Interpolated Univariate Spline method in the SciPy Interpolation package.

  • This tool can take advantage of the increased performance available in systems that use multiple CPUs (or multi-core CPUs). The tool will default to run using 50 percent of the processors available; however, the number of CPUs used can be increased or decreased using the Parallel Processing Factor environment. The increased processing speed is most noticeable when creating large space-time cubes.

Parameters

LabelExplanationData Type
Input Features

The input point or polygon feature class that will be converted to a space-time cube.

Feature Layer
Output Space Time Cube

The output netCDF data cube that will be created.

File
Location ID

An integer or text field containing the ID number for each unique location.

Field
Temporal Aggregation

Specifies whether the data will be aggregated temporally.

  • Unchecked—The space-time cube will be created using the existing temporal structure of the input features. For example, you have yearly data and want to create a cube with a yearly time-step interval. This is the default.
  • Checked—The space-time cube will aggregate the features temporally based on the time-step interval provided. For example, the data has been collected daily and you want to create a cube with a weekly time-step interval.
Boolean
Time Field

The field containing the time stamp for each row in the dataset. This field must be of type date.

Field
Time Step Interval
(Optional)

The number of seconds, minutes, hours, days, weeks, or years that will represent a single time step. Examples of valid entries for this parameter are 1 Weeks, 13 Days, or 1 Months.

If you do not apply temporal aggregation (Temporal Aggregation is unchecked), set this parameter to the existing temporal structure of the data.

If you apply temporal aggregation (Temporal Aggregation is checked), set this parameter to the time-step interval you want to create. All features within the same time-step interval will be aggregated.

Time Unit
Time Step Alignment
(Optional)

Specifies how the cube structure will align based on a given time-step interval.

  • End timeTime steps will align to the last time event and aggregate back in time. This is the default.
  • Start timeTime steps will align to the first time event and aggregate forward in time.
  • Reference timeTime steps will align to the specified date and time. If all points in the input features have a time stamp larger than the reference time provided (or it falls exactly on the start time of the input features), the time-step interval will begin with that reference time and aggregate forward in time (as occurs with a start time alignment). If all points in the input features have a time stamp smaller than the reference time provided (or it falls exactly on the end time of the input features), the time-step interval will end with that reference time and aggregate backward in time (as occurs with an end time alignment). If the reference time provided is in the middle of the time extent of the data, a time-step interval will be created ending with the reference time provided (as occurs with an end time alignment). Additional intervals will be created both before and after the reference time until the full time extent of the data is covered.
String
Reference Time
(Optional)

The date and time that will be used to align the time-step intervals. To bin the data weekly from Monday to Sunday, for example, set a reference time of Sunday at midnight to ensure that bins break between Sunday and Monday at midnight.

Date
Variables
(Optional)

The numeric field containing attribute values that will be brought into the space-time cube.

The following are the available fill types:

  • DROP_LOCATIONS—Locations with missing data for any of the variables will be excluded from the output space-time cube.
  • ZEROS—Empty bins will be filled with zeros.
  • SPATIAL_NEIGHBORS—Empty bins will be filled with the average value of spatial neighbors.
  • SPACE_TIME_NEIGHBORS—Empty bins will be filled with the average value of space-time neighbors.
  • TEMPORAL_TREND—Empty bins will be filled using an interpolated univariate spline algorithm.

Note:

Null values in any of the variable records will result in an empty bin. If null values are present in the input features, it is recommended that you run the Fill Missing Values tool first.

Value Table
Summary Fields
(Optional)

The numeric field containing attribute values that will be used to calculate the specified statistic when aggregating into a space-time cube. Multiple statistic and field combinations can be specified. Null values in any of the fields specified will result in that feature being excluded from the output cube. If null values are present in the input features, it is recommended that you run the Fill Missing Values tool before creating a space-time cube.

The following are the available statistic types:

  • SUM—The total value for the specified field within each bin will be calculated.
  • MEAN—The average for the specified field within each bin will be calculated.
  • MIN—The smallest value for all records of the specified field within each bin will be identified.
  • MAX—The largest value for all records of the specified field within each bin will be identified.
  • STD—The standard deviation for values in the specified field within each bin will be calculated.
  • MEDIAN—The sorted middle value of all records of the specified field within each bin will be calculated.

The following are the available fill types:

  • ZEROS—Empty bins will be filled with zeros.
  • SPATIAL_NEIGHBORS—Empty bins will be filled with the average value of spatial neighbors
  • SPACE_TIME_NEIGHBORS—Empty bins will be filled with the average value of space-time neighbors.
  • TEMPORAL_TREND—Empty bins will be filled using an interpolated univariate spline algorithm.

Note:

Null values in any of the summary field records will result in those features being excluded from the output cube. If null values are present in the input features, it is recommended that you run the Fill Missing Values tool first. If, after running the Fill Missing Values tool, null values are still present and having the count of points in each bin is part of your analysis strategy, consider creating separate cubes: one for the count (without summary fields) and one for the summary fields. If the set of null values is different for each summary field, you can create a separate cube for each summary field.

Value Table
Related Table
(Optional)

The table or table view to be related to the input features.

Table View
Related Location ID
(Optional)

An integer or text field in the related table that contains the location ID on which the relate will be based.

Field

arcpy.stpm.CreateSpaceTimeCubeDefinedLocations(in_features, output_cube, location_id, temporal_aggregation, time_field, {time_step_interval}, {time_step_alignment}, {reference_time}, {variables}, {summary_fields}, {in_related_table}, {related_location_id})
NameExplanationData Type
in_features

The input point or polygon feature class that will be converted to a space-time cube.

Feature Layer
output_cube

The output netCDF data cube that will be created.

File
location_id

An integer or text field containing the ID number for each unique location.

Field
temporal_aggregation
  • APPLY_TEMPORAL_AGGREGATIONThe space-time cube will aggregate the features temporally based on the time-step interval provided. For example, the data has been collected daily and you want to create a cube with a weekly time-step interval.
  • NO_TEMPORAL_AGGREGATIONThe space-time cube will be created using the existing temporal structure of the input features. For example, you have yearly data and want to create a cube with a yearly time-step interval. This is the default.
Boolean
time_field

The field containing the time stamp for each row in the dataset. This field must be of type date.

Field
time_step_interval
(Optional)

The number of seconds, minutes, hours, days, weeks, or years that will represent a single time step. Examples of valid entries for this parameter are 1 Weeks, 13 Days, or 1 Months.

If you do not apply temporal aggregation (temporal_aggregation = "NO TEMPORAL_AGGREGATION"), set this parameter to the existing temporal structure of the data.

If you apply temporal aggregation (temporal_aggregation = "APPLY TEMPORAL_AGGREGATION"), set this parameter to the time-step interval you want to create. All features within the same time-step interval will be aggregated.

Time Unit
time_step_alignment
(Optional)

Specifies how the cube structure will align based on a given time_step_interval value.

  • END_TIMETime steps will align to the last time event and aggregate back in time. This is the default.
  • START_TIMETime steps will align to the first time event and aggregate forward in time.
  • REFERENCE_TIMETime steps will align to the specified date and time. If all points in the input features have a time stamp larger than the reference time provided (or it falls exactly on the start time of the input features), the time-step interval will begin with that reference time and aggregate forward in time (as occurs with a start time alignment). If all points in the input features have a time stamp smaller than the reference time provided (or it falls exactly on the end time of the input features), the time-step interval will end with that reference time and aggregate backward in time (as occurs with an end time alignment). If the reference time provided is in the middle of the time extent of the data, a time-step interval will be created ending with the reference time provided (as occurs with an end time alignment). Additional intervals will be created both before and after the reference time until the full time extent of the data is covered.
String
reference_time
(Optional)

The date and time that will be used to align the time-step intervals. To bin the data weekly from Monday to Sunday, for example, set a reference time of Sunday at midnight to ensure that bins break between Sunday and Monday at midnight.

Date
variables
[[Field, Fill Empty Bins with],...]
(Optional)

The numeric field containing attribute values that will be brought into the space-time cube.

The following are the available fill types:

  • DROP_LOCATIONS—Locations with missing data for any of the variables will be excluded from the output space-time cube.
  • ZEROS—Empty bins will be filled with zeros.
  • SPATIAL_NEIGHBORS—Empty bins will be filled with the average value of spatial neighbors.
  • SPACE_TIME_NEIGHBORS—Empty bins will be filled with the average value of space-time neighbors.
  • TEMPORAL_TREND—Empty bins will be filled using an interpolated univariate spline algorithm.

Note:

Null values in any of the variable records will result in an empty bin. If null values are present in the input features, it is recommended that you run the Fill Missing Values tool first.

Value Table
summary_fields
[[Field, Statistic, Fill Empty Bins with],...]
(Optional)

The numeric field containing attribute values that will be used to calculate the specified statistic when aggregating into a space-time cube. Multiple statistic and field combinations can be specified. Null values in any of the fields specified will result in that feature being excluded from the output cube. If null values are present in the input features, it is recommended that you run the Fill Missing Values tool before creating a space-time cube.

The following are the available statistic types:

  • SUM—The total value for the specified field within each bin will be calculated.
  • MEAN—The average for the specified field within each bin will be calculated.
  • MIN—The smallest value for all records of the specified field within each bin will be identified.
  • MAX—The largest value for all records of the specified field within each bin will be identified.
  • STD—The standard deviation for values in the specified field within each bin will be calculated.
  • MEDIAN—The sorted middle value of all records of the specified field within each bin will be calculated.

The following are the available fill types:

  • ZEROS—Empty bins will be filled with zeros.
  • SPATIAL_NEIGHBORS—Empty bins will be filled with the average value of spatial neighbors
  • SPACE_TIME_NEIGHBORS—Empty bins will be filled with the average value of space-time neighbors.
  • TEMPORAL_TREND—Empty bins will be filled using an interpolated univariate spline algorithm.

Note:

Null values in any of the summary field records will result in those features being excluded from the output cube. If null values are present in the input features, it is recommended that you run the Fill Missing Values tool first. If, after running the Fill Missing Values tool, null values are still present and having the count of points in each bin is part of your analysis strategy, consider creating separate cubes: one for the count (without summary fields) and one for the summary fields. If the set of null values is different for each summary field, you can create a separate cube for each summary field.

Value Table
in_related_table
(Optional)

The table or table view to be related to the input features.

Table View
related_location_id
(Optional)

An integer or text field in the related table that contains the location ID on which the relate will be based.

Field

Code sample

CreateSpaceTimeCubeDefinedLocations example 1 (Python window)

The following Python window script demonstrates how to use the CreateSpaceTimeCubeDefinedLocations function.

import arcpy
arcpy.env.workspace = r"C:\STPM\Chicago.gdb"
arcpy.stpm.CreateSpaceTimeCubeDefinedLocations("Chicago_Data", r"C:\STPM\Chicago_Cube.nc", "MYID",
                                               "NO_TEMPORAL_AGGREGATION", "TIME", "1 Months",
                                               "END_TIME", "", "COUNT ZEROS")
CreateSpaceTimeCubeDefinedLocations example 2 (stand-alone script)

The following stand-alone Python script demonstrates how to use the CreateSpaceTimeCubeDefinedLocations function.

# Fill missing values using a feature set and related table
# Use the results to create a space-time cube from defined locations
# Run Emerging Hot Spot Analysis on the data
# Visualize the results in 3d

# Import system modules
import arcpy

# Set overwriteOutput property to overwrite existing output, by default
arcpy.env.overwriteOutput = True

# Local variables ...
arcpy.env.workspace = r"C:\STPM\Chicago.gdb"

try:

    # Fill missing values in a feature class containing block group polygon shapes and a related table containing the incidents
    # Since some of the values are missing, fill them using the temporal trend method

    arcpy.stpm.FillMissingValues("Chicago_Feature", "Chicago_FilledFeature", "COUNT", "TEMPORAL_TREND", "", "", NoneNone,
                                 "TIME", "", "MYID", "Chicago_Table", "MYID", "", "", "", "Chicago_FilledTable")



    # Create a defined location space-time cube using a related table
    # Using a reference time at the start of the month to force binning fall on month breaks
    # Using temporal aggregation to sum multiple entries into one month
    # Using the method drop location if missing values since you already filled using Fill Missing Values
    arcpy.stpm.CreateSpaceTimeCubeDefinedLocations("Chicago_FilledFeature", r"C:\STPM\Chicago_Cube.nc", "MYID",
                                                   "APPLY_TEMPORAL_AGGREGATION", "TIME", "1 Months", "REFERENCE_TIME",
                                                   "10/1/2015", "", "COUNT SUM DROP_LOCATIONS", "Chicago_FilledTable",
                                                   "MYID")

    # Run an emerging hot spot analysis on the defined locations cube
    # Using contiguity edges so only block groups that bound each other are considered neighbors
    arcpy.stpm.EmergingHotSpotAnalysis(r"C:\STPM\Chicago_Cube.nc", "COUNT_SUM_NONE",
                                       "Chicago_Cube_EmergingHotSpot", "", 1, "",
                                       "CONTIGUITY_EDGES_ONLY")

    # Use Visualize Cube in 3d to see the hot spot results for each time slice
    arcpy.stpm.VisualizeSpaceTimeCube3D(r"C:\STPM\Chicago_Cube.nc", "COUNT_SUM_NONE", "HOT_AND_COLD_SPOT_RESULTS",
                                        "Chicago_Cube_Visualize3d")

except arcpy.ExecuteError:
    # If any error occurred when running the tool, print the messages
    print(arcpy.GetMessages())